2024-04-23T14:14:40.108 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-23T14:14:40.109 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/pool-perm} --name rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi --owner scheduled_rishabh@teuthology --verbose --archive /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669587 -- /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669587/orig.config.yaml 2024-04-23T14:14:40.153 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T14:14:40.251 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669587 branch: wip-rishabh-testing-20240416.193735-3 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/pool-perm} email: null first_in_suite: false job_id: '7669587' kernel: flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-rishabh-testing-20240416.193735-3 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d44fddd910b5e94da0f2357af4be37d7bd495ae0 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d44fddd910b5e94da0f2357af4be37d7bd495ae0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240416.193735-3 sha1: d44fddd910b5e94da0f2357af4be37d7bd495ae0 owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7954 sha1: d44fddd910b5e94da0f2357af4be37d7bd495ae0 sleep_before_teardown: 0 subset: 364/512 suite: fs suite_branch: wip-rishabh-testing-20240416.193735-3 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d44fddd910b5e94da0f2357af4be37d7bd495ae0 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE/aCshXzWt2yH2pHsDWDUnQMtkY17vqfGEhcBIgWVJVdM6pUoFiqJpdrUwnY+nyeH6VB0CdTY6TY8BHoavBEJY= smithi151.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG95C7i0TtHFsiFmo24RegfJZIQu1ezh/b6jEqzBhGM/8ylKezjQ6tKnnv+fWihm7hbm1I4T7PxgWGf5ucNRTaQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_pool_perm teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-23_14:04:26 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-23T14:14:40.251 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa; will attempt to use it 2024-04-23T14:14:40.252 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa/tasks 2024-04-23T14:14:40.252 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-23T14:14:40.254 INFO:teuthology.task.internal:Checking packages... 2024-04-23T14:14:40.277 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'd44fddd910b5e94da0f2357af4be37d7bd495ae0' 2024-04-23T14:14:40.278 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-23T14:14:40.278 INFO:teuthology.packaging:ref: None 2024-04-23T14:14:40.278 INFO:teuthology.packaging:tag: None 2024-04-23T14:14:40.278 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240416.193735-3 2024-04-23T14:14:40.278 INFO:teuthology.packaging:sha1: d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:14:40.278 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-rishabh-testing-20240416.193735-3 2024-04-23T14:14:40.505 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3126-gd44fddd9-1jammy 2024-04-23T14:14:40.507 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-23T14:14:40.516 INFO:teuthology.task.internal:no buildpackages task found 2024-04-23T14:14:40.516 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-23T14:14:40.539 INFO:teuthology.task.internal:Saving configuration 2024-04-23T14:14:40.555 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-23T14:14:40.579 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T14:14:40.601 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669587', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 14:05:43.108657', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFVZMw4ZVVUW/TsCi0WaZl1kJe7hZYWbiw3y0eiEBV9h'} 2024-04-23T14:14:40.625 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669587', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 14:05:43.109955', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPME8j751QeVuiKPkz566YFLMoLPgg9pn3JuAHAtlbst'} 2024-04-23T14:14:40.625 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-23T14:14:40.632 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-23T14:14:40.632 INFO:teuthology.task.internal:roles: ubuntu@smithi151.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-23T14:14:40.632 INFO:teuthology.run_tasks:Running task console_log... 2024-04-23T14:14:40.719 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fed4b3cb4c0>, signals=[15]) 2024-04-23T14:14:40.720 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-23T14:14:40.727 INFO:teuthology.task.internal:Opening connections... 2024-04-23T14:14:40.727 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:14:40.729 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:14:40.810 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi151.front.sepia.ceph.com 2024-04-23T14:14:40.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:14:40.890 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-23T14:14:40.896 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-04-23T14:14:40.901 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-04-23T14:14:40.901 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-04-23T14:14:40.947 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-23T14:14:40.947 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="Ubuntu" 2024-04-23T14:14:40.947 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="22.04" 2024-04-23T14:14:40.947 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-23T14:14:40.947 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_CODENAME=jammy 2024-04-23T14:14:40.947 INFO:teuthology.orchestra.run.smithi100.stdout:ID=ubuntu 2024-04-23T14:14:40.947 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE=debian 2024-04-23T14:14:40.947 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-23T14:14:40.947 INFO:teuthology.orchestra.run.smithi100.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-23T14:14:40.947 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-23T14:14:40.947 INFO:teuthology.orchestra.run.smithi100.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-23T14:14:40.947 INFO:teuthology.orchestra.run.smithi100.stdout:UBUNTU_CODENAME=jammy 2024-04-23T14:14:40.948 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-04-23T14:14:40.991 DEBUG:teuthology.orchestra.run.smithi151:> uname -m 2024-04-23T14:14:40.997 INFO:teuthology.orchestra.run.smithi151.stdout:x86_64 2024-04-23T14:14:40.997 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release 2024-04-23T14:14:41.043 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-23T14:14:41.043 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="Ubuntu" 2024-04-23T14:14:41.043 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="22.04" 2024-04-23T14:14:41.043 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-23T14:14:41.043 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_CODENAME=jammy 2024-04-23T14:14:41.043 INFO:teuthology.orchestra.run.smithi151.stdout:ID=ubuntu 2024-04-23T14:14:41.043 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE=debian 2024-04-23T14:14:41.043 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-23T14:14:41.043 INFO:teuthology.orchestra.run.smithi151.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-23T14:14:41.044 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-23T14:14:41.044 INFO:teuthology.orchestra.run.smithi151.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-23T14:14:41.044 INFO:teuthology.orchestra.run.smithi151.stdout:UBUNTU_CODENAME=jammy 2024-04-23T14:14:41.044 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2024-04-23T14:14:41.073 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-23T14:14:41.083 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-23T14:14:41.105 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-23T14:14:41.106 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2024-04-23T14:14:41.108 DEBUG:teuthology.orchestra.run.smithi151:> test '!' -e /home/ubuntu/cephtest 2024-04-23T14:14:41.113 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-23T14:14:41.133 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-23T14:14:41.134 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2024-04-23T14:14:41.157 DEBUG:teuthology.orchestra.run.smithi151:> test -z $(ls -A /var/lib/ceph) 2024-04-23T14:14:41.175 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-23T14:14:41.233 INFO:teuthology.run_tasks:Running task kernel... 2024-04-23T14:14:41.248 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-23T14:14:41.248 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, timeout 300 2024-04-23T14:14:41.249 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T14:14:41.249 DEBUG:teuthology.orchestra.run.smithi151:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T14:14:41.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:14:41.254 DEBUG:teuthology.task.kernel:sha1 for mon.a is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-23T14:14:41.254 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-23T14:14:41.254 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-04-23T14:14:41.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:14:41.255 DEBUG:teuthology.task.kernel:sha1 for mon.b is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-23T14:14:41.255 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-23T14:14:41.256 DEBUG:teuthology.orchestra.run.smithi151:> uname -r 2024-04-23T14:14:41.300 INFO:teuthology.orchestra.run.smithi100.stdout:5.15.0-56-generic 2024-04-23T14:14:41.300 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-23T14:14:41.300 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-23T14:14:41.300 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-23T14:14:41.304 INFO:teuthology.orchestra.run.smithi151.stdout:5.15.0-56-generic 2024-04-23T14:14:41.305 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-23T14:14:41.305 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-23T14:14:41.305 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-23T14:14:41.450 INFO:teuthology.task.kernel:Installing linux-firmware on mon.b... 2024-04-23T14:14:41.450 DEBUG:teuthology.orchestra.run.smithi151:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-04-23T14:14:41.471 INFO:teuthology.orchestra.run.smithi151.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-04-23T14:14:41.523 INFO:teuthology.task.kernel:Installing linux-firmware on mon.a... 2024-04-23T14:14:41.523 DEBUG:teuthology.orchestra.run.smithi100:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-04-23T14:14:41.548 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-04-23T14:14:41.607 INFO:teuthology.orchestra.run.smithi151.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-04-23T14:14:41.607 INFO:teuthology.orchestra.run.smithi151.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-04-23T14:14:41.607 INFO:teuthology.orchestra.run.smithi151.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-04-23T14:14:41.607 INFO:teuthology.orchestra.run.smithi151.stderr:hint: 2024-04-23T14:14:41.607 INFO:teuthology.orchestra.run.smithi151.stderr:hint: git config --global init.defaultBranch 2024-04-23T14:14:41.608 INFO:teuthology.orchestra.run.smithi151.stderr:hint: 2024-04-23T14:14:41.608 INFO:teuthology.orchestra.run.smithi151.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-04-23T14:14:41.608 INFO:teuthology.orchestra.run.smithi151.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-04-23T14:14:41.608 INFO:teuthology.orchestra.run.smithi151.stderr:hint: 2024-04-23T14:14:41.608 INFO:teuthology.orchestra.run.smithi151.stderr:hint: git branch -m 2024-04-23T14:14:41.609 INFO:teuthology.orchestra.run.smithi151.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-04-23T14:14:41.611 DEBUG:teuthology.orchestra.run.smithi151:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-04-23T14:14:41.655 INFO:teuthology.orchestra.run.smithi100.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-04-23T14:14:41.655 INFO:teuthology.orchestra.run.smithi100.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-04-23T14:14:41.655 INFO:teuthology.orchestra.run.smithi100.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-04-23T14:14:41.655 INFO:teuthology.orchestra.run.smithi100.stderr:hint: 2024-04-23T14:14:41.655 INFO:teuthology.orchestra.run.smithi100.stderr:hint: git config --global init.defaultBranch 2024-04-23T14:14:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:hint: 2024-04-23T14:14:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-04-23T14:14:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-04-23T14:14:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:hint: 2024-04-23T14:14:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:hint: git branch -m 2024-04-23T14:14:41.657 INFO:teuthology.orchestra.run.smithi100.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-04-23T14:14:41.658 DEBUG:teuthology.orchestra.run.smithi100:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-04-23T14:14:41.679 DEBUG:teuthology.orchestra.run.smithi151:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-04-23T14:14:41.723 DEBUG:teuthology.orchestra.run.smithi100:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-04-23T14:14:41.738 DEBUG:teuthology.orchestra.run.smithi151:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-04-23T14:14:41.780 DEBUG:teuthology.orchestra.run.smithi100:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-04-23T14:17:13.070 INFO:teuthology.orchestra.run.smithi151.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-04-23T14:17:13.070 INFO:teuthology.orchestra.run.smithi151.stderr: * [new branch] main -> origin/main 2024-04-23T14:17:13.103 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20190312 -> 20190312 2024-04-23T14:17:13.103 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20190416 -> 20190416 2024-04-23T14:17:13.103 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20190514 -> 20190514 2024-04-23T14:17:13.103 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20190618 -> 20190618 2024-04-23T14:17:13.103 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20190717 -> 20190717 2024-04-23T14:17:13.103 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20190815 -> 20190815 2024-04-23T14:17:13.104 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20190923 -> 20190923 2024-04-23T14:17:13.104 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20191022 -> 20191022 2024-04-23T14:17:13.104 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20191215 -> 20191215 2024-04-23T14:17:13.104 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20200122 -> 20200122 2024-04-23T14:17:13.104 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20200316 -> 20200316 2024-04-23T14:17:13.106 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20200421 -> 20200421 2024-04-23T14:17:13.106 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20200519 -> 20200519 2024-04-23T14:17:13.106 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20200619 -> 20200619 2024-04-23T14:17:13.106 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20200721 -> 20200721 2024-04-23T14:17:13.107 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20200817 -> 20200817 2024-04-23T14:17:13.107 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20200918 -> 20200918 2024-04-23T14:17:13.107 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20201022 -> 20201022 2024-04-23T14:17:13.107 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20201118 -> 20201118 2024-04-23T14:17:13.107 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20201218 -> 20201218 2024-04-23T14:17:13.107 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20210208 -> 20210208 2024-04-23T14:17:13.107 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20210315 -> 20210315 2024-04-23T14:17:13.107 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20210511 -> 20210511 2024-04-23T14:17:13.107 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20210716 -> 20210716 2024-04-23T14:17:13.107 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20210818 -> 20210818 2024-04-23T14:17:13.107 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20210919 -> 20210919 2024-04-23T14:17:13.107 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20211027 -> 20211027 2024-04-23T14:17:13.107 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20211216 -> 20211216 2024-04-23T14:17:13.107 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20220209 -> 20220209 2024-04-23T14:17:13.108 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20220310 -> 20220310 2024-04-23T14:17:13.108 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20220411 -> 20220411 2024-04-23T14:17:13.108 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20220509 -> 20220509 2024-04-23T14:17:13.110 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20220610 -> 20220610 2024-04-23T14:17:13.110 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20220708 -> 20220708 2024-04-23T14:17:13.110 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20220815 -> 20220815 2024-04-23T14:17:13.110 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20220913 -> 20220913 2024-04-23T14:17:13.110 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20221012 -> 20221012 2024-04-23T14:17:13.110 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20221109 -> 20221109 2024-04-23T14:17:13.111 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20221214 -> 20221214 2024-04-23T14:17:13.111 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20230117 -> 20230117 2024-04-23T14:17:13.111 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20230210 -> 20230210 2024-04-23T14:17:13.111 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20230310 -> 20230310 2024-04-23T14:17:13.111 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20230404 -> 20230404 2024-04-23T14:17:13.111 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20230515 -> 20230515 2024-04-23T14:17:13.111 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20230625 -> 20230625 2024-04-23T14:17:13.111 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20230804 -> 20230804 2024-04-23T14:17:13.111 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20230919 -> 20230919 2024-04-23T14:17:13.111 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20231030 -> 20231030 2024-04-23T14:17:13.111 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20231111 -> 20231111 2024-04-23T14:17:13.111 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20231211 -> 20231211 2024-04-23T14:17:13.111 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20240115 -> 20240115 2024-04-23T14:17:13.111 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20240220 -> 20240220 2024-04-23T14:17:13.111 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20240312 -> 20240312 2024-04-23T14:17:13.111 INFO:teuthology.orchestra.run.smithi151.stderr: * [new tag] 20240410 -> 20240410 2024-04-23T14:17:17.384 INFO:teuthology.orchestra.run.smithi151.stderr:Updating files: 48% (1777/3633) Updating files: 49% (1781/3633) Updating files: 50% (1817/3633) Updating files: 51% (1853/3633) Updating files: 52% (1890/3633) Updating files: 53% (1926/3633) Updating files: 54% (1962/3633) Updating files: 55% (1999/3633) Updating files: 56% (2035/3633) Updating files: 57% (2071/3633) Updating files: 58% (2108/3633) Updating files: 59% (2144/3633) Updating files: 60% (2180/3633) Updating files: 61% (2217/3633) Updating files: 62% (2253/3633) Updating files: 62% (2260/3633) Updating files: 63% (2289/3633) Updating files: 64% (2326/3633) Updating files: 65% (2362/3633) Updating files: 66% (2398/3633) Updating files: 67% (2435/3633) Updating files: 68% (2471/3633) Updating files: 69% (2507/3633) Updating files: 70% (2544/3633) Updating files: 71% (2580/3633) Updating files: 71% (2614/3633) Updating files: 72% (2616/3633) Updating files: 73% (2653/3633) Updating files: 74% (2689/3633) Updating files: 75% (2725/3633) Updating files: 76% (2762/3633) Updating files: 77% (2798/3633) Updating files: 78% (2834/3633) Updating files: 79% (2871/3633) Updating files: 80% (2907/3633) Updating files: 81% (2943/3633) Updating files: 82% (2980/3633) Updating files: 83% (3016/3633) Updating files: 84% (3052/3633) Updating files: 84% (3063/3633) Updating files: 85% (3089/3633) Updating files: 86% (3125/3633) Updating files: 87% (3161/3633) Updating files: 88% (3198/3633) Updating files: 89% (3234/3633) Updating files: 90% (3270/3633) Updating files: 91% (3307/3633) Updating files: 92% (3343/3633) Updating files: 93% (3379/3633) Updating files: 94% (3416/3633) Updating files: 95% (3452/3633) Updating files: 96% (3488/3633) Updating files: 97% (3525/3633) Updating files: 98% (3561/3633) Updating files: 99% (3597/3633) Updating files: 100% (3633/3633) Updating files: 100% (3633/3633), done. 2024-04-23T14:17:17.389 INFO:teuthology.orchestra.run.smithi151.stdout:HEAD is now at e6309bb2 Merge branch 'robot/patch-0-1713581904' into 'main' 2024-04-23T14:17:17.411 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-23T14:17:17.412 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-23T14:17:17.637 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ 2024-04-23T14:17:17.638 DEBUG:teuthology.orchestra.run.smithi151:> rm -f /tmp/linux-image.deb && echo linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ --input-file=- 2024-04-23T14:17:17.640 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi151.front.sepia.ceph.com to complete... 2024-04-23T14:17:20.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23 14:17:19 URL:https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb [44805284/44805284] -> "/tmp/linux-image.deb" [1] 2024-04-23T14:17:20.098 INFO:teuthology.orchestra.run.smithi151.stderr:FINISHED --2024-04-23 14:17:19-- 2024-04-23T14:17:20.098 INFO:teuthology.orchestra.run.smithi151.stderr:Total wall clock time: 2.4s 2024-04-23T14:17:20.099 INFO:teuthology.orchestra.run.smithi151.stderr:Downloaded: 1 files, 43M in 2.0s (20.9 MB/s) 2024-04-23T14:17:20.102 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-23T14:17:20.103 DEBUG:teuthology.orchestra.run.smithi151:> sudo dpkg -i /tmp/linux-image.deb 2024-04-23T14:17:20.372 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package linux-image-6.9.0-rc3-g806ec36b2112. 2024-04-23T14:17:22.524 INFO:teuthology.orchestra.run.smithi151.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-04-23T14:17:22.529 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-04-23T14:17:22.733 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-23T14:17:27.131 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-23T14:17:27.244 INFO:teuthology.orchestra.run.smithi151.stderr:update-initramfs: Generating /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-23T14:17:28.435 INFO:teuthology.orchestra.run.smithi100.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-04-23T14:17:28.435 INFO:teuthology.orchestra.run.smithi100.stderr: * [new branch] main -> origin/main 2024-04-23T14:17:28.469 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20190312 -> 20190312 2024-04-23T14:17:28.469 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20190416 -> 20190416 2024-04-23T14:17:28.469 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20190514 -> 20190514 2024-04-23T14:17:28.469 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20190618 -> 20190618 2024-04-23T14:17:28.469 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20190717 -> 20190717 2024-04-23T14:17:28.469 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20190815 -> 20190815 2024-04-23T14:17:28.469 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20190923 -> 20190923 2024-04-23T14:17:28.469 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20191022 -> 20191022 2024-04-23T14:17:28.469 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20191215 -> 20191215 2024-04-23T14:17:28.469 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20200122 -> 20200122 2024-04-23T14:17:28.469 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20200316 -> 20200316 2024-04-23T14:17:28.475 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20200421 -> 20200421 2024-04-23T14:17:28.475 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20200519 -> 20200519 2024-04-23T14:17:28.475 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20200619 -> 20200619 2024-04-23T14:17:28.475 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20200721 -> 20200721 2024-04-23T14:17:28.475 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20200817 -> 20200817 2024-04-23T14:17:28.475 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20200918 -> 20200918 2024-04-23T14:17:28.475 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20201022 -> 20201022 2024-04-23T14:17:28.475 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20201118 -> 20201118 2024-04-23T14:17:28.475 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20201218 -> 20201218 2024-04-23T14:17:28.476 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20210208 -> 20210208 2024-04-23T14:17:28.476 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20210315 -> 20210315 2024-04-23T14:17:28.476 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20210511 -> 20210511 2024-04-23T14:17:28.476 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20210716 -> 20210716 2024-04-23T14:17:28.476 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20210818 -> 20210818 2024-04-23T14:17:28.476 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20210919 -> 20210919 2024-04-23T14:17:28.476 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20211027 -> 20211027 2024-04-23T14:17:28.476 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20211216 -> 20211216 2024-04-23T14:17:28.476 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20220209 -> 20220209 2024-04-23T14:17:28.476 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20220310 -> 20220310 2024-04-23T14:17:28.476 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20220411 -> 20220411 2024-04-23T14:17:28.476 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20220509 -> 20220509 2024-04-23T14:17:28.476 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20220610 -> 20220610 2024-04-23T14:17:28.476 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20220708 -> 20220708 2024-04-23T14:17:28.476 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20220815 -> 20220815 2024-04-23T14:17:28.477 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20220913 -> 20220913 2024-04-23T14:17:28.477 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20221012 -> 20221012 2024-04-23T14:17:28.477 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20221109 -> 20221109 2024-04-23T14:17:28.477 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20221214 -> 20221214 2024-04-23T14:17:28.477 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20230117 -> 20230117 2024-04-23T14:17:28.477 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20230210 -> 20230210 2024-04-23T14:17:28.477 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20230310 -> 20230310 2024-04-23T14:17:28.477 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20230404 -> 20230404 2024-04-23T14:17:28.477 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20230515 -> 20230515 2024-04-23T14:17:28.477 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20230625 -> 20230625 2024-04-23T14:17:28.477 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20230804 -> 20230804 2024-04-23T14:17:28.477 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20230919 -> 20230919 2024-04-23T14:17:28.477 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20231030 -> 20231030 2024-04-23T14:17:28.477 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20231111 -> 20231111 2024-04-23T14:17:28.478 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20231211 -> 20231211 2024-04-23T14:17:28.478 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20240115 -> 20240115 2024-04-23T14:17:28.478 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20240220 -> 20240220 2024-04-23T14:17:28.478 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20240312 -> 20240312 2024-04-23T14:17:28.478 INFO:teuthology.orchestra.run.smithi100.stderr: * [new tag] 20240410 -> 20240410 2024-04-23T14:17:32.766 INFO:teuthology.orchestra.run.smithi100.stderr:Updating files: 48% (1765/3633) Updating files: 49% (1781/3633) Updating files: 50% (1817/3633) Updating files: 51% (1853/3633) Updating files: 52% (1890/3633) Updating files: 53% (1926/3633) Updating files: 54% (1962/3633) Updating files: 55% (1999/3633) Updating files: 56% (2035/3633) Updating files: 57% (2071/3633) Updating files: 58% (2108/3633) Updating files: 59% (2144/3633) Updating files: 60% (2180/3633) Updating files: 61% (2217/3633) Updating files: 62% (2253/3633) Updating files: 62% (2256/3633) Updating files: 63% (2289/3633) Updating files: 64% (2326/3633) Updating files: 65% (2362/3633) Updating files: 66% (2398/3633) Updating files: 67% (2435/3633) Updating files: 68% (2471/3633) Updating files: 69% (2507/3633) Updating files: 70% (2544/3633) Updating files: 71% (2580/3633) Updating files: 71% (2614/3633) Updating files: 72% (2616/3633) Updating files: 73% (2653/3633) Updating files: 74% (2689/3633) Updating files: 75% (2725/3633) Updating files: 76% (2762/3633) Updating files: 77% (2798/3633) Updating files: 78% (2834/3633) Updating files: 79% (2871/3633) Updating files: 80% (2907/3633) Updating files: 81% (2943/3633) Updating files: 82% (2980/3633) Updating files: 83% (3016/3633) Updating files: 84% (3052/3633) Updating files: 84% (3059/3633) Updating files: 85% (3089/3633) Updating files: 86% (3125/3633) Updating files: 87% (3161/3633) Updating files: 88% (3198/3633) Updating files: 89% (3234/3633) Updating files: 90% (3270/3633) Updating files: 91% (3307/3633) Updating files: 92% (3343/3633) Updating files: 93% (3379/3633) Updating files: 94% (3416/3633) Updating files: 95% (3452/3633) Updating files: 96% (3488/3633) Updating files: 97% (3525/3633) Updating files: 98% (3561/3633) Updating files: 99% (3597/3633) Updating files: 100% (3633/3633) Updating files: 100% (3633/3633), done. 2024-04-23T14:17:32.771 INFO:teuthology.orchestra.run.smithi100.stdout:HEAD is now at e6309bb2 Merge branch 'robot/patch-0-1713581904' into 'main' 2024-04-23T14:17:32.792 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-23T14:17:32.793 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-23T14:17:32.942 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ 2024-04-23T14:17:32.942 DEBUG:teuthology.orchestra.run.smithi100:> rm -f /tmp/linux-image.deb && echo linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ --input-file=- 2024-04-23T14:17:32.945 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi100.front.sepia.ceph.com to complete... 2024-04-23T14:17:34.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23 14:17:34 URL:https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb [44805284/44805284] -> "/tmp/linux-image.deb" [1] 2024-04-23T14:17:34.740 INFO:teuthology.orchestra.run.smithi100.stderr:FINISHED --2024-04-23 14:17:34-- 2024-04-23T14:17:34.741 INFO:teuthology.orchestra.run.smithi100.stderr:Total wall clock time: 1.7s 2024-04-23T14:17:34.741 INFO:teuthology.orchestra.run.smithi100.stderr:Downloaded: 1 files, 43M in 1.6s (27.6 MB/s) 2024-04-23T14:17:34.745 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-23T14:17:34.745 DEBUG:teuthology.orchestra.run.smithi100:> sudo dpkg -i /tmp/linux-image.deb 2024-04-23T14:17:35.181 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-image-6.9.0-rc3-g806ec36b2112. 2024-04-23T14:17:37.222 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-04-23T14:17:37.226 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-04-23T14:17:37.338 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-23T14:17:41.130 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-23T14:17:41.238 INFO:teuthology.orchestra.run.smithi100.stderr:update-initramfs: Generating /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-23T14:17:50.808 INFO:teuthology.orchestra.run.smithi151.stderr:Sourcing file `/etc/default/grub' 2024-04-23T14:17:50.832 INFO:teuthology.orchestra.run.smithi151.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-23T14:17:50.859 INFO:teuthology.orchestra.run.smithi151.stderr:Generating grub configuration file ... 2024-04-23T14:17:51.239 INFO:teuthology.orchestra.run.smithi151.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-23T14:17:51.258 INFO:teuthology.orchestra.run.smithi151.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-23T14:17:51.562 INFO:teuthology.orchestra.run.smithi151.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-23T14:17:51.573 INFO:teuthology.orchestra.run.smithi151.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-23T14:17:51.631 INFO:teuthology.orchestra.run.smithi151.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-23T14:17:51.637 INFO:teuthology.orchestra.run.smithi151.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-23T14:17:51.842 INFO:teuthology.orchestra.run.smithi151.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-23T14:17:51.842 INFO:teuthology.orchestra.run.smithi151.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-23T14:17:51.842 INFO:teuthology.orchestra.run.smithi151.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-23T14:17:51.886 INFO:teuthology.orchestra.run.smithi151.stderr:done 2024-04-23T14:17:52.080 DEBUG:teuthology.orchestra.run.smithi151:> dpkg-deb -c /tmp/linux-image.deb 2024-04-23T14:17:52.087 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./ 2024-04-23T14:17:52.087 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./boot/ 2024-04-23T14:17:52.087 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 7002012 2024-04-19 08:55 ./boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-23T14:17:52.104 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 172073 2024-04-19 08:55 ./boot/config-6.9.0-rc3-g806ec36b2112 2024-04-23T14:17:52.105 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13120512 2024-04-19 08:55 ./boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-23T14:17:52.128 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/ 2024-04-23T14:17:52.128 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/ 2024-04-23T14:17:52.128 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postinst.d/ 2024-04-23T14:17:52.128 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postrm.d/ 2024-04-23T14:17:52.128 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/preinst.d/ 2024-04-23T14:17:52.128 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/prerm.d/ 2024-04-23T14:17:52.129 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/ 2024-04-23T14:17:52.129 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/ 2024-04-23T14:17:52.129 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/ 2024-04-23T14:17:52.129 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/ 2024-04-23T14:17:52.129 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/ 2024-04-23T14:17:52.129 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/ 2024-04-23T14:17:52.129 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ 2024-04-23T14:17:52.129 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 507569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/aesni-intel.ko 2024-04-23T14:17:52.129 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-23T14:17:52.129 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 34313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-23T14:17:52.130 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-23T14:17:52.130 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-23T14:17:52.130 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 109905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-23T14:17:52.130 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 155249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-23T14:17:52.130 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-23T14:17:52.130 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-23T14:17:52.131 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 87009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-23T14:17:52.131 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-23T14:17:52.131 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 50305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-23T14:17:52.131 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 50377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-23T14:17:52.131 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 54513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-23T14:17:52.131 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 44657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-23T14:17:52.131 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 48049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-23T14:17:52.131 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 62873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-23T14:17:52.131 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 53569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-23T14:17:52.131 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 34761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-23T14:17:52.131 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-23T14:17:52.131 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/ 2024-04-23T14:17:52.131 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/ 2024-04-23T14:17:52.132 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/ 2024-04-23T14:17:52.132 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-23T14:17:52.132 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpuid.ko 2024-04-23T14:17:52.132 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/msr.ko 2024-04-23T14:17:52.132 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/ 2024-04-23T14:17:52.132 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 444425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-23T14:17:52.132 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 1035881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-23T14:17:52.132 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 2849609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-23T14:17:52.213 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ 2024-04-23T14:17:52.213 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 65025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/af_alg.ko 2024-04-23T14:17:52.214 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 27961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_hash.ko 2024-04-23T14:17:52.214 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_skcipher.ko 2024-04-23T14:17:52.214 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-23T14:17:52.214 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-23T14:17:52.214 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-23T14:17:52.214 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/ 2024-04-23T14:17:52.214 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-04-23T14:17:52.214 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/ 2024-04-23T14:17:52.214 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-23T14:17:52.214 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-23T14:17:52.214 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-23T14:17:52.214 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-23T14:17:52.214 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-23T14:17:52.215 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-23T14:17:52.215 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-23T14:17:52.215 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 27953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-23T14:17:52.215 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-23T14:17:52.215 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-23T14:17:52.215 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-23T14:17:52.215 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-23T14:17:52.215 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-23T14:17:52.215 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-23T14:17:52.215 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-23T14:17:52.215 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 43281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-23T14:17:52.215 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-23T14:17:52.215 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-23T14:17:52.216 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 59129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cryptd.ko 2024-04-23T14:17:52.216 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 48113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-23T14:17:52.216 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 32169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_simd.ko 2024-04-23T14:17:52.216 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 30881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-23T14:17:52.216 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-23T14:17:52.216 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-23T14:17:52.216 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-23T14:17:52.216 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-23T14:17:52.216 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 30689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-23T14:17:52.216 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-23T14:17:52.216 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-23T14:17:52.216 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lrw.ko 2024-04-23T14:17:52.216 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-23T14:17:52.216 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-23T14:17:52.217 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-23T14:17:52.217 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-23T14:17:52.217 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-23T14:17:52.217 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-23T14:17:52.217 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-23T14:17:52.217 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 30673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-23T14:17:52.217 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-23T14:17:52.217 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 169065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-23T14:17:52.217 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-23T14:17:52.217 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-23T14:17:52.217 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-23T14:17:52.217 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-23T14:17:52.217 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-23T14:17:52.217 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-23T14:17:52.218 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-23T14:17:52.218 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xts.ko 2024-04-23T14:17:52.218 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-23T14:17:52.218 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ 2024-04-23T14:17:52.218 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ 2024-04-23T14:17:52.218 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_extlog.ko 2024-04-23T14:17:52.218 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 31921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-23T14:17:52.218 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 32281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-23T14:17:52.218 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/ 2024-04-23T14:17:52.218 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 44705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/einj.ko 2024-04-23T14:17:52.218 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-23T14:17:52.218 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-23T14:17:52.218 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-23T14:17:52.218 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-23T14:17:52.219 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 119033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-23T14:17:52.219 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ 2024-04-23T14:17:52.219 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-23T14:17:52.219 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 96825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci.ko 2024-04-23T14:17:52.219 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-23T14:17:52.219 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 107737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci.ko 2024-04-23T14:17:52.219 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 42001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-23T14:17:52.219 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-23T14:17:52.219 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-23T14:17:52.219 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-23T14:17:52.219 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-23T14:17:52.219 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-23T14:17:52.219 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-23T14:17:52.220 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-23T14:17:52.220 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-23T14:17:52.220 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-23T14:17:52.220 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-23T14:17:52.220 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-23T14:17:52.220 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 38913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-23T14:17:52.220 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-23T14:17:52.220 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-23T14:17:52.220 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-23T14:17:52.220 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 41289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-23T14:17:52.220 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-23T14:17:52.220 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 60257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_legacy.ko 2024-04-23T14:17:52.220 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-23T14:17:52.220 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-23T14:17:52.221 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-23T14:17:52.221 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-23T14:17:52.221 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-23T14:17:52.221 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-23T14:17:52.221 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-23T14:17:52.221 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-23T14:17:52.221 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-23T14:17:52.221 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-23T14:17:52.221 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 43009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-23T14:17:52.221 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-23T14:17:52.221 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-23T14:17:52.221 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_radisys.ko 2024-04-23T14:17:52.221 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-23T14:17:52.221 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rz1000.ko 2024-04-23T14:17:52.222 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-23T14:17:52.222 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-23T14:17:52.222 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-23T14:17:52.222 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-23T14:17:52.222 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-23T14:17:52.222 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-23T14:17:52.222 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-23T14:17:52.222 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 34889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-23T14:17:52.222 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 102121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-23T14:17:52.222 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 85409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-23T14:17:52.222 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-23T14:17:52.222 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-23T14:17:52.222 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 34025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-23T14:17:52.222 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 45905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-23T14:17:52.223 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-23T14:17:52.223 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-23T14:17:52.223 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 53353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-23T14:17:52.223 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-23T14:17:52.223 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 41049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-23T14:17:52.223 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-23T14:17:52.223 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ 2024-04-23T14:17:52.223 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-23T14:17:52.223 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/ 2024-04-23T14:17:52.223 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 163969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-23T14:17:52.223 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/ 2024-04-23T14:17:52.223 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/ 2024-04-23T14:17:52.223 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 140865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-23T14:17:52.223 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/ 2024-04-23T14:17:52.223 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 1241657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-23T14:17:52.224 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 203289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-23T14:17:52.224 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/ 2024-04-23T14:17:52.224 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 115265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-23T14:17:52.224 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 143433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-23T14:17:52.224 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/ 2024-04-23T14:17:52.224 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 126617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-23T14:17:52.224 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 119873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-23T14:17:52.224 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 313217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-23T14:17:52.224 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/ 2024-04-23T14:17:52.224 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 117233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-23T14:17:52.224 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/ 2024-04-23T14:17:52.224 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 70385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-23T14:17:52.224 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ 2024-04-23T14:17:52.225 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/applicom.ko 2024-04-23T14:17:52.225 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-23T14:17:52.225 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/ 2024-04-23T14:17:52.225 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-23T14:17:52.225 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-23T14:17:52.225 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-23T14:17:52.225 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-23T14:17:52.225 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-23T14:17:52.225 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ 2024-04-23T14:17:52.225 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 37257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-23T14:17:52.225 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 186921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-23T14:17:52.225 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 32521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-23T14:17:52.225 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 179561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-23T14:17:52.225 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 85441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-23T14:17:52.225 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 65313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-23T14:17:52.226 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 43857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-23T14:17:52.226 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/ 2024-04-23T14:17:52.226 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 103409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-23T14:17:52.226 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 31833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/nvram.ko 2024-04-23T14:17:52.226 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 38977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-23T14:17:52.226 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 59241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-23T14:17:52.226 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/ 2024-04-23T14:17:52.226 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-23T14:17:52.226 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-04-23T14:17:52.226 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-04-23T14:17:52.226 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 39193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-04-23T14:17:52.226 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 34169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-23T14:17:52.226 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-23T14:17:52.226 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-04-23T14:17:52.227 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/ 2024-04-23T14:17:52.227 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_class.ko 2024-04-23T14:17:52.227 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 62545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_core.ko 2024-04-23T14:17:52.227 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-04-23T14:17:52.227 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/ 2024-04-23T14:17:52.227 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-palmas.ko 2024-04-23T14:17:52.227 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 57289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-si5351.ko 2024-04-23T14:17:52.227 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-twl6040.ko 2024-04-23T14:17:52.227 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-wm831x.ko 2024-04-23T14:17:52.227 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/ 2024-04-23T14:17:52.227 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-23T14:17:52.227 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-23T14:17:52.227 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-23T14:17:52.228 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ 2024-04-23T14:17:52.228 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ 2024-04-23T14:17:52.228 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 89257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-23T14:17:52.228 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 333657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-23T14:17:52.228 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/ 2024-04-23T14:17:52.228 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/ 2024-04-23T14:17:52.228 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/ 2024-04-23T14:17:52.228 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 751713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-23T14:17:52.228 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-04-23T14:17:52.228 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-23T14:17:52.228 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-23T14:17:52.228 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-23T14:17:52.228 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/ 2024-04-23T14:17:52.228 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 90441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-23T14:17:52.229 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/ 2024-04-23T14:17:52.229 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 36105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-23T14:17:52.229 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ 2024-04-23T14:17:52.229 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/ 2024-04-23T14:17:52.229 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-23T14:17:52.229 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ 2024-04-23T14:17:52.229 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 146769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-23T14:17:52.229 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ 2024-04-23T14:17:52.229 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 96145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-23T14:17:52.229 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 41993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-23T14:17:52.229 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 51105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-23T14:17:52.229 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-23T14:17:52.229 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-23T14:17:52.229 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-23T14:17:52.230 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-23T14:17:52.230 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-23T14:17:52.230 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 64161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-23T14:17:52.230 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-23T14:17:52.230 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-23T14:17:52.230 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 68697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-23T14:17:52.230 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-23T14:17:52.230 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/ 2024-04-23T14:17:52.230 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-gpio.ko 2024-04-23T14:17:52.230 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 31369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max14577.ko 2024-04-23T14:17:52.230 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max77693.ko 2024-04-23T14:17:52.230 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max8997.ko 2024-04-23T14:17:52.230 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 30961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-palmas.ko 2024-04-23T14:17:52.230 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 27201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-rt8973a.ko 2024-04-23T14:17:52.231 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 31993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-sm5502.ko 2024-04-23T14:17:52.231 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/ 2024-04-23T14:17:52.231 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 42953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/dmi-sysfs.ko 2024-04-23T14:17:52.231 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/ 2024-04-23T14:17:52.231 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/efi-pstore.ko 2024-04-23T14:17:52.231 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-23T14:17:52.231 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/ 2024-04-23T14:17:52.231 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-adp5520.ko 2024-04-23T14:17:52.231 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-amd8111.ko 2024-04-23T14:17:52.231 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-arizona.ko 2024-04-23T14:17:52.231 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9052.ko 2024-04-23T14:17:52.231 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9055.ko 2024-04-23T14:17:52.231 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-dln2.ko 2024-04-23T14:17:52.232 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 81073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-f7188x.ko 2024-04-23T14:17:52.232 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 40761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-generic.ko 2024-04-23T14:17:52.232 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-23T14:17:52.232 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-04-23T14:17:52.232 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-kempld.ko 2024-04-23T14:17:52.232 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-lp3943.ko 2024-04-23T14:17:52.232 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max7300.ko 2024-04-23T14:17:52.232 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max730x.ko 2024-04-23T14:17:52.232 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max732x.ko 2024-04-23T14:17:52.232 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-04-23T14:17:52.232 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pca953x.ko 2024-04-23T14:17:52.232 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 31881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pcf857x.ko 2024-04-23T14:17:52.232 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-rdc321x.ko 2024-04-23T14:17:52.232 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch.ko 2024-04-23T14:17:52.232 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch311x.ko 2024-04-23T14:17:52.233 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-tps65912.ko 2024-04-23T14:17:52.233 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl4030.ko 2024-04-23T14:17:52.233 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl6040.ko 2024-04-23T14:17:52.233 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-23T14:17:52.233 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-vx855.ko 2024-04-23T14:17:52.233 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm831x.ko 2024-04-23T14:17:52.233 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8350.ko 2024-04-23T14:17:52.233 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8994.ko 2024-04-23T14:17:52.233 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/ 2024-04-23T14:17:52.233 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-a4tech.ko 2024-04-23T14:17:52.233 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-apple.ko 2024-04-23T14:17:52.233 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-23T14:17:52.233 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-23T14:17:52.233 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-23T14:17:52.234 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-belkin.ko 2024-04-23T14:17:52.234 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cherry.ko 2024-04-23T14:17:52.234 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-chicony.ko 2024-04-23T14:17:52.234 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 74393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cp2112.ko 2024-04-23T14:17:52.234 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cypress.ko 2024-04-23T14:17:52.234 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-23T14:17:52.234 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-23T14:17:52.234 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-23T14:17:52.234 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-23T14:17:52.234 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ezkey.ko 2024-04-23T14:17:52.234 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-23T14:17:52.234 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-generic.ko 2024-04-23T14:17:52.234 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-23T14:17:52.235 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-23T14:17:52.235 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-23T14:17:52.235 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-23T14:17:52.235 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-23T14:17:52.235 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-23T14:17:52.235 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-23T14:17:52.235 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kensington.ko 2024-04-23T14:17:52.235 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-23T14:17:52.235 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-23T14:17:52.235 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-23T14:17:52.235 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-23T14:17:52.235 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 54217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-23T14:17:52.235 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 37553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lg-g15.ko 2024-04-23T14:17:52.235 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 68081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-23T14:17:52.236 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 121369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-23T14:17:52.236 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech.ko 2024-04-23T14:17:52.236 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 36689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-magicmouse.ko 2024-04-23T14:17:52.236 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-microsoft.ko 2024-04-23T14:17:52.236 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-monterey.ko 2024-04-23T14:17:52.236 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 59001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-23T14:17:52.236 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 39697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ntrig.ko 2024-04-23T14:17:52.236 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-23T14:17:52.236 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-23T14:17:52.236 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-23T14:17:52.236 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 91241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-23T14:17:52.236 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-23T14:17:52.236 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-23T14:17:52.236 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-23T14:17:52.237 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-23T14:17:52.237 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-23T14:17:52.237 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-23T14:17:52.237 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 38865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-23T14:17:52.237 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 36305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-23T14:17:52.237 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 38073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-23T14:17:52.237 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-23T14:17:52.237 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-23T14:17:52.237 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-23T14:17:52.237 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-23T14:17:52.237 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-23T14:17:52.237 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-23T14:17:52.237 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-23T14:17:52.237 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-23T14:17:52.238 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 42305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-23T14:17:52.238 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 43609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-23T14:17:52.238 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-23T14:17:52.238 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 62129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-23T14:17:52.238 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-23T14:17:52.238 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 30017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-23T14:17:52.238 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-23T14:17:52.238 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-23T14:17:52.238 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-23T14:17:52.238 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-23T14:17:52.238 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-23T14:17:52.238 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-23T14:17:52.238 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 81113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-23T14:17:52.239 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-23T14:17:52.239 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 184745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-23T14:17:52.239 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-23T14:17:52.239 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-23T14:17:52.239 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-23T14:17:52.239 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 319921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid.ko 2024-04-23T14:17:52.239 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 39529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-23T14:17:52.239 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/ 2024-04-23T14:17:52.239 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 153633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbhid.ko 2024-04-23T14:17:52.239 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbkbd.ko 2024-04-23T14:17:52.239 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbmouse.ko 2024-04-23T14:17:52.239 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 261945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-23T14:17:52.239 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/ 2024-04-23T14:17:52.240 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 75065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-23T14:17:52.240 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 110681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-23T14:17:52.240 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 382033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-23T14:17:52.240 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ 2024-04-23T14:17:52.240 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 77153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-23T14:17:52.240 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 67009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-23T14:17:52.240 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-23T14:17:52.240 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-23T14:17:52.240 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-23T14:17:52.240 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-23T14:17:52.240 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 38097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-23T14:17:52.240 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 115057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-23T14:17:52.240 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-23T14:17:52.240 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 55225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-23T14:17:52.241 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 31441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-23T14:17:52.241 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-23T14:17:52.241 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-23T14:17:52.241 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-23T14:17:52.241 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 89025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-23T14:17:52.241 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 49153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-23T14:17:52.241 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 81473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-23T14:17:52.241 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-23T14:17:52.241 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 48177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-23T14:17:52.241 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 68865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-23T14:17:52.241 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 60193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-23T14:17:52.241 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 69009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-23T14:17:52.241 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 47977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-23T14:17:52.241 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-23T14:17:52.242 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-23T14:17:52.242 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9052-hwmon.ko 2024-04-23T14:17:52.242 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9055-hwmon.ko 2024-04-23T14:17:52.242 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-23T14:17:52.242 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 113201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-23T14:17:52.242 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-23T14:17:52.242 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-23T14:17:52.242 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-23T14:17:52.242 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 40177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc2103.ko 2024-04-23T14:17:52.242 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 34649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-23T14:17:52.242 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 78921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-23T14:17:52.242 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 108961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-23T14:17:52.242 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 50697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-23T14:17:52.243 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-23T14:17:52.243 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 65769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-23T14:17:52.243 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-23T14:17:52.243 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 30585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-23T14:17:52.243 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 59761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-23T14:17:52.243 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 56313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-23T14:17:52.243 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hih6130.ko 2024-04-23T14:17:52.243 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-23T14:17:52.243 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-23T14:17:52.243 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 27633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-23T14:17:52.243 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 42585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-23T14:17:52.243 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-23T14:17:52.243 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-23T14:17:52.243 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-23T14:17:52.244 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 156465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-23T14:17:52.244 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/jc42.ko 2024-04-23T14:17:52.244 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-23T14:17:52.244 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-23T14:17:52.244 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-23T14:17:52.244 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 57769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-23T14:17:52.244 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-23T14:17:52.244 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 44121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-23T14:17:52.244 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-23T14:17:52.244 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 41473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-23T14:17:52.244 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 37049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-23T14:17:52.244 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-23T14:17:52.244 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 86497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-23T14:17:52.244 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 55641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-23T14:17:52.245 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-23T14:17:52.245 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-23T14:17:52.245 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 131505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-23T14:17:52.245 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 40225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-23T14:17:52.245 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-23T14:17:52.245 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-23T14:17:52.245 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-23T14:17:52.245 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-23T14:17:52.245 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-23T14:17:52.245 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-23T14:17:52.245 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-23T14:17:52.245 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-23T14:17:52.245 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-23T14:17:52.245 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 36769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-23T14:17:52.246 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-23T14:17:52.246 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-23T14:17:52.246 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-23T14:17:52.246 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-23T14:17:52.246 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-23T14:17:52.246 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 39729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-23T14:17:52.246 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mc13783-adc.ko 2024-04-23T14:17:52.246 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-23T14:17:52.246 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-04-23T14:17:52.246 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 56505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-23T14:17:52.246 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 141353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-23T14:17:52.246 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 72737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-23T14:17:52.246 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 67841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-23T14:17:52.246 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 97777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-23T14:17:52.247 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 61921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-23T14:17:52.247 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-23T14:17:52.247 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ 2024-04-23T14:17:52.247 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-23T14:17:52.247 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-23T14:17:52.247 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-23T14:17:52.247 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-23T14:17:52.247 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-23T14:17:52.247 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-23T14:17:52.247 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-23T14:17:52.247 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 101233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-23T14:17:52.247 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-23T14:17:52.247 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 36177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-23T14:17:52.248 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-23T14:17:52.248 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-23T14:17:52.248 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-23T14:17:52.248 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-23T14:17:52.248 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 32945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-23T14:17:52.248 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 34561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-23T14:17:52.248 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 40849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-23T14:17:52.248 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-23T14:17:52.248 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-23T14:17:52.248 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 48713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-23T14:17:52.248 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-23T14:17:52.248 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 46473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-23T14:17:52.248 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-23T14:17:52.248 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 31417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-23T14:17:52.249 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-23T14:17:52.249 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-23T14:17:52.249 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-23T14:17:52.249 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-23T14:17:52.249 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-23T14:17:52.249 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 50217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-23T14:17:52.249 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 59665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-23T14:17:52.249 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 62737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-23T14:17:52.249 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 71169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-23T14:17:52.249 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 96577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-23T14:17:52.249 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 85913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-23T14:17:52.249 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 75129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-23T14:17:52.250 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 83425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-23T14:17:52.250 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 116777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-23T14:17:52.250 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 134441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-23T14:17:52.250 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-23T14:17:52.250 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 43385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-23T14:17:52.250 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-04-23T14:17:52.250 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-04-23T14:17:52.250 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/ 2024-04-23T14:17:52.250 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/ 2024-04-23T14:17:52.250 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 27353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-23T14:17:52.250 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-23T14:17:52.250 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/ 2024-04-23T14:17:52.250 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-04-23T14:17:52.251 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-04-23T14:17:52.251 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 32905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-04-23T14:17:52.251 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-23T14:17:52.251 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 30361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-23T14:17:52.251 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-23T14:17:52.251 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-04-23T14:17:52.251 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 7729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-23T14:17:52.251 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-04-23T14:17:52.251 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 64233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-23T14:17:52.251 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-23T14:17:52.251 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-23T14:17:52.251 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-23T14:17:52.251 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-04-23T14:17:52.252 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-04-23T14:17:52.252 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 76969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-23T14:17:52.252 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-23T14:17:52.252 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 52937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-23T14:17:52.252 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-04-23T14:17:52.252 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-23T14:17:52.252 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-23T14:17:52.252 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-04-23T14:17:52.252 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-23T14:17:52.252 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-23T14:17:52.252 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 57217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-23T14:17:52.252 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-04-23T14:17:52.252 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-23T14:17:52.252 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-23T14:17:52.253 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-04-23T14:17:52.253 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 32585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-04-23T14:17:52.253 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-23T14:17:52.253 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-04-23T14:17:52.253 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-23T14:17:52.253 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-23T14:17:52.253 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-23T14:17:52.253 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-23T14:17:52.253 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 59193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-04-23T14:17:52.253 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 27849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-23T14:17:52.253 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-23T14:17:52.253 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-23T14:17:52.253 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/ 2024-04-23T14:17:52.253 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-04-23T14:17:52.254 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-04-23T14:17:52.254 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 36569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-04-23T14:17:52.254 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ 2024-04-23T14:17:52.254 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ 2024-04-23T14:17:52.254 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 329697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-23T14:17:52.254 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 1119913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-23T14:17:52.254 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 83905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-23T14:17:52.254 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 408433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-23T14:17:52.254 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 120921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-23T14:17:52.254 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 347217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-23T14:17:52.254 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 84945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-23T14:17:52.254 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ 2024-04-23T14:17:52.254 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/ 2024-04-23T14:17:52.255 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 590929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-23T14:17:52.255 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/ 2024-04-23T14:17:52.255 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 514641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-23T14:17:52.255 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/ 2024-04-23T14:17:52.255 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 911665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-23T14:17:52.255 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ 2024-04-23T14:17:52.255 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 375545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-23T14:17:52.255 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ 2024-04-23T14:17:52.255 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 197121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-04-23T14:17:52.255 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/ 2024-04-23T14:17:52.255 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 212553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-04-23T14:17:52.255 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ 2024-04-23T14:17:52.255 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ 2024-04-23T14:17:52.255 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 334401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-23T14:17:52.256 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ 2024-04-23T14:17:52.256 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 120673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-23T14:17:52.256 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ 2024-04-23T14:17:52.256 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 132585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-23T14:17:52.256 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ 2024-04-23T14:17:52.256 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 157937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-23T14:17:52.256 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ 2024-04-23T14:17:52.256 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 156673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-23T14:17:52.256 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ 2024-04-23T14:17:52.256 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/evbug.ko 2024-04-23T14:17:52.256 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 27609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ff-memless.ko 2024-04-23T14:17:52.256 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/ 2024-04-23T14:17:52.256 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-23T14:17:52.256 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-23T14:17:52.257 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 50713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-23T14:17:52.257 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 49441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-23T14:17:52.257 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/ 2024-04-23T14:17:52.257 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-04-23T14:17:52.257 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-04-23T14:17:52.257 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-04-23T14:17:52.257 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-04-23T14:17:52.257 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 45473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-23T14:17:52.257 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-04-23T14:17:52.257 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lkkbd.ko 2024-04-23T14:17:52.257 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 31857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8323.ko 2024-04-23T14:17:52.257 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8333.ko 2024-04-23T14:17:52.257 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-04-23T14:17:52.258 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-04-23T14:17:52.258 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-04-23T14:17:52.258 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-04-23T14:17:52.258 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/newtonkbd.ko 2024-04-23T14:17:52.258 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-04-23T14:17:52.258 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt1070.ko 2024-04-23T14:17:52.258 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt2160.ko 2024-04-23T14:17:52.258 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-04-23T14:17:52.258 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/stowaway.ko 2024-04-23T14:17:52.258 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/sunkbd.ko 2024-04-23T14:17:52.258 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-04-23T14:17:52.258 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-04-23T14:17:52.258 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-04-23T14:17:52.258 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/xtkbd.ko 2024-04-23T14:17:52.259 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/matrix-keymap.ko 2024-04-23T14:17:52.259 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/ 2024-04-23T14:17:52.259 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 40617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-23T14:17:52.259 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 39553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-23T14:17:52.259 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 160017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-23T14:17:52.259 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 114273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-23T14:17:52.259 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/gpio_mouse.ko 2024-04-23T14:17:52.259 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 428185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/psmouse.ko 2024-04-23T14:17:52.259 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-23T14:17:52.259 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 27865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-23T14:17:52.259 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 27193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-23T14:17:52.259 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-23T14:17:52.259 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/ 2024-04-23T14:17:52.260 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 140969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-23T14:17:52.260 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ 2024-04-23T14:17:52.260 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-23T14:17:52.260 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-23T14:17:52.260 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ct82c710.ko 2024-04-23T14:17:52.260 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-23T14:17:52.260 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/parkbd.ko 2024-04-23T14:17:52.260 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/pcips2.ko 2024-04-23T14:17:52.260 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ps2mult.ko 2024-04-23T14:17:52.260 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-23T14:17:52.260 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serport.ko 2024-04-23T14:17:52.260 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-23T14:17:52.260 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/ 2024-04-23T14:17:52.260 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-88pm860x.ko 2024-04-23T14:17:52.261 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-adp5520.ko 2024-04-23T14:17:52.261 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 77265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-bd2802.ko 2024-04-23T14:17:52.261 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 31097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-23T14:17:52.261 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da903x.ko 2024-04-23T14:17:52.261 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da9052.ko 2024-04-23T14:17:52.261 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-gpio.ko 2024-04-23T14:17:52.261 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-23T14:17:52.261 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 36705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3533.ko 2024-04-23T14:17:52.261 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm355x.ko 2024-04-23T14:17:52.261 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3642.ko 2024-04-23T14:17:52.261 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-23T14:17:52.261 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp8788.ko 2024-04-23T14:17:52.261 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-max8997.ko 2024-04-23T14:17:52.261 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-mc13783.ko 2024-04-23T14:17:52.262 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-menf21bmc.ko 2024-04-23T14:17:52.262 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca9532.ko 2024-04-23T14:17:52.262 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca955x.ko 2024-04-23T14:17:52.262 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca963x.ko 2024-04-23T14:17:52.262 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 38113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-23T14:17:52.262 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-tca6507.ko 2024-04-23T14:17:52.262 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm831x-status.ko 2024-04-23T14:17:52.262 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm8350.ko 2024-04-23T14:17:52.262 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ 2024-04-23T14:17:52.262 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-23T14:17:52.262 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-23T14:17:52.262 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-23T14:17:52.262 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 8641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-23T14:17:52.263 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-23T14:17:52.263 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-23T14:17:52.263 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-23T14:17:52.263 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-23T14:17:52.263 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-23T14:17:52.263 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/ 2024-04-23T14:17:52.263 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/ 2024-04-23T14:17:52.263 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 790753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-23T14:17:52.263 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-23T14:17:52.263 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 116825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bufio.ko 2024-04-23T14:17:52.263 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 58281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-23T14:17:52.263 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 199041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-23T14:17:52.263 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 134945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-23T14:17:52.263 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-23T14:17:52.264 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 67497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-era.ko 2024-04-23T14:17:52.264 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-23T14:17:52.264 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 50457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-23T14:17:52.264 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 41145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log.ko 2024-04-23T14:17:52.264 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 60977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-mirror.ko 2024-04-23T14:17:52.264 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 97209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-23T14:17:52.264 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-23T14:17:52.264 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 85689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-23T14:17:52.264 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-region-hash.ko 2024-04-23T14:17:52.264 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-23T14:17:52.264 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-23T14:17:52.264 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-snapshot.ko 2024-04-23T14:17:52.264 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-23T14:17:52.264 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 234457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-23T14:17:52.265 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 66425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-23T14:17:52.265 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-zero.ko 2024-04-23T14:17:52.265 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/ 2024-04-23T14:17:52.265 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 264329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-23T14:17:52.265 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 43137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-23T14:17:52.265 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 125009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-23T14:17:52.265 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 157681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-23T14:17:52.265 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 444665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-23T14:17:52.265 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/ 2024-04-23T14:17:52.265 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/ 2024-04-23T14:17:52.265 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 214841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-23T14:17:52.265 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 77985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-23T14:17:52.265 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 54785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-23T14:17:52.265 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 44633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-23T14:17:52.266 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 150761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-23T14:17:52.266 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 97281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-23T14:17:52.266 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 63873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-23T14:17:52.266 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/ 2024-04-23T14:17:52.266 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm800.ko 2024-04-23T14:17:52.266 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm805.ko 2024-04-23T14:17:52.266 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm80x.ko 2024-04-23T14:17:52.266 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona-i2c.ko 2024-04-23T14:17:52.266 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 148633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona.ko 2024-04-23T14:17:52.266 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/bcm590xx.ko 2024-04-23T14:17:52.266 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/cros_ec_dev.ko 2024-04-23T14:17:52.266 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 41009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/dln2.ko 2024-04-23T14:17:52.266 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/janz-cmodio.ko 2024-04-23T14:17:52.266 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 59089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/kempld-core.ko 2024-04-23T14:17:52.267 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 31897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-core.ko 2024-04-23T14:17:52.267 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-04-23T14:17:52.267 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lp3943.ko 2024-04-23T14:17:52.267 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 60497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-23T14:17:52.267 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-23T14:17:52.267 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/max8907.ko 2024-04-23T14:17:52.267 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-core.ko 2024-04-23T14:17:52.267 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-04-23T14:17:52.267 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/menf21bmc.ko 2024-04-23T14:17:52.267 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-adc.ko 2024-04-23T14:17:52.267 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-gpio.ko 2024-04-23T14:17:52.267 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633.ko 2024-04-23T14:17:52.267 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-04-23T14:17:52.267 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/retu-mfd.ko 2024-04-23T14:17:52.268 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 99425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/si476x-core.ko 2024-04-23T14:17:52.268 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 58057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-23T14:17:52.268 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 60681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps65010.ko 2024-04-23T14:17:52.268 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps6507x.ko 2024-04-23T14:17:52.268 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-23T14:17:52.268 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-23T14:17:52.268 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-23T14:17:52.268 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 52361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wm8994.ko 2024-04-23T14:17:52.268 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ 2024-04-23T14:17:52.268 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-04-23T14:17:52.268 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 64265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot.ko 2024-04-23T14:17:52.268 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/ 2024-04-23T14:17:52.268 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 57521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-23T14:17:52.269 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-23T14:17:52.269 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-23T14:17:52.269 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 58633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-23T14:17:52.269 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/ 2024-04-23T14:17:52.269 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-04-23T14:17:52.269 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 47449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/core.ko 2024-04-23T14:17:52.269 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/ 2024-04-23T14:17:52.269 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-23T14:17:52.269 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ds1682.ko 2024-04-23T14:17:52.269 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/dummy-irq.ko 2024-04-23T14:17:52.269 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/ 2024-04-23T14:17:52.269 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/echo.ko 2024-04-23T14:17:52.269 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/ 2024-04-23T14:17:52.269 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 43041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-23T14:17:52.270 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-23T14:17:52.270 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-23T14:17:52.270 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 41185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-23T14:17:52.270 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/ 2024-04-23T14:17:52.270 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 191649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-04-23T14:17:52.270 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hmc6352.ko 2024-04-23T14:17:52.270 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 37761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-23T14:17:52.270 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ 2024-04-23T14:17:52.270 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 92345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-04-23T14:17:52.270 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ics932s401.ko 2024-04-23T14:17:52.270 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-23T14:17:52.270 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-23T14:17:52.270 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/ 2024-04-23T14:17:52.270 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 54865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-23T14:17:52.271 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-23T14:17:52.271 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/ 2024-04-23T14:17:52.271 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 154169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-23T14:17:52.271 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 69809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-23T14:17:52.271 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 389033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-23T14:17:52.271 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 37961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/phantom.ko 2024-04-23T14:17:52.271 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/ 2024-04-23T14:17:52.271 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 91145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/st_drv.ko 2024-04-23T14:17:52.271 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-23T14:17:52.271 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 34457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-23T14:17:52.271 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-23T14:17:52.271 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 55793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-23T14:17:52.271 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/ 2024-04-23T14:17:52.271 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 234905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-23T14:17:52.272 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ 2024-04-23T14:17:52.272 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/ 2024-04-23T14:17:52.272 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 620961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-23T14:17:52.272 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/ 2024-04-23T14:17:52.272 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/ 2024-04-23T14:17:52.272 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-23T14:17:52.272 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 49545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-23T14:17:52.272 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-23T14:17:52.272 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/ 2024-04-23T14:17:52.272 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 587065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-23T14:17:52.272 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-23T14:17:52.272 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-23T14:17:52.272 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ 2024-04-23T14:17:52.272 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/ 2024-04-23T14:17:52.273 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 53753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-23T14:17:52.273 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 49201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-23T14:17:52.273 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 158649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-23T14:17:52.273 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 85953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-23T14:17:52.273 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ 2024-04-23T14:17:52.273 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 44161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-23T14:17:52.273 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-23T14:17:52.273 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-23T14:17:52.273 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 109521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-23T14:17:52.273 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/ 2024-04-23T14:17:52.273 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 64705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-23T14:17:52.273 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/ 2024-04-23T14:17:52.273 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 84329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-23T14:17:52.274 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/ 2024-04-23T14:17:52.274 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 70737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-23T14:17:52.274 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/ 2024-04-23T14:17:52.274 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 52705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-23T14:17:52.274 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/ 2024-04-23T14:17:52.274 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 65857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-23T14:17:52.274 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 56057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-23T14:17:52.274 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 115881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-23T14:17:52.274 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/ 2024-04-23T14:17:52.274 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/ 2024-04-23T14:17:52.274 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 124497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-23T14:17:52.274 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-04-23T14:17:52.274 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 128409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-23T14:17:52.274 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-04-23T14:17:52.275 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 110345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-23T14:17:52.275 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/ 2024-04-23T14:17:52.275 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 101857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-23T14:17:52.275 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 88953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-23T14:17:52.275 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/ 2024-04-23T14:17:52.275 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 105633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-23T14:17:52.275 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 228985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-23T14:17:52.275 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-04-23T14:17:52.275 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 1733353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-23T14:17:52.275 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 167185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-23T14:17:52.275 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/ 2024-04-23T14:17:52.275 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 161417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-23T14:17:52.275 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 440625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-23T14:17:52.275 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/ 2024-04-23T14:17:52.276 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/ 2024-04-23T14:17:52.276 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 432201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-23T14:17:52.357 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/ 2024-04-23T14:17:52.357 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-04-23T14:17:52.357 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 257201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-23T14:17:52.357 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-04-23T14:17:52.357 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 446177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-23T14:17:52.357 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-04-23T14:17:52.358 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 1311433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-23T14:17:52.358 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-04-23T14:17:52.358 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 206689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-23T14:17:52.358 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-04-23T14:17:52.358 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 40161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-23T14:17:52.358 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/ 2024-04-23T14:17:52.358 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/ 2024-04-23T14:17:52.358 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 269313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-23T14:17:52.358 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/ 2024-04-23T14:17:52.358 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/ 2024-04-23T14:17:52.358 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 85385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-23T14:17:52.358 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 77457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-23T14:17:52.358 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 169089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-23T14:17:52.359 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-23T14:17:52.359 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 76209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-23T14:17:52.359 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 38065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-23T14:17:52.359 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/ 2024-04-23T14:17:52.359 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 75945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-23T14:17:52.359 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 78705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-23T14:17:52.359 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-23T14:17:52.359 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 30409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ec_bhf.ko 2024-04-23T14:17:52.359 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/ 2024-04-23T14:17:52.359 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/ 2024-04-23T14:17:52.359 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 356377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-23T14:17:52.359 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 61737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-23T14:17:52.360 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 63185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-23T14:17:52.360 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/ 2024-04-23T14:17:52.360 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 51513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-04-23T14:17:52.360 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ 2024-04-23T14:17:52.360 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 115569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-23T14:17:52.360 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/ 2024-04-23T14:17:52.360 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 366217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-23T14:17:52.360 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/ 2024-04-23T14:17:52.360 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 755793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-23T14:17:52.360 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/ 2024-04-23T14:17:52.360 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 324577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-23T14:17:52.360 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/ 2024-04-23T14:17:52.360 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 1196297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-23T14:17:52.360 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/ 2024-04-23T14:17:52.361 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 362593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-23T14:17:52.361 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/ 2024-04-23T14:17:52.361 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 671801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-23T14:17:52.361 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/ 2024-04-23T14:17:52.361 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 121537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-23T14:17:52.361 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-04-23T14:17:52.361 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 966705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-23T14:17:52.361 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-04-23T14:17:52.361 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 193585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-23T14:17:52.361 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 108633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-23T14:17:52.361 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/ 2024-04-23T14:17:52.361 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-23T14:17:52.361 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 126881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-23T14:17:52.362 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 150777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-23T14:17:52.362 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/ 2024-04-23T14:17:52.362 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-04-23T14:17:52.362 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 905033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-23T14:17:52.362 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 344145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-23T14:17:52.362 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-04-23T14:17:52.362 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-04-23T14:17:52.362 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 1412417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-23T14:17:52.362 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ 2024-04-23T14:17:52.362 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-04-23T14:17:52.362 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 54497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-04-23T14:17:52.362 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-04-23T14:17:52.362 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 117353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-23T14:17:52.362 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/ 2024-04-23T14:17:52.363 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-04-23T14:17:52.363 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 131425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-23T14:17:52.363 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ 2024-04-23T14:17:52.363 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 91465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-23T14:17:52.363 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 56689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-23T14:17:52.363 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/ 2024-04-23T14:17:52.363 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 181081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-23T14:17:52.363 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/ 2024-04-23T14:17:52.363 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 171417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-23T14:17:52.363 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/ 2024-04-23T14:17:52.363 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 58249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-23T14:17:52.363 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 50121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-23T14:17:52.363 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/ 2024-04-23T14:17:52.363 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-04-23T14:17:52.364 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 300817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-23T14:17:52.364 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 105689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-23T14:17:52.364 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-04-23T14:17:52.364 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 885129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-23T14:17:52.364 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/ 2024-04-23T14:17:52.364 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 52305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-23T14:17:52.364 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/ 2024-04-23T14:17:52.364 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 81641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-23T14:17:52.364 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 100689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-23T14:17:52.364 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 34073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-23T14:17:52.364 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 264161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-23T14:17:52.364 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/ 2024-04-23T14:17:52.364 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-04-23T14:17:52.364 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 142449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-04-23T14:17:52.365 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/ 2024-04-23T14:17:52.365 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 1350385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-23T14:17:52.365 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/ 2024-04-23T14:17:52.365 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 57953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-23T14:17:52.365 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/ 2024-04-23T14:17:52.365 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 66505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-23T14:17:52.365 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 80065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-23T14:17:52.365 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/ 2024-04-23T14:17:52.365 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 74289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-23T14:17:52.365 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 79457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-23T14:17:52.365 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 100785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-23T14:17:52.365 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 83441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-23T14:17:52.365 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/ 2024-04-23T14:17:52.366 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-04-23T14:17:52.366 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-04-23T14:17:52.366 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 46985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-23T14:17:52.366 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-04-23T14:17:52.366 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 644537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-23T14:17:52.366 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/ 2024-04-23T14:17:52.366 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 128497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-23T14:17:52.366 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 207993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-23T14:17:52.366 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 82633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-23T14:17:52.366 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 91353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-23T14:17:52.366 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/ 2024-04-23T14:17:52.366 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 59865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-23T14:17:52.366 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/ 2024-04-23T14:17:52.367 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 110729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-23T14:17:52.367 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/ 2024-04-23T14:17:52.367 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 98689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-23T14:17:52.367 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 102513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-23T14:17:52.367 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/ 2024-04-23T14:17:52.367 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 61497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-23T14:17:52.367 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 40609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-23T14:17:52.367 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/ 2024-04-23T14:17:52.367 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 69913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-23T14:17:52.367 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/ 2024-04-23T14:17:52.367 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 49449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/defxx.ko 2024-04-23T14:17:52.367 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/ 2024-04-23T14:17:52.367 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 275801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/skfp.ko 2024-04-23T14:17:52.367 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 91105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-23T14:17:52.368 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/ 2024-04-23T14:17:52.368 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 290441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-23T14:17:52.368 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/ 2024-04-23T14:17:52.368 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-23T14:17:52.368 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 30001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-23T14:17:52.368 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ 2024-04-23T14:17:52.368 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 108329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-23T14:17:52.368 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 78065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-23T14:17:52.368 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-23T14:17:52.368 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/ 2024-04-23T14:17:52.368 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-23T14:17:52.368 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-23T14:17:52.368 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-gpio.ko 2024-04-23T14:17:52.368 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-23T14:17:52.369 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-23T14:17:52.369 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-23T14:17:52.369 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 62681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-23T14:17:52.369 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-23T14:17:52.369 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/ 2024-04-23T14:17:52.369 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-23T14:17:52.369 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 48313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-23T14:17:52.369 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ 2024-04-23T14:17:52.369 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-23T14:17:52.369 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 61985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-23T14:17:52.369 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 52865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-23T14:17:52.369 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-23T14:17:52.369 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 53409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-23T14:17:52.369 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-23T14:17:52.370 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-23T14:17:52.370 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-23T14:17:52.370 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 31313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-23T14:17:52.370 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-23T14:17:52.370 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 103377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-23T14:17:52.370 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 96025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-23T14:17:52.370 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-23T14:17:52.370 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 121105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-23T14:17:52.370 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-23T14:17:52.370 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 62113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-23T14:17:52.370 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-23T14:17:52.370 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-23T14:17:52.370 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-23T14:17:52.370 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/ 2024-04-23T14:17:52.371 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 55353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/plip.ko 2024-04-23T14:17:52.371 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ 2024-04-23T14:17:52.371 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-23T14:17:52.371 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 36273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-23T14:17:52.371 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-23T14:17:52.371 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-23T14:17:52.371 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-23T14:17:52.371 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 53233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-23T14:17:52.371 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-23T14:17:52.371 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-23T14:17:52.371 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sb1000.ko 2024-04-23T14:17:52.371 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/ 2024-04-23T14:17:52.371 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 52073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-23T14:17:52.371 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 42409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-23T14:17:52.372 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 80161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-23T14:17:52.372 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/ 2024-04-23T14:17:52.372 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 137321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-23T14:17:52.372 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-23T14:17:52.372 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-23T14:17:52.372 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 36209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-23T14:17:52.372 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-23T14:17:52.372 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-23T14:17:52.372 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 108089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-23T14:17:52.372 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/ 2024-04-23T14:17:52.372 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 176497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-23T14:17:52.372 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/ 2024-04-23T14:17:52.372 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 383593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-23T14:17:52.372 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/ 2024-04-23T14:17:52.373 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 175537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-23T14:17:52.373 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ 2024-04-23T14:17:52.373 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-23T14:17:52.373 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/ 2024-04-23T14:17:52.373 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/ 2024-04-23T14:17:52.373 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 335857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-23T14:17:52.373 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 126553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-23T14:17:52.373 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/ 2024-04-23T14:17:52.373 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 168145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-23T14:17:52.373 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-23T14:17:52.373 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 103889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-23T14:17:52.373 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 30313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-23T14:17:52.373 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/ 2024-04-23T14:17:52.373 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/ 2024-04-23T14:17:52.374 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-23T14:17:52.374 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-04-23T14:17:52.374 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-04-23T14:17:52.374 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pci-stub.ko 2024-04-23T14:17:52.374 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 55329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-23T14:17:52.374 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/ 2024-04-23T14:17:52.374 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 31529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-23T14:17:52.374 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 179961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia.ko 2024-04-23T14:17:52.374 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 78161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_core.ko 2024-04-23T14:17:52.374 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 41945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-04-23T14:17:52.374 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 40457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-23T14:17:52.374 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 133753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-23T14:17:52.374 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/ 2024-04-23T14:17:52.374 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/ 2024-04-23T14:17:52.375 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-04-23T14:17:52.375 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/ 2024-04-23T14:17:52.375 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-04-23T14:17:52.375 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/ 2024-04-23T14:17:52.375 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/ 2024-04-23T14:17:52.375 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 78193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-23T14:17:52.375 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/ 2024-04-23T14:17:52.375 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/ 2024-04-23T14:17:52.375 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 61041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-04-23T14:17:52.375 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec.ko 2024-04-23T14:17:52.375 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-04-23T14:17:52.375 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 30217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-04-23T14:17:52.375 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-04-23T14:17:52.375 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-04-23T14:17:52.375 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-04-23T14:17:52.376 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ 2024-04-23T14:17:52.376 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 87481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-23T14:17:52.376 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 44105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-23T14:17:52.376 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 48257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-23T14:17:52.376 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-23T14:17:52.376 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-23T14:17:52.376 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 164769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-23T14:17:52.376 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 46185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-23T14:17:52.376 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 62073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-23T14:17:52.376 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-23T14:17:52.376 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 51881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-23T14:17:52.376 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-23T14:17:52.376 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 42761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-23T14:17:52.376 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ibm_rtl.ko 2024-04-23T14:17:52.377 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/ 2024-04-23T14:17:52.377 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-23T14:17:52.377 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-04-23T14:17:52.377 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 54001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-23T14:17:52.377 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-23T14:17:52.377 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-23T14:17:52.377 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 43753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-23T14:17:52.377 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-23T14:17:52.377 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-23T14:17:52.377 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-23T14:17:52.377 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-23T14:17:52.377 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-23T14:17:52.377 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-23T14:17:52.377 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 61417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-23T14:17:52.378 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/ 2024-04-23T14:17:52.378 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ 2024-04-23T14:17:52.378 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 36513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_battery.ko 2024-04-23T14:17:52.378 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 40425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_charger.ko 2024-04-23T14:17:52.378 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 68833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq2415x_charger.ko 2024-04-23T14:17:52.378 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 73073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24190_charger.ko 2024-04-23T14:17:52.378 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24735-charger.ko 2024-04-23T14:17:52.378 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-04-23T14:17:52.378 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9030_battery.ko 2024-04-23T14:17:52.378 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9052-battery.ko 2024-04-23T14:17:52.378 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 34489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2760_battery.ko 2024-04-23T14:17:52.378 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2780_battery.ko 2024-04-23T14:17:52.378 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2781_battery.ko 2024-04-23T14:17:52.379 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2782_battery.ko 2024-04-23T14:17:52.379 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/gpio-charger.ko 2024-04-23T14:17:52.379 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/isp1704_charger.ko 2024-04-23T14:17:52.379 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/lp8727_charger.ko 2024-04-23T14:17:52.379 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max14577_charger.ko 2024-04-23T14:17:52.379 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 37833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max17042_battery.ko 2024-04-23T14:17:52.379 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8903_charger.ko 2024-04-23T14:17:52.379 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 27689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8925_power.ko 2024-04-23T14:17:52.379 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/pcf50633-charger.ko 2024-04-23T14:17:52.379 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 44841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/sbs-battery.ko 2024-04-23T14:17:52.379 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 34905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/test_power.ko 2024-04-23T14:17:52.379 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/tps65090-charger.ko 2024-04-23T14:17:52.379 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_backup.ko 2024-04-23T14:17:52.379 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_power.ko 2024-04-23T14:17:52.379 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 27441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm8350_power.ko 2024-04-23T14:17:52.380 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/ 2024-04-23T14:17:52.380 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 73297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-23T14:17:52.380 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-23T14:17:52.380 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/ 2024-04-23T14:17:52.380 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/ 2024-04-23T14:17:52.380 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-23T14:17:52.380 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-23T14:17:52.380 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-23T14:17:52.380 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 49817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-23T14:17:52.380 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ 2024-04-23T14:17:52.380 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 91393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-23T14:17:52.380 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-23T14:17:52.380 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/ 2024-04-23T14:17:52.380 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/ 2024-04-23T14:17:52.381 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/ 2024-04-23T14:17:52.381 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 67513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-23T14:17:52.381 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 47137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-23T14:17:52.381 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/ 2024-04-23T14:17:52.381 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm80x.ko 2024-04-23T14:17:52.381 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm860x.ko 2024-04-23T14:17:52.381 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-23T14:17:52.381 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9052.ko 2024-04-23T14:17:52.381 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9055.ko 2024-04-23T14:17:52.381 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9063.ko 2024-04-23T14:17:52.381 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-23T14:17:52.381 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 82225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-23T14:17:52.381 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-23T14:17:52.381 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-23T14:17:52.382 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-23T14:17:52.382 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-23T14:17:52.382 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-23T14:17:52.382 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-23T14:17:52.382 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-23T14:17:52.382 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-23T14:17:52.382 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-23T14:17:52.382 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-23T14:17:52.382 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 44129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-23T14:17:52.382 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-lp8788.ko 2024-04-23T14:17:52.382 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 46913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-23T14:17:52.382 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-23T14:17:52.382 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-23T14:17:52.382 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t86.ko 2024-04-23T14:17:52.383 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-23T14:17:52.383 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8907.ko 2024-04-23T14:17:52.383 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8925.ko 2024-04-23T14:17:52.383 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 31337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8997.ko 2024-04-23T14:17:52.383 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8998.ko 2024-04-23T14:17:52.383 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-04-23T14:17:52.383 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-23T14:17:52.383 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-palmas.ko 2024-04-23T14:17:52.383 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 56777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-23T14:17:52.383 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf50633.ko 2024-04-23T14:17:52.383 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 32657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-23T14:17:52.385 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-23T14:17:52.385 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 36065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-23T14:17:52.385 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-23T14:17:52.386 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rc5t583.ko 2024-04-23T14:17:52.386 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-23T14:17:52.386 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-23T14:17:52.386 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 34529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-23T14:17:52.386 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-23T14:17:52.386 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-23T14:17:52.386 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-s35390a.ko 2024-04-23T14:17:52.386 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-23T14:17:52.386 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps6586x.ko 2024-04-23T14:17:52.386 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps65910.ko 2024-04-23T14:17:52.386 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm831x.ko 2024-04-23T14:17:52.386 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm8350.ko 2024-04-23T14:17:52.386 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-23T14:17:52.386 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ 2024-04-23T14:17:52.387 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 90377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-23T14:17:52.387 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 69593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-23T14:17:52.387 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 65945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-23T14:17:52.387 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 164569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-23T14:17:52.387 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 31617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-23T14:17:52.387 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/ 2024-04-23T14:17:52.387 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 340721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-23T14:17:52.387 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 120337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-23T14:17:52.387 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/ 2024-04-23T14:17:52.387 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 468081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-23T14:17:52.387 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 380673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-23T14:17:52.387 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/ 2024-04-23T14:17:52.387 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 243705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-04-23T14:17:52.387 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-23T14:17:52.388 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/ 2024-04-23T14:17:52.388 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 135617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-23T14:17:52.388 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-23T14:17:52.388 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/ 2024-04-23T14:17:52.388 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 304993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-23T14:17:52.388 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/ 2024-04-23T14:17:52.388 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 1527113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-23T14:17:52.388 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/ 2024-04-23T14:17:52.388 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 297801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-23T14:17:52.388 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/ 2024-04-23T14:17:52.388 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 144401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-23T14:17:52.388 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 47457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-23T14:17:52.388 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/ 2024-04-23T14:17:52.389 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 390713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-23T14:17:52.389 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/ 2024-04-23T14:17:52.389 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-04-23T14:17:52.389 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 88881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-23T14:17:52.389 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-04-23T14:17:52.389 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 117481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-23T14:17:52.389 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 173017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-23T14:17:52.389 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 91569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-23T14:17:52.389 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 45401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-23T14:17:52.389 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/ 2024-04-23T14:17:52.389 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 267577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-23T14:17:52.389 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-23T14:17:52.389 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/ 2024-04-23T14:17:52.389 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 95329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-23T14:17:52.390 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 194017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-23T14:17:52.390 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fdomain.ko 2024-04-23T14:17:52.390 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/ 2024-04-23T14:17:52.390 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 366529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-23T14:17:52.390 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 251505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-23T14:17:52.390 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 65761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-23T14:17:52.390 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 57097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-23T14:17:52.390 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 44689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/imm.ko 2024-04-23T14:17:52.390 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 55153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-23T14:17:52.390 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 327841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-23T14:17:52.390 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 129785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-23T14:17:52.390 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/ 2024-04-23T14:17:52.390 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 397785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-23T14:17:52.391 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 32017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-23T14:17:52.391 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 63121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-23T14:17:52.391 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/ 2024-04-23T14:17:52.391 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 386849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-23T14:17:52.391 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 178177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-23T14:17:52.391 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 64809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-23T14:17:52.391 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/ 2024-04-23T14:17:52.391 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 274009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-23T14:17:52.391 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/ 2024-04-23T14:17:52.391 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 2736001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-23T14:17:52.391 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/ 2024-04-23T14:17:52.391 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 88529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-23T14:17:52.391 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 40145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-23T14:17:52.391 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 389177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-23T14:17:52.392 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 104769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-23T14:17:52.392 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/ 2024-04-23T14:17:52.392 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 836945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-23T14:17:52.392 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/ 2024-04-23T14:17:52.392 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 158729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-23T14:17:52.392 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 96537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-23T14:17:52.392 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/ 2024-04-23T14:17:52.392 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 101625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-04-23T14:17:52.392 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-04-23T14:17:52.392 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-04-23T14:17:52.392 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-04-23T14:17:52.392 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/ 2024-04-23T14:17:52.392 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 443745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-23T14:17:52.392 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 152033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-23T14:17:52.393 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 41625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ppa.ko 2024-04-23T14:17:52.393 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 66977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-23T14:17:52.393 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/ 2024-04-23T14:17:52.393 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 2324809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-23T14:17:52.393 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 103601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-23T14:17:52.393 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/ 2024-04-23T14:17:52.393 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 697337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-23T14:17:52.393 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qlogicfas408.ko 2024-04-23T14:17:52.393 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-23T14:17:52.393 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 322865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-23T14:17:52.393 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 219857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-23T14:17:52.393 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 357153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-23T14:17:52.393 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 121185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-23T14:17:52.393 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 93481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-23T14:17:52.394 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 55401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-23T14:17:52.394 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-23T14:17:52.394 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 138121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-23T14:17:52.394 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 57401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-23T14:17:52.394 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/ 2024-04-23T14:17:52.394 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 205025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-23T14:17:52.394 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 55425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-23T14:17:52.394 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 59185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-23T14:17:52.394 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 48777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-23T14:17:52.394 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 59417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/xen-scsifront.ko 2024-04-23T14:17:52.394 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ 2024-04-23T14:17:52.394 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 174913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-23T14:17:52.394 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/ 2024-04-23T14:17:52.395 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/ 2024-04-23T14:17:52.395 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 920169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-23T14:17:52.395 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/ 2024-04-23T14:17:52.395 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 60321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-23T14:17:52.395 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 48129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-23T14:17:52.395 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 54233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-23T14:17:52.395 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 1170033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-23T14:17:52.395 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-23T14:17:52.395 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 149721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-23T14:17:52.395 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/ 2024-04-23T14:17:52.395 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 90057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-23T14:17:52.395 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/ 2024-04-23T14:17:52.395 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/ 2024-04-23T14:17:52.395 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/ 2024-04-23T14:17:52.395 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-23T14:17:52.396 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-23T14:17:52.396 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-23T14:17:52.396 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-23T14:17:52.396 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-23T14:17:52.396 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-23T14:17:52.396 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-23T14:17:52.396 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-23T14:17:52.396 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-23T14:17:52.396 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-23T14:17:52.396 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-23T14:17:52.396 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-23T14:17:52.396 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 46065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-23T14:17:52.396 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-23T14:17:52.396 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-23T14:17:52.397 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-04-23T14:17:52.397 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-23T14:17:52.397 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-23T14:17:52.397 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 30577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-23T14:17:52.397 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ 2024-04-23T14:17:52.397 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ 2024-04-23T14:17:52.397 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 99313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-23T14:17:52.397 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 69729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/moxa.ko 2024-04-23T14:17:52.397 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 61649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/mxser.ko 2024-04-23T14:17:52.397 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 134401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-23T14:17:52.397 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 37905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-23T14:17:52.397 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-23T14:17:52.397 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/ 2024-04-23T14:17:52.398 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/ 2024-04-23T14:17:52.398 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 44057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-04-23T14:17:52.398 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 65121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-23T14:17:52.398 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-04-23T14:17:52.398 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_uart.ko 2024-04-23T14:17:52.398 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-23T14:17:52.398 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 91665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-04-23T14:17:52.398 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/ 2024-04-23T14:17:52.398 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 120577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-23T14:17:52.398 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/rp2.ko 2024-04-23T14:17:52.398 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 65641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/sc16is7xx.ko 2024-04-23T14:17:52.398 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/ 2024-04-23T14:17:52.398 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ 2024-04-23T14:17:52.398 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 693081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-23T14:17:52.399 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ 2024-04-23T14:17:52.399 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 34449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-23T14:17:52.399 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-04-23T14:17:52.399 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/ 2024-04-23T14:17:52.399 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 55617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-23T14:17:52.399 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-23T14:17:52.399 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-23T14:17:52.399 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_dmem_genirq.ko 2024-04-23T14:17:52.399 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_mf624.ko 2024-04-23T14:17:52.399 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_netx.ko 2024-04-23T14:17:52.399 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-23T14:17:52.399 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-04-23T14:17:52.399 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-23T14:17:52.399 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/ 2024-04-23T14:17:52.399 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/ 2024-04-23T14:17:52.400 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 90321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/c67x00.ko 2024-04-23T14:17:52.400 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ 2024-04-23T14:17:52.400 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 189209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-04-23T14:17:52.400 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-04-23T14:17:52.400 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-04-23T14:17:52.400 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-04-23T14:17:52.400 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-04-23T14:17:52.400 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/ 2024-04-23T14:17:52.400 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 101545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-23T14:17:52.400 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 63073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-23T14:17:52.400 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 56721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-23T14:17:52.400 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 107417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-23T14:17:52.400 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ 2024-04-23T14:17:52.400 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ulpi.ko 2024-04-23T14:17:52.401 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/ 2024-04-23T14:17:52.401 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-04-23T14:17:52.401 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/ 2024-04-23T14:17:52.401 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-04-23T14:17:52.401 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-04-23T14:17:52.401 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 574593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3.ko 2024-04-23T14:17:52.401 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/ 2024-04-23T14:17:52.401 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/ 2024-04-23T14:17:52.401 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 72249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_ether.ko 2024-04-23T14:17:52.401 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 61953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_serial.ko 2024-04-23T14:17:52.401 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-04-23T14:17:52.401 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 49737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-04-23T14:17:52.401 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-04-23T14:17:52.401 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-04-23T14:17:52.402 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 131345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-04-23T14:17:52.402 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 62321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-04-23T14:17:52.402 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 146617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-04-23T14:17:52.402 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 70689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-04-23T14:17:52.402 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-04-23T14:17:52.402 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 37649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-04-23T14:17:52.402 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 70017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-04-23T14:17:52.402 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 101249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-04-23T14:17:52.402 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-04-23T14:17:52.402 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 71257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-04-23T14:17:52.402 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 84937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-04-23T14:17:52.402 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/ 2024-04-23T14:17:52.402 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-04-23T14:17:52.402 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-04-23T14:17:52.403 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-04-23T14:17:52.403 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-04-23T14:17:52.403 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-04-23T14:17:52.403 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-04-23T14:17:52.403 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-04-23T14:17:52.403 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-04-23T14:17:52.403 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-04-23T14:17:52.403 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-04-23T14:17:52.403 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-04-23T14:17:52.403 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-04-23T14:17:52.403 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 78073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-04-23T14:17:52.403 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-04-23T14:17:52.403 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 204049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/libcomposite.ko 2024-04-23T14:17:52.403 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/ 2024-04-23T14:17:52.404 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-04-23T14:17:52.404 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/ 2024-04-23T14:17:52.404 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 182561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-04-23T14:17:52.404 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 45545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-04-23T14:17:52.404 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 66513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-04-23T14:17:52.404 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 85217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-04-23T14:17:52.404 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 69313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-04-23T14:17:52.404 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 90313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2272.ko 2024-04-23T14:17:52.404 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 90361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2280.ko 2024-04-23T14:17:52.404 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 94841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-04-23T14:17:52.404 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-04-23T14:17:52.404 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 87465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-04-23T14:17:52.404 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 95441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-04-23T14:17:52.404 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 195969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-04-23T14:17:52.405 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ 2024-04-23T14:17:52.405 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/bcma-hcd.ko 2024-04-23T14:17:52.405 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 108321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/isp116x-hcd.ko 2024-04-23T14:17:52.405 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 104377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-04-23T14:17:52.405 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 102273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-04-23T14:17:52.405 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 60105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-23T14:17:52.405 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811_cs.ko 2024-04-23T14:17:52.405 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ssb-hcd.ko 2024-04-23T14:17:52.405 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-04-23T14:17:52.405 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/ 2024-04-23T14:17:52.405 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 40409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-23T14:17:52.405 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-23T14:17:52.405 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ 2024-04-23T14:17:52.405 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 47353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-23T14:17:52.406 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-23T14:17:52.406 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-04-23T14:17:52.406 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cytherm.ko 2024-04-23T14:17:52.406 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ehset.ko 2024-04-23T14:17:52.406 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-23T14:17:52.406 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-23T14:17:52.406 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-23T14:17:52.406 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-23T14:17:52.406 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 42729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-23T14:17:52.406 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-23T14:17:52.406 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 42241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-23T14:17:52.406 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 42233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-23T14:17:52.406 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/lvstest.ko 2024-04-23T14:17:52.406 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/ 2024-04-23T14:17:52.407 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 85313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-23T14:17:52.407 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-23T14:17:52.407 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-23T14:17:52.407 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-23T14:17:52.407 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 27809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-23T14:17:52.407 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 84265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbtest.ko 2024-04-23T14:17:52.407 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 52065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-23T14:17:52.407 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 32345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-23T14:17:52.407 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/ 2024-04-23T14:17:52.407 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 91529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/usbmon.ko 2024-04-23T14:17:52.407 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/ 2024-04-23T14:17:52.407 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 396209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/musb_hdrc.ko 2024-04-23T14:17:52.407 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/ 2024-04-23T14:17:52.408 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-23T14:17:52.408 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-04-23T14:17:52.408 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-isp1301.ko 2024-04-23T14:17:52.408 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-tahvo.ko 2024-04-23T14:17:52.408 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ 2024-04-23T14:17:52.408 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-23T14:17:52.408 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-23T14:17:52.408 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 30681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-23T14:17:52.408 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 44329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-23T14:17:52.408 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-23T14:17:52.408 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-23T14:17:52.408 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 66873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-23T14:17:52.408 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 58329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-23T14:17:52.408 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-23T14:17:52.409 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 53009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/f81232.ko 2024-04-23T14:17:52.409 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 188609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-23T14:17:52.409 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 61489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-23T14:17:52.409 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 152305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-23T14:17:52.409 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 142833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-23T14:17:52.409 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 69217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-23T14:17:52.409 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-23T14:17:52.409 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-23T14:17:52.409 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-23T14:17:52.409 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 122329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-23T14:17:52.409 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 41297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-23T14:17:52.409 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-23T14:17:52.409 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 38049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-23T14:17:52.409 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 38041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-23T14:17:52.410 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/metro-usb.ko 2024-04-23T14:17:52.410 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-23T14:17:52.410 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 89969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-23T14:17:52.410 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 65073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mxuport.ko 2024-04-23T14:17:52.410 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-23T14:17:52.410 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-23T14:17:52.410 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-23T14:17:52.410 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 193857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-23T14:17:52.410 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 45337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-23T14:17:52.410 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 61761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-23T14:17:52.410 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-23T14:17:52.410 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 39985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-23T14:17:52.410 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 40529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-23T14:17:52.410 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-23T14:17:52.411 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 59345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-23T14:17:52.411 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-23T14:17:52.411 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 34065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-23T14:17:52.411 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-23T14:17:52.411 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 70497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-23T14:17:52.411 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-23T14:17:52.411 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-23T14:17:52.411 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 39321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-23T14:17:52.411 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 151137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usbserial.ko 2024-04-23T14:17:52.411 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-23T14:17:52.411 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 43577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-23T14:17:52.411 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/wishbone-serial.ko 2024-04-23T14:17:52.411 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-23T14:17:52.411 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ 2024-04-23T14:17:52.412 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 64049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-23T14:17:52.412 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 30209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-23T14:17:52.412 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-23T14:17:52.412 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-23T14:17:52.412 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 41073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-23T14:17:52.412 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-23T14:17:52.412 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-23T14:17:52.412 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-23T14:17:52.412 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-23T14:17:52.412 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-23T14:17:52.412 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-23T14:17:52.412 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-23T14:17:52.412 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-23T14:17:52.412 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 41785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-23T14:17:52.413 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 201193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-23T14:17:52.413 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/ 2024-04-23T14:17:52.413 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 76041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-23T14:17:52.413 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 85257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-23T14:17:52.413 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 122321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-23T14:17:52.413 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/ 2024-04-23T14:17:52.413 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/ 2024-04-23T14:17:52.413 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 206921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-23T14:17:52.413 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-23T14:17:52.413 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 134265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-23T14:17:52.413 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 93673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-23T14:17:52.413 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/ 2024-04-23T14:17:52.413 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 138721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-23T14:17:52.413 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-23T14:17:52.414 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 71281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-23T14:17:52.414 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 107905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-23T14:17:52.414 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/ 2024-04-23T14:17:52.414 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/ 2024-04-23T14:17:52.414 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-04-23T14:17:52.414 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/aat2870_bl.ko 2024-04-23T14:17:52.414 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp5520_bl.ko 2024-04-23T14:17:52.414 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 41169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8860_bl.ko 2024-04-23T14:17:52.414 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 51745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8870_bl.ko 2024-04-23T14:17:52.414 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-23T14:17:52.414 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/as3711_bl.ko 2024-04-23T14:17:52.414 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/bd6107.ko 2024-04-23T14:17:52.414 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da903x_bl.ko 2024-04-23T14:17:52.415 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da9052_bl.ko 2024-04-23T14:17:52.415 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/gpio_backlight.ko 2024-04-23T14:17:52.415 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/kb3886_bl.ko 2024-04-23T14:17:52.415 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-23T14:17:52.415 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3533_bl.ko 2024-04-23T14:17:52.415 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3639_bl.ko 2024-04-23T14:17:52.415 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lv5207lp.ko 2024-04-23T14:17:52.415 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/max8925_bl.ko 2024-04-23T14:17:52.415 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pandora_bl.ko 2024-04-23T14:17:52.415 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-04-23T14:17:52.415 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-23T14:17:52.415 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/wm831x_bl.ko 2024-04-23T14:17:52.415 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/ 2024-04-23T14:17:52.415 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ 2024-04-23T14:17:52.415 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-23T14:17:52.416 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 46801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-23T14:17:52.416 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/matrox_w1.ko 2024-04-23T14:17:52.416 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/w1-gpio.ko 2024-04-23T14:17:52.416 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/ 2024-04-23T14:17:52.416 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-23T14:17:52.416 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 27257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-23T14:17:52.416 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-23T14:17:52.416 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-23T14:17:52.416 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-23T14:17:52.416 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-23T14:17:52.416 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-23T14:17:52.417 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-23T14:17:52.417 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-23T14:17:52.417 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 8313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-23T14:17:52.417 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 70369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-23T14:17:52.417 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 129433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-23T14:17:52.417 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ 2024-04-23T14:17:52.417 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/acquirewdt.ko 2024-04-23T14:17:52.417 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/advantechwdt.ko 2024-04-23T14:17:52.417 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-23T14:17:52.417 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 27401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-23T14:17:52.417 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/cpu5wdt.ko 2024-04-23T14:17:52.417 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9052_wdt.ko 2024-04-23T14:17:52.417 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9055_wdt.ko 2024-04-23T14:17:52.417 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9063_wdt.ko 2024-04-23T14:17:52.418 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/dw_wdt.ko 2024-04-23T14:17:52.418 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/eurotechwdt.ko 2024-04-23T14:17:52.418 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-23T14:17:52.418 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 32857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-23T14:17:52.418 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-23T14:17:52.418 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-23T14:17:52.418 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-23T14:17:52.418 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-23T14:17:52.418 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-23T14:17:52.418 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-23T14:17:52.418 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-23T14:17:52.418 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-23T14:17:52.418 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/kempld_wdt.ko 2024-04-23T14:17:52.419 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-23T14:17:52.419 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/mena21_wdt.ko 2024-04-23T14:17:52.419 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-04-23T14:17:52.419 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-23T14:17:52.419 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-04-23T14:17:52.419 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 27721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pc87413_wdt.ko 2024-04-23T14:17:52.419 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 46953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-23T14:17:52.419 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 37193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-23T14:17:52.419 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/retu_wdt.ko 2024-04-23T14:17:52.419 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-04-23T14:17:52.419 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-04-23T14:17:52.419 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-23T14:17:52.419 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sc1200wdt.ko 2024-04-23T14:17:52.419 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 27489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-23T14:17:52.420 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-04-23T14:17:52.420 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-23T14:17:52.420 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-23T14:17:52.420 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/twl4030_wdt.ko 2024-04-23T14:17:52.420 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-23T14:17:52.420 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-23T14:17:52.420 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-23T14:17:52.420 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 32481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-23T14:17:52.420 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wafer5823wdt.ko 2024-04-23T14:17:52.420 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-23T14:17:52.420 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm831x_wdt.ko 2024-04-23T14:17:52.420 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm8350_wdt.ko 2024-04-23T14:17:52.420 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-23T14:17:52.420 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/ 2024-04-23T14:17:52.420 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-23T14:17:52.421 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 30649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-23T14:17:52.421 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-23T14:17:52.421 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/ 2024-04-23T14:17:52.421 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 213913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-23T14:17:52.421 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-23T14:17:52.421 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 85713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-scsiback.ko 2024-04-23T14:17:52.421 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/ 2024-04-23T14:17:52.421 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-23T14:17:52.421 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ 2024-04-23T14:17:52.421 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/ 2024-04-23T14:17:52.421 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 192617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-23T14:17:52.421 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/ 2024-04-23T14:17:52.421 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 78153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/adfs.ko 2024-04-23T14:17:52.421 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/ 2024-04-23T14:17:52.422 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 197849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-23T14:17:52.422 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/ 2024-04-23T14:17:52.422 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 1261113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/kafs.ko 2024-04-23T14:17:52.422 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/ 2024-04-23T14:17:52.422 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 127377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/autofs4.ko 2024-04-23T14:17:52.422 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/ 2024-04-23T14:17:52.422 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-23T14:17:52.422 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/ 2024-04-23T14:17:52.422 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 50473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/bfs.ko 2024-04-23T14:17:52.422 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 45561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-23T14:17:52.422 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/ 2024-04-23T14:17:52.422 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 4489593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-23T14:17:52.422 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/ 2024-04-23T14:17:52.423 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 376857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-23T14:17:52.423 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ 2024-04-23T14:17:52.423 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 1586065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-23T14:17:52.423 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/ 2024-04-23T14:17:52.423 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 130073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-23T14:17:52.423 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/ 2024-04-23T14:17:52.423 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 139081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/configfs.ko 2024-04-23T14:17:52.423 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/ 2024-04-23T14:17:52.423 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-23T14:17:52.423 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/ 2024-04-23T14:17:52.423 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 813225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-23T14:17:52.423 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/ 2024-04-23T14:17:52.423 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 46409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/efs.ko 2024-04-23T14:17:52.423 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/ 2024-04-23T14:17:52.424 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 1906817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-23T14:17:52.424 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/ 2024-04-23T14:17:52.424 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-23T14:17:52.424 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/ 2024-04-23T14:17:52.424 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/freevxfs.ko 2024-04-23T14:17:52.424 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/ 2024-04-23T14:17:52.424 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-23T14:17:52.424 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/ 2024-04-23T14:17:52.424 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 1486521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-23T14:17:52.424 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/ 2024-04-23T14:17:52.424 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 183305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-23T14:17:52.424 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/ 2024-04-23T14:17:52.424 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 282097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-23T14:17:52.424 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/ 2024-04-23T14:17:52.425 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 249009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/hpfs.ko 2024-04-23T14:17:52.425 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/ 2024-04-23T14:17:52.425 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 93033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-23T14:17:52.425 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/ 2024-04-23T14:17:52.425 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 533073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-23T14:17:52.425 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/ 2024-04-23T14:17:52.425 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 361369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-23T14:17:52.425 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/ 2024-04-23T14:17:52.425 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 112609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-23T14:17:52.425 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/ 2024-04-23T14:17:52.425 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 591585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-23T14:17:52.425 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/ 2024-04-23T14:17:52.425 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/ 2024-04-23T14:17:52.425 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 102561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-23T14:17:52.425 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/ 2024-04-23T14:17:52.426 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 74353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-23T14:17:52.426 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/ 2024-04-23T14:17:52.426 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 147689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-23T14:17:52.426 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 1320249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-23T14:17:52.476 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 82105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv2.ko 2024-04-23T14:17:52.476 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 151641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-23T14:17:52.476 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 2518505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-23T14:17:52.476 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/ 2024-04-23T14:17:52.476 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-23T14:17:52.476 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-23T14:17:52.476 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/ 2024-04-23T14:17:52.476 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 2024049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-23T14:17:52.476 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/ 2024-04-23T14:17:52.476 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 734025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-23T14:17:52.477 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/ 2024-04-23T14:17:52.477 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-23T14:17:52.477 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-23T14:17:52.477 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-23T14:17:52.477 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-23T14:17:52.477 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-23T14:17:52.477 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-23T14:17:52.477 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-23T14:17:52.477 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-23T14:17:52.477 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-23T14:17:52.477 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-23T14:17:52.477 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-23T14:17:52.477 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ascii.ko 2024-04-23T14:17:52.477 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-23T14:17:52.478 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-23T14:17:52.478 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-23T14:17:52.478 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-23T14:17:52.478 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-23T14:17:52.478 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-23T14:17:52.478 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-23T14:17:52.478 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-23T14:17:52.478 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-23T14:17:52.478 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-23T14:17:52.478 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-23T14:17:52.478 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-23T14:17:52.478 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-23T14:17:52.478 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-23T14:17:52.478 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-23T14:17:52.479 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-23T14:17:52.479 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-23T14:17:52.479 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-23T14:17:52.479 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 95625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-23T14:17:52.479 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 144209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-23T14:17:52.479 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 167137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-23T14:17:52.479 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 119513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-23T14:17:52.479 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-23T14:17:52.479 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-23T14:17:52.479 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-23T14:17:52.479 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-23T14:17:52.479 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-23T14:17:52.479 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-23T14:17:52.479 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-23T14:17:52.480 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-23T14:17:52.480 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-23T14:17:52.480 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-23T14:17:52.480 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-23T14:17:52.480 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-23T14:17:52.480 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-23T14:17:52.480 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-23T14:17:52.480 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-23T14:17:52.480 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-23T14:17:52.480 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-23T14:17:52.480 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ 2024-04-23T14:17:52.480 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ 2024-04-23T14:17:52.480 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 302073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-23T14:17:52.480 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ 2024-04-23T14:17:52.481 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 548169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-23T14:17:52.481 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ 2024-04-23T14:17:52.481 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-23T14:17:52.481 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 3245617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-23T14:17:52.481 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 22113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-23T14:17:52.481 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 39313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-23T14:17:52.481 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 43073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-23T14:17:52.481 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/ 2024-04-23T14:17:52.481 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 67489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/omfs.ko 2024-04-23T14:17:52.481 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/ 2024-04-23T14:17:52.481 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 462873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-23T14:17:52.481 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ 2024-04-23T14:17:52.481 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 67977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-23T14:17:52.482 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/ 2024-04-23T14:17:52.482 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/qnx4.ko 2024-04-23T14:17:52.482 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/ 2024-04-23T14:17:52.482 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 56097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/qnx6.ko 2024-04-23T14:17:52.482 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/ 2024-04-23T14:17:52.482 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 40313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_tree.ko 2024-04-23T14:17:52.482 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v1.ko 2024-04-23T14:17:52.482 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v2.ko 2024-04-23T14:17:52.482 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/ 2024-04-23T14:17:52.482 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 684841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-23T14:17:52.482 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/ 2024-04-23T14:17:52.482 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 34673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-23T14:17:52.482 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/ 2024-04-23T14:17:52.482 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/ 2024-04-23T14:17:52.483 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 3319041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-23T14:17:52.483 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/ 2024-04-23T14:17:52.483 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 8073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-23T14:17:52.483 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-23T14:17:52.483 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/ 2024-04-23T14:17:52.483 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 164185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-23T14:17:52.483 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/ 2024-04-23T14:17:52.483 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 115977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-23T14:17:52.483 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/ 2024-04-23T14:17:52.483 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 310017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-23T14:17:52.483 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ 2024-04-23T14:17:52.483 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 198233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-23T14:17:52.483 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/ 2024-04-23T14:17:52.483 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 5192705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-23T14:17:52.484 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/ 2024-04-23T14:17:52.484 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/ 2024-04-23T14:17:52.484 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 97625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/locktorture.ko 2024-04-23T14:17:52.484 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/ 2024-04-23T14:17:52.484 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/test_udelay.ko 2024-04-23T14:17:52.484 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 72665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-23T14:17:52.484 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ 2024-04-23T14:17:52.484 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 7313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-23T14:17:52.484 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 7081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc7.ko 2024-04-23T14:17:52.484 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 7953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-23T14:17:52.484 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/ 2024-04-23T14:17:52.484 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 8009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-23T14:17:52.484 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 38985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-23T14:17:52.485 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/interval_tree_test.ko 2024-04-23T14:17:52.485 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-23T14:17:52.485 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-23T14:17:52.485 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/ 2024-04-23T14:17:52.485 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 36169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-23T14:17:52.485 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-23T14:17:52.485 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/ 2024-04-23T14:17:52.485 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 7281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-23T14:17:52.485 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/memory-notifier-error-inject.ko 2024-04-23T14:17:52.485 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/notifier-error-inject.ko 2024-04-23T14:17:52.485 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/percpu_test.ko 2024-04-23T14:17:52.485 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/pm-notifier-error-inject.ko 2024-04-23T14:17:52.485 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/ 2024-04-23T14:17:52.485 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 152025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-23T14:17:52.485 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/rbtree_test.ko 2024-04-23T14:17:52.486 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/ 2024-04-23T14:17:52.486 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-23T14:17:52.486 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 40969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test-kstrtox.ko 2024-04-23T14:17:52.486 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 4912441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_bpf.ko 2024-04-23T14:17:52.486 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 61169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_firmware.ko 2024-04-23T14:17:52.486 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 7121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_module.ko 2024-04-23T14:17:52.486 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_user_copy.ko 2024-04-23T14:17:52.486 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-23T14:17:52.486 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-23T14:17:52.486 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-23T14:17:52.486 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/ 2024-04-23T14:17:52.486 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/xz_dec_test.ko 2024-04-23T14:17:52.486 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/ 2024-04-23T14:17:52.487 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-23T14:17:52.487 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ 2024-04-23T14:17:52.487 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/ 2024-04-23T14:17:52.487 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 66969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-23T14:17:52.487 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 7577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-23T14:17:52.487 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-23T14:17:52.487 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-23T14:17:52.487 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-23T14:17:52.487 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-23T14:17:52.487 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-23T14:17:52.487 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-23T14:17:52.487 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/ 2024-04-23T14:17:52.487 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-23T14:17:52.487 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-23T14:17:52.488 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-23T14:17:52.488 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-23T14:17:52.488 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-23T14:17:52.488 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/ 2024-04-23T14:17:52.488 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 106793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-23T14:17:52.488 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/ 2024-04-23T14:17:52.488 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 236793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-23T14:17:52.488 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 44449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-23T14:17:52.488 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 32209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-23T14:17:52.488 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 39513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-23T14:17:52.488 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/ 2024-04-23T14:17:52.488 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 879697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-23T14:17:52.488 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/ 2024-04-23T14:17:52.488 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 84761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-23T14:17:52.488 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 1030873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-23T14:17:52.489 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ 2024-04-23T14:17:52.489 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 8905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-23T14:17:52.489 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-23T14:17:52.489 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-23T14:17:52.489 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-23T14:17:52.489 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-23T14:17:52.489 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-23T14:17:52.489 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-23T14:17:52.489 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-23T14:17:52.489 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-23T14:17:52.489 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 8841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-23T14:17:52.489 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 8777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-23T14:17:52.489 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-23T14:17:52.489 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-23T14:17:52.490 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-23T14:17:52.490 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-23T14:17:52.490 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-23T14:17:52.490 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-23T14:17:52.490 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-23T14:17:52.490 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-23T14:17:52.490 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-23T14:17:52.490 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 70529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-23T14:17:52.490 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-23T14:17:52.490 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-23T14:17:52.490 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/ 2024-04-23T14:17:52.490 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 1243505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-23T14:17:52.490 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/ 2024-04-23T14:17:52.490 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 153801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-23T14:17:52.491 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/ 2024-04-23T14:17:52.491 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 244889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-23T14:17:52.491 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-23T14:17:52.491 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 71033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-23T14:17:52.491 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 74993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-23T14:17:52.491 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/ 2024-04-23T14:17:52.491 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 507705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-23T14:17:52.491 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-23T14:17:52.491 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-23T14:17:52.491 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 8873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-23T14:17:52.491 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-23T14:17:52.491 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/ 2024-04-23T14:17:52.491 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 128873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/hsr.ko 2024-04-23T14:17:52.491 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ 2024-04-23T14:17:52.492 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ 2024-04-23T14:17:52.492 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-23T14:17:52.492 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 357425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-23T14:17:52.492 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 59393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-23T14:17:52.492 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ 2024-04-23T14:17:52.492 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-23T14:17:52.492 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 46417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-23T14:17:52.492 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 60233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-23T14:17:52.492 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-23T14:17:52.492 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 65665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-23T14:17:52.492 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 75489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-23T14:17:52.492 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 77721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-23T14:17:52.492 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 43705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-23T14:17:52.492 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-23T14:17:52.493 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 34745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-23T14:17:52.493 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ 2024-04-23T14:17:52.493 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 45521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-23T14:17:52.493 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-23T14:17:52.493 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-23T14:17:52.493 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 50169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ip_tables.ko 2024-04-23T14:17:52.493 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-23T14:17:52.493 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-04-23T14:17:52.493 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-23T14:17:52.493 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-23T14:17:52.493 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-23T14:17:52.493 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-04-23T14:17:52.493 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-23T14:17:52.493 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-23T14:17:52.493 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-23T14:17:52.494 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-23T14:17:52.494 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-23T14:17:52.494 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-23T14:17:52.494 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 32473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-23T14:17:52.494 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-23T14:17:52.494 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-23T14:17:52.494 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-04-23T14:17:52.494 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-23T14:17:52.494 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-23T14:17:52.494 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-23T14:17:52.494 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-23T14:17:52.494 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-23T14:17:52.494 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-23T14:17:52.494 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-23T14:17:52.495 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-23T14:17:52.495 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-23T14:17:52.495 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-23T14:17:52.495 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-23T14:17:52.495 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-23T14:17:52.495 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-23T14:17:52.495 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-23T14:17:52.495 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-23T14:17:52.495 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-23T14:17:52.495 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-23T14:17:52.495 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-23T14:17:52.495 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-23T14:17:52.495 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-23T14:17:52.495 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 54441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-23T14:17:52.496 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-23T14:17:52.496 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ 2024-04-23T14:17:52.496 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 31073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-23T14:17:52.496 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 47433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-23T14:17:52.496 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-23T14:17:52.496 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 103177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_gre.ko 2024-04-23T14:17:52.496 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 118881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-23T14:17:52.496 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-23T14:17:52.496 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 71089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-23T14:17:52.496 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-23T14:17:52.496 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/mip6.ko 2024-04-23T14:17:52.496 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ 2024-04-23T14:17:52.496 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 50673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-23T14:17:52.496 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-04-23T14:17:52.497 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-23T14:17:52.497 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-23T14:17:52.497 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-23T14:17:52.497 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 8721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-23T14:17:52.497 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-23T14:17:52.497 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-23T14:17:52.497 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-23T14:17:52.497 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-23T14:17:52.497 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-23T14:17:52.497 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-23T14:17:52.497 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-23T14:17:52.497 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-23T14:17:52.497 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-23T14:17:52.497 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-23T14:17:52.498 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-23T14:17:52.498 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 46313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-23T14:17:52.498 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-23T14:17:52.498 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 29817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-23T14:17:52.498 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-23T14:17:52.498 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-23T14:17:52.498 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-23T14:17:52.498 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-23T14:17:52.498 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-23T14:17:52.498 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 94089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-23T14:17:52.498 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 30297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-23T14:17:52.498 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-23T14:17:52.498 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/ 2024-04-23T14:17:52.499 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 106369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-23T14:17:52.499 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/ 2024-04-23T14:17:52.499 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 153217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-23T14:17:52.499 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-23T14:17:52.499 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 31985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-23T14:17:52.499 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 43641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-23T14:17:52.499 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 44993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-23T14:17:52.499 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 42049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-23T14:17:52.499 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 66241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-23T14:17:52.499 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/ 2024-04-23T14:17:52.499 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 63345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/lapb.ko 2024-04-23T14:17:52.499 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/ 2024-04-23T14:17:52.499 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 27385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-23T14:17:52.499 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 332889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc2.ko 2024-04-23T14:17:52.499 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/ 2024-04-23T14:17:52.500 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 388873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-23T14:17:52.500 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/ 2024-04-23T14:17:52.500 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-23T14:17:52.500 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ 2024-04-23T14:17:52.500 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ 2024-04-23T14:17:52.500 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 147745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-23T14:17:52.500 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 34473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-23T14:17:52.500 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-23T14:17:52.500 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-23T14:17:52.500 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 124041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-23T14:17:52.500 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 124345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-23T14:17:52.500 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 126681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-23T14:17:52.500 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 126473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-23T14:17:52.500 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 137833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-23T14:17:52.501 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 72353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-23T14:17:52.501 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 133681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-23T14:17:52.501 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 139073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-23T14:17:52.501 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 138265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-23T14:17:52.501 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 136177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-23T14:17:52.501 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 139897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-23T14:17:52.501 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 37129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-23T14:17:52.501 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ 2024-04-23T14:17:52.501 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 534113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-23T14:17:52.501 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-23T14:17:52.501 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-23T14:17:52.501 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-23T14:17:52.501 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-23T14:17:52.502 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-23T14:17:52.502 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-23T14:17:52.502 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 8865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-23T14:17:52.502 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-23T14:17:52.502 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-23T14:17:52.502 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-23T14:17:52.502 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-23T14:17:52.502 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-23T14:17:52.502 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-23T14:17:52.502 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 55217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-23T14:17:52.502 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 496817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-23T14:17:52.502 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-23T14:17:52.502 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-23T14:17:52.503 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 37865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-23T14:17:52.503 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 134377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-23T14:17:52.503 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-23T14:17:52.503 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-23T14:17:52.503 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 140801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-23T14:17:52.503 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 38305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-23T14:17:52.503 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-23T14:17:52.503 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 79873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-23T14:17:52.503 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-23T14:17:52.503 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-23T14:17:52.503 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-23T14:17:52.503 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 117201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-23T14:17:52.504 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-23T14:17:52.504 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 46105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-23T14:17:52.504 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 118369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-23T14:17:52.504 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-23T14:17:52.504 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-23T14:17:52.504 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-23T14:17:52.504 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 34609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-23T14:17:52.504 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-23T14:17:52.504 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 53393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-23T14:17:52.504 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 807985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-23T14:17:52.504 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 53049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-23T14:17:52.504 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 36481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-23T14:17:52.504 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 37209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-04-23T14:17:52.504 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 49297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-23T14:17:52.504 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-23T14:17:52.505 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 65145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-23T14:17:52.505 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-23T14:17:52.505 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 41769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-23T14:17:52.505 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 50657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-23T14:17:52.505 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-23T14:17:52.505 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-23T14:17:52.505 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-23T14:17:52.505 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-23T14:17:52.505 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-23T14:17:52.505 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-23T14:17:52.505 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-23T14:17:52.505 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-23T14:17:52.505 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-23T14:17:52.505 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_osf.ko 2024-04-23T14:17:52.506 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-23T14:17:52.506 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-23T14:17:52.506 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-23T14:17:52.506 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-23T14:17:52.506 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-23T14:17:52.506 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 24993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_synproxy.ko 2024-04-23T14:17:52.506 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tproxy.ko 2024-04-23T14:17:52.506 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 108393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/x_tables.ko 2024-04-23T14:17:52.506 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-23T14:17:52.506 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-23T14:17:52.506 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-23T14:17:52.506 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-23T14:17:52.506 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 21865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-23T14:17:52.507 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-23T14:17:52.507 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-23T14:17:52.507 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-23T14:17:52.507 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 45217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-23T14:17:52.507 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-23T14:17:52.507 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-23T14:17:52.507 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-23T14:17:52.507 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-23T14:17:52.507 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-23T14:17:52.507 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-23T14:17:52.507 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-23T14:17:52.507 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-23T14:17:52.507 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-23T14:17:52.507 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-23T14:17:52.508 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-23T14:17:52.508 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-23T14:17:52.508 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-23T14:17:52.508 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-23T14:17:52.508 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-23T14:17:52.508 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-23T14:17:52.508 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-23T14:17:52.508 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-23T14:17:52.508 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 8465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-23T14:17:52.508 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-23T14:17:52.508 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-23T14:17:52.508 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-23T14:17:52.508 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-23T14:17:52.508 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-23T14:17:52.508 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 8913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-23T14:17:52.509 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-23T14:17:52.509 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-23T14:17:52.509 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-23T14:17:52.509 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-23T14:17:52.509 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-23T14:17:52.509 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 52817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-23T14:17:52.509 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-23T14:17:52.509 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-23T14:17:52.509 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-23T14:17:52.509 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-23T14:17:52.509 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-23T14:17:52.509 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-23T14:17:52.509 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-23T14:17:52.509 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-23T14:17:52.509 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 8449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-23T14:17:52.510 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-23T14:17:52.510 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-23T14:17:52.510 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 16161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-23T14:17:52.510 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-23T14:17:52.510 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-23T14:17:52.510 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-23T14:17:52.510 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-23T14:17:52.510 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-23T14:17:52.510 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-23T14:17:52.510 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-23T14:17:52.510 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 11881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-23T14:17:52.510 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-23T14:17:52.510 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 43465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-23T14:17:52.511 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-23T14:17:52.511 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 36489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-23T14:17:52.511 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-23T14:17:52.511 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-23T14:17:52.511 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-23T14:17:52.511 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-23T14:17:52.511 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-23T14:17:52.511 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpudp.ko 2024-04-23T14:17:52.511 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-23T14:17:52.511 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-23T14:17:52.511 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/ 2024-04-23T14:17:52.511 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 17233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-23T14:17:52.511 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/ 2024-04-23T14:17:52.511 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-23T14:17:52.512 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/ 2024-04-23T14:17:52.512 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 488825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-23T14:17:52.512 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-23T14:17:52.512 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-23T14:17:52.512 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-23T14:17:52.512 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/ 2024-04-23T14:17:52.512 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-23T14:17:52.512 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/ 2024-04-23T14:17:52.512 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 142025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/phonet.ko 2024-04-23T14:17:52.512 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 68817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/pn_pep.ko 2024-04-23T14:17:52.512 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/ 2024-04-23T14:17:52.512 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 318617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-23T14:17:52.512 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 202305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-23T14:17:52.512 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 85105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-23T14:17:52.513 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/ 2024-04-23T14:17:52.513 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 1027897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/rxrpc.ko 2024-04-23T14:17:52.513 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/ 2024-04-23T14:17:52.513 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 39001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-23T14:17:52.513 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 28721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-23T14:17:52.513 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 45409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-23T14:17:52.513 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-23T14:17:52.513 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 53465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-23T14:17:52.513 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 38705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-23T14:17:52.513 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-23T14:17:52.513 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 43417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-23T14:17:52.513 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 45673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-23T14:17:52.513 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 32129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-23T14:17:52.513 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 41457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-23T14:17:52.514 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_cgroup.ko 2024-04-23T14:17:52.514 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 37305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-23T14:17:52.514 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 39601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-23T14:17:52.514 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 49321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-23T14:17:52.514 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 79145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-23T14:17:52.514 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 8713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-23T14:17:52.514 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-23T14:17:52.514 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 48785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-23T14:17:52.514 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-23T14:17:52.514 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-23T14:17:52.514 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 8377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-23T14:17:52.514 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-23T14:17:52.514 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 25097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-23T14:17:52.515 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 43481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-23T14:17:52.515 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 43153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-23T14:17:52.515 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 39177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq_codel.ko 2024-04-23T14:17:52.515 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-23T14:17:52.515 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 62817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-23T14:17:52.515 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-23T14:17:52.515 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 81833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-23T14:17:52.515 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-23T14:17:52.515 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 36625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-23T14:17:52.515 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-23T14:17:52.515 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-23T14:17:52.515 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 44793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-23T14:17:52.515 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 32833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-23T14:17:52.515 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 13505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-23T14:17:52.516 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-23T14:17:52.516 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 60249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-23T14:17:52.516 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-23T14:17:52.516 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 33433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-23T14:17:52.516 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-23T14:17:52.516 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 35777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-23T14:17:52.516 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 34161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-23T14:17:52.516 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/ 2024-04-23T14:17:52.516 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 1187313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-23T14:17:52.516 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-23T14:17:52.516 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/ 2024-04-23T14:17:52.516 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/ 2024-04-23T14:17:52.516 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 432409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-23T14:17:52.516 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 70113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-23T14:17:52.517 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 1940889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-23T14:17:52.517 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/ 2024-04-23T14:17:52.517 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 890729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-23T14:17:52.517 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/ 2024-04-23T14:17:52.517 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 1122337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-23T14:17:52.517 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-23T14:17:52.517 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/ 2024-04-23T14:17:52.517 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 18801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-23T14:17:52.517 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/ 2024-04-23T14:17:52.517 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 105609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-23T14:17:52.517 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 90265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-23T14:17:52.517 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 123769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-23T14:17:52.517 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-23T14:17:52.517 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-23T14:17:52.518 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/ 2024-04-23T14:17:52.518 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 165841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/x25.ko 2024-04-23T14:17:52.518 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/ 2024-04-23T14:17:52.518 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_algo.ko 2024-04-23T14:17:52.518 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-23T14:17:52.518 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 131865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_user.ko 2024-04-23T14:17:52.518 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 644769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-23T14:17:52.518 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 628542 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-23T14:17:52.518 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 7706 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-23T14:17:52.518 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 23609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-23T14:17:52.518 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 9941 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-23T14:17:52.518 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 70990 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-23T14:17:52.518 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 146959 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-23T14:17:52.518 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 232835 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-23T14:17:52.519 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 210 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-23T14:17:52.519 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 72602 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-23T14:17:52.519 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 879 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-23T14:17:52.519 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 258450 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-23T14:17:52.519 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 313019 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-23T14:17:52.519 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/ 2024-04-23T14:17:52.519 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/ 2024-04-23T14:17:52.519 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/ 2024-04-23T14:17:52.519 INFO:teuthology.orchestra.run.smithi151.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/ 2024-04-23T14:17:52.519 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 172 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/changelog.Debian.gz 2024-04-23T14:17:52.519 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r-- root/root 691 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/copyright 2024-04-23T14:17:52.524 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-23T14:17:52.524 INFO:teuthology.task.kernel:searching for kernel 6.9.0-rc3-g806ec36b2112 2024-04-23T14:17:52.525 DEBUG:teuthology.orchestra.run.smithi151:> egrep '(submenu|menuentry.*6.9.0-rc3-g806ec36b2112).*{' /boot/grub/grub.cfg 2024-04-23T14:17:52.532 INFO:teuthology.orchestra.run.smithi151.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-23T14:17:52.532 INFO:teuthology.orchestra.run.smithi151.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-23T14:17:52.533 INFO:teuthology.orchestra.run.smithi151.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-23T14:17:52.534 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-04-23T14:17:52.534 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 2024-04-23T14:17:52.534 DEBUG:teuthology.orchestra.run.smithi151:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-04-23T14:17:52.576 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi151.front.sepia.ceph.com to complete... 2024-04-23T14:17:52.963 INFO:teuthology.orchestra.run.smithi151.stderr:Sourcing file `/etc/default/grub' 2024-04-23T14:17:52.964 INFO:teuthology.orchestra.run.smithi151.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-23T14:17:52.990 INFO:teuthology.orchestra.run.smithi151.stderr:Generating grub configuration file ... 2024-04-23T14:17:53.218 INFO:teuthology.orchestra.run.smithi151.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-23T14:17:53.225 INFO:teuthology.orchestra.run.smithi151.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-23T14:17:53.469 INFO:teuthology.orchestra.run.smithi151.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-23T14:17:53.475 INFO:teuthology.orchestra.run.smithi151.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-23T14:17:53.514 INFO:teuthology.orchestra.run.smithi151.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-23T14:17:53.519 INFO:teuthology.orchestra.run.smithi151.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-23T14:17:53.648 INFO:teuthology.orchestra.run.smithi151.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-23T14:17:53.648 INFO:teuthology.orchestra.run.smithi151.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-23T14:17:53.648 INFO:teuthology.orchestra.run.smithi151.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-23T14:17:53.659 INFO:teuthology.orchestra.run.smithi151.stderr:done 2024-04-23T14:18:03.125 INFO:teuthology.orchestra.run.smithi100.stderr:Sourcing file `/etc/default/grub' 2024-04-23T14:18:03.145 INFO:teuthology.orchestra.run.smithi100.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-23T14:18:03.175 INFO:teuthology.orchestra.run.smithi100.stderr:Generating grub configuration file ... 2024-04-23T14:18:03.490 INFO:teuthology.orchestra.run.smithi100.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-23T14:18:03.504 INFO:teuthology.orchestra.run.smithi100.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-23T14:18:03.798 INFO:teuthology.orchestra.run.smithi100.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-23T14:18:03.804 INFO:teuthology.orchestra.run.smithi100.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-23T14:18:03.863 INFO:teuthology.orchestra.run.smithi100.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-23T14:18:03.868 INFO:teuthology.orchestra.run.smithi100.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-23T14:18:04.015 INFO:teuthology.orchestra.run.smithi100.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-23T14:18:04.015 INFO:teuthology.orchestra.run.smithi100.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-23T14:18:04.015 INFO:teuthology.orchestra.run.smithi100.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-23T14:18:04.061 INFO:teuthology.orchestra.run.smithi100.stderr:done 2024-04-23T14:18:04.230 DEBUG:teuthology.orchestra.run.smithi100:> dpkg-deb -c /tmp/linux-image.deb 2024-04-23T14:18:04.238 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./ 2024-04-23T14:18:04.238 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./boot/ 2024-04-23T14:18:04.238 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 7002012 2024-04-19 08:55 ./boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-23T14:18:04.256 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 172073 2024-04-19 08:55 ./boot/config-6.9.0-rc3-g806ec36b2112 2024-04-23T14:18:04.256 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13120512 2024-04-19 08:55 ./boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-23T14:18:04.279 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/ 2024-04-23T14:18:04.279 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/ 2024-04-23T14:18:04.279 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postinst.d/ 2024-04-23T14:18:04.279 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postrm.d/ 2024-04-23T14:18:04.279 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/preinst.d/ 2024-04-23T14:18:04.280 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/prerm.d/ 2024-04-23T14:18:04.280 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/ 2024-04-23T14:18:04.280 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/ 2024-04-23T14:18:04.280 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/ 2024-04-23T14:18:04.280 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/ 2024-04-23T14:18:04.280 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/ 2024-04-23T14:18:04.280 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/ 2024-04-23T14:18:04.280 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ 2024-04-23T14:18:04.280 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 507569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/aesni-intel.ko 2024-04-23T14:18:04.280 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-23T14:18:04.280 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 34313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-23T14:18:04.280 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-23T14:18:04.280 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-23T14:18:04.281 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 109905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-23T14:18:04.281 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 155249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-23T14:18:04.281 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-23T14:18:04.281 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-23T14:18:04.281 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 87009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-23T14:18:04.281 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-23T14:18:04.281 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 50305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-23T14:18:04.281 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 50377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-23T14:18:04.281 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 54513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-23T14:18:04.281 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 44657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-23T14:18:04.281 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 48049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-23T14:18:04.281 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 62873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-23T14:18:04.281 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 53569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-23T14:18:04.281 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 34761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-23T14:18:04.282 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-23T14:18:04.282 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/ 2024-04-23T14:18:04.282 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/ 2024-04-23T14:18:04.282 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/ 2024-04-23T14:18:04.282 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-23T14:18:04.282 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpuid.ko 2024-04-23T14:18:04.282 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/msr.ko 2024-04-23T14:18:04.282 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/ 2024-04-23T14:18:04.282 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 444425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-23T14:18:04.283 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 1035881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-23T14:18:04.283 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 2849609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-23T14:18:04.303 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ 2024-04-23T14:18:04.303 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 65025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/af_alg.ko 2024-04-23T14:18:04.303 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 27961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_hash.ko 2024-04-23T14:18:04.303 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_skcipher.ko 2024-04-23T14:18:04.303 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-23T14:18:04.303 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-23T14:18:04.303 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-23T14:18:04.303 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/ 2024-04-23T14:18:04.303 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-04-23T14:18:04.304 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/ 2024-04-23T14:18:04.304 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-23T14:18:04.304 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-23T14:18:04.304 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-23T14:18:04.304 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-23T14:18:04.304 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-23T14:18:04.304 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-23T14:18:04.304 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-23T14:18:04.304 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 27953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-23T14:18:04.304 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-23T14:18:04.304 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-23T14:18:04.304 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-23T14:18:04.304 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-23T14:18:04.304 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-23T14:18:04.305 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-23T14:18:04.305 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-23T14:18:04.305 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 43281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-23T14:18:04.305 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-23T14:18:04.305 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-23T14:18:04.305 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 59129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cryptd.ko 2024-04-23T14:18:04.305 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 48113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-23T14:18:04.305 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 32169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_simd.ko 2024-04-23T14:18:04.305 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 30881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-23T14:18:04.305 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-23T14:18:04.305 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-23T14:18:04.305 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-23T14:18:04.305 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-23T14:18:04.305 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 30689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-23T14:18:04.306 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-23T14:18:04.306 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-23T14:18:04.306 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lrw.ko 2024-04-23T14:18:04.306 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-23T14:18:04.306 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-23T14:18:04.306 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-23T14:18:04.306 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-23T14:18:04.306 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-23T14:18:04.306 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-23T14:18:04.306 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-23T14:18:04.306 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 30673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-23T14:18:04.306 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-23T14:18:04.306 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 169065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-23T14:18:04.306 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-23T14:18:04.307 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-23T14:18:04.307 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-23T14:18:04.307 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-23T14:18:04.307 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-23T14:18:04.307 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-23T14:18:04.307 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-23T14:18:04.307 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xts.ko 2024-04-23T14:18:04.307 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-23T14:18:04.307 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ 2024-04-23T14:18:04.307 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ 2024-04-23T14:18:04.307 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_extlog.ko 2024-04-23T14:18:04.307 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 31921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-23T14:18:04.307 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 32281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-23T14:18:04.307 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/ 2024-04-23T14:18:04.307 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 44705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/einj.ko 2024-04-23T14:18:04.308 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-23T14:18:04.308 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-23T14:18:04.308 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-23T14:18:04.308 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-23T14:18:04.308 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 119033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-23T14:18:04.308 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ 2024-04-23T14:18:04.308 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-23T14:18:04.308 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 96825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci.ko 2024-04-23T14:18:04.308 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-23T14:18:04.308 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 107737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci.ko 2024-04-23T14:18:04.308 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 42001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-23T14:18:04.308 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-23T14:18:04.308 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-23T14:18:04.309 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-23T14:18:04.309 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-23T14:18:04.309 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-23T14:18:04.309 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-23T14:18:04.309 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-23T14:18:04.309 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-23T14:18:04.309 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-23T14:18:04.309 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-23T14:18:04.309 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-23T14:18:04.309 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 38913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-23T14:18:04.309 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-23T14:18:04.309 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-23T14:18:04.309 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-23T14:18:04.309 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 41289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-23T14:18:04.310 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-23T14:18:04.310 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 60257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_legacy.ko 2024-04-23T14:18:04.310 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-23T14:18:04.310 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-23T14:18:04.310 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-23T14:18:04.310 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-23T14:18:04.310 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-23T14:18:04.310 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-23T14:18:04.310 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-23T14:18:04.310 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-23T14:18:04.310 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-23T14:18:04.310 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-23T14:18:04.310 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 43009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-23T14:18:04.310 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-23T14:18:04.311 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-23T14:18:04.311 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_radisys.ko 2024-04-23T14:18:04.311 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-23T14:18:04.311 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rz1000.ko 2024-04-23T14:18:04.311 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-23T14:18:04.311 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-23T14:18:04.311 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-23T14:18:04.311 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-23T14:18:04.311 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-23T14:18:04.311 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-23T14:18:04.311 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-23T14:18:04.311 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 34889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-23T14:18:04.311 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 102121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-23T14:18:04.311 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 85409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-23T14:18:04.312 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-23T14:18:04.312 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-23T14:18:04.312 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 34025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-23T14:18:04.312 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 45905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-23T14:18:04.312 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-23T14:18:04.312 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-23T14:18:04.312 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 53353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-23T14:18:04.312 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-23T14:18:04.312 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 41049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-23T14:18:04.312 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-23T14:18:04.312 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ 2024-04-23T14:18:04.312 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-23T14:18:04.312 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/ 2024-04-23T14:18:04.312 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 163969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-23T14:18:04.312 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/ 2024-04-23T14:18:04.313 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/ 2024-04-23T14:18:04.313 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 140865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-23T14:18:04.313 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/ 2024-04-23T14:18:04.313 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 1241657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-23T14:18:04.313 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 203289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-23T14:18:04.313 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/ 2024-04-23T14:18:04.313 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 115265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-23T14:18:04.313 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 143433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-23T14:18:04.313 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/ 2024-04-23T14:18:04.313 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 126617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-23T14:18:04.313 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 119873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-23T14:18:04.313 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 313217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-23T14:18:04.313 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/ 2024-04-23T14:18:04.313 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 117233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-23T14:18:04.314 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/ 2024-04-23T14:18:04.314 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 70385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-23T14:18:04.314 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ 2024-04-23T14:18:04.314 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/applicom.ko 2024-04-23T14:18:04.314 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-23T14:18:04.314 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/ 2024-04-23T14:18:04.314 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-23T14:18:04.314 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-23T14:18:04.314 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-23T14:18:04.314 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-23T14:18:04.314 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-23T14:18:04.314 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ 2024-04-23T14:18:04.314 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 37257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-23T14:18:04.315 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 186921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-23T14:18:04.315 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 32521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-23T14:18:04.315 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 179561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-23T14:18:04.315 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 85441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-23T14:18:04.315 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 65313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-23T14:18:04.315 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 43857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-23T14:18:04.315 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/ 2024-04-23T14:18:04.315 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 103409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-23T14:18:04.315 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 31833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/nvram.ko 2024-04-23T14:18:04.315 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 38977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-23T14:18:04.315 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 59241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-23T14:18:04.315 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/ 2024-04-23T14:18:04.315 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-23T14:18:04.315 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-04-23T14:18:04.316 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-04-23T14:18:04.316 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 39193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-04-23T14:18:04.316 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 34169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-23T14:18:04.316 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-23T14:18:04.316 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-04-23T14:18:04.316 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/ 2024-04-23T14:18:04.316 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_class.ko 2024-04-23T14:18:04.316 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 62545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_core.ko 2024-04-23T14:18:04.316 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-04-23T14:18:04.316 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/ 2024-04-23T14:18:04.316 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-palmas.ko 2024-04-23T14:18:04.316 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 57289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-si5351.ko 2024-04-23T14:18:04.317 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-twl6040.ko 2024-04-23T14:18:04.317 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-wm831x.ko 2024-04-23T14:18:04.318 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/ 2024-04-23T14:18:04.318 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-23T14:18:04.318 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-23T14:18:04.318 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-23T14:18:04.318 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ 2024-04-23T14:18:04.318 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ 2024-04-23T14:18:04.318 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 89257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-23T14:18:04.318 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 333657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-23T14:18:04.318 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/ 2024-04-23T14:18:04.318 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/ 2024-04-23T14:18:04.318 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/ 2024-04-23T14:18:04.318 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 751713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-23T14:18:04.370 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-04-23T14:18:04.371 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-23T14:18:04.371 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-23T14:18:04.371 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-23T14:18:04.371 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/ 2024-04-23T14:18:04.371 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 90441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-23T14:18:04.371 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/ 2024-04-23T14:18:04.371 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 36105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-23T14:18:04.371 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ 2024-04-23T14:18:04.371 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/ 2024-04-23T14:18:04.371 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-23T14:18:04.371 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ 2024-04-23T14:18:04.372 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 146769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-23T14:18:04.372 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ 2024-04-23T14:18:04.372 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 96145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-23T14:18:04.372 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 41993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-23T14:18:04.372 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 51105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-23T14:18:04.372 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-23T14:18:04.372 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-23T14:18:04.372 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-23T14:18:04.372 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-23T14:18:04.372 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-23T14:18:04.372 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 64161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-23T14:18:04.372 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-23T14:18:04.372 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-23T14:18:04.372 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 68697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-23T14:18:04.373 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-23T14:18:04.373 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/ 2024-04-23T14:18:04.373 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-gpio.ko 2024-04-23T14:18:04.373 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 31369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max14577.ko 2024-04-23T14:18:04.373 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max77693.ko 2024-04-23T14:18:04.373 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max8997.ko 2024-04-23T14:18:04.373 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 30961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-palmas.ko 2024-04-23T14:18:04.373 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 27201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-rt8973a.ko 2024-04-23T14:18:04.373 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 31993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-sm5502.ko 2024-04-23T14:18:04.373 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/ 2024-04-23T14:18:04.373 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 42953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/dmi-sysfs.ko 2024-04-23T14:18:04.373 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/ 2024-04-23T14:18:04.373 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/efi-pstore.ko 2024-04-23T14:18:04.373 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-23T14:18:04.374 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/ 2024-04-23T14:18:04.374 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-adp5520.ko 2024-04-23T14:18:04.374 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-amd8111.ko 2024-04-23T14:18:04.374 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-arizona.ko 2024-04-23T14:18:04.374 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9052.ko 2024-04-23T14:18:04.374 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9055.ko 2024-04-23T14:18:04.374 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-dln2.ko 2024-04-23T14:18:04.374 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 81073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-f7188x.ko 2024-04-23T14:18:04.374 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 40761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-generic.ko 2024-04-23T14:18:04.374 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-23T14:18:04.374 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-04-23T14:18:04.374 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-kempld.ko 2024-04-23T14:18:04.374 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-lp3943.ko 2024-04-23T14:18:04.374 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max7300.ko 2024-04-23T14:18:04.375 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max730x.ko 2024-04-23T14:18:04.375 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max732x.ko 2024-04-23T14:18:04.375 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-04-23T14:18:04.375 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pca953x.ko 2024-04-23T14:18:04.375 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 31881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pcf857x.ko 2024-04-23T14:18:04.375 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-rdc321x.ko 2024-04-23T14:18:04.375 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch.ko 2024-04-23T14:18:04.375 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch311x.ko 2024-04-23T14:18:04.375 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-tps65912.ko 2024-04-23T14:18:04.375 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl4030.ko 2024-04-23T14:18:04.375 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl6040.ko 2024-04-23T14:18:04.375 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-23T14:18:04.375 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-vx855.ko 2024-04-23T14:18:04.376 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm831x.ko 2024-04-23T14:18:04.376 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8350.ko 2024-04-23T14:18:04.376 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8994.ko 2024-04-23T14:18:04.376 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/ 2024-04-23T14:18:04.376 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-a4tech.ko 2024-04-23T14:18:04.376 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-apple.ko 2024-04-23T14:18:04.376 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-23T14:18:04.376 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-23T14:18:04.376 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-23T14:18:04.376 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-belkin.ko 2024-04-23T14:18:04.376 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cherry.ko 2024-04-23T14:18:04.376 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-chicony.ko 2024-04-23T14:18:04.376 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 74393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cp2112.ko 2024-04-23T14:18:04.377 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cypress.ko 2024-04-23T14:18:04.377 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-23T14:18:04.377 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-23T14:18:04.377 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-23T14:18:04.377 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-23T14:18:04.377 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ezkey.ko 2024-04-23T14:18:04.377 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-23T14:18:04.377 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-generic.ko 2024-04-23T14:18:04.377 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-23T14:18:04.377 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-23T14:18:04.377 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-23T14:18:04.377 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-23T14:18:04.377 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-23T14:18:04.377 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-23T14:18:04.377 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-23T14:18:04.378 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kensington.ko 2024-04-23T14:18:04.378 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-23T14:18:04.378 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-23T14:18:04.378 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-23T14:18:04.378 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-23T14:18:04.378 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 54217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-23T14:18:04.378 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 37553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lg-g15.ko 2024-04-23T14:18:04.378 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 68081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-23T14:18:04.378 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 121369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-23T14:18:04.378 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech.ko 2024-04-23T14:18:04.378 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 36689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-magicmouse.ko 2024-04-23T14:18:04.378 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-microsoft.ko 2024-04-23T14:18:04.378 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-monterey.ko 2024-04-23T14:18:04.379 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 59001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-23T14:18:04.379 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 39697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ntrig.ko 2024-04-23T14:18:04.379 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-23T14:18:04.379 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-23T14:18:04.379 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-23T14:18:04.379 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 91241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-23T14:18:04.379 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-23T14:18:04.379 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-23T14:18:04.379 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-23T14:18:04.379 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-23T14:18:04.379 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-23T14:18:04.379 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-23T14:18:04.379 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 38865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-23T14:18:04.379 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 36305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-23T14:18:04.379 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 38073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-23T14:18:04.380 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-23T14:18:04.380 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-23T14:18:04.380 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-23T14:18:04.380 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-23T14:18:04.380 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-23T14:18:04.380 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-23T14:18:04.380 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-23T14:18:04.380 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-23T14:18:04.380 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 42305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-23T14:18:04.380 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 43609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-23T14:18:04.380 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-23T14:18:04.380 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 62129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-23T14:18:04.380 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-23T14:18:04.381 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 30017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-23T14:18:04.381 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-23T14:18:04.381 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-23T14:18:04.381 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-23T14:18:04.381 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-23T14:18:04.381 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-23T14:18:04.381 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-23T14:18:04.381 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 81113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-23T14:18:04.381 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-23T14:18:04.381 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 184745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-23T14:18:04.381 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-23T14:18:04.381 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-23T14:18:04.381 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-23T14:18:04.381 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 319921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid.ko 2024-04-23T14:18:04.382 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 39529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-23T14:18:04.382 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/ 2024-04-23T14:18:04.382 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 153633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbhid.ko 2024-04-23T14:18:04.382 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbkbd.ko 2024-04-23T14:18:04.382 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbmouse.ko 2024-04-23T14:18:04.382 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 261945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-23T14:18:04.382 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/ 2024-04-23T14:18:04.382 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 75065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-23T14:18:04.382 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 110681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-23T14:18:04.382 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 382033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-23T14:18:04.382 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ 2024-04-23T14:18:04.382 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 77153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-23T14:18:04.382 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 67009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-23T14:18:04.382 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-23T14:18:04.383 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-23T14:18:04.383 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-23T14:18:04.383 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-23T14:18:04.383 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 38097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-23T14:18:04.383 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 115057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-23T14:18:04.383 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-23T14:18:04.383 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 55225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-23T14:18:04.383 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 31441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-23T14:18:04.383 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-23T14:18:04.383 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-23T14:18:04.383 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-23T14:18:04.383 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 89025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-23T14:18:04.383 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 49153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-23T14:18:04.384 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 81473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-23T14:18:04.384 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-23T14:18:04.384 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 48177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-23T14:18:04.384 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 68865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-23T14:18:04.384 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 60193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-23T14:18:04.384 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 69009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-23T14:18:04.384 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 47977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-23T14:18:04.384 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-23T14:18:04.384 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-23T14:18:04.384 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9052-hwmon.ko 2024-04-23T14:18:04.384 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9055-hwmon.ko 2024-04-23T14:18:04.384 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-23T14:18:04.384 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 113201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-23T14:18:04.384 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-23T14:18:04.385 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-23T14:18:04.385 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-23T14:18:04.385 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 40177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc2103.ko 2024-04-23T14:18:04.385 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 34649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-23T14:18:04.385 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 78921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-23T14:18:04.385 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 108961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-23T14:18:04.385 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 50697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-23T14:18:04.385 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-23T14:18:04.385 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 65769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-23T14:18:04.385 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-23T14:18:04.385 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 30585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-23T14:18:04.385 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 59761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-23T14:18:04.385 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 56313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-23T14:18:04.385 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hih6130.ko 2024-04-23T14:18:04.386 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-23T14:18:04.386 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-23T14:18:04.386 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 27633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-23T14:18:04.386 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 42585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-23T14:18:04.386 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-23T14:18:04.386 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-23T14:18:04.386 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-23T14:18:04.386 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 156465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-23T14:18:04.386 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/jc42.ko 2024-04-23T14:18:04.386 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-23T14:18:04.386 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-23T14:18:04.386 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-23T14:18:04.386 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 57769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-23T14:18:04.386 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-23T14:18:04.387 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 44121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-23T14:18:04.387 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-23T14:18:04.387 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 41473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-23T14:18:04.387 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 37049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-23T14:18:04.387 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-23T14:18:04.387 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 86497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-23T14:18:04.387 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 55641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-23T14:18:04.387 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-23T14:18:04.387 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-23T14:18:04.387 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 131505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-23T14:18:04.387 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 40225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-23T14:18:04.387 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-23T14:18:04.387 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-23T14:18:04.387 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-23T14:18:04.388 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-23T14:18:04.388 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-23T14:18:04.388 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-23T14:18:04.388 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-23T14:18:04.388 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-23T14:18:04.388 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-23T14:18:04.388 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 36769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-23T14:18:04.388 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-23T14:18:04.388 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-23T14:18:04.388 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-23T14:18:04.388 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-23T14:18:04.388 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-23T14:18:04.388 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 39729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-23T14:18:04.388 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mc13783-adc.ko 2024-04-23T14:18:04.389 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-23T14:18:04.389 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-04-23T14:18:04.389 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 56505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-23T14:18:04.389 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 141353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-23T14:18:04.389 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 72737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-23T14:18:04.389 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 67841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-23T14:18:04.389 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 97777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-23T14:18:04.389 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 61921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-23T14:18:04.389 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-23T14:18:04.389 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ 2024-04-23T14:18:04.389 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-23T14:18:04.389 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-23T14:18:04.389 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-23T14:18:04.389 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-23T14:18:04.390 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-23T14:18:04.390 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-23T14:18:04.390 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-23T14:18:04.390 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 101233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-23T14:18:04.390 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-23T14:18:04.390 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 36177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-23T14:18:04.390 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-23T14:18:04.390 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-23T14:18:04.390 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-23T14:18:04.390 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-23T14:18:04.390 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 32945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-23T14:18:04.390 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 34561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-23T14:18:04.390 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 40849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-23T14:18:04.391 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-23T14:18:04.391 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-23T14:18:04.391 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 48713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-23T14:18:04.391 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-23T14:18:04.391 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 46473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-23T14:18:04.391 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-23T14:18:04.391 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 31417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-23T14:18:04.391 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-23T14:18:04.391 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-23T14:18:04.391 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-23T14:18:04.391 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-23T14:18:04.391 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-23T14:18:04.391 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 50217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-23T14:18:04.391 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 59665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-23T14:18:04.392 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 62737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-23T14:18:04.392 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 71169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-23T14:18:04.392 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 96577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-23T14:18:04.392 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 85913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-23T14:18:04.392 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 75129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-23T14:18:04.392 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 83425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-23T14:18:04.392 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 116777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-23T14:18:04.392 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 134441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-23T14:18:04.392 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-23T14:18:04.392 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 43385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-23T14:18:04.392 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-04-23T14:18:04.392 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-04-23T14:18:04.392 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/ 2024-04-23T14:18:04.392 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/ 2024-04-23T14:18:04.393 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 27353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-23T14:18:04.393 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-23T14:18:04.393 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/ 2024-04-23T14:18:04.393 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-04-23T14:18:04.393 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-04-23T14:18:04.393 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 32905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-04-23T14:18:04.393 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-23T14:18:04.393 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 30361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-23T14:18:04.393 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-23T14:18:04.393 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-04-23T14:18:04.393 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 7729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-23T14:18:04.393 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-04-23T14:18:04.393 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 64233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-23T14:18:04.393 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-23T14:18:04.393 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-23T14:18:04.394 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-23T14:18:04.394 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-04-23T14:18:04.394 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-04-23T14:18:04.394 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 76969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-23T14:18:04.394 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-23T14:18:04.394 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 52937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-23T14:18:04.394 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-04-23T14:18:04.394 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-23T14:18:04.394 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-23T14:18:04.394 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-04-23T14:18:04.394 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-23T14:18:04.394 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-23T14:18:04.394 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 57217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-23T14:18:04.395 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-04-23T14:18:04.395 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-23T14:18:04.395 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-23T14:18:04.395 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-04-23T14:18:04.395 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 32585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-04-23T14:18:04.395 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-23T14:18:04.395 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-04-23T14:18:04.395 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-23T14:18:04.395 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-23T14:18:04.395 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-23T14:18:04.395 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-23T14:18:04.395 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 59193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-04-23T14:18:04.395 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 27849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-23T14:18:04.395 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-23T14:18:04.396 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-23T14:18:04.396 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/ 2024-04-23T14:18:04.396 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-04-23T14:18:04.396 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-04-23T14:18:04.396 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 36569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-04-23T14:18:04.396 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ 2024-04-23T14:18:04.396 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ 2024-04-23T14:18:04.396 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 329697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-23T14:18:04.396 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 1119913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-23T14:18:04.396 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 83905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-23T14:18:04.396 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 408433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-23T14:18:04.396 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 120921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-23T14:18:04.396 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 347217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-23T14:18:04.396 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 84945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-23T14:18:04.397 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ 2024-04-23T14:18:04.397 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/ 2024-04-23T14:18:04.397 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 590929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-23T14:18:04.397 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/ 2024-04-23T14:18:04.397 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 514641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-23T14:18:04.397 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/ 2024-04-23T14:18:04.397 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 911665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-23T14:18:04.397 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ 2024-04-23T14:18:04.397 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 375545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-23T14:18:04.397 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ 2024-04-23T14:18:04.397 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 197121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-04-23T14:18:04.397 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/ 2024-04-23T14:18:04.397 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 212553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-04-23T14:18:04.398 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ 2024-04-23T14:18:04.398 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ 2024-04-23T14:18:04.398 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 334401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-23T14:18:04.398 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ 2024-04-23T14:18:04.398 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 120673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-23T14:18:04.398 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ 2024-04-23T14:18:04.398 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 132585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-23T14:18:04.398 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ 2024-04-23T14:18:04.398 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 157937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-23T14:18:04.398 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ 2024-04-23T14:18:04.398 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 156673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-23T14:18:04.398 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ 2024-04-23T14:18:04.398 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/evbug.ko 2024-04-23T14:18:04.398 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 27609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ff-memless.ko 2024-04-23T14:18:04.399 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/ 2024-04-23T14:18:04.399 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-23T14:18:04.399 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-23T14:18:04.399 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 50713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-23T14:18:04.399 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 49441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-23T14:18:04.399 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/ 2024-04-23T14:18:04.399 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-04-23T14:18:04.399 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-04-23T14:18:04.399 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-04-23T14:18:04.399 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-04-23T14:18:04.399 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 45473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-23T14:18:04.399 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-04-23T14:18:04.399 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lkkbd.ko 2024-04-23T14:18:04.399 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 31857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8323.ko 2024-04-23T14:18:04.400 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8333.ko 2024-04-23T14:18:04.400 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-04-23T14:18:04.400 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-04-23T14:18:04.400 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-04-23T14:18:04.400 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-04-23T14:18:04.400 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/newtonkbd.ko 2024-04-23T14:18:04.400 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-04-23T14:18:04.400 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt1070.ko 2024-04-23T14:18:04.400 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt2160.ko 2024-04-23T14:18:04.400 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-04-23T14:18:04.400 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/stowaway.ko 2024-04-23T14:18:04.400 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/sunkbd.ko 2024-04-23T14:18:04.400 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-04-23T14:18:04.400 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-04-23T14:18:04.401 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-04-23T14:18:04.401 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/xtkbd.ko 2024-04-23T14:18:04.401 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/matrix-keymap.ko 2024-04-23T14:18:04.401 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/ 2024-04-23T14:18:04.401 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 40617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-23T14:18:04.401 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 39553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-23T14:18:04.401 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 160017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-23T14:18:04.401 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 114273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-23T14:18:04.401 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/gpio_mouse.ko 2024-04-23T14:18:04.401 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 428185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/psmouse.ko 2024-04-23T14:18:04.401 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-23T14:18:04.401 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 27865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-23T14:18:04.401 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 27193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-23T14:18:04.401 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-23T14:18:04.401 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/ 2024-04-23T14:18:04.402 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 140969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-23T14:18:04.402 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ 2024-04-23T14:18:04.402 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-23T14:18:04.402 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-23T14:18:04.402 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ct82c710.ko 2024-04-23T14:18:04.402 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-23T14:18:04.402 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/parkbd.ko 2024-04-23T14:18:04.402 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/pcips2.ko 2024-04-23T14:18:04.402 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ps2mult.ko 2024-04-23T14:18:04.402 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-23T14:18:04.402 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serport.ko 2024-04-23T14:18:04.402 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-23T14:18:04.402 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/ 2024-04-23T14:18:04.402 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-88pm860x.ko 2024-04-23T14:18:04.403 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-adp5520.ko 2024-04-23T14:18:04.403 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 77265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-bd2802.ko 2024-04-23T14:18:04.403 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 31097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-23T14:18:04.403 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da903x.ko 2024-04-23T14:18:04.403 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da9052.ko 2024-04-23T14:18:04.403 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-gpio.ko 2024-04-23T14:18:04.403 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-23T14:18:04.403 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 36705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3533.ko 2024-04-23T14:18:04.403 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm355x.ko 2024-04-23T14:18:04.403 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3642.ko 2024-04-23T14:18:04.403 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-23T14:18:04.403 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp8788.ko 2024-04-23T14:18:04.403 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-max8997.ko 2024-04-23T14:18:04.403 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-mc13783.ko 2024-04-23T14:18:04.404 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-menf21bmc.ko 2024-04-23T14:18:04.404 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca9532.ko 2024-04-23T14:18:04.404 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca955x.ko 2024-04-23T14:18:04.404 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca963x.ko 2024-04-23T14:18:04.404 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 38113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-23T14:18:04.404 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-tca6507.ko 2024-04-23T14:18:04.404 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm831x-status.ko 2024-04-23T14:18:04.404 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm8350.ko 2024-04-23T14:18:04.404 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ 2024-04-23T14:18:04.404 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-23T14:18:04.404 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-23T14:18:04.404 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-23T14:18:04.404 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 8641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-23T14:18:04.404 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-23T14:18:04.404 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-23T14:18:04.405 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-23T14:18:04.405 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-23T14:18:04.405 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-23T14:18:04.405 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/ 2024-04-23T14:18:04.405 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/ 2024-04-23T14:18:04.405 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 790753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-23T14:18:04.405 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-23T14:18:04.405 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 116825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bufio.ko 2024-04-23T14:18:04.405 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 58281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-23T14:18:04.405 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 199041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-23T14:18:04.405 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 134945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-23T14:18:04.405 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-23T14:18:04.405 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 67497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-era.ko 2024-04-23T14:18:04.405 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-23T14:18:04.406 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 50457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-23T14:18:04.406 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 41145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log.ko 2024-04-23T14:18:04.406 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 60977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-mirror.ko 2024-04-23T14:18:04.406 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 97209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-23T14:18:04.406 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-23T14:18:04.406 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 85689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-23T14:18:04.406 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-region-hash.ko 2024-04-23T14:18:04.406 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-23T14:18:04.406 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-23T14:18:04.406 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-snapshot.ko 2024-04-23T14:18:04.406 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-23T14:18:04.406 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 234457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-23T14:18:04.406 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 66425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-23T14:18:04.406 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-zero.ko 2024-04-23T14:18:04.407 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/ 2024-04-23T14:18:04.407 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 264329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-23T14:18:04.407 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 43137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-23T14:18:04.407 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 125009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-23T14:18:04.407 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 157681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-23T14:18:04.407 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 444665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-23T14:18:04.407 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/ 2024-04-23T14:18:04.407 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/ 2024-04-23T14:18:04.407 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 214841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-23T14:18:04.407 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 77985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-23T14:18:04.407 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 54785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-23T14:18:04.407 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 44633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-23T14:18:04.407 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 150761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-23T14:18:04.407 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 97281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-23T14:18:04.408 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 63873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-23T14:18:04.408 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/ 2024-04-23T14:18:04.408 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm800.ko 2024-04-23T14:18:04.408 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm805.ko 2024-04-23T14:18:04.408 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm80x.ko 2024-04-23T14:18:04.408 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona-i2c.ko 2024-04-23T14:18:04.408 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 148633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona.ko 2024-04-23T14:18:04.408 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/bcm590xx.ko 2024-04-23T14:18:04.408 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/cros_ec_dev.ko 2024-04-23T14:18:04.408 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 41009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/dln2.ko 2024-04-23T14:18:04.408 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/janz-cmodio.ko 2024-04-23T14:18:04.408 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 59089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/kempld-core.ko 2024-04-23T14:18:04.408 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 31897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-core.ko 2024-04-23T14:18:04.408 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-04-23T14:18:04.409 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lp3943.ko 2024-04-23T14:18:04.409 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 60497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-23T14:18:04.409 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-23T14:18:04.409 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/max8907.ko 2024-04-23T14:18:04.409 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-core.ko 2024-04-23T14:18:04.409 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-04-23T14:18:04.409 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/menf21bmc.ko 2024-04-23T14:18:04.409 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-adc.ko 2024-04-23T14:18:04.409 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-gpio.ko 2024-04-23T14:18:04.409 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633.ko 2024-04-23T14:18:04.409 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-04-23T14:18:04.409 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/retu-mfd.ko 2024-04-23T14:18:04.409 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 99425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/si476x-core.ko 2024-04-23T14:18:04.409 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 58057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-23T14:18:04.410 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 60681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps65010.ko 2024-04-23T14:18:04.410 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps6507x.ko 2024-04-23T14:18:04.410 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-23T14:18:04.410 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-23T14:18:04.410 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-23T14:18:04.410 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 52361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wm8994.ko 2024-04-23T14:18:04.410 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ 2024-04-23T14:18:04.410 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-04-23T14:18:04.410 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 64265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot.ko 2024-04-23T14:18:04.410 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/ 2024-04-23T14:18:04.410 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 57521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-23T14:18:04.410 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-23T14:18:04.410 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-23T14:18:04.410 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 58633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-23T14:18:04.411 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/ 2024-04-23T14:18:04.411 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-04-23T14:18:04.411 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 47449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/core.ko 2024-04-23T14:18:04.411 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/ 2024-04-23T14:18:04.411 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-23T14:18:04.411 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ds1682.ko 2024-04-23T14:18:04.411 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/dummy-irq.ko 2024-04-23T14:18:04.411 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/ 2024-04-23T14:18:04.411 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/echo.ko 2024-04-23T14:18:04.411 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/ 2024-04-23T14:18:04.411 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 43041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-23T14:18:04.411 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-23T14:18:04.411 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-23T14:18:04.411 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 41185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-23T14:18:04.412 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/ 2024-04-23T14:18:04.412 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 191649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-04-23T14:18:04.412 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hmc6352.ko 2024-04-23T14:18:04.412 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 37761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-23T14:18:04.412 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ 2024-04-23T14:18:04.412 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 92345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-04-23T14:18:04.412 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ics932s401.ko 2024-04-23T14:18:04.412 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-23T14:18:04.412 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-23T14:18:04.412 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/ 2024-04-23T14:18:04.412 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 54865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-23T14:18:04.412 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-23T14:18:04.412 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/ 2024-04-23T14:18:04.412 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 154169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-23T14:18:04.412 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 69809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-23T14:18:04.413 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 389033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-23T14:18:04.413 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 37961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/phantom.ko 2024-04-23T14:18:04.413 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/ 2024-04-23T14:18:04.413 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 91145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/st_drv.ko 2024-04-23T14:18:04.413 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-23T14:18:04.413 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 34457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-23T14:18:04.413 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-23T14:18:04.413 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 55793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-23T14:18:04.413 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/ 2024-04-23T14:18:04.413 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 234905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-23T14:18:04.413 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ 2024-04-23T14:18:04.413 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/ 2024-04-23T14:18:04.413 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 620961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-23T14:18:04.413 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/ 2024-04-23T14:18:04.413 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/ 2024-04-23T14:18:04.414 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-23T14:18:04.414 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 49545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-23T14:18:04.414 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-23T14:18:04.414 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/ 2024-04-23T14:18:04.414 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 587065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-23T14:18:04.414 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-23T14:18:04.414 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-23T14:18:04.414 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ 2024-04-23T14:18:04.414 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/ 2024-04-23T14:18:04.414 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 53753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-23T14:18:04.414 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 49201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-23T14:18:04.414 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 158649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-23T14:18:04.414 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 85953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-23T14:18:04.414 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ 2024-04-23T14:18:04.415 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 44161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-23T14:18:04.415 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-23T14:18:04.415 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-23T14:18:04.415 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 109521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-23T14:18:04.415 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/ 2024-04-23T14:18:04.415 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 64705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-23T14:18:04.415 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/ 2024-04-23T14:18:04.415 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 84329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-23T14:18:04.415 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/ 2024-04-23T14:18:04.415 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 70737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-23T14:18:04.415 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/ 2024-04-23T14:18:04.415 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 52705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-23T14:18:04.415 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/ 2024-04-23T14:18:04.416 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 65857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-23T14:18:04.416 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 56057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-23T14:18:04.416 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 115881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-23T14:18:04.416 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/ 2024-04-23T14:18:04.416 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/ 2024-04-23T14:18:04.416 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 124497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-23T14:18:04.416 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-04-23T14:18:04.416 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 128409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-23T14:18:04.416 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-04-23T14:18:04.416 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 110345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-23T14:18:04.416 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/ 2024-04-23T14:18:04.416 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 101857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-23T14:18:04.416 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 88953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-23T14:18:04.417 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/ 2024-04-23T14:18:04.417 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 105633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-23T14:18:04.417 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 228985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-23T14:18:04.417 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-04-23T14:18:04.417 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 1733353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-23T14:18:04.417 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 167185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-23T14:18:04.417 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/ 2024-04-23T14:18:04.417 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 161417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-23T14:18:04.417 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 440625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-23T14:18:04.417 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/ 2024-04-23T14:18:04.417 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/ 2024-04-23T14:18:04.417 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 432201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-23T14:18:04.417 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/ 2024-04-23T14:18:04.417 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-04-23T14:18:04.418 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 257201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-23T14:18:04.418 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-04-23T14:18:04.418 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 446177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-23T14:18:04.418 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-04-23T14:18:04.418 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 1311433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-23T14:18:04.457 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-04-23T14:18:04.457 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 206689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-23T14:18:04.457 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-04-23T14:18:04.457 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 40161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-23T14:18:04.457 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/ 2024-04-23T14:18:04.457 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/ 2024-04-23T14:18:04.457 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 269313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-23T14:18:04.458 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/ 2024-04-23T14:18:04.458 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/ 2024-04-23T14:18:04.458 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 85385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-23T14:18:04.458 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 77457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-23T14:18:04.458 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 169089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-23T14:18:04.458 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-23T14:18:04.458 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 76209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-23T14:18:04.458 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 38065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-23T14:18:04.458 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/ 2024-04-23T14:18:04.458 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 75945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-23T14:18:04.459 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 78705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-23T14:18:04.459 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-23T14:18:04.459 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 30409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ec_bhf.ko 2024-04-23T14:18:04.459 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/ 2024-04-23T14:18:04.459 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/ 2024-04-23T14:18:04.459 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 356377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-23T14:18:04.459 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 61737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-23T14:18:04.459 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 63185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-23T14:18:04.459 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/ 2024-04-23T14:18:04.459 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 51513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-04-23T14:18:04.459 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ 2024-04-23T14:18:04.460 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 115569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-23T14:18:04.460 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/ 2024-04-23T14:18:04.460 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 366217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-23T14:18:04.460 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/ 2024-04-23T14:18:04.460 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 755793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-23T14:18:04.460 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/ 2024-04-23T14:18:04.460 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 324577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-23T14:18:04.460 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/ 2024-04-23T14:18:04.460 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 1196297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-23T14:18:04.460 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/ 2024-04-23T14:18:04.460 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 362593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-23T14:18:04.460 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/ 2024-04-23T14:18:04.461 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 671801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-23T14:18:04.461 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/ 2024-04-23T14:18:04.461 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 121537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-23T14:18:04.461 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-04-23T14:18:04.461 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 966705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-23T14:18:04.461 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-04-23T14:18:04.461 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 193585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-23T14:18:04.461 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 108633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-23T14:18:04.461 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/ 2024-04-23T14:18:04.461 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-23T14:18:04.461 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 126881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-23T14:18:04.461 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 150777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-23T14:18:04.462 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/ 2024-04-23T14:18:04.462 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-04-23T14:18:04.462 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 905033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-23T14:18:04.462 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 344145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-23T14:18:04.462 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-04-23T14:18:04.462 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-04-23T14:18:04.462 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 1412417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-23T14:18:04.462 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ 2024-04-23T14:18:04.462 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-04-23T14:18:04.462 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 54497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-04-23T14:18:04.463 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-04-23T14:18:04.463 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 117353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-23T14:18:04.463 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/ 2024-04-23T14:18:04.463 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-04-23T14:18:04.463 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 131425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-23T14:18:04.463 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ 2024-04-23T14:18:04.463 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 91465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-23T14:18:04.463 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 56689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-23T14:18:04.463 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/ 2024-04-23T14:18:04.463 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 181081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-23T14:18:04.463 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/ 2024-04-23T14:18:04.463 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 171417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-23T14:18:04.463 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/ 2024-04-23T14:18:04.463 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 58249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-23T14:18:04.464 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 50121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-23T14:18:04.464 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/ 2024-04-23T14:18:04.464 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-04-23T14:18:04.464 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 300817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-23T14:18:04.464 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 105689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-23T14:18:04.464 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-04-23T14:18:04.464 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 885129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-23T14:18:04.464 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/ 2024-04-23T14:18:04.464 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 52305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-23T14:18:04.464 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/ 2024-04-23T14:18:04.464 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 81641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-23T14:18:04.464 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 100689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-23T14:18:04.464 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 34073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-23T14:18:04.464 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 264161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-23T14:18:04.465 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/ 2024-04-23T14:18:04.465 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-04-23T14:18:04.465 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 142449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-04-23T14:18:04.465 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/ 2024-04-23T14:18:04.465 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 1350385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-23T14:18:04.465 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/ 2024-04-23T14:18:04.465 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 57953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-23T14:18:04.465 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/ 2024-04-23T14:18:04.465 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 66505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-23T14:18:04.465 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 80065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-23T14:18:04.465 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/ 2024-04-23T14:18:04.465 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 74289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-23T14:18:04.465 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 79457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-23T14:18:04.465 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 100785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-23T14:18:04.466 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 83441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-23T14:18:04.466 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/ 2024-04-23T14:18:04.466 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-04-23T14:18:04.466 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-04-23T14:18:04.466 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 46985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-23T14:18:04.466 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-04-23T14:18:04.466 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 644537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-23T14:18:04.466 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/ 2024-04-23T14:18:04.466 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 128497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-23T14:18:04.466 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 207993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-23T14:18:04.466 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 82633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-23T14:18:04.466 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 91353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-23T14:18:04.466 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/ 2024-04-23T14:18:04.466 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 59865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-23T14:18:04.467 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/ 2024-04-23T14:18:04.467 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 110729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-23T14:18:04.467 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/ 2024-04-23T14:18:04.467 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 98689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-23T14:18:04.467 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 102513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-23T14:18:04.467 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/ 2024-04-23T14:18:04.467 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 61497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-23T14:18:04.467 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 40609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-23T14:18:04.467 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/ 2024-04-23T14:18:04.467 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 69913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-23T14:18:04.467 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/ 2024-04-23T14:18:04.467 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 49449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/defxx.ko 2024-04-23T14:18:04.467 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/ 2024-04-23T14:18:04.467 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 275801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/skfp.ko 2024-04-23T14:18:04.468 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 91105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-23T14:18:04.468 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/ 2024-04-23T14:18:04.468 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 290441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-23T14:18:04.468 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/ 2024-04-23T14:18:04.468 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-23T14:18:04.468 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 30001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-23T14:18:04.468 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ 2024-04-23T14:18:04.468 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 108329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-23T14:18:04.468 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 78065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-23T14:18:04.468 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-23T14:18:04.468 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/ 2024-04-23T14:18:04.468 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-23T14:18:04.468 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-23T14:18:04.468 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-gpio.ko 2024-04-23T14:18:04.469 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-23T14:18:04.469 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-23T14:18:04.469 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-23T14:18:04.469 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 62681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-23T14:18:04.469 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-23T14:18:04.469 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/ 2024-04-23T14:18:04.469 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-23T14:18:04.469 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 48313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-23T14:18:04.469 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ 2024-04-23T14:18:04.469 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-23T14:18:04.469 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 61985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-23T14:18:04.469 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 52865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-23T14:18:04.469 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-23T14:18:04.469 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 53409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-23T14:18:04.470 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-23T14:18:04.470 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-23T14:18:04.470 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-23T14:18:04.470 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 31313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-23T14:18:04.470 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-23T14:18:04.470 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 103377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-23T14:18:04.470 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 96025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-23T14:18:04.470 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-23T14:18:04.470 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 121105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-23T14:18:04.470 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-23T14:18:04.470 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 62113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-23T14:18:04.470 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-23T14:18:04.470 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-23T14:18:04.470 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-23T14:18:04.470 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/ 2024-04-23T14:18:04.471 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 55353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/plip.ko 2024-04-23T14:18:04.471 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ 2024-04-23T14:18:04.471 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-23T14:18:04.471 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 36273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-23T14:18:04.471 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-23T14:18:04.471 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-23T14:18:04.471 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-23T14:18:04.471 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 53233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-23T14:18:04.471 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-23T14:18:04.471 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-23T14:18:04.471 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sb1000.ko 2024-04-23T14:18:04.471 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/ 2024-04-23T14:18:04.471 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 52073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-23T14:18:04.471 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 42409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-23T14:18:04.472 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 80161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-23T14:18:04.472 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/ 2024-04-23T14:18:04.472 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 137321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-23T14:18:04.472 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-23T14:18:04.472 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-23T14:18:04.472 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 36209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-23T14:18:04.472 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-23T14:18:04.472 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-23T14:18:04.472 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 108089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-23T14:18:04.472 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/ 2024-04-23T14:18:04.472 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 176497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-23T14:18:04.472 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/ 2024-04-23T14:18:04.472 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 383593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-23T14:18:04.472 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/ 2024-04-23T14:18:04.473 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 175537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-23T14:18:04.473 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ 2024-04-23T14:18:04.473 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-23T14:18:04.473 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/ 2024-04-23T14:18:04.473 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/ 2024-04-23T14:18:04.473 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 335857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-23T14:18:04.473 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 126553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-23T14:18:04.473 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/ 2024-04-23T14:18:04.473 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 168145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-23T14:18:04.473 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-23T14:18:04.473 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 103889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-23T14:18:04.473 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 30313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-23T14:18:04.473 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/ 2024-04-23T14:18:04.473 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/ 2024-04-23T14:18:04.473 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-23T14:18:04.474 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-04-23T14:18:04.474 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-04-23T14:18:04.474 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pci-stub.ko 2024-04-23T14:18:04.474 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 55329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-23T14:18:04.474 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/ 2024-04-23T14:18:04.474 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 31529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-23T14:18:04.474 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 179961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia.ko 2024-04-23T14:18:04.474 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 78161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_core.ko 2024-04-23T14:18:04.474 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 41945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-04-23T14:18:04.474 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 40457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-23T14:18:04.474 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 133753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-23T14:18:04.474 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/ 2024-04-23T14:18:04.474 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/ 2024-04-23T14:18:04.474 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-04-23T14:18:04.475 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/ 2024-04-23T14:18:04.475 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-04-23T14:18:04.475 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/ 2024-04-23T14:18:04.475 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/ 2024-04-23T14:18:04.475 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 78193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-23T14:18:04.475 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/ 2024-04-23T14:18:04.475 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/ 2024-04-23T14:18:04.475 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 61041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-04-23T14:18:04.475 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec.ko 2024-04-23T14:18:04.475 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-04-23T14:18:04.475 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 30217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-04-23T14:18:04.475 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-04-23T14:18:04.475 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-04-23T14:18:04.475 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-04-23T14:18:04.476 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ 2024-04-23T14:18:04.476 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 87481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-23T14:18:04.476 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 44105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-23T14:18:04.476 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 48257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-23T14:18:04.476 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-23T14:18:04.476 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-23T14:18:04.476 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 164769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-23T14:18:04.476 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 46185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-23T14:18:04.476 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 62073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-23T14:18:04.476 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-23T14:18:04.476 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 51881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-23T14:18:04.476 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-23T14:18:04.476 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 42761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-23T14:18:04.476 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ibm_rtl.ko 2024-04-23T14:18:04.476 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/ 2024-04-23T14:18:04.477 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-23T14:18:04.477 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-04-23T14:18:04.477 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 54001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-23T14:18:04.477 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-23T14:18:04.477 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-23T14:18:04.477 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 43753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-23T14:18:04.477 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-23T14:18:04.477 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-23T14:18:04.477 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-23T14:18:04.477 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-23T14:18:04.477 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-23T14:18:04.477 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-23T14:18:04.477 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 61417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-23T14:18:04.477 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/ 2024-04-23T14:18:04.478 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ 2024-04-23T14:18:04.478 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 36513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_battery.ko 2024-04-23T14:18:04.478 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 40425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_charger.ko 2024-04-23T14:18:04.478 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 68833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq2415x_charger.ko 2024-04-23T14:18:04.478 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 73073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24190_charger.ko 2024-04-23T14:18:04.478 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24735-charger.ko 2024-04-23T14:18:04.478 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-04-23T14:18:04.478 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9030_battery.ko 2024-04-23T14:18:04.478 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9052-battery.ko 2024-04-23T14:18:04.478 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 34489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2760_battery.ko 2024-04-23T14:18:04.478 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2780_battery.ko 2024-04-23T14:18:04.478 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2781_battery.ko 2024-04-23T14:18:04.478 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2782_battery.ko 2024-04-23T14:18:04.478 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/gpio-charger.ko 2024-04-23T14:18:04.479 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/isp1704_charger.ko 2024-04-23T14:18:04.479 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/lp8727_charger.ko 2024-04-23T14:18:04.479 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max14577_charger.ko 2024-04-23T14:18:04.479 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 37833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max17042_battery.ko 2024-04-23T14:18:04.479 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8903_charger.ko 2024-04-23T14:18:04.479 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 27689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8925_power.ko 2024-04-23T14:18:04.479 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/pcf50633-charger.ko 2024-04-23T14:18:04.479 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 44841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/sbs-battery.ko 2024-04-23T14:18:04.479 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 34905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/test_power.ko 2024-04-23T14:18:04.479 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/tps65090-charger.ko 2024-04-23T14:18:04.479 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_backup.ko 2024-04-23T14:18:04.479 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_power.ko 2024-04-23T14:18:04.479 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 27441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm8350_power.ko 2024-04-23T14:18:04.479 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/ 2024-04-23T14:18:04.480 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 73297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-23T14:18:04.480 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-23T14:18:04.480 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/ 2024-04-23T14:18:04.480 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/ 2024-04-23T14:18:04.480 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-23T14:18:04.480 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-23T14:18:04.480 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-23T14:18:04.480 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 49817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-23T14:18:04.480 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ 2024-04-23T14:18:04.480 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 91393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-23T14:18:04.480 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-23T14:18:04.480 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/ 2024-04-23T14:18:04.480 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/ 2024-04-23T14:18:04.480 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/ 2024-04-23T14:18:04.481 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 67513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-23T14:18:04.481 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 47137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-23T14:18:04.481 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/ 2024-04-23T14:18:04.481 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm80x.ko 2024-04-23T14:18:04.481 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm860x.ko 2024-04-23T14:18:04.481 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-23T14:18:04.481 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9052.ko 2024-04-23T14:18:04.481 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9055.ko 2024-04-23T14:18:04.481 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9063.ko 2024-04-23T14:18:04.481 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-23T14:18:04.481 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 82225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-23T14:18:04.481 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-23T14:18:04.481 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-23T14:18:04.481 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-23T14:18:04.482 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-23T14:18:04.482 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-23T14:18:04.482 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-23T14:18:04.482 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-23T14:18:04.482 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-23T14:18:04.482 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-23T14:18:04.482 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-23T14:18:04.482 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 44129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-23T14:18:04.482 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-lp8788.ko 2024-04-23T14:18:04.482 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 46913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-23T14:18:04.482 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-23T14:18:04.482 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-23T14:18:04.482 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t86.ko 2024-04-23T14:18:04.482 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-23T14:18:04.483 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8907.ko 2024-04-23T14:18:04.483 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8925.ko 2024-04-23T14:18:04.483 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 31337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8997.ko 2024-04-23T14:18:04.483 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8998.ko 2024-04-23T14:18:04.483 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-04-23T14:18:04.483 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-23T14:18:04.483 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-palmas.ko 2024-04-23T14:18:04.483 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 56777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-23T14:18:04.483 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf50633.ko 2024-04-23T14:18:04.483 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 32657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-23T14:18:04.483 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-23T14:18:04.483 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 36065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-23T14:18:04.483 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-23T14:18:04.483 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rc5t583.ko 2024-04-23T14:18:04.483 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-23T14:18:04.484 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-23T14:18:04.484 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 34529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-23T14:18:04.484 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-23T14:18:04.484 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-23T14:18:04.484 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-s35390a.ko 2024-04-23T14:18:04.484 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-23T14:18:04.484 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps6586x.ko 2024-04-23T14:18:04.484 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps65910.ko 2024-04-23T14:18:04.484 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm831x.ko 2024-04-23T14:18:04.484 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm8350.ko 2024-04-23T14:18:04.484 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-23T14:18:04.484 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ 2024-04-23T14:18:04.484 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 90377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-23T14:18:04.484 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 69593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-23T14:18:04.485 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 65945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-23T14:18:04.485 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 164569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-23T14:18:04.485 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 31617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-23T14:18:04.485 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/ 2024-04-23T14:18:04.485 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 340721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-23T14:18:04.485 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 120337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-23T14:18:04.485 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/ 2024-04-23T14:18:04.485 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 468081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-23T14:18:04.485 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 380673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-23T14:18:04.485 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/ 2024-04-23T14:18:04.485 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 243705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-04-23T14:18:04.485 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-23T14:18:04.485 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/ 2024-04-23T14:18:04.485 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 135617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-23T14:18:04.485 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-23T14:18:04.486 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/ 2024-04-23T14:18:04.486 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 304993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-23T14:18:04.486 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/ 2024-04-23T14:18:04.486 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 1527113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-23T14:18:04.486 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/ 2024-04-23T14:18:04.486 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 297801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-23T14:18:04.486 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/ 2024-04-23T14:18:04.486 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 144401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-23T14:18:04.486 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 47457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-23T14:18:04.486 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/ 2024-04-23T14:18:04.486 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 390713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-23T14:18:04.486 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/ 2024-04-23T14:18:04.486 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-04-23T14:18:04.486 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 88881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-23T14:18:04.487 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-04-23T14:18:04.487 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 117481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-23T14:18:04.487 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 173017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-23T14:18:04.487 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 91569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-23T14:18:04.487 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 45401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-23T14:18:04.487 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/ 2024-04-23T14:18:04.487 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 267577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-23T14:18:04.487 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-23T14:18:04.487 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/ 2024-04-23T14:18:04.487 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 95329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-23T14:18:04.487 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 194017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-23T14:18:04.487 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fdomain.ko 2024-04-23T14:18:04.488 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/ 2024-04-23T14:18:04.488 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 366529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-23T14:18:04.488 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 251505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-23T14:18:04.488 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 65761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-23T14:18:04.488 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 57097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-23T14:18:04.488 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 44689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/imm.ko 2024-04-23T14:18:04.488 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 55153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-23T14:18:04.488 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 327841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-23T14:18:04.488 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 129785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-23T14:18:04.488 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/ 2024-04-23T14:18:04.488 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 397785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-23T14:18:04.488 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 32017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-23T14:18:04.488 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 63121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-23T14:18:04.488 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/ 2024-04-23T14:18:04.489 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 386849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-23T14:18:04.489 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 178177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-23T14:18:04.489 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 64809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-23T14:18:04.489 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/ 2024-04-23T14:18:04.489 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 274009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-23T14:18:04.489 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/ 2024-04-23T14:18:04.489 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 2736001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-23T14:18:04.489 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/ 2024-04-23T14:18:04.489 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 88529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-23T14:18:04.489 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 40145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-23T14:18:04.489 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 389177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-23T14:18:04.489 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 104769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-23T14:18:04.489 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/ 2024-04-23T14:18:04.489 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 836945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-23T14:18:04.490 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/ 2024-04-23T14:18:04.490 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 158729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-23T14:18:04.490 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 96537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-23T14:18:04.490 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/ 2024-04-23T14:18:04.490 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 101625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-04-23T14:18:04.490 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-04-23T14:18:04.490 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-04-23T14:18:04.490 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-04-23T14:18:04.490 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/ 2024-04-23T14:18:04.490 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 443745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-23T14:18:04.490 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 152033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-23T14:18:04.490 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 41625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ppa.ko 2024-04-23T14:18:04.490 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 66977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-23T14:18:04.491 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/ 2024-04-23T14:18:04.491 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 2324809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-23T14:18:04.491 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 103601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-23T14:18:04.491 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/ 2024-04-23T14:18:04.491 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 697337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-23T14:18:04.491 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qlogicfas408.ko 2024-04-23T14:18:04.491 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-23T14:18:04.491 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 322865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-23T14:18:04.491 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 219857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-23T14:18:04.491 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 357153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-23T14:18:04.491 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 121185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-23T14:18:04.491 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 93481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-23T14:18:04.491 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 55401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-23T14:18:04.492 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-23T14:18:04.492 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 138121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-23T14:18:04.492 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 57401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-23T14:18:04.492 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/ 2024-04-23T14:18:04.492 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 205025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-23T14:18:04.492 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 55425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-23T14:18:04.492 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 59185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-23T14:18:04.492 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 48777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-23T14:18:04.492 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 59417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/xen-scsifront.ko 2024-04-23T14:18:04.492 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ 2024-04-23T14:18:04.492 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 174913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-23T14:18:04.492 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/ 2024-04-23T14:18:04.492 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/ 2024-04-23T14:18:04.492 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 920169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-23T14:18:04.493 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/ 2024-04-23T14:18:04.493 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 60321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-23T14:18:04.493 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 48129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-23T14:18:04.493 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 54233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-23T14:18:04.493 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 1170033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-23T14:18:04.493 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-23T14:18:04.493 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 149721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-23T14:18:04.493 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/ 2024-04-23T14:18:04.493 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 90057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-23T14:18:04.493 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/ 2024-04-23T14:18:04.493 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/ 2024-04-23T14:18:04.493 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/ 2024-04-23T14:18:04.493 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-23T14:18:04.493 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-23T14:18:04.494 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-23T14:18:04.494 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-23T14:18:04.494 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-23T14:18:04.494 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-23T14:18:04.494 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-23T14:18:04.534 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-23T14:18:04.534 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-23T14:18:04.534 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-23T14:18:04.534 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-23T14:18:04.534 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-23T14:18:04.534 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 46065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-23T14:18:04.534 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-23T14:18:04.535 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-23T14:18:04.535 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-04-23T14:18:04.535 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-23T14:18:04.535 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-23T14:18:04.535 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 30577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-23T14:18:04.535 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ 2024-04-23T14:18:04.535 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ 2024-04-23T14:18:04.535 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 99313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-23T14:18:04.535 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 69729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/moxa.ko 2024-04-23T14:18:04.535 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 61649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/mxser.ko 2024-04-23T14:18:04.535 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 134401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-23T14:18:04.535 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 37905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-23T14:18:04.535 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-23T14:18:04.536 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/ 2024-04-23T14:18:04.536 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/ 2024-04-23T14:18:04.536 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 44057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-04-23T14:18:04.536 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 65121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-23T14:18:04.536 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-04-23T14:18:04.536 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_uart.ko 2024-04-23T14:18:04.536 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-23T14:18:04.536 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 91665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-04-23T14:18:04.536 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/ 2024-04-23T14:18:04.536 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 120577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-23T14:18:04.536 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/rp2.ko 2024-04-23T14:18:04.536 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 65641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/sc16is7xx.ko 2024-04-23T14:18:04.536 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/ 2024-04-23T14:18:04.537 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ 2024-04-23T14:18:04.537 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 693081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-23T14:18:04.537 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ 2024-04-23T14:18:04.537 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 34449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-23T14:18:04.537 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-04-23T14:18:04.537 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/ 2024-04-23T14:18:04.537 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 55617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-23T14:18:04.537 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-23T14:18:04.537 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-23T14:18:04.537 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_dmem_genirq.ko 2024-04-23T14:18:04.537 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_mf624.ko 2024-04-23T14:18:04.537 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_netx.ko 2024-04-23T14:18:04.537 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-23T14:18:04.537 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-04-23T14:18:04.538 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-23T14:18:04.538 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/ 2024-04-23T14:18:04.538 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/ 2024-04-23T14:18:04.538 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 90321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/c67x00.ko 2024-04-23T14:18:04.538 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ 2024-04-23T14:18:04.538 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 189209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-04-23T14:18:04.538 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-04-23T14:18:04.538 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-04-23T14:18:04.538 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-04-23T14:18:04.538 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-04-23T14:18:04.538 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/ 2024-04-23T14:18:04.538 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 101545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-23T14:18:04.538 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 63073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-23T14:18:04.538 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 56721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-23T14:18:04.539 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 107417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-23T14:18:04.539 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ 2024-04-23T14:18:04.539 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ulpi.ko 2024-04-23T14:18:04.539 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/ 2024-04-23T14:18:04.539 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-04-23T14:18:04.539 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/ 2024-04-23T14:18:04.539 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-04-23T14:18:04.539 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-04-23T14:18:04.539 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 574593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3.ko 2024-04-23T14:18:04.539 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/ 2024-04-23T14:18:04.539 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/ 2024-04-23T14:18:04.539 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 72249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_ether.ko 2024-04-23T14:18:04.539 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 61953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_serial.ko 2024-04-23T14:18:04.540 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-04-23T14:18:04.540 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 49737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-04-23T14:18:04.540 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-04-23T14:18:04.540 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-04-23T14:18:04.540 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 131345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-04-23T14:18:04.540 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 62321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-04-23T14:18:04.540 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 146617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-04-23T14:18:04.540 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 70689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-04-23T14:18:04.540 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-04-23T14:18:04.540 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 37649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-04-23T14:18:04.540 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 70017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-04-23T14:18:04.540 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 101249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-04-23T14:18:04.540 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-04-23T14:18:04.541 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 71257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-04-23T14:18:04.541 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 84937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-04-23T14:18:04.541 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/ 2024-04-23T14:18:04.541 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-04-23T14:18:04.541 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-04-23T14:18:04.541 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-04-23T14:18:04.541 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-04-23T14:18:04.541 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-04-23T14:18:04.541 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-04-23T14:18:04.541 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-04-23T14:18:04.541 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-04-23T14:18:04.541 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-04-23T14:18:04.541 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-04-23T14:18:04.541 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-04-23T14:18:04.542 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-04-23T14:18:04.542 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 78073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-04-23T14:18:04.542 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-04-23T14:18:04.542 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 204049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/libcomposite.ko 2024-04-23T14:18:04.542 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/ 2024-04-23T14:18:04.542 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-04-23T14:18:04.542 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/ 2024-04-23T14:18:04.542 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 182561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-04-23T14:18:04.542 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 45545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-04-23T14:18:04.542 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 66513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-04-23T14:18:04.542 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 85217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-04-23T14:18:04.542 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 69313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-04-23T14:18:04.542 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 90313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2272.ko 2024-04-23T14:18:04.543 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 90361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2280.ko 2024-04-23T14:18:04.543 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 94841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-04-23T14:18:04.543 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-04-23T14:18:04.543 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 87465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-04-23T14:18:04.543 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 95441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-04-23T14:18:04.543 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 195969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-04-23T14:18:04.543 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ 2024-04-23T14:18:04.543 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/bcma-hcd.ko 2024-04-23T14:18:04.543 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 108321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/isp116x-hcd.ko 2024-04-23T14:18:04.543 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 104377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-04-23T14:18:04.543 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 102273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-04-23T14:18:04.543 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 60105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-23T14:18:04.544 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811_cs.ko 2024-04-23T14:18:04.544 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ssb-hcd.ko 2024-04-23T14:18:04.544 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-04-23T14:18:04.544 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/ 2024-04-23T14:18:04.544 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 40409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-23T14:18:04.544 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-23T14:18:04.544 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ 2024-04-23T14:18:04.544 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 47353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-23T14:18:04.544 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-23T14:18:04.544 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-04-23T14:18:04.544 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cytherm.ko 2024-04-23T14:18:04.544 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ehset.ko 2024-04-23T14:18:04.545 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-23T14:18:04.545 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-23T14:18:04.545 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-23T14:18:04.545 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-23T14:18:04.545 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 42729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-23T14:18:04.545 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-23T14:18:04.545 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 42241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-23T14:18:04.545 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 42233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-23T14:18:04.545 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/lvstest.ko 2024-04-23T14:18:04.545 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/ 2024-04-23T14:18:04.545 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 85313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-23T14:18:04.545 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-23T14:18:04.546 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-23T14:18:04.546 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-23T14:18:04.546 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 27809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-23T14:18:04.546 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 84265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbtest.ko 2024-04-23T14:18:04.546 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 52065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-23T14:18:04.546 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 32345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-23T14:18:04.546 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/ 2024-04-23T14:18:04.546 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 91529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/usbmon.ko 2024-04-23T14:18:04.546 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/ 2024-04-23T14:18:04.546 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 396209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/musb_hdrc.ko 2024-04-23T14:18:04.546 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/ 2024-04-23T14:18:04.546 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-23T14:18:04.546 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-04-23T14:18:04.547 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-isp1301.ko 2024-04-23T14:18:04.547 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-tahvo.ko 2024-04-23T14:18:04.547 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ 2024-04-23T14:18:04.547 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-23T14:18:04.547 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-23T14:18:04.547 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 30681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-23T14:18:04.547 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 44329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-23T14:18:04.547 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-23T14:18:04.547 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-23T14:18:04.547 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 66873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-23T14:18:04.547 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 58329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-23T14:18:04.547 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-23T14:18:04.547 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 53009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/f81232.ko 2024-04-23T14:18:04.548 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 188609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-23T14:18:04.548 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 61489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-23T14:18:04.548 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 152305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-23T14:18:04.548 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 142833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-23T14:18:04.548 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 69217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-23T14:18:04.548 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-23T14:18:04.548 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-23T14:18:04.548 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-23T14:18:04.548 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 122329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-23T14:18:04.548 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 41297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-23T14:18:04.548 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-23T14:18:04.548 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 38049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-23T14:18:04.548 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 38041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-23T14:18:04.549 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/metro-usb.ko 2024-04-23T14:18:04.549 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-23T14:18:04.549 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 89969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-23T14:18:04.549 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 65073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mxuport.ko 2024-04-23T14:18:04.549 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-23T14:18:04.549 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-23T14:18:04.549 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-23T14:18:04.549 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 193857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-23T14:18:04.549 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 45337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-23T14:18:04.549 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 61761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-23T14:18:04.549 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-23T14:18:04.549 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 39985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-23T14:18:04.549 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 40529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-23T14:18:04.550 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-23T14:18:04.550 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 59345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-23T14:18:04.550 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-23T14:18:04.550 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 34065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-23T14:18:04.550 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-23T14:18:04.550 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 70497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-23T14:18:04.550 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-23T14:18:04.550 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-23T14:18:04.550 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 39321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-23T14:18:04.550 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 151137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usbserial.ko 2024-04-23T14:18:04.550 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-23T14:18:04.551 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 43577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-23T14:18:04.551 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/wishbone-serial.ko 2024-04-23T14:18:04.551 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-23T14:18:04.551 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ 2024-04-23T14:18:04.551 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 64049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-23T14:18:04.551 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 30209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-23T14:18:04.551 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-23T14:18:04.551 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-23T14:18:04.551 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 41073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-23T14:18:04.551 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-23T14:18:04.551 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-23T14:18:04.551 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-23T14:18:04.551 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-23T14:18:04.551 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-23T14:18:04.552 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-23T14:18:04.552 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-23T14:18:04.552 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-23T14:18:04.552 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 41785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-23T14:18:04.552 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 201193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-23T14:18:04.552 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/ 2024-04-23T14:18:04.552 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 76041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-23T14:18:04.552 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 85257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-23T14:18:04.552 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 122321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-23T14:18:04.552 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/ 2024-04-23T14:18:04.552 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/ 2024-04-23T14:18:04.552 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 206921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-23T14:18:04.552 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-23T14:18:04.552 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 134265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-23T14:18:04.552 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 93673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-23T14:18:04.553 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/ 2024-04-23T14:18:04.553 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 138721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-23T14:18:04.553 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-23T14:18:04.553 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 71281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-23T14:18:04.553 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 107905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-23T14:18:04.553 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/ 2024-04-23T14:18:04.553 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/ 2024-04-23T14:18:04.553 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-04-23T14:18:04.553 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/aat2870_bl.ko 2024-04-23T14:18:04.553 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp5520_bl.ko 2024-04-23T14:18:04.553 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 41169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8860_bl.ko 2024-04-23T14:18:04.554 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 51745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8870_bl.ko 2024-04-23T14:18:04.554 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-23T14:18:04.554 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/as3711_bl.ko 2024-04-23T14:18:04.554 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/bd6107.ko 2024-04-23T14:18:04.554 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da903x_bl.ko 2024-04-23T14:18:04.554 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da9052_bl.ko 2024-04-23T14:18:04.554 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/gpio_backlight.ko 2024-04-23T14:18:04.554 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/kb3886_bl.ko 2024-04-23T14:18:04.554 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-23T14:18:04.554 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3533_bl.ko 2024-04-23T14:18:04.555 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3639_bl.ko 2024-04-23T14:18:04.555 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lv5207lp.ko 2024-04-23T14:18:04.555 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/max8925_bl.ko 2024-04-23T14:18:04.555 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pandora_bl.ko 2024-04-23T14:18:04.555 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-04-23T14:18:04.555 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-23T14:18:04.555 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/wm831x_bl.ko 2024-04-23T14:18:04.555 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/ 2024-04-23T14:18:04.555 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ 2024-04-23T14:18:04.555 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-23T14:18:04.555 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 46801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-23T14:18:04.555 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/matrox_w1.ko 2024-04-23T14:18:04.555 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/w1-gpio.ko 2024-04-23T14:18:04.556 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/ 2024-04-23T14:18:04.556 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-23T14:18:04.556 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 27257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-23T14:18:04.556 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-23T14:18:04.556 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-23T14:18:04.556 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-23T14:18:04.556 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-23T14:18:04.556 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-23T14:18:04.556 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-23T14:18:04.556 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-23T14:18:04.556 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 8313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-23T14:18:04.556 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 70369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-23T14:18:04.556 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 129433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-23T14:18:04.556 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ 2024-04-23T14:18:04.557 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/acquirewdt.ko 2024-04-23T14:18:04.557 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/advantechwdt.ko 2024-04-23T14:18:04.557 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-23T14:18:04.557 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 27401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-23T14:18:04.557 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/cpu5wdt.ko 2024-04-23T14:18:04.557 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9052_wdt.ko 2024-04-23T14:18:04.557 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9055_wdt.ko 2024-04-23T14:18:04.557 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9063_wdt.ko 2024-04-23T14:18:04.557 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/dw_wdt.ko 2024-04-23T14:18:04.557 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/eurotechwdt.ko 2024-04-23T14:18:04.557 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-23T14:18:04.557 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 32857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-23T14:18:04.557 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-23T14:18:04.557 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-23T14:18:04.558 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-23T14:18:04.558 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-23T14:18:04.558 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-23T14:18:04.558 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-23T14:18:04.558 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-23T14:18:04.558 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-23T14:18:04.558 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/kempld_wdt.ko 2024-04-23T14:18:04.558 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-23T14:18:04.558 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/mena21_wdt.ko 2024-04-23T14:18:04.558 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-04-23T14:18:04.558 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-23T14:18:04.558 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-04-23T14:18:04.558 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 27721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pc87413_wdt.ko 2024-04-23T14:18:04.559 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 46953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-23T14:18:04.559 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 37193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-23T14:18:04.559 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/retu_wdt.ko 2024-04-23T14:18:04.559 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-04-23T14:18:04.559 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-04-23T14:18:04.559 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-23T14:18:04.559 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sc1200wdt.ko 2024-04-23T14:18:04.559 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 27489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-23T14:18:04.559 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-04-23T14:18:04.559 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-23T14:18:04.559 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-23T14:18:04.559 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/twl4030_wdt.ko 2024-04-23T14:18:04.559 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-23T14:18:04.559 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-23T14:18:04.560 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-23T14:18:04.560 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 32481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-23T14:18:04.560 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wafer5823wdt.ko 2024-04-23T14:18:04.560 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-23T14:18:04.560 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm831x_wdt.ko 2024-04-23T14:18:04.560 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm8350_wdt.ko 2024-04-23T14:18:04.560 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-23T14:18:04.560 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/ 2024-04-23T14:18:04.560 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-23T14:18:04.560 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 30649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-23T14:18:04.560 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-23T14:18:04.560 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/ 2024-04-23T14:18:04.560 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 213913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-23T14:18:04.560 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-23T14:18:04.561 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 85713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-scsiback.ko 2024-04-23T14:18:04.561 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/ 2024-04-23T14:18:04.561 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-23T14:18:04.561 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ 2024-04-23T14:18:04.561 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/ 2024-04-23T14:18:04.561 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 192617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-23T14:18:04.561 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/ 2024-04-23T14:18:04.561 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 78153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/adfs.ko 2024-04-23T14:18:04.561 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/ 2024-04-23T14:18:04.561 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 197849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-23T14:18:04.561 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/ 2024-04-23T14:18:04.561 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 1261113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/kafs.ko 2024-04-23T14:18:04.561 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/ 2024-04-23T14:18:04.561 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 127377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/autofs4.ko 2024-04-23T14:18:04.562 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/ 2024-04-23T14:18:04.562 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-23T14:18:04.562 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/ 2024-04-23T14:18:04.562 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 50473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/bfs.ko 2024-04-23T14:18:04.562 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 45561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-23T14:18:04.562 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/ 2024-04-23T14:18:04.562 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 4489593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-23T14:18:04.562 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/ 2024-04-23T14:18:04.562 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 376857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-23T14:18:04.562 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ 2024-04-23T14:18:04.562 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 1586065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-23T14:18:04.562 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/ 2024-04-23T14:18:04.562 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 130073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-23T14:18:04.562 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/ 2024-04-23T14:18:04.563 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 139081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/configfs.ko 2024-04-23T14:18:04.563 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/ 2024-04-23T14:18:04.563 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-23T14:18:04.563 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/ 2024-04-23T14:18:04.563 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 813225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-23T14:18:04.563 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/ 2024-04-23T14:18:04.563 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 46409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/efs.ko 2024-04-23T14:18:04.563 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/ 2024-04-23T14:18:04.563 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 1906817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-23T14:18:04.563 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/ 2024-04-23T14:18:04.563 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-23T14:18:04.563 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/ 2024-04-23T14:18:04.563 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/freevxfs.ko 2024-04-23T14:18:04.563 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/ 2024-04-23T14:18:04.564 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-23T14:18:04.564 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/ 2024-04-23T14:18:04.564 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 1486521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-23T14:18:04.564 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/ 2024-04-23T14:18:04.564 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 183305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-23T14:18:04.564 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/ 2024-04-23T14:18:04.564 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 282097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-23T14:18:04.564 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/ 2024-04-23T14:18:04.564 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 249009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/hpfs.ko 2024-04-23T14:18:04.564 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/ 2024-04-23T14:18:04.564 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 93033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-23T14:18:04.564 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/ 2024-04-23T14:18:04.564 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 533073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-23T14:18:04.564 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/ 2024-04-23T14:18:04.565 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 361369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-23T14:18:04.565 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/ 2024-04-23T14:18:04.565 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 112609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-23T14:18:04.565 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/ 2024-04-23T14:18:04.565 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 591585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-23T14:18:04.565 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/ 2024-04-23T14:18:04.565 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/ 2024-04-23T14:18:04.565 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 102561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-23T14:18:04.565 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/ 2024-04-23T14:18:04.565 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 74353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-23T14:18:04.565 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/ 2024-04-23T14:18:04.565 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 147689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-23T14:18:04.565 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 1320249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-23T14:18:04.565 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 82105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv2.ko 2024-04-23T14:18:04.566 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 151641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-23T14:18:04.566 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 2518505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-23T14:18:04.566 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/ 2024-04-23T14:18:04.566 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-23T14:18:04.566 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-23T14:18:04.566 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/ 2024-04-23T14:18:04.566 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 2024049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-23T14:18:04.566 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/ 2024-04-23T14:18:04.566 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 734025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-23T14:18:04.566 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/ 2024-04-23T14:18:04.566 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-23T14:18:04.566 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-23T14:18:04.566 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-23T14:18:04.566 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-23T14:18:04.567 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-23T14:18:04.567 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-23T14:18:04.567 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-23T14:18:04.567 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-23T14:18:04.567 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-23T14:18:04.567 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-23T14:18:04.567 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-23T14:18:04.567 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ascii.ko 2024-04-23T14:18:04.567 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-23T14:18:04.567 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-23T14:18:04.567 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-23T14:18:04.567 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-23T14:18:04.567 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-23T14:18:04.567 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-23T14:18:04.568 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-23T14:18:04.568 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-23T14:18:04.568 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-23T14:18:04.568 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-23T14:18:04.568 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-23T14:18:04.568 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-23T14:18:04.568 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-23T14:18:04.568 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-23T14:18:04.568 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-23T14:18:04.568 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-23T14:18:04.568 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-23T14:18:04.568 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-23T14:18:04.568 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 95625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-23T14:18:04.569 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 144209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-23T14:18:04.569 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 167137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-23T14:18:04.569 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 119513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-23T14:18:04.569 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-23T14:18:04.569 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-23T14:18:04.569 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-23T14:18:04.569 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-23T14:18:04.569 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-23T14:18:04.569 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-23T14:18:04.569 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-23T14:18:04.569 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-23T14:18:04.569 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-23T14:18:04.569 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-23T14:18:04.569 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-23T14:18:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-23T14:18:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-23T14:18:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-23T14:18:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-23T14:18:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-23T14:18:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-23T14:18:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ 2024-04-23T14:18:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ 2024-04-23T14:18:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 302073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-23T14:18:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ 2024-04-23T14:18:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 548169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-23T14:18:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ 2024-04-23T14:18:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-23T14:18:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 3245617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-23T14:18:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 22113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-23T14:18:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 39313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-23T14:18:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 43073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-23T14:18:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/ 2024-04-23T14:18:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 67489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/omfs.ko 2024-04-23T14:18:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/ 2024-04-23T14:18:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 462873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-23T14:18:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ 2024-04-23T14:18:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 67977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-23T14:18:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/ 2024-04-23T14:18:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/qnx4.ko 2024-04-23T14:18:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/ 2024-04-23T14:18:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 56097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/qnx6.ko 2024-04-23T14:18:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/ 2024-04-23T14:18:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 40313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_tree.ko 2024-04-23T14:18:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v1.ko 2024-04-23T14:18:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v2.ko 2024-04-23T14:18:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/ 2024-04-23T14:18:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 684841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-23T14:18:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/ 2024-04-23T14:18:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 34673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-23T14:18:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/ 2024-04-23T14:18:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/ 2024-04-23T14:18:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 3319041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-23T14:18:04.615 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/ 2024-04-23T14:18:04.615 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 8073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-23T14:18:04.615 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-23T14:18:04.615 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/ 2024-04-23T14:18:04.615 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 164185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-23T14:18:04.615 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/ 2024-04-23T14:18:04.615 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 115977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-23T14:18:04.615 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/ 2024-04-23T14:18:04.615 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 310017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-23T14:18:04.615 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ 2024-04-23T14:18:04.615 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 198233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-23T14:18:04.616 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/ 2024-04-23T14:18:04.616 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 5192705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-23T14:18:04.616 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/ 2024-04-23T14:18:04.616 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/ 2024-04-23T14:18:04.616 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 97625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/locktorture.ko 2024-04-23T14:18:04.616 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/ 2024-04-23T14:18:04.616 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/test_udelay.ko 2024-04-23T14:18:04.616 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 72665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-23T14:18:04.616 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ 2024-04-23T14:18:04.616 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 7313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-23T14:18:04.616 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 7081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc7.ko 2024-04-23T14:18:04.616 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 7953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-23T14:18:04.616 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/ 2024-04-23T14:18:04.617 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 8009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-23T14:18:04.617 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 38985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-23T14:18:04.617 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/interval_tree_test.ko 2024-04-23T14:18:04.617 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-23T14:18:04.617 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-23T14:18:04.617 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/ 2024-04-23T14:18:04.617 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 36169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-23T14:18:04.617 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-23T14:18:04.617 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/ 2024-04-23T14:18:04.617 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 7281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-23T14:18:04.617 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/memory-notifier-error-inject.ko 2024-04-23T14:18:04.617 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/notifier-error-inject.ko 2024-04-23T14:18:04.617 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/percpu_test.ko 2024-04-23T14:18:04.617 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/pm-notifier-error-inject.ko 2024-04-23T14:18:04.618 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/ 2024-04-23T14:18:04.618 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 152025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-23T14:18:04.618 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/rbtree_test.ko 2024-04-23T14:18:04.618 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/ 2024-04-23T14:18:04.618 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-23T14:18:04.618 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 40969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test-kstrtox.ko 2024-04-23T14:18:04.618 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 4912441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_bpf.ko 2024-04-23T14:18:04.618 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 61169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_firmware.ko 2024-04-23T14:18:04.618 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 7121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_module.ko 2024-04-23T14:18:04.618 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_user_copy.ko 2024-04-23T14:18:04.618 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-23T14:18:04.619 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-23T14:18:04.619 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-23T14:18:04.619 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/ 2024-04-23T14:18:04.619 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/xz_dec_test.ko 2024-04-23T14:18:04.619 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/ 2024-04-23T14:18:04.619 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-23T14:18:04.619 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ 2024-04-23T14:18:04.619 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/ 2024-04-23T14:18:04.619 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 66969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-23T14:18:04.619 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 7577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-23T14:18:04.619 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-23T14:18:04.619 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-23T14:18:04.619 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-23T14:18:04.619 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-23T14:18:04.620 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-23T14:18:04.620 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-23T14:18:04.620 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/ 2024-04-23T14:18:04.620 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-23T14:18:04.620 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-23T14:18:04.620 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-23T14:18:04.620 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-23T14:18:04.620 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-23T14:18:04.620 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/ 2024-04-23T14:18:04.620 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 106793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-23T14:18:04.620 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/ 2024-04-23T14:18:04.620 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 236793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-23T14:18:04.620 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 44449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-23T14:18:04.620 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 32209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-23T14:18:04.621 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 39513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-23T14:18:04.621 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/ 2024-04-23T14:18:04.621 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 879697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-23T14:18:04.621 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/ 2024-04-23T14:18:04.621 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 84761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-23T14:18:04.621 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 1030873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-23T14:18:04.621 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ 2024-04-23T14:18:04.621 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 8905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-23T14:18:04.621 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-23T14:18:04.621 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-23T14:18:04.621 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-23T14:18:04.621 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-23T14:18:04.621 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-23T14:18:04.621 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-23T14:18:04.622 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-23T14:18:04.622 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-23T14:18:04.622 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 8841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-23T14:18:04.622 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 8777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-23T14:18:04.622 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-23T14:18:04.622 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-23T14:18:04.622 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-23T14:18:04.622 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-23T14:18:04.622 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-23T14:18:04.622 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-23T14:18:04.622 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-23T14:18:04.622 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-23T14:18:04.622 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-23T14:18:04.623 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 70529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-23T14:18:04.623 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-23T14:18:04.623 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-23T14:18:04.623 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/ 2024-04-23T14:18:04.623 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 1243505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-23T14:18:04.623 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/ 2024-04-23T14:18:04.623 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 153801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-23T14:18:04.623 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/ 2024-04-23T14:18:04.623 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 244889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-23T14:18:04.623 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-23T14:18:04.623 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 71033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-23T14:18:04.623 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 74993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-23T14:18:04.623 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/ 2024-04-23T14:18:04.623 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 507705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-23T14:18:04.623 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-23T14:18:04.623 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-23T14:18:04.624 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 8873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-23T14:18:04.624 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-23T14:18:04.624 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/ 2024-04-23T14:18:04.624 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 128873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/hsr.ko 2024-04-23T14:18:04.624 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ 2024-04-23T14:18:04.624 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ 2024-04-23T14:18:04.624 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-23T14:18:04.624 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 357425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-23T14:18:04.624 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 59393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-23T14:18:04.624 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ 2024-04-23T14:18:04.624 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-23T14:18:04.624 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 46417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-23T14:18:04.624 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 60233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-23T14:18:04.624 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-23T14:18:04.625 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 65665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-23T14:18:04.625 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 75489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-23T14:18:04.625 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 77721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-23T14:18:04.625 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 43705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-23T14:18:04.625 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-23T14:18:04.625 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 34745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-23T14:18:04.625 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ 2024-04-23T14:18:04.625 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 45521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-23T14:18:04.625 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-23T14:18:04.625 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-23T14:18:04.625 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 50169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ip_tables.ko 2024-04-23T14:18:04.625 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-23T14:18:04.625 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-04-23T14:18:04.626 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-23T14:18:04.626 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-23T14:18:04.626 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-23T14:18:04.626 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-04-23T14:18:04.626 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-23T14:18:04.626 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-23T14:18:04.626 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-23T14:18:04.626 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-23T14:18:04.626 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-23T14:18:04.626 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-23T14:18:04.626 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 32473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-23T14:18:04.626 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-23T14:18:04.626 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-23T14:18:04.627 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-04-23T14:18:04.627 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-23T14:18:04.627 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-23T14:18:04.627 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-23T14:18:04.627 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-23T14:18:04.627 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-23T14:18:04.627 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-23T14:18:04.627 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-23T14:18:04.627 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-23T14:18:04.627 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-23T14:18:04.627 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-23T14:18:04.627 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-23T14:18:04.627 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-23T14:18:04.628 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-23T14:18:04.628 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-23T14:18:04.628 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-23T14:18:04.628 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-23T14:18:04.628 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-23T14:18:04.628 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-23T14:18:04.628 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-23T14:18:04.628 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-23T14:18:04.628 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 54441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-23T14:18:04.628 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-23T14:18:04.628 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ 2024-04-23T14:18:04.628 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 31073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-23T14:18:04.628 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 47433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-23T14:18:04.628 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-23T14:18:04.629 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 103177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_gre.ko 2024-04-23T14:18:04.629 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 118881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-23T14:18:04.629 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-23T14:18:04.629 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 71089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-23T14:18:04.629 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-23T14:18:04.629 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/mip6.ko 2024-04-23T14:18:04.629 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ 2024-04-23T14:18:04.629 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 50673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-23T14:18:04.629 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-04-23T14:18:04.629 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-23T14:18:04.629 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-23T14:18:04.629 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-23T14:18:04.629 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 8721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-23T14:18:04.629 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-23T14:18:04.630 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-23T14:18:04.630 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-23T14:18:04.630 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-23T14:18:04.630 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-23T14:18:04.630 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-23T14:18:04.630 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-23T14:18:04.630 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-23T14:18:04.630 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-23T14:18:04.630 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-23T14:18:04.630 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-23T14:18:04.630 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 46313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-23T14:18:04.630 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-23T14:18:04.630 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 29817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-23T14:18:04.630 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-23T14:18:04.631 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-23T14:18:04.631 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-23T14:18:04.631 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-23T14:18:04.631 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-23T14:18:04.631 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 94089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-23T14:18:04.631 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 30297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-23T14:18:04.631 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-23T14:18:04.631 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/ 2024-04-23T14:18:04.631 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 106369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-23T14:18:04.631 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/ 2024-04-23T14:18:04.631 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 153217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-23T14:18:04.631 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-23T14:18:04.631 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 31985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-23T14:18:04.631 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 43641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-23T14:18:04.631 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 44993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-23T14:18:04.632 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 42049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-23T14:18:04.632 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 66241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-23T14:18:04.632 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/ 2024-04-23T14:18:04.632 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 63345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/lapb.ko 2024-04-23T14:18:04.632 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/ 2024-04-23T14:18:04.632 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 27385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-23T14:18:04.632 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 332889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc2.ko 2024-04-23T14:18:04.632 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/ 2024-04-23T14:18:04.632 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 388873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-23T14:18:04.632 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/ 2024-04-23T14:18:04.632 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-23T14:18:04.632 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ 2024-04-23T14:18:04.632 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ 2024-04-23T14:18:04.632 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 147745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-23T14:18:04.632 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 34473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-23T14:18:04.633 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-23T14:18:04.633 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-23T14:18:04.633 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 124041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-23T14:18:04.633 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 124345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-23T14:18:04.633 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 126681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-23T14:18:04.633 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 126473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-23T14:18:04.633 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 137833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-23T14:18:04.633 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 72353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-23T14:18:04.633 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 133681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-23T14:18:04.633 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 139073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-23T14:18:04.633 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 138265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-23T14:18:04.633 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 136177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-23T14:18:04.633 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 139897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-23T14:18:04.633 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 37129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-23T14:18:04.633 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ 2024-04-23T14:18:04.634 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 534113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-23T14:18:04.634 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-23T14:18:04.634 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-23T14:18:04.634 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-23T14:18:04.634 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-23T14:18:04.634 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-23T14:18:04.634 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-23T14:18:04.634 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 8865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-23T14:18:04.634 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-23T14:18:04.634 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-23T14:18:04.634 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-23T14:18:04.634 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-23T14:18:04.634 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-23T14:18:04.635 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-23T14:18:04.635 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 55217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-23T14:18:04.635 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 496817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-23T14:18:04.635 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-23T14:18:04.635 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-23T14:18:04.635 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 37865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-23T14:18:04.635 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 134377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-23T14:18:04.635 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-23T14:18:04.635 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-23T14:18:04.635 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 140801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-23T14:18:04.635 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 38305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-23T14:18:04.635 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-23T14:18:04.635 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 79873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-23T14:18:04.635 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-23T14:18:04.635 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-23T14:18:04.636 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-23T14:18:04.636 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 117201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-23T14:18:04.636 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-23T14:18:04.636 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 46105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-23T14:18:04.636 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 118369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-23T14:18:04.636 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-23T14:18:04.636 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-23T14:18:04.636 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-23T14:18:04.636 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 34609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-23T14:18:04.636 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-23T14:18:04.636 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 53393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-23T14:18:04.636 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 807985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-23T14:18:04.636 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 53049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-23T14:18:04.637 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 36481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-23T14:18:04.637 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 37209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-04-23T14:18:04.637 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 49297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-23T14:18:04.637 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-23T14:18:04.637 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 65145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-23T14:18:04.637 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-23T14:18:04.637 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 41769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-23T14:18:04.637 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 50657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-23T14:18:04.637 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-23T14:18:04.637 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-23T14:18:04.637 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-23T14:18:04.637 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-23T14:18:04.637 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-23T14:18:04.638 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-23T14:18:04.638 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-23T14:18:04.638 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-23T14:18:04.638 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-23T14:18:04.638 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_osf.ko 2024-04-23T14:18:04.638 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-23T14:18:04.638 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-23T14:18:04.638 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-23T14:18:04.638 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-23T14:18:04.638 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-23T14:18:04.638 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 24993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_synproxy.ko 2024-04-23T14:18:04.638 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tproxy.ko 2024-04-23T14:18:04.638 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 108393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/x_tables.ko 2024-04-23T14:18:04.639 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-23T14:18:04.639 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-23T14:18:04.639 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-23T14:18:04.639 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-23T14:18:04.639 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 21865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-23T14:18:04.639 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-23T14:18:04.639 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-23T14:18:04.639 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-23T14:18:04.639 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 45217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-23T14:18:04.639 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-23T14:18:04.639 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-23T14:18:04.639 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-23T14:18:04.639 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-23T14:18:04.639 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-23T14:18:04.639 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-23T14:18:04.640 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-23T14:18:04.640 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-23T14:18:04.640 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-23T14:18:04.640 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-23T14:18:04.640 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-23T14:18:04.640 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-23T14:18:04.640 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-23T14:18:04.640 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-23T14:18:04.640 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-23T14:18:04.640 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-23T14:18:04.640 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-23T14:18:04.640 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-23T14:18:04.640 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 8465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-23T14:18:04.640 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-23T14:18:04.641 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-23T14:18:04.641 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-23T14:18:04.641 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-23T14:18:04.641 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-23T14:18:04.641 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 8913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-23T14:18:04.641 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-23T14:18:04.641 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-23T14:18:04.641 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-23T14:18:04.641 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-23T14:18:04.641 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-23T14:18:04.641 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 52817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-23T14:18:04.641 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-23T14:18:04.641 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-23T14:18:04.641 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-23T14:18:04.642 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-23T14:18:04.642 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-23T14:18:04.642 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-23T14:18:04.642 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-23T14:18:04.642 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-23T14:18:04.642 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 8449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-23T14:18:04.642 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-23T14:18:04.642 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-23T14:18:04.642 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 16161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-23T14:18:04.642 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-23T14:18:04.642 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-23T14:18:04.642 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-23T14:18:04.643 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-23T14:18:04.643 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-23T14:18:04.643 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-23T14:18:04.643 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-23T14:18:04.643 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 11881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-23T14:18:04.643 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-23T14:18:04.643 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 43465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-23T14:18:04.643 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-23T14:18:04.643 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 36489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-23T14:18:04.643 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-23T14:18:04.643 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-23T14:18:04.643 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-23T14:18:04.643 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-23T14:18:04.644 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-23T14:18:04.644 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpudp.ko 2024-04-23T14:18:04.644 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-23T14:18:04.644 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-23T14:18:04.644 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/ 2024-04-23T14:18:04.644 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 17233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-23T14:18:04.644 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/ 2024-04-23T14:18:04.644 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-23T14:18:04.644 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/ 2024-04-23T14:18:04.644 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 488825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-23T14:18:04.644 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-23T14:18:04.644 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-23T14:18:04.644 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-23T14:18:04.644 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/ 2024-04-23T14:18:04.644 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-23T14:18:04.645 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/ 2024-04-23T14:18:04.645 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 142025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/phonet.ko 2024-04-23T14:18:04.645 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 68817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/pn_pep.ko 2024-04-23T14:18:04.645 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/ 2024-04-23T14:18:04.645 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 318617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-23T14:18:04.645 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 202305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-23T14:18:04.645 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 85105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-23T14:18:04.645 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/ 2024-04-23T14:18:04.645 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 1027897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/rxrpc.ko 2024-04-23T14:18:04.645 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/ 2024-04-23T14:18:04.645 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 39001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-23T14:18:04.645 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 28721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-23T14:18:04.645 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 45409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-23T14:18:04.646 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-23T14:18:04.646 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 53465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-23T14:18:04.646 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 38705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-23T14:18:04.646 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-23T14:18:04.646 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 43417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-23T14:18:04.646 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 45673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-23T14:18:04.646 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 32129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-23T14:18:04.646 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 41457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-23T14:18:04.646 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_cgroup.ko 2024-04-23T14:18:04.646 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 37305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-23T14:18:04.646 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 39601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-23T14:18:04.646 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 49321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-23T14:18:04.646 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 79145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-23T14:18:04.646 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 8713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-23T14:18:04.647 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-23T14:18:04.647 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 48785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-23T14:18:04.647 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-23T14:18:04.647 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-23T14:18:04.647 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 8377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-23T14:18:04.647 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-23T14:18:04.647 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 25097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-23T14:18:04.647 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 43481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-23T14:18:04.647 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 43153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-23T14:18:04.647 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 39177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq_codel.ko 2024-04-23T14:18:04.647 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-23T14:18:04.647 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 62817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-23T14:18:04.647 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-23T14:18:04.648 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 81833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-23T14:18:04.648 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-23T14:18:04.648 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 36625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-23T14:18:04.648 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-23T14:18:04.648 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-23T14:18:04.648 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 44793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-23T14:18:04.648 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 32833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-23T14:18:04.648 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 13505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-23T14:18:04.648 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-23T14:18:04.648 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 60249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-23T14:18:04.648 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-23T14:18:04.648 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 33433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-23T14:18:04.649 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-23T14:18:04.649 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 35777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-23T14:18:04.649 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 34161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-23T14:18:04.649 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/ 2024-04-23T14:18:04.649 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 1187313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-23T14:18:04.649 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-23T14:18:04.649 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/ 2024-04-23T14:18:04.649 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/ 2024-04-23T14:18:04.649 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 432409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-23T14:18:04.649 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 70113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-23T14:18:04.649 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 1940889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-23T14:18:04.649 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/ 2024-04-23T14:18:04.649 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 890729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-23T14:18:04.649 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/ 2024-04-23T14:18:04.649 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 1122337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-23T14:18:04.650 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-23T14:18:04.650 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/ 2024-04-23T14:18:04.650 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 18801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-23T14:18:04.650 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/ 2024-04-23T14:18:04.650 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 105609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-23T14:18:04.650 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 90265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-23T14:18:04.650 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 123769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-23T14:18:04.650 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-23T14:18:04.650 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-23T14:18:04.650 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/ 2024-04-23T14:18:04.650 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 165841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/x25.ko 2024-04-23T14:18:04.650 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/ 2024-04-23T14:18:04.650 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_algo.ko 2024-04-23T14:18:04.650 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-23T14:18:04.651 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 131865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_user.ko 2024-04-23T14:18:04.651 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 644769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-23T14:18:04.651 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 628542 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-23T14:18:04.651 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 7706 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-23T14:18:04.651 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 23609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-23T14:18:04.651 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 9941 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-23T14:18:04.651 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 70990 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-23T14:18:04.651 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 146959 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-23T14:18:04.651 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 232835 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-23T14:18:04.651 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 210 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-23T14:18:04.651 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 72602 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-23T14:18:04.651 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 879 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-23T14:18:04.651 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 258450 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-23T14:18:04.651 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 313019 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-23T14:18:04.652 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/ 2024-04-23T14:18:04.652 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/ 2024-04-23T14:18:04.652 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/ 2024-04-23T14:18:04.652 INFO:teuthology.orchestra.run.smithi100.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/ 2024-04-23T14:18:04.652 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 172 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/changelog.Debian.gz 2024-04-23T14:18:04.652 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r-- root/root 691 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/copyright 2024-04-23T14:18:04.657 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-23T14:18:04.657 INFO:teuthology.task.kernel:searching for kernel 6.9.0-rc3-g806ec36b2112 2024-04-23T14:18:04.657 DEBUG:teuthology.orchestra.run.smithi100:> egrep '(submenu|menuentry.*6.9.0-rc3-g806ec36b2112).*{' /boot/grub/grub.cfg 2024-04-23T14:18:04.664 INFO:teuthology.orchestra.run.smithi100.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-23T14:18:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-23T14:18:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-23T14:18:04.665 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-04-23T14:18:04.665 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 2024-04-23T14:18:04.665 DEBUG:teuthology.orchestra.run.smithi100:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-04-23T14:18:04.710 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi100.front.sepia.ceph.com to complete... 2024-04-23T14:18:05.072 INFO:teuthology.orchestra.run.smithi100.stderr:Sourcing file `/etc/default/grub' 2024-04-23T14:18:05.073 INFO:teuthology.orchestra.run.smithi100.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-23T14:18:05.098 INFO:teuthology.orchestra.run.smithi100.stderr:Generating grub configuration file ... 2024-04-23T14:18:05.275 INFO:teuthology.orchestra.run.smithi100.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-23T14:18:05.282 INFO:teuthology.orchestra.run.smithi100.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-23T14:18:05.496 INFO:teuthology.orchestra.run.smithi100.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-23T14:18:05.536 INFO:teuthology.orchestra.run.smithi100.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-23T14:18:05.541 INFO:teuthology.orchestra.run.smithi100.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-23T14:18:05.546 INFO:teuthology.orchestra.run.smithi100.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-23T14:18:05.656 INFO:teuthology.orchestra.run.smithi100.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-23T14:18:05.657 INFO:teuthology.orchestra.run.smithi100.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-23T14:18:05.657 INFO:teuthology.orchestra.run.smithi100.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-23T14:18:05.666 INFO:teuthology.orchestra.run.smithi100.stderr:done 2024-04-23T14:18:24.826 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-23T14:18:24.827 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-23T14:18:24.827 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:18:36.772 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-23T14:18:36.772 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-23T14:18:36.773 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:18:43.362 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-04-23T14:18:52.370 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-23T14:18:52.371 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:18:55.134 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-04-23T14:18:55.426 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-04-23T14:19:04.142 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-23T14:19:04.143 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:19:07.198 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-04-23T14:19:07.427 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-23T14:19:07.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:19:19.202 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-23T14:19:19.203 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:19:25.854 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-04-23T14:19:37.634 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-04-23T14:19:40.858 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-23T14:19:40.859 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:19:40.862 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-04-23T14:19:52.646 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-23T14:19:52.647 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:19:52.848 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-04-23T14:19:53.540 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-23T14:19:53.541 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g806ec36b2112"... 2024-04-23T14:19:53.541 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-04-23T14:19:53.590 INFO:teuthology.orchestra.run.smithi100.stdout:6.9.0-rc3-g806ec36b2112 2024-04-23T14:19:53.590 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-23T14:19:53.590 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-23T14:19:53.591 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-23T14:19:54.592 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-23T14:19:54.592 DEBUG:teuthology.orchestra.run.smithi100:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-23T14:19:54.685 INFO:teuthology.orchestra.run.smithi100.stdout:ttyS1 2024-04-23T14:19:54.742 DEBUG:teuthology.parallel:result is None 2024-04-23T14:19:58.864 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-23T14:19:58.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:19:59.198 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-04-23T14:19:59.856 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-23T14:19:59.856 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g806ec36b2112"... 2024-04-23T14:19:59.856 DEBUG:teuthology.orchestra.run.smithi151:> uname -r 2024-04-23T14:19:59.906 INFO:teuthology.orchestra.run.smithi151.stdout:6.9.0-rc3-g806ec36b2112 2024-04-23T14:19:59.907 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-23T14:19:59.907 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-23T14:19:59.907 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-23T14:20:00.908 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-23T14:20:00.909 DEBUG:teuthology.orchestra.run.smithi151:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-23T14:20:00.999 INFO:teuthology.orchestra.run.smithi151.stdout:ttyS1 2024-04-23T14:20:01.038 DEBUG:teuthology.parallel:result is None 2024-04-23T14:20:01.039 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-23T14:20:01.053 INFO:teuthology.task.internal:Creating test directory... 2024-04-23T14:20:01.053 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T14:20:01.058 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T14:20:01.066 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-23T14:20:01.093 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-23T14:20:01.126 INFO:teuthology.task.internal:Creating archive directory... 2024-04-23T14:20:01.126 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T14:20:01.129 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T14:20:01.155 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-23T14:20:01.165 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-23T14:20:01.165 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T14:20:01.193 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T14:20:01.221 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T14:20:01.222 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T14:20:01.235 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T14:20:01.236 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T14:20:01.239 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-23T14:20:01.249 INFO:teuthology.task.internal:Configuring sudo... 2024-04-23T14:20:01.249 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T14:20:01.289 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T14:20:01.310 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-23T14:20:01.321 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-23T14:20:01.321 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T14:20:01.357 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T14:20:01.366 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T14:20:01.419 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T14:20:01.470 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:20:01.470 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T14:20:01.544 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T14:20:01.553 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T14:20:01.603 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:20:01.603 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T14:20:01.669 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2024-04-23T14:20:01.672 DEBUG:teuthology.orchestra.run.smithi151:> sudo service rsyslog restart 2024-04-23T14:20:01.801 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-23T14:20:01.809 INFO:teuthology.task.internal:Starting timer... 2024-04-23T14:20:01.809 INFO:teuthology.run_tasks:Running task pcp... 2024-04-23T14:20:01.850 INFO:teuthology.run_tasks:Running task selinux... 2024-04-23T14:20:01.877 DEBUG:teuthology.task.selinux:Excluding smithi100: OS 'ubuntu' does not support SELinux 2024-04-23T14:20:01.877 DEBUG:teuthology.task.selinux:Excluding smithi151: OS 'ubuntu' does not support SELinux 2024-04-23T14:20:01.878 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-23T14:20:01.878 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-23T14:20:01.878 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-23T14:20:01.878 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-23T14:20:01.913 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-23T14:20:01.930 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-23T14:20:01.932 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi100.front.sepia.ceph.com,smithi151.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-23T14:27:39.889 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi100.front.sepia.ceph.com'), Remote(name='ubuntu@smithi151.front.sepia.ceph.com')] 2024-04-23T14:27:39.891 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-23T14:27:39.892 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:27:39.984 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-04-23T14:27:40.088 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-23T14:27:40.089 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-23T14:27:40.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:27:40.171 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-04-23T14:27:40.281 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-23T14:27:40.281 INFO:teuthology.run_tasks:Running task clock... 2024-04-23T14:27:40.292 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-23T14:27:40.293 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T14:27:40.293 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T14:27:40.296 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T14:27:40.296 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T14:27:40.331 INFO:teuthology.orchestra.run.smithi100.stdout:23 Apr 14:27:40 ntpd[17253]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-23T14:27:40.331 INFO:teuthology.orchestra.run.smithi100.stdout:23 Apr 14:27:40 ntpd[17253]: Command line: ntpd -gq 2024-04-23T14:27:40.331 INFO:teuthology.orchestra.run.smithi100.stdout:23 Apr 14:27:40 ntpd[17253]: ---------------------------------------------------- 2024-04-23T14:27:40.331 INFO:teuthology.orchestra.run.smithi100.stdout:23 Apr 14:27:40 ntpd[17253]: ntp-4 is maintained by Network Time Foundation, 2024-04-23T14:27:40.332 INFO:teuthology.orchestra.run.smithi100.stdout:23 Apr 14:27:40 ntpd[17253]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-23T14:27:40.332 INFO:teuthology.orchestra.run.smithi100.stdout:23 Apr 14:27:40 ntpd[17253]: corporation. Support and training for ntp-4 are 2024-04-23T14:27:40.332 INFO:teuthology.orchestra.run.smithi100.stdout:23 Apr 14:27:40 ntpd[17253]: available at https://www.nwtime.org/support 2024-04-23T14:27:40.332 INFO:teuthology.orchestra.run.smithi100.stdout:23 Apr 14:27:40 ntpd[17253]: ---------------------------------------------------- 2024-04-23T14:27:40.333 INFO:teuthology.orchestra.run.smithi100.stdout:23 Apr 14:27:40 ntpd[17253]: proto: precision = 0.044 usec (-24) 2024-04-23T14:27:40.333 INFO:teuthology.orchestra.run.smithi100.stdout:23 Apr 14:27:40 ntpd[17253]: basedate set to 2022-02-04 2024-04-23T14:27:40.333 INFO:teuthology.orchestra.run.smithi100.stdout:23 Apr 14:27:40 ntpd[17253]: gps base set to 2022-02-06 (week 2196) 2024-04-23T14:27:40.333 INFO:teuthology.orchestra.run.smithi100.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-23T14:27:40.334 INFO:teuthology.orchestra.run.smithi100.stdout:23 Apr 14:27:40 ntpd[17253]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-23T14:27:40.334 INFO:teuthology.orchestra.run.smithi100.stdout:23 Apr 14:27:40 ntpd[17253]: restrict ::: KOD does nothing without LIMITED. 2024-04-23T14:27:40.334 INFO:teuthology.orchestra.run.smithi100.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-23T14:27:40.335 INFO:teuthology.orchestra.run.smithi100.stdout:23 Apr 14:27:40 ntpd[17253]: Listen and drop on 0 v6wildcard [::]:123 2024-04-23T14:27:40.335 INFO:teuthology.orchestra.run.smithi100.stdout:23 Apr 14:27:40 ntpd[17253]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-23T14:27:40.335 INFO:teuthology.orchestra.run.smithi100.stdout:23 Apr 14:27:40 ntpd[17253]: Listen normally on 2 lo 127.0.0.1:123 2024-04-23T14:27:40.335 INFO:teuthology.orchestra.run.smithi100.stdout:23 Apr 14:27:40 ntpd[17253]: Listen normally on 3 enp3s0f1 172.21.15.100:123 2024-04-23T14:27:40.335 INFO:teuthology.orchestra.run.smithi100.stdout:23 Apr 14:27:40 ntpd[17253]: Listen normally on 4 lo [::1]:123 2024-04-23T14:27:40.336 INFO:teuthology.orchestra.run.smithi100.stdout:23 Apr 14:27:40 ntpd[17253]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d733%5]:123 2024-04-23T14:27:40.336 INFO:teuthology.orchestra.run.smithi100.stdout:23 Apr 14:27:40 ntpd[17253]: Listening on routing socket on fd #22 for interface updates 2024-04-23T14:27:40.364 INFO:teuthology.orchestra.run.smithi151.stdout:23 Apr 14:27:40 ntpd[17210]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-23T14:27:40.364 INFO:teuthology.orchestra.run.smithi151.stdout:23 Apr 14:27:40 ntpd[17210]: Command line: ntpd -gq 2024-04-23T14:27:40.365 INFO:teuthology.orchestra.run.smithi151.stdout:23 Apr 14:27:40 ntpd[17210]: ---------------------------------------------------- 2024-04-23T14:27:40.365 INFO:teuthology.orchestra.run.smithi151.stdout:23 Apr 14:27:40 ntpd[17210]: ntp-4 is maintained by Network Time Foundation, 2024-04-23T14:27:40.365 INFO:teuthology.orchestra.run.smithi151.stdout:23 Apr 14:27:40 ntpd[17210]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-23T14:27:40.365 INFO:teuthology.orchestra.run.smithi151.stdout:23 Apr 14:27:40 ntpd[17210]: corporation. Support and training for ntp-4 are 2024-04-23T14:27:40.365 INFO:teuthology.orchestra.run.smithi151.stdout:23 Apr 14:27:40 ntpd[17210]: available at https://www.nwtime.org/support 2024-04-23T14:27:40.365 INFO:teuthology.orchestra.run.smithi151.stdout:23 Apr 14:27:40 ntpd[17210]: ---------------------------------------------------- 2024-04-23T14:27:40.365 INFO:teuthology.orchestra.run.smithi151.stdout:23 Apr 14:27:40 ntpd[17210]: proto: precision = 0.042 usec (-24) 2024-04-23T14:27:40.365 INFO:teuthology.orchestra.run.smithi151.stdout:23 Apr 14:27:40 ntpd[17210]: basedate set to 2022-02-04 2024-04-23T14:27:40.365 INFO:teuthology.orchestra.run.smithi151.stdout:23 Apr 14:27:40 ntpd[17210]: gps base set to 2022-02-06 (week 2196) 2024-04-23T14:27:40.366 INFO:teuthology.orchestra.run.smithi151.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-23T14:27:40.367 INFO:teuthology.orchestra.run.smithi151.stdout:23 Apr 14:27:40 ntpd[17210]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-23T14:27:40.367 INFO:teuthology.orchestra.run.smithi151.stdout:23 Apr 14:27:40 ntpd[17210]: restrict ::: KOD does nothing without LIMITED. 2024-04-23T14:27:40.367 INFO:teuthology.orchestra.run.smithi151.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-23T14:27:40.368 INFO:teuthology.orchestra.run.smithi151.stdout:23 Apr 14:27:40 ntpd[17210]: Listen and drop on 0 v6wildcard [::]:123 2024-04-23T14:27:40.368 INFO:teuthology.orchestra.run.smithi151.stdout:23 Apr 14:27:40 ntpd[17210]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-23T14:27:40.368 INFO:teuthology.orchestra.run.smithi151.stdout:23 Apr 14:27:40 ntpd[17210]: Listen normally on 2 lo 127.0.0.1:123 2024-04-23T14:27:40.368 INFO:teuthology.orchestra.run.smithi151.stdout:23 Apr 14:27:40 ntpd[17210]: Listen normally on 3 enp3s0f1 172.21.15.151:123 2024-04-23T14:27:40.368 INFO:teuthology.orchestra.run.smithi151.stdout:23 Apr 14:27:40 ntpd[17210]: Listen normally on 4 lo [::1]:123 2024-04-23T14:27:40.368 INFO:teuthology.orchestra.run.smithi151.stdout:23 Apr 14:27:40 ntpd[17210]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6f63%5]:123 2024-04-23T14:27:40.368 INFO:teuthology.orchestra.run.smithi151.stdout:23 Apr 14:27:40 ntpd[17210]: Listening on routing socket on fd #22 for interface updates 2024-04-23T14:27:41.364 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:41 ntpd[17210]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:41.364 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:41 ntpd[17210]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:41.364 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:41 ntpd[17210]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:41.364 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:41 ntpd[17210]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:41.364 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:41 ntpd[17210]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:41.364 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:41 ntpd[17210]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:41.364 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:41 ntpd[17210]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-23T14:27:41.364 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:41 ntpd[17210]: can't open /var/log/ntpstats/peerstats.20240423: Permission denied 2024-04-23T14:27:42.332 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:42 ntpd[17253]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:42.332 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:42 ntpd[17253]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:42.332 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:42 ntpd[17253]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:42.332 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:42 ntpd[17253]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:42.332 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:42 ntpd[17253]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-23T14:27:42.332 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:42 ntpd[17253]: can't open /var/log/ntpstats/peerstats.20240423: Permission denied 2024-04-23T14:27:43.364 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:43 ntpd[17210]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:43.364 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:43 ntpd[17210]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:43.364 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:43 ntpd[17210]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:43.364 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:43 ntpd[17210]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:43.364 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:43 ntpd[17210]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-23T14:27:43.364 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:43 ntpd[17210]: can't open /var/log/ntpstats/peerstats.20240423: Permission denied 2024-04-23T14:27:44.332 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:44 ntpd[17253]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:44.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:44 ntpd[17253]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:44.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:44 ntpd[17253]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:44.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:44 ntpd[17253]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:44.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:44 ntpd[17253]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:44.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:44 ntpd[17253]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:44.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:44 ntpd[17253]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:44.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:44 ntpd[17253]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:44.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:44 ntpd[17253]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-23T14:27:44.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:44 ntpd[17253]: can't open /var/log/ntpstats/peerstats.20240423: Permission denied 2024-04-23T14:27:44.334 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:44 ntpd[17253]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:44.334 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:44 ntpd[17253]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:44.334 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:44 ntpd[17253]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-23T14:27:44.334 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:44 ntpd[17253]: can't open /var/log/ntpstats/peerstats.20240423: Permission denied 2024-04-23T14:27:44.364 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:44 ntpd[17210]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:44.365 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:44 ntpd[17210]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:44.365 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:44 ntpd[17210]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:44.365 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:44 ntpd[17210]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:44.365 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:44 ntpd[17210]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-23T14:27:44.365 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:44 ntpd[17210]: can't open /var/log/ntpstats/peerstats.20240423: Permission denied 2024-04-23T14:27:45.332 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:45 ntpd[17253]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:45.332 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:45 ntpd[17253]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:45.332 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:45 ntpd[17253]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:45.332 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:45 ntpd[17253]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:45.332 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:45 ntpd[17253]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-23T14:27:45.332 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:45 ntpd[17253]: can't open /var/log/ntpstats/peerstats.20240423: Permission denied 2024-04-23T14:27:45.364 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:45 ntpd[17210]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:45.365 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:45 ntpd[17210]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:45.365 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:45 ntpd[17210]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:45.365 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:45 ntpd[17210]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:45.365 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:45 ntpd[17210]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:45.365 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:45 ntpd[17210]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:45.366 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:45 ntpd[17210]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-23T14:27:45.366 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:45 ntpd[17210]: can't open /var/log/ntpstats/peerstats.20240423: Permission denied 2024-04-23T14:27:45.366 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:45 ntpd[17210]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:45.366 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:45 ntpd[17210]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:45.366 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:45 ntpd[17210]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-23T14:27:45.366 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:45 ntpd[17210]: can't open /var/log/ntpstats/peerstats.20240423: Permission denied 2024-04-23T14:27:46.332 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:46 ntpd[17253]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:46.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:46 ntpd[17253]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:46.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:46 ntpd[17253]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:46.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:46 ntpd[17253]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:46.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:46 ntpd[17253]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:46.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:46 ntpd[17253]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:46.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:46 ntpd[17253]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-23T14:27:46.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:46 ntpd[17253]: can't open /var/log/ntpstats/peerstats.20240423: Permission denied 2024-04-23T14:27:46.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:46 ntpd[17253]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:46.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:46 ntpd[17253]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:46.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:46 ntpd[17253]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-23T14:27:46.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:46 ntpd[17253]: can't open /var/log/ntpstats/peerstats.20240423: Permission denied 2024-04-23T14:27:46.369 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:46 ntpd[17210]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:46.369 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:46 ntpd[17210]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:46.369 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:46 ntpd[17210]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:46.369 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:46 ntpd[17210]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:46.369 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:46 ntpd[17210]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-23T14:27:46.369 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:46 ntpd[17210]: can't open /var/log/ntpstats/peerstats.20240423: Permission denied 2024-04-23T14:27:47.332 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:47 ntpd[17253]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:47.332 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:47 ntpd[17253]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:47.332 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:47 ntpd[17253]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:47.332 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:47 ntpd[17253]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:47.332 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:47 ntpd[17253]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-23T14:27:47.332 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:47 ntpd[17253]: can't open /var/log/ntpstats/peerstats.20240423: Permission denied 2024-04-23T14:27:47.364 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:47 ntpd[17210]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:47.364 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:47 ntpd[17210]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:47.364 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:47 ntpd[17210]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:47.364 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:47 ntpd[17210]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:47.364 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:47 ntpd[17210]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:47.365 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:47 ntpd[17210]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:47.365 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:47 ntpd[17210]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-23T14:27:47.365 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:47 ntpd[17210]: can't open /var/log/ntpstats/peerstats.20240423: Permission denied 2024-04-23T14:27:47.365 INFO:teuthology.orchestra.run.smithi151.stdout:23 Apr 14:27:47 ntpd[17210]: ntpd: time slew +0.001257 s 2024-04-23T14:27:47.365 INFO:teuthology.orchestra.run.smithi151.stdout:ntpd: time slew +0.001257s 2024-04-23T14:27:47.365 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:47 ntpd[17210]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-23T14:27:47.366 INFO:teuthology.orchestra.run.smithi151.stderr:23 Apr 14:27:47 ntpd[17210]: can't open /var/log/ntpstats/loopstats.20240423: Permission denied 2024-04-23T14:27:47.476 INFO:teuthology.orchestra.run.smithi151.stdout: remote refid st t when poll reach delay offset jitter 2024-04-23T14:27:47.476 INFO:teuthology.orchestra.run.smithi151.stdout:============================================================================== 2024-04-23T14:27:47.476 INFO:teuthology.orchestra.run.smithi151.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-23T14:27:47.476 INFO:teuthology.orchestra.run.smithi151.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-23T14:27:47.476 INFO:teuthology.orchestra.run.smithi151.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-23T14:27:47.476 INFO:teuthology.orchestra.run.smithi151.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-23T14:27:48.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:48 ntpd[17253]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:48.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:48 ntpd[17253]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:48.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:48 ntpd[17253]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:48.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:48 ntpd[17253]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:48.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:48 ntpd[17253]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:48.333 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:48 ntpd[17253]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:48.334 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:48 ntpd[17253]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-23T14:27:48.334 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:48 ntpd[17253]: can't open /var/log/ntpstats/peerstats.20240423: Permission denied 2024-04-23T14:27:48.334 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:48 ntpd[17253]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-23T14:27:48.334 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:48 ntpd[17253]: can't open /var/log/ntpstats/rawstats.20240423: Permission denied 2024-04-23T14:27:48.334 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:48 ntpd[17253]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-23T14:27:48.334 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:48 ntpd[17253]: can't open /var/log/ntpstats/peerstats.20240423: Permission denied 2024-04-23T14:27:48.335 INFO:teuthology.orchestra.run.smithi100.stdout:23 Apr 14:27:48 ntpd[17253]: ntpd: time slew -0.001038 s 2024-04-23T14:27:48.335 INFO:teuthology.orchestra.run.smithi100.stdout:ntpd: time slew -0.001038s 2024-04-23T14:27:48.335 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:48 ntpd[17253]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-23T14:27:48.335 INFO:teuthology.orchestra.run.smithi100.stderr:23 Apr 14:27:48 ntpd[17253]: can't open /var/log/ntpstats/loopstats.20240423: Permission denied 2024-04-23T14:27:48.443 INFO:teuthology.orchestra.run.smithi100.stdout: remote refid st t when poll reach delay offset jitter 2024-04-23T14:27:48.443 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================== 2024-04-23T14:27:48.443 INFO:teuthology.orchestra.run.smithi100.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-23T14:27:48.443 INFO:teuthology.orchestra.run.smithi100.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-23T14:27:48.443 INFO:teuthology.orchestra.run.smithi100.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-23T14:27:48.443 INFO:teuthology.orchestra.run.smithi100.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-23T14:27:48.444 INFO:teuthology.run_tasks:Running task install... 2024-04-23T14:27:48.453 DEBUG:teuthology.task.install:project ceph 2024-04-23T14:27:48.453 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0'}} 2024-04-23T14:27:48.453 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0'} 2024-04-23T14:27:48.454 INFO:teuthology.task.install:Using flavor: default 2024-04-23T14:27:48.463 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-23T14:27:48.463 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-23T14:27:48.464 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-key list | grep Ceph 2024-04-23T14:27:48.465 DEBUG:teuthology.orchestra.run.smithi151:> sudo apt-key list | grep Ceph 2024-04-23T14:27:48.568 INFO:teuthology.orchestra.run.smithi151.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-23T14:27:48.589 INFO:teuthology.orchestra.run.smithi100.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-23T14:27:48.619 INFO:teuthology.orchestra.run.smithi151.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-23T14:27:48.619 INFO:teuthology.orchestra.run.smithi151.stdout:uid [ unknown] Ceph.com (release key) 2024-04-23T14:27:48.620 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-23T14:27:48.620 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-23T14:27:48.621 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:27:48.640 INFO:teuthology.orchestra.run.smithi100.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-23T14:27:48.640 INFO:teuthology.orchestra.run.smithi100.stdout:uid [ unknown] Ceph.com (release key) 2024-04-23T14:27:48.641 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-23T14:27:48.641 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-23T14:27:48.641 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:27:48.902 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default/ 2024-04-23T14:27:48.902 INFO:teuthology.task.install.deb:Package version is 19.0.0-3126-gd44fddd9-1jammy 2024-04-23T14:27:48.942 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default/ 2024-04-23T14:27:48.942 INFO:teuthology.task.install.deb:Package version is 19.0.0-3126-gd44fddd9-1jammy 2024-04-23T14:27:49.068 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:27:49.068 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-23T14:27:49.088 DEBUG:teuthology.orchestra.run.smithi151:> sudo apt-get update 2024-04-23T14:27:49.091 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:27:49.091 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-23T14:27:49.114 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get update 2024-04-23T14:27:49.321 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-23T14:27:49.321 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-23T14:27:49.335 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-23T14:27:49.335 INFO:teuthology.orchestra.run.smithi100.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-23T14:27:49.347 INFO:teuthology.orchestra.run.smithi151.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-23T14:27:49.352 INFO:teuthology.orchestra.run.smithi100.stdout:Ign:3 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy InRelease 2024-04-23T14:27:49.362 INFO:teuthology.orchestra.run.smithi100.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-23T14:27:49.375 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-23T14:27:49.399 INFO:teuthology.orchestra.run.smithi100.stdout:Ign:6 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-23T14:27:49.422 INFO:teuthology.orchestra.run.smithi100.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,413 B] 2024-04-23T14:27:49.458 INFO:teuthology.orchestra.run.smithi151.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy InRelease 2024-04-23T14:27:49.460 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-04-23T14:27:49.511 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-23T14:27:49.518 INFO:teuthology.orchestra.run.smithi151.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-23T14:27:49.549 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-23T14:27:49.626 INFO:teuthology.orchestra.run.smithi151.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-23T14:27:49.695 INFO:teuthology.orchestra.run.smithi151.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,413 B] 2024-04-23T14:27:49.723 INFO:teuthology.orchestra.run.smithi151.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,608 kB] 2024-04-23T14:27:49.740 INFO:teuthology.orchestra.run.smithi100.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,608 kB] 2024-04-23T14:27:49.785 INFO:teuthology.orchestra.run.smithi151.stdout:Get:10 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-04-23T14:27:50.185 INFO:teuthology.orchestra.run.smithi151.stdout:Fetched 1,756 kB in 1s (2,015 kB/s) 2024-04-23T14:27:50.203 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 1,866 kB in 1s (2,166 kB/s) 2024-04-23T14:27:51.169 INFO:teuthology.orchestra.run.smithi151.stdout:Reading package lists... 2024-04-23T14:27:51.180 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-04-23T14:27:51.184 DEBUG:teuthology.orchestra.run.smithi151:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3126-gd44fddd9-1jammy cephadm=19.0.0-3126-gd44fddd9-1jammy ceph-mds=19.0.0-3126-gd44fddd9-1jammy ceph-mgr=19.0.0-3126-gd44fddd9-1jammy ceph-common=19.0.0-3126-gd44fddd9-1jammy ceph-fuse=19.0.0-3126-gd44fddd9-1jammy ceph-test=19.0.0-3126-gd44fddd9-1jammy ceph-volume=19.0.0-3126-gd44fddd9-1jammy radosgw=19.0.0-3126-gd44fddd9-1jammy python3-rados=19.0.0-3126-gd44fddd9-1jammy python3-rgw=19.0.0-3126-gd44fddd9-1jammy python3-cephfs=19.0.0-3126-gd44fddd9-1jammy python3-rbd=19.0.0-3126-gd44fddd9-1jammy libcephfs2=19.0.0-3126-gd44fddd9-1jammy libcephfs-dev=19.0.0-3126-gd44fddd9-1jammy librados2=19.0.0-3126-gd44fddd9-1jammy librbd1=19.0.0-3126-gd44fddd9-1jammy rbd-fuse=19.0.0-3126-gd44fddd9-1jammy python3-cephfs=19.0.0-3126-gd44fddd9-1jammy cephfs-shell=19.0.0-3126-gd44fddd9-1jammy cephfs-top=19.0.0-3126-gd44fddd9-1jammy cephfs-mirror=19.0.0-3126-gd44fddd9-1jammy 2024-04-23T14:27:51.194 DEBUG:teuthology.orchestra.run.smithi100:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3126-gd44fddd9-1jammy cephadm=19.0.0-3126-gd44fddd9-1jammy ceph-mds=19.0.0-3126-gd44fddd9-1jammy ceph-mgr=19.0.0-3126-gd44fddd9-1jammy ceph-common=19.0.0-3126-gd44fddd9-1jammy ceph-fuse=19.0.0-3126-gd44fddd9-1jammy ceph-test=19.0.0-3126-gd44fddd9-1jammy ceph-volume=19.0.0-3126-gd44fddd9-1jammy radosgw=19.0.0-3126-gd44fddd9-1jammy python3-rados=19.0.0-3126-gd44fddd9-1jammy python3-rgw=19.0.0-3126-gd44fddd9-1jammy python3-cephfs=19.0.0-3126-gd44fddd9-1jammy python3-rbd=19.0.0-3126-gd44fddd9-1jammy libcephfs2=19.0.0-3126-gd44fddd9-1jammy libcephfs-dev=19.0.0-3126-gd44fddd9-1jammy librados2=19.0.0-3126-gd44fddd9-1jammy librbd1=19.0.0-3126-gd44fddd9-1jammy rbd-fuse=19.0.0-3126-gd44fddd9-1jammy python3-cephfs=19.0.0-3126-gd44fddd9-1jammy cephfs-shell=19.0.0-3126-gd44fddd9-1jammy cephfs-top=19.0.0-3126-gd44fddd9-1jammy cephfs-mirror=19.0.0-3126-gd44fddd9-1jammy 2024-04-23T14:27:51.257 INFO:teuthology.orchestra.run.smithi151.stdout:Reading package lists... 2024-04-23T14:27:51.269 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-04-23T14:27:51.431 INFO:teuthology.orchestra.run.smithi151.stdout:Building dependency tree... 2024-04-23T14:27:51.432 INFO:teuthology.orchestra.run.smithi151.stdout:Reading state information... 2024-04-23T14:27:51.442 INFO:teuthology.orchestra.run.smithi100.stdout:Building dependency tree... 2024-04-23T14:27:51.442 INFO:teuthology.orchestra.run.smithi100.stdout:Reading state information... 2024-04-23T14:27:51.603 INFO:teuthology.orchestra.run.smithi151.stdout:The following packages were automatically installed and are no longer required: 2024-04-23T14:27:51.603 INFO:teuthology.orchestra.run.smithi151.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-23T14:27:51.603 INFO:teuthology.orchestra.run.smithi151.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-23T14:27:51.604 INFO:teuthology.orchestra.run.smithi151.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-23T14:27:51.604 INFO:teuthology.orchestra.run.smithi151.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-23T14:27:51.604 INFO:teuthology.orchestra.run.smithi151.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-23T14:27:51.604 INFO:teuthology.orchestra.run.smithi151.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-23T14:27:51.604 INFO:teuthology.orchestra.run.smithi151.stdout:The following additional packages will be installed: 2024-04-23T14:27:51.604 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-23T14:27:51.606 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-23T14:27:51.606 INFO:teuthology.orchestra.run.smithi151.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-23T14:27:51.606 INFO:teuthology.orchestra.run.smithi151.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-23T14:27:51.606 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-23T14:27:51.607 INFO:teuthology.orchestra.run.smithi151.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-23T14:27:51.607 INFO:teuthology.orchestra.run.smithi151.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-23T14:27:51.607 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-23T14:27:51.607 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-23T14:27:51.607 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-23T14:27:51.607 INFO:teuthology.orchestra.run.smithi151.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-23T14:27:51.607 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-23T14:27:51.607 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-23T14:27:51.607 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-23T14:27:51.607 INFO:teuthology.orchestra.run.smithi151.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-23T14:27:51.607 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-23T14:27:51.607 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-23T14:27:51.607 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-23T14:27:51.607 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-23T14:27:51.608 INFO:teuthology.orchestra.run.smithi151.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-23T14:27:51.608 INFO:teuthology.orchestra.run.smithi151.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-23T14:27:51.608 INFO:teuthology.orchestra.run.smithi151.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-23T14:27:51.608 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-23T14:27:51.608 INFO:teuthology.orchestra.run.smithi151.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-23T14:27:51.609 INFO:teuthology.orchestra.run.smithi151.stdout:Suggested packages: 2024-04-23T14:27:51.609 INFO:teuthology.orchestra.run.smithi151.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-23T14:27:51.609 INFO:teuthology.orchestra.run.smithi151.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-23T14:27:51.609 INFO:teuthology.orchestra.run.smithi151.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-23T14:27:51.609 INFO:teuthology.orchestra.run.smithi151.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-23T14:27:51.609 INFO:teuthology.orchestra.run.smithi151.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-23T14:27:51.609 INFO:teuthology.orchestra.run.smithi151.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-23T14:27:51.609 INFO:teuthology.orchestra.run.smithi151.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-23T14:27:51.609 INFO:teuthology.orchestra.run.smithi151.stdout:Recommended packages: 2024-04-23T14:27:51.609 INFO:teuthology.orchestra.run.smithi151.stdout: btrfs-tools 2024-04-23T14:27:51.617 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages were automatically installed and are no longer required: 2024-04-23T14:27:51.617 INFO:teuthology.orchestra.run.smithi100.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-23T14:27:51.617 INFO:teuthology.orchestra.run.smithi100.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-23T14:27:51.617 INFO:teuthology.orchestra.run.smithi100.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-23T14:27:51.617 INFO:teuthology.orchestra.run.smithi100.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-23T14:27:51.618 INFO:teuthology.orchestra.run.smithi100.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-23T14:27:51.618 INFO:teuthology.orchestra.run.smithi100.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-23T14:27:51.618 INFO:teuthology.orchestra.run.smithi100.stdout:The following additional packages will be installed: 2024-04-23T14:27:51.618 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-23T14:27:51.619 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-23T14:27:51.619 INFO:teuthology.orchestra.run.smithi100.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-23T14:27:51.619 INFO:teuthology.orchestra.run.smithi100.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-23T14:27:51.621 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-23T14:27:51.621 INFO:teuthology.orchestra.run.smithi100.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-23T14:27:51.621 INFO:teuthology.orchestra.run.smithi100.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-23T14:27:51.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-23T14:27:51.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-23T14:27:51.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-23T14:27:51.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-23T14:27:51.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-23T14:27:51.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-23T14:27:51.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-23T14:27:51.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-23T14:27:51.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-23T14:27:51.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-23T14:27:51.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-23T14:27:51.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-23T14:27:51.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-23T14:27:51.623 INFO:teuthology.orchestra.run.smithi100.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-23T14:27:51.623 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-23T14:27:51.623 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-23T14:27:51.623 INFO:teuthology.orchestra.run.smithi100.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-23T14:27:51.623 INFO:teuthology.orchestra.run.smithi100.stdout:Suggested packages: 2024-04-23T14:27:51.623 INFO:teuthology.orchestra.run.smithi100.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-23T14:27:51.623 INFO:teuthology.orchestra.run.smithi100.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-23T14:27:51.623 INFO:teuthology.orchestra.run.smithi100.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-23T14:27:51.623 INFO:teuthology.orchestra.run.smithi100.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-23T14:27:51.623 INFO:teuthology.orchestra.run.smithi100.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-23T14:27:51.623 INFO:teuthology.orchestra.run.smithi100.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-23T14:27:51.623 INFO:teuthology.orchestra.run.smithi100.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-23T14:27:51.623 INFO:teuthology.orchestra.run.smithi100.stdout:Recommended packages: 2024-04-23T14:27:51.623 INFO:teuthology.orchestra.run.smithi100.stdout: btrfs-tools 2024-04-23T14:27:51.757 INFO:teuthology.orchestra.run.smithi151.stdout:The following NEW packages will be installed: 2024-04-23T14:27:51.757 INFO:teuthology.orchestra.run.smithi151.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-23T14:27:51.757 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-23T14:27:51.757 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-23T14:27:51.758 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-23T14:27:51.758 INFO:teuthology.orchestra.run.smithi151.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-23T14:27:51.758 INFO:teuthology.orchestra.run.smithi151.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-23T14:27:51.758 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-23T14:27:51.758 INFO:teuthology.orchestra.run.smithi151.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-23T14:27:51.758 INFO:teuthology.orchestra.run.smithi151.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-23T14:27:51.758 INFO:teuthology.orchestra.run.smithi151.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-23T14:27:51.758 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-23T14:27:51.758 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-23T14:27:51.758 INFO:teuthology.orchestra.run.smithi151.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-23T14:27:51.758 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-23T14:27:51.758 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-23T14:27:51.758 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-23T14:27:51.758 INFO:teuthology.orchestra.run.smithi151.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-23T14:27:51.759 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-23T14:27:51.759 INFO:teuthology.orchestra.run.smithi151.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-23T14:27:51.759 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-23T14:27:51.759 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-23T14:27:51.759 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-23T14:27:51.759 INFO:teuthology.orchestra.run.smithi151.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-23T14:27:51.759 INFO:teuthology.orchestra.run.smithi151.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-23T14:27:51.759 INFO:teuthology.orchestra.run.smithi151.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-23T14:27:51.759 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-23T14:27:51.760 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-23T14:27:51.760 INFO:teuthology.orchestra.run.smithi151.stdout: socat unzip xmlstarlet zip 2024-04-23T14:27:51.769 INFO:teuthology.orchestra.run.smithi100.stdout:The following NEW packages will be installed: 2024-04-23T14:27:51.770 INFO:teuthology.orchestra.run.smithi100.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-23T14:27:51.770 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-23T14:27:51.770 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-23T14:27:51.770 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-23T14:27:51.772 INFO:teuthology.orchestra.run.smithi100.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-23T14:27:51.772 INFO:teuthology.orchestra.run.smithi100.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-23T14:27:51.772 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-23T14:27:51.772 INFO:teuthology.orchestra.run.smithi100.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-23T14:27:51.772 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-23T14:27:51.772 INFO:teuthology.orchestra.run.smithi100.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-23T14:27:51.772 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-23T14:27:51.772 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-23T14:27:51.772 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-23T14:27:51.772 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-23T14:27:51.772 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-23T14:27:51.772 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-23T14:27:51.772 INFO:teuthology.orchestra.run.smithi100.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-23T14:27:51.773 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-23T14:27:51.773 INFO:teuthology.orchestra.run.smithi100.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-23T14:27:51.773 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-23T14:27:51.773 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-23T14:27:51.773 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-23T14:27:51.773 INFO:teuthology.orchestra.run.smithi100.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-23T14:27:51.773 INFO:teuthology.orchestra.run.smithi100.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-23T14:27:51.773 INFO:teuthology.orchestra.run.smithi100.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-23T14:27:51.773 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-23T14:27:51.773 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-23T14:27:51.773 INFO:teuthology.orchestra.run.smithi100.stdout: socat unzip xmlstarlet zip 2024-04-23T14:27:51.853 INFO:teuthology.orchestra.run.smithi151.stdout:0 upgraded, 124 newly installed, 0 to remove and 305 not upgraded. 2024-04-23T14:27:51.853 INFO:teuthology.orchestra.run.smithi151.stdout:Need to get 182 MB of archives. 2024-04-23T14:27:51.853 INFO:teuthology.orchestra.run.smithi151.stdout:After this operation, 830 MB of additional disk space will be used. 2024-04-23T14:27:51.853 INFO:teuthology.orchestra.run.smithi151.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-23T14:27:51.872 INFO:teuthology.orchestra.run.smithi100.stdout:0 upgraded, 124 newly installed, 0 to remove and 305 not upgraded. 2024-04-23T14:27:51.872 INFO:teuthology.orchestra.run.smithi100.stdout:Need to get 182 MB of archives. 2024-04-23T14:27:51.872 INFO:teuthology.orchestra.run.smithi100.stdout:After this operation, 830 MB of additional disk space will be used. 2024-04-23T14:27:51.872 INFO:teuthology.orchestra.run.smithi100.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-23T14:27:51.887 INFO:teuthology.orchestra.run.smithi151.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-23T14:27:51.900 INFO:teuthology.orchestra.run.smithi151.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-23T14:27:51.903 INFO:teuthology.orchestra.run.smithi100.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-23T14:27:51.921 INFO:teuthology.orchestra.run.smithi100.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-23T14:27:51.988 INFO:teuthology.orchestra.run.smithi151.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-23T14:27:52.000 INFO:teuthology.orchestra.run.smithi100.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-23T14:27:52.026 INFO:teuthology.orchestra.run.smithi151.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-23T14:27:52.035 INFO:teuthology.orchestra.run.smithi151.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-23T14:27:52.047 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-23T14:27:52.051 INFO:teuthology.orchestra.run.smithi100.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-23T14:27:52.052 INFO:teuthology.orchestra.run.smithi151.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-23T14:27:52.087 INFO:teuthology.orchestra.run.smithi100.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-23T14:27:52.175 INFO:teuthology.orchestra.run.smithi151.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-23T14:27:52.180 INFO:teuthology.orchestra.run.smithi151.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-23T14:27:52.210 INFO:teuthology.orchestra.run.smithi151.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-23T14:27:52.220 INFO:teuthology.orchestra.run.smithi151.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-23T14:27:52.221 INFO:teuthology.orchestra.run.smithi151.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-23T14:27:52.234 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-23T14:27:52.242 INFO:teuthology.orchestra.run.smithi151.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-23T14:27:52.243 INFO:teuthology.orchestra.run.smithi100.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-23T14:27:52.251 INFO:teuthology.orchestra.run.smithi151.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-23T14:27:52.256 INFO:teuthology.orchestra.run.smithi151.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-23T14:27:52.258 INFO:teuthology.orchestra.run.smithi151.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-23T14:27:52.259 INFO:teuthology.orchestra.run.smithi151.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-23T14:27:52.264 INFO:teuthology.orchestra.run.smithi100.stdout:Get:10 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3126-gd44fddd9-1jammy [3,553 kB] 2024-04-23T14:27:52.266 INFO:teuthology.orchestra.run.smithi151.stdout:Get:18 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3126-gd44fddd9-1jammy [3,553 kB] 2024-04-23T14:27:52.322 INFO:teuthology.orchestra.run.smithi100.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-23T14:27:52.337 INFO:teuthology.orchestra.run.smithi100.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-23T14:27:52.338 INFO:teuthology.orchestra.run.smithi100.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-23T14:27:52.344 INFO:teuthology.orchestra.run.smithi100.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-23T14:27:52.346 INFO:teuthology.orchestra.run.smithi100.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-23T14:27:52.346 INFO:teuthology.orchestra.run.smithi151.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-23T14:27:52.348 INFO:teuthology.orchestra.run.smithi100.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-23T14:27:52.349 INFO:teuthology.orchestra.run.smithi151.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-23T14:27:52.349 INFO:teuthology.orchestra.run.smithi100.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-23T14:27:52.349 INFO:teuthology.orchestra.run.smithi100.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-23T14:27:52.350 INFO:teuthology.orchestra.run.smithi151.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-23T14:27:52.353 INFO:teuthology.orchestra.run.smithi100.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-23T14:27:52.353 INFO:teuthology.orchestra.run.smithi151.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-23T14:27:52.353 INFO:teuthology.orchestra.run.smithi151.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-23T14:27:52.353 INFO:teuthology.orchestra.run.smithi151.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-23T14:27:52.353 INFO:teuthology.orchestra.run.smithi151.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-23T14:27:52.354 INFO:teuthology.orchestra.run.smithi151.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-23T14:27:52.354 INFO:teuthology.orchestra.run.smithi151.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-23T14:27:52.376 INFO:teuthology.orchestra.run.smithi100.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-23T14:27:52.379 INFO:teuthology.orchestra.run.smithi100.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-23T14:27:52.380 INFO:teuthology.orchestra.run.smithi100.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-23T14:27:52.381 INFO:teuthology.orchestra.run.smithi151.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-23T14:27:52.389 INFO:teuthology.orchestra.run.smithi151.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-23T14:27:52.390 INFO:teuthology.orchestra.run.smithi151.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-23T14:27:52.395 INFO:teuthology.orchestra.run.smithi151.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-23T14:27:52.395 INFO:teuthology.orchestra.run.smithi151.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-23T14:27:52.396 INFO:teuthology.orchestra.run.smithi151.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-23T14:27:52.396 INFO:teuthology.orchestra.run.smithi151.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-23T14:27:52.397 INFO:teuthology.orchestra.run.smithi151.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-23T14:27:52.398 INFO:teuthology.orchestra.run.smithi151.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-23T14:27:52.409 INFO:teuthology.orchestra.run.smithi100.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-23T14:27:52.409 INFO:teuthology.orchestra.run.smithi100.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-23T14:27:52.409 INFO:teuthology.orchestra.run.smithi100.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-23T14:27:52.409 INFO:teuthology.orchestra.run.smithi100.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-23T14:27:52.409 INFO:teuthology.orchestra.run.smithi100.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-23T14:27:52.410 INFO:teuthology.orchestra.run.smithi100.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-23T14:27:52.410 INFO:teuthology.orchestra.run.smithi100.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-23T14:27:52.410 INFO:teuthology.orchestra.run.smithi151.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-23T14:27:52.426 INFO:teuthology.orchestra.run.smithi151.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-23T14:27:52.427 INFO:teuthology.orchestra.run.smithi151.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-23T14:27:52.428 INFO:teuthology.orchestra.run.smithi151.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-23T14:27:52.438 INFO:teuthology.orchestra.run.smithi151.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-23T14:27:52.438 INFO:teuthology.orchestra.run.smithi151.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-23T14:27:52.439 INFO:teuthology.orchestra.run.smithi151.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-23T14:27:52.441 INFO:teuthology.orchestra.run.smithi151.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-23T14:27:52.448 INFO:teuthology.orchestra.run.smithi151.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-23T14:27:52.449 INFO:teuthology.orchestra.run.smithi100.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-23T14:27:52.455 INFO:teuthology.orchestra.run.smithi151.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-23T14:27:52.455 INFO:teuthology.orchestra.run.smithi100.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-23T14:27:52.457 INFO:teuthology.orchestra.run.smithi100.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-23T14:27:52.474 INFO:teuthology.orchestra.run.smithi151.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-23T14:27:52.476 INFO:teuthology.orchestra.run.smithi151.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-23T14:27:52.477 INFO:teuthology.orchestra.run.smithi151.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-23T14:27:52.478 INFO:teuthology.orchestra.run.smithi100.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-23T14:27:52.478 INFO:teuthology.orchestra.run.smithi100.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-23T14:27:52.479 INFO:teuthology.orchestra.run.smithi100.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-23T14:27:52.480 INFO:teuthology.orchestra.run.smithi100.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-23T14:27:52.480 INFO:teuthology.orchestra.run.smithi100.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-23T14:27:52.481 INFO:teuthology.orchestra.run.smithi151.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-23T14:27:52.482 INFO:teuthology.orchestra.run.smithi151.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-23T14:27:52.482 INFO:teuthology.orchestra.run.smithi100.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-23T14:27:52.483 INFO:teuthology.orchestra.run.smithi151.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-23T14:27:52.484 INFO:teuthology.orchestra.run.smithi100.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-23T14:27:52.488 INFO:teuthology.orchestra.run.smithi151.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-23T14:27:52.523 INFO:teuthology.orchestra.run.smithi100.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-23T14:27:52.533 INFO:teuthology.orchestra.run.smithi100.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-23T14:27:52.533 INFO:teuthology.orchestra.run.smithi100.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-23T14:27:52.576 INFO:teuthology.orchestra.run.smithi100.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-23T14:27:52.576 INFO:teuthology.orchestra.run.smithi100.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-23T14:27:52.579 INFO:teuthology.orchestra.run.smithi100.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-23T14:27:52.583 INFO:teuthology.orchestra.run.smithi100.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-23T14:27:52.583 INFO:teuthology.orchestra.run.smithi100.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-23T14:27:52.585 INFO:teuthology.orchestra.run.smithi100.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-23T14:27:52.586 INFO:teuthology.orchestra.run.smithi100.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-23T14:27:52.593 INFO:teuthology.orchestra.run.smithi151.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-23T14:27:52.594 INFO:teuthology.orchestra.run.smithi151.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-23T14:27:52.597 INFO:teuthology.orchestra.run.smithi151.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-23T14:27:52.597 INFO:teuthology.orchestra.run.smithi151.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-23T14:27:52.598 INFO:teuthology.orchestra.run.smithi151.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-23T14:27:52.630 INFO:teuthology.orchestra.run.smithi100.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-23T14:27:52.630 INFO:teuthology.orchestra.run.smithi100.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-23T14:27:52.631 INFO:teuthology.orchestra.run.smithi100.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-23T14:27:52.677 INFO:teuthology.orchestra.run.smithi100.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-23T14:27:52.678 INFO:teuthology.orchestra.run.smithi151.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-23T14:27:52.680 INFO:teuthology.orchestra.run.smithi151.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-23T14:27:52.680 INFO:teuthology.orchestra.run.smithi151.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-23T14:27:52.697 INFO:teuthology.orchestra.run.smithi151.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-23T14:27:52.704 INFO:teuthology.orchestra.run.smithi151.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-23T14:27:52.704 INFO:teuthology.orchestra.run.smithi151.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-23T14:27:52.705 INFO:teuthology.orchestra.run.smithi151.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-23T14:27:52.706 INFO:teuthology.orchestra.run.smithi151.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-23T14:27:52.707 INFO:teuthology.orchestra.run.smithi151.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-23T14:27:52.741 INFO:teuthology.orchestra.run.smithi151.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-23T14:27:52.742 INFO:teuthology.orchestra.run.smithi151.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-23T14:27:52.746 INFO:teuthology.orchestra.run.smithi151.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-23T14:27:52.749 INFO:teuthology.orchestra.run.smithi151.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-23T14:27:52.772 INFO:teuthology.orchestra.run.smithi100.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-23T14:27:52.773 INFO:teuthology.orchestra.run.smithi100.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-23T14:27:52.774 INFO:teuthology.orchestra.run.smithi100.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-23T14:27:52.774 INFO:teuthology.orchestra.run.smithi100.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-23T14:27:52.775 INFO:teuthology.orchestra.run.smithi100.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-23T14:27:52.791 INFO:teuthology.orchestra.run.smithi151.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-23T14:27:52.792 INFO:teuthology.orchestra.run.smithi151.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-23T14:27:52.798 INFO:teuthology.orchestra.run.smithi151.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-23T14:27:52.802 INFO:teuthology.orchestra.run.smithi151.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-23T14:27:52.803 INFO:teuthology.orchestra.run.smithi151.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-23T14:27:52.804 INFO:teuthology.orchestra.run.smithi151.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-23T14:27:52.820 INFO:teuthology.orchestra.run.smithi100.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-23T14:27:52.822 INFO:teuthology.orchestra.run.smithi100.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-23T14:27:52.823 INFO:teuthology.orchestra.run.smithi100.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-23T14:27:52.847 INFO:teuthology.orchestra.run.smithi151.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-23T14:27:52.848 INFO:teuthology.orchestra.run.smithi151.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-23T14:27:52.850 INFO:teuthology.orchestra.run.smithi151.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-23T14:27:52.853 INFO:teuthology.orchestra.run.smithi151.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-23T14:27:52.887 INFO:teuthology.orchestra.run.smithi151.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-23T14:27:52.888 INFO:teuthology.orchestra.run.smithi151.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-23T14:27:52.889 INFO:teuthology.orchestra.run.smithi151.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-23T14:27:52.891 INFO:teuthology.orchestra.run.smithi151.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-23T14:27:52.892 INFO:teuthology.orchestra.run.smithi151.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-23T14:27:52.894 INFO:teuthology.orchestra.run.smithi100.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-23T14:27:52.894 INFO:teuthology.orchestra.run.smithi100.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-23T14:27:52.895 INFO:teuthology.orchestra.run.smithi100.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-23T14:27:52.896 INFO:teuthology.orchestra.run.smithi100.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-23T14:27:52.896 INFO:teuthology.orchestra.run.smithi100.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-23T14:27:52.897 INFO:teuthology.orchestra.run.smithi100.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-23T14:27:52.897 INFO:teuthology.orchestra.run.smithi100.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-23T14:27:52.898 INFO:teuthology.orchestra.run.smithi100.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-23T14:27:52.901 INFO:teuthology.orchestra.run.smithi151.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-23T14:27:52.902 INFO:teuthology.orchestra.run.smithi100.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-23T14:27:52.915 INFO:teuthology.orchestra.run.smithi151.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-23T14:27:52.916 INFO:teuthology.orchestra.run.smithi151.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-23T14:27:52.919 INFO:teuthology.orchestra.run.smithi151.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-23T14:27:52.920 INFO:teuthology.orchestra.run.smithi151.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-23T14:27:52.934 INFO:teuthology.orchestra.run.smithi100.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-23T14:27:52.949 INFO:teuthology.orchestra.run.smithi100.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-23T14:27:52.949 INFO:teuthology.orchestra.run.smithi100.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-23T14:27:52.953 INFO:teuthology.orchestra.run.smithi100.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-23T14:27:52.956 INFO:teuthology.orchestra.run.smithi100.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-23T14:27:52.956 INFO:teuthology.orchestra.run.smithi100.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-23T14:27:52.958 INFO:teuthology.orchestra.run.smithi100.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-23T14:27:52.958 INFO:teuthology.orchestra.run.smithi100.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-23T14:27:52.963 INFO:teuthology.orchestra.run.smithi100.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-23T14:27:52.969 INFO:teuthology.orchestra.run.smithi100.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-23T14:27:52.985 INFO:teuthology.orchestra.run.smithi100.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-23T14:27:52.985 INFO:teuthology.orchestra.run.smithi100.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-23T14:27:52.986 INFO:teuthology.orchestra.run.smithi100.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-23T14:27:52.986 INFO:teuthology.orchestra.run.smithi100.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-23T14:27:52.988 INFO:teuthology.orchestra.run.smithi100.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-23T14:27:52.989 INFO:teuthology.orchestra.run.smithi100.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-23T14:27:53.000 INFO:teuthology.orchestra.run.smithi100.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-23T14:27:53.001 INFO:teuthology.orchestra.run.smithi100.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-23T14:27:53.001 INFO:teuthology.orchestra.run.smithi100.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-23T14:27:53.024 INFO:teuthology.orchestra.run.smithi100.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-23T14:27:53.025 INFO:teuthology.orchestra.run.smithi100.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-23T14:27:53.138 INFO:teuthology.orchestra.run.smithi100.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3126-gd44fddd9-1jammy [3,224 kB] 2024-04-23T14:27:53.152 INFO:teuthology.orchestra.run.smithi151.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3126-gd44fddd9-1jammy [3,224 kB] 2024-04-23T14:27:53.640 INFO:teuthology.orchestra.run.smithi100.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3126-gd44fddd9-1jammy [712 kB] 2024-04-23T14:27:53.809 INFO:teuthology.orchestra.run.smithi151.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3126-gd44fddd9-1jammy [712 kB] 2024-04-23T14:27:53.816 INFO:teuthology.orchestra.run.smithi151.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3126-gd44fddd9-1jammy [356 kB] 2024-04-23T14:27:53.819 INFO:teuthology.orchestra.run.smithi100.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3126-gd44fddd9-1jammy [356 kB] 2024-04-23T14:27:53.855 INFO:teuthology.orchestra.run.smithi100.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3126-gd44fddd9-1jammy [32.8 kB] 2024-04-23T14:27:53.855 INFO:teuthology.orchestra.run.smithi100.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3126-gd44fddd9-1jammy [180 kB] 2024-04-23T14:27:53.862 INFO:teuthology.orchestra.run.smithi100.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3126-gd44fddd9-1jammy [66.9 kB] 2024-04-23T14:27:53.865 INFO:teuthology.orchestra.run.smithi100.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3126-gd44fddd9-1jammy [330 kB] 2024-04-23T14:27:53.883 INFO:teuthology.orchestra.run.smithi151.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3126-gd44fddd9-1jammy [32.8 kB] 2024-04-23T14:27:53.884 INFO:teuthology.orchestra.run.smithi151.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3126-gd44fddd9-1jammy [180 kB] 2024-04-23T14:27:53.937 INFO:teuthology.orchestra.run.smithi100.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3126-gd44fddd9-1jammy [6,889 kB] 2024-04-23T14:27:53.945 INFO:teuthology.orchestra.run.smithi151.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3126-gd44fddd9-1jammy [66.9 kB] 2024-04-23T14:27:53.949 INFO:teuthology.orchestra.run.smithi151.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3126-gd44fddd9-1jammy [330 kB] 2024-04-23T14:27:54.022 INFO:teuthology.orchestra.run.smithi151.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3126-gd44fddd9-1jammy [6,889 kB] 2024-04-23T14:27:55.458 INFO:teuthology.orchestra.run.smithi100.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3126-gd44fddd9-1jammy [112 kB] 2024-04-23T14:27:55.504 INFO:teuthology.orchestra.run.smithi100.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3126-gd44fddd9-1jammy [466 kB] 2024-04-23T14:27:55.584 INFO:teuthology.orchestra.run.smithi100.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3126-gd44fddd9-1jammy [25.8 MB] 2024-04-23T14:27:55.663 INFO:teuthology.orchestra.run.smithi151.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3126-gd44fddd9-1jammy [112 kB] 2024-04-23T14:27:55.666 INFO:teuthology.orchestra.run.smithi151.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3126-gd44fddd9-1jammy [466 kB] 2024-04-23T14:27:55.812 INFO:teuthology.orchestra.run.smithi151.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3126-gd44fddd9-1jammy [25.8 MB] 2024-04-23T14:28:02.444 INFO:teuthology.orchestra.run.smithi151.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3126-gd44fddd9-1jammy [5,094 kB] 2024-04-23T14:28:03.691 INFO:teuthology.orchestra.run.smithi151.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3126-gd44fddd9-1jammy [244 kB] 2024-04-23T14:28:03.717 INFO:teuthology.orchestra.run.smithi100.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3126-gd44fddd9-1jammy [5,094 kB] 2024-04-23T14:28:03.753 INFO:teuthology.orchestra.run.smithi151.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3126-gd44fddd9-1jammy [124 kB] 2024-04-23T14:28:03.798 INFO:teuthology.orchestra.run.smithi151.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3126-gd44fddd9-1jammy [1,525 kB] 2024-04-23T14:28:04.188 INFO:teuthology.orchestra.run.smithi151.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3126-gd44fddd9-1jammy [6,258 kB] 2024-04-23T14:28:05.580 INFO:teuthology.orchestra.run.smithi100.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3126-gd44fddd9-1jammy [244 kB] 2024-04-23T14:28:05.664 INFO:teuthology.orchestra.run.smithi100.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3126-gd44fddd9-1jammy [124 kB] 2024-04-23T14:28:05.710 INFO:teuthology.orchestra.run.smithi100.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3126-gd44fddd9-1jammy [1,525 kB] 2024-04-23T14:28:05.806 INFO:teuthology.orchestra.run.smithi151.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3126-gd44fddd9-1jammy [22.8 MB] 2024-04-23T14:28:06.253 INFO:teuthology.orchestra.run.smithi100.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3126-gd44fddd9-1jammy [6,258 kB] 2024-04-23T14:28:08.305 INFO:teuthology.orchestra.run.smithi100.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3126-gd44fddd9-1jammy [22.8 MB] 2024-04-23T14:28:11.340 INFO:teuthology.orchestra.run.smithi151.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3126-gd44fddd9-1jammy [14.1 kB] 2024-04-23T14:28:11.340 INFO:teuthology.orchestra.run.smithi151.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3126-gd44fddd9-1jammy [870 kB] 2024-04-23T14:28:11.547 INFO:teuthology.orchestra.run.smithi151.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3126-gd44fddd9-1jammy [2,436 kB] 2024-04-23T14:28:12.153 INFO:teuthology.orchestra.run.smithi151.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3126-gd44fddd9-1jammy [791 kB] 2024-04-23T14:28:12.336 INFO:teuthology.orchestra.run.smithi151.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3126-gd44fddd9-1jammy [148 kB] 2024-04-23T14:28:12.392 INFO:teuthology.orchestra.run.smithi151.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3126-gd44fddd9-1jammy [2,297 kB] 2024-04-23T14:28:12.873 INFO:teuthology.orchestra.run.smithi151.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3126-gd44fddd9-1jammy [8,625 kB] 2024-04-23T14:28:13.178 INFO:teuthology.orchestra.run.smithi100.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3126-gd44fddd9-1jammy [14.1 kB] 2024-04-23T14:28:13.178 INFO:teuthology.orchestra.run.smithi100.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3126-gd44fddd9-1jammy [870 kB] 2024-04-23T14:28:13.321 INFO:teuthology.orchestra.run.smithi100.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3126-gd44fddd9-1jammy [2,436 kB] 2024-04-23T14:28:13.760 INFO:teuthology.orchestra.run.smithi100.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3126-gd44fddd9-1jammy [791 kB] 2024-04-23T14:28:13.897 INFO:teuthology.orchestra.run.smithi100.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3126-gd44fddd9-1jammy [148 kB] 2024-04-23T14:28:13.921 INFO:teuthology.orchestra.run.smithi100.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3126-gd44fddd9-1jammy [2,297 kB] 2024-04-23T14:28:14.310 INFO:teuthology.orchestra.run.smithi100.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3126-gd44fddd9-1jammy [8,625 kB] 2024-04-23T14:28:15.059 INFO:teuthology.orchestra.run.smithi151.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3126-gd44fddd9-1jammy [14.2 kB] 2024-04-23T14:28:15.059 INFO:teuthology.orchestra.run.smithi151.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3126-gd44fddd9-1jammy [51.6 MB] 2024-04-23T14:28:15.635 INFO:teuthology.orchestra.run.smithi100.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3126-gd44fddd9-1jammy [14.2 kB] 2024-04-23T14:28:15.635 INFO:teuthology.orchestra.run.smithi100.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3126-gd44fddd9-1jammy [51.6 MB] 2024-04-23T14:28:21.791 INFO:teuthology.orchestra.run.smithi100.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3126-gd44fddd9-1jammy [124 kB] 2024-04-23T14:28:21.792 INFO:teuthology.orchestra.run.smithi100.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-3126-gd44fddd9-1jammy [232 kB] 2024-04-23T14:28:21.821 INFO:teuthology.orchestra.run.smithi100.stdout:Get:120 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-3126-gd44fddd9-1jammy [33.9 kB] 2024-04-23T14:28:21.821 INFO:teuthology.orchestra.run.smithi100.stdout:Get:121 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-3126-gd44fddd9-1jammy [23.4 kB] 2024-04-23T14:28:21.825 INFO:teuthology.orchestra.run.smithi100.stdout:Get:122 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3126-gd44fddd9-1jammy [38.9 kB] 2024-04-23T14:28:21.826 INFO:teuthology.orchestra.run.smithi100.stdout:Get:123 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3126-gd44fddd9-1jammy [13.6 MB] 2024-04-23T14:28:23.573 INFO:teuthology.orchestra.run.smithi100.stdout:Get:124 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3126-gd44fddd9-1jammy [91.4 kB] 2024-04-23T14:28:24.213 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 182 MB in 32s (5,710 kB/s) 2024-04-23T14:28:24.818 INFO:teuthology.orchestra.run.smithi151.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3126-gd44fddd9-1jammy [124 kB] 2024-04-23T14:28:24.820 INFO:teuthology.orchestra.run.smithi151.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-3126-gd44fddd9-1jammy [232 kB] 2024-04-23T14:28:24.860 INFO:teuthology.orchestra.run.smithi151.stdout:Get:120 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-3126-gd44fddd9-1jammy [33.9 kB] 2024-04-23T14:28:24.862 INFO:teuthology.orchestra.run.smithi151.stdout:Get:121 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-3126-gd44fddd9-1jammy [23.4 kB] 2024-04-23T14:28:24.874 INFO:teuthology.orchestra.run.smithi151.stdout:Get:122 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3126-gd44fddd9-1jammy [38.9 kB] 2024-04-23T14:28:24.875 INFO:teuthology.orchestra.run.smithi151.stdout:Get:123 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3126-gd44fddd9-1jammy [13.6 MB] 2024-04-23T14:28:25.129 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-23T14:28:25.202 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132922 files and directories currently installed.) 2024-04-23T14:28:25.206 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-23T14:28:25.252 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-23T14:28:25.538 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-23T14:28:25.553 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-23T14:28:25.596 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-23T14:28:25.907 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cmd2. 2024-04-23T14:28:25.922 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-23T14:28:25.966 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-23T14:28:26.344 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-23T14:28:26.359 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-23T14:28:26.448 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-23T14:28:26.814 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-23T14:28:26.829 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-23T14:28:26.872 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-23T14:28:27.275 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-23T14:28:27.284 INFO:teuthology.orchestra.run.smithi151.stdout:Get:124 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3126-gd44fddd9-1jammy [91.4 kB] 2024-04-23T14:28:27.290 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-23T14:28:27.350 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-23T14:28:27.931 INFO:teuthology.orchestra.run.smithi151.stdout:Fetched 182 MB in 36s (5,112 kB/s) 2024-04-23T14:28:27.996 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-23T14:28:28.013 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-23T14:28:28.062 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-23T14:28:28.078 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-23T14:28:28.145 INFO:teuthology.orchestra.run.smithi151.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132922 files and directories currently installed.) 2024-04-23T14:28:28.148 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-23T14:28:28.194 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-23T14:28:28.497 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-23T14:28:28.512 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-23T14:28:28.515 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-23T14:28:28.530 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-23T14:28:28.555 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-23T14:28:28.573 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-23T14:28:28.851 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-cmd2. 2024-04-23T14:28:28.864 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-23T14:28:28.909 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-23T14:28:28.968 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-23T14:28:28.983 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-23T14:28:29.034 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-23T14:28:29.238 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-23T14:28:29.252 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-23T14:28:29.338 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-23T14:28:29.471 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-23T14:28:29.486 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-23T14:28:29.537 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-23T14:28:29.876 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-23T14:28:29.890 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-23T14:28:29.924 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package librados2. 2024-04-23T14:28:29.934 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-23T14:28:29.938 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../010-librados2_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:29.982 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librados2 (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:30.229 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-23T14:28:30.243 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-23T14:28:30.288 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-23T14:28:30.452 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package librbd1. 2024-04-23T14:28:30.466 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../011-librbd1_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:30.527 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librbd1 (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:30.725 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-23T14:28:30.738 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-23T14:28:30.783 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-23T14:28:31.004 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libcephfs2. 2024-04-23T14:28:31.019 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:31.071 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libcephfs2 (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:31.222 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-23T14:28:31.234 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-23T14:28:31.271 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-23T14:28:31.390 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rados. 2024-04-23T14:28:31.405 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../013-python3-rados_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:31.457 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rados (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:31.592 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-23T14:28:31.604 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-23T14:28:31.641 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-23T14:28:31.818 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-23T14:28:31.832 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-3126-gd44fddd9-1jammy_all.deb ... 2024-04-23T14:28:31.876 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-ceph-argparse (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:32.045 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-23T14:28:32.058 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-23T14:28:32.095 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-23T14:28:32.129 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cephfs. 2024-04-23T14:28:32.144 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:32.187 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cephfs (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:32.390 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package librados2. 2024-04-23T14:28:32.408 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../010-librados2_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:32.440 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking librados2 (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:32.465 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-23T14:28:32.479 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-3126-gd44fddd9-1jammy_all.deb ... 2024-04-23T14:28:32.514 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-ceph-common (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:32.817 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-prettytable. 2024-04-23T14:28:32.832 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-23T14:28:32.869 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package librbd1. 2024-04-23T14:28:32.883 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../011-librbd1_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:32.892 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-23T14:28:32.927 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking librbd1 (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:33.105 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rbd. 2024-04-23T14:28:33.118 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:33.161 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rbd (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:33.389 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libcephfs2. 2024-04-23T14:28:33.403 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:33.447 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libcephfs2 (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:33.481 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-23T14:28:33.496 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-23T14:28:33.539 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-23T14:28:33.734 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-rados. 2024-04-23T14:28:33.748 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../013-python3-rados_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:33.784 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-rados (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:33.834 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-23T14:28:33.848 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-23T14:28:33.892 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-23T14:28:34.095 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-23T14:28:34.109 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-3126-gd44fddd9-1jammy_all.deb ... 2024-04-23T14:28:34.154 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-ceph-argparse (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:34.178 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-23T14:28:34.193 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-23T14:28:34.229 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-23T14:28:34.482 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-cephfs. 2024-04-23T14:28:34.496 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:34.523 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package lua5.1. 2024-04-23T14:28:34.533 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-23T14:28:34.548 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-cephfs (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:34.573 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-23T14:28:34.901 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package lua-any. 2024-04-23T14:28:34.913 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-23T14:28:34.919 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-23T14:28:34.933 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-3126-gd44fddd9-1jammy_all.deb ... 2024-04-23T14:28:34.951 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-23T14:28:34.985 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-ceph-common (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:35.203 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package zip. 2024-04-23T14:28:35.214 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-23T14:28:35.253 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking zip (3.0-12build2) ... 2024-04-23T14:28:35.398 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-prettytable. 2024-04-23T14:28:35.412 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-23T14:28:35.464 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-23T14:28:35.506 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package unzip. 2024-04-23T14:28:35.517 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-23T14:28:35.550 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-23T14:28:35.776 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-rbd. 2024-04-23T14:28:35.790 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:35.843 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-rbd (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:35.977 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package luarocks. 2024-04-23T14:28:35.991 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-23T14:28:36.035 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-23T14:28:36.238 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-23T14:28:36.252 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-23T14:28:36.305 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-23T14:28:36.434 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package librgw2. 2024-04-23T14:28:36.447 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../027-librgw2_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:36.501 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librgw2 (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:36.650 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-23T14:28:36.664 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-23T14:28:36.716 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-23T14:28:37.058 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rgw. 2024-04-23T14:28:37.070 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:37.070 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-23T14:28:37.084 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-23T14:28:37.120 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rgw (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:37.128 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-23T14:28:37.515 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-23T14:28:37.516 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package lua5.1. 2024-04-23T14:28:37.527 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-23T14:28:37.529 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-23T14:28:37.573 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-23T14:28:37.582 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-23T14:28:37.911 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package lua-any. 2024-04-23T14:28:37.925 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-23T14:28:37.969 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-23T14:28:38.068 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libradosstriper1. 2024-04-23T14:28:38.080 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:38.125 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libradosstriper1 (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:38.226 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package zip. 2024-04-23T14:28:38.237 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-23T14:28:38.281 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking zip (3.0-12build2) ... 2024-04-23T14:28:38.627 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package unzip. 2024-04-23T14:28:38.640 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-23T14:28:38.662 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-common. 2024-04-23T14:28:38.675 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../031-ceph-common_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:38.685 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-23T14:28:38.721 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-common (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:39.081 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package luarocks. 2024-04-23T14:28:39.095 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-23T14:28:39.140 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-23T14:28:39.532 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package librgw2. 2024-04-23T14:28:39.545 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../027-librgw2_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:39.582 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking librgw2 (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:39.951 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-base. 2024-04-23T14:28:39.963 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../032-ceph-base_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:40.026 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-base (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:40.070 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-rgw. 2024-04-23T14:28:40.078 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:40.119 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-rgw (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:40.456 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-23T14:28:40.467 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-23T14:28:40.497 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-23T14:28:40.605 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-23T14:28:40.617 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-23T14:28:40.655 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-23T14:28:40.842 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libradosstriper1. 2024-04-23T14:28:40.853 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:40.901 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libradosstriper1 (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:40.916 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cheroot. 2024-04-23T14:28:40.927 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-23T14:28:40.966 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-23T14:28:41.261 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-23T14:28:41.273 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-23T14:28:41.302 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-23T14:28:41.330 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-common. 2024-04-23T14:28:41.341 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../031-ceph-common_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:41.380 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-common (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:41.622 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-23T14:28:41.634 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-23T14:28:41.672 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-23T14:28:41.974 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-23T14:28:41.987 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-23T14:28:42.024 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-23T14:28:42.311 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-tempora. 2024-04-23T14:28:42.323 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-23T14:28:42.360 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-23T14:28:42.647 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-portend. 2024-04-23T14:28:42.659 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-23T14:28:42.697 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-23T14:28:42.704 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-base. 2024-04-23T14:28:42.716 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../032-ceph-base_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:42.770 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-base (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:43.000 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-23T14:28:43.012 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-23T14:28:43.049 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-23T14:28:43.336 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-23T14:28:43.348 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-23T14:28:43.359 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-23T14:28:43.369 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-23T14:28:43.385 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-23T14:28:43.417 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-23T14:28:43.758 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-natsort. 2024-04-23T14:28:43.768 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-23T14:28:43.787 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-cheroot. 2024-04-23T14:28:43.798 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-23T14:28:43.805 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-23T14:28:43.846 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-23T14:28:44.117 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-logutils. 2024-04-23T14:28:44.129 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-23T14:28:44.169 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-23T14:28:44.258 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-23T14:28:44.269 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-23T14:28:44.316 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-23T14:28:44.462 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-23T14:28:44.474 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-23T14:28:44.511 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-23T14:28:44.753 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-23T14:28:44.765 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-23T14:28:44.811 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-23T14:28:44.947 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-mako. 2024-04-23T14:28:44.959 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-23T14:28:44.997 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-23T14:28:45.326 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-23T14:28:45.338 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-23T14:28:45.375 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-23T14:28:45.382 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-23T14:28:45.393 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-23T14:28:45.440 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-23T14:28:45.678 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-23T14:28:45.689 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-23T14:28:45.728 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-23T14:28:45.811 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-tempora. 2024-04-23T14:28:45.822 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-23T14:28:45.869 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-23T14:28:46.031 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-webob. 2024-04-23T14:28:46.042 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-23T14:28:46.081 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-23T14:28:46.264 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-portend. 2024-04-23T14:28:46.275 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-23T14:28:46.315 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-23T14:28:46.375 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-waitress. 2024-04-23T14:28:46.386 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-23T14:28:46.420 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-23T14:28:46.659 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-23T14:28:46.670 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-23T14:28:46.679 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-tempita. 2024-04-23T14:28:46.689 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-23T14:28:46.718 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-23T14:28:46.728 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-23T14:28:47.071 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-23T14:28:47.072 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-paste. 2024-04-23T14:28:47.082 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-23T14:28:47.083 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-23T14:28:47.123 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-23T14:28:47.129 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-23T14:28:47.434 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-23T14:28:47.447 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-23T14:28:47.484 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-23T14:28:47.559 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-natsort. 2024-04-23T14:28:47.569 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-23T14:28:47.617 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-23T14:28:47.779 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-23T14:28:47.790 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-23T14:28:47.828 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-23T14:28:47.979 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-logutils. 2024-04-23T14:28:47.990 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-23T14:28:48.038 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-23T14:28:48.090 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-webtest. 2024-04-23T14:28:48.102 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-23T14:28:48.148 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-23T14:28:48.358 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-23T14:28:48.369 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-23T14:28:48.418 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pecan. 2024-04-23T14:28:48.429 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-23T14:28:48.433 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-23T14:28:48.467 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-23T14:28:48.753 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-mako. 2024-04-23T14:28:48.764 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-23T14:28:48.803 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-23T14:28:48.814 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-certifi. 2024-04-23T14:28:48.825 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-23T14:28:48.864 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-23T14:28:49.132 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-23T14:28:49.143 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-23T14:28:49.182 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-23T14:28:49.200 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-idna. 2024-04-23T14:28:49.211 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-23T14:28:49.275 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-23T14:28:49.536 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-23T14:28:49.547 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-23T14:28:49.594 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-23T14:28:49.595 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-urllib3. 2024-04-23T14:28:49.606 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-23T14:28:49.645 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-23T14:28:49.948 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-webob. 2024-04-23T14:28:49.959 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-23T14:28:49.972 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-requests. 2024-04-23T14:28:49.986 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-23T14:28:50.006 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-23T14:28:50.031 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-23T14:28:50.384 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-23T14:28:50.396 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-23T14:28:50.426 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-waitress. 2024-04-23T14:28:50.433 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-23T14:28:50.437 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-23T14:28:50.487 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-23T14:28:50.728 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-23T14:28:50.741 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-3126-gd44fddd9-1jammy_all.deb ... 2024-04-23T14:28:50.778 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:50.864 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-tempita. 2024-04-23T14:28:50.875 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-23T14:28:50.913 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-23T14:28:51.074 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-23T14:28:51.086 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:51.124 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:51.283 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-paste. 2024-04-23T14:28:51.295 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-23T14:28:51.342 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-23T14:28:51.478 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-23T14:28:51.490 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-23T14:28:51.527 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-23T14:28:51.721 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-23T14:28:51.732 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-23T14:28:51.779 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-23T14:28:51.830 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-openssl. 2024-04-23T14:28:51.842 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-23T14:28:51.880 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-23T14:28:52.149 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-23T14:28:52.160 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-23T14:28:52.183 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr. 2024-04-23T14:28:52.196 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:52.207 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-23T14:28:52.224 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:52.528 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-webtest. 2024-04-23T14:28:52.540 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-23T14:28:52.578 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-23T14:28:52.611 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mon. 2024-04-23T14:28:52.624 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:52.661 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mon (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:52.923 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pecan. 2024-04-23T14:28:52.935 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-23T14:28:52.973 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-23T14:28:53.314 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-osd. 2024-04-23T14:28:53.321 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-certifi. 2024-04-23T14:28:53.326 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:53.331 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-23T14:28:53.371 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-23T14:28:53.506 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-osd (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:53.666 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-idna. 2024-04-23T14:28:53.680 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-23T14:28:53.724 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-23T14:28:54.070 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-urllib3. 2024-04-23T14:28:54.082 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-23T14:28:54.120 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-23T14:28:54.213 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph. 2024-04-23T14:28:54.220 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../068-ceph_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:54.259 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:54.499 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-requests. 2024-04-23T14:28:54.511 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-23T14:28:54.557 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-23T14:28:54.586 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-fuse. 2024-04-23T14:28:54.598 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:54.636 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-fuse (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:54.960 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-23T14:28:54.973 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-23T14:28:55.019 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-23T14:28:55.023 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mds. 2024-04-23T14:28:55.035 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:55.089 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mds (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:55.365 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-23T14:28:55.377 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-3126-gd44fddd9-1jammy_all.deb ... 2024-04-23T14:28:55.423 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:55.425 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package cephadm. 2024-04-23T14:28:55.438 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../071-cephadm_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:55.467 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking cephadm (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:55.778 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-23T14:28:55.790 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-23T14:28:55.812 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-23T14:28:55.824 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:55.861 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-23T14:28:55.862 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:56.156 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-23T14:28:56.167 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-23T14:28:56.174 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-23T14:28:56.186 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-23T14:28:56.205 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-23T14:28:56.216 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-23T14:28:56.519 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-openssl. 2024-04-23T14:28:56.531 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-23T14:28:56.569 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-23T14:28:56.891 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr. 2024-04-23T14:28:56.904 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:56.933 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:57.177 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-babel. 2024-04-23T14:28:57.188 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-23T14:28:57.220 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-23T14:28:57.337 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mon. 2024-04-23T14:28:57.349 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:57.387 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mon (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:57.548 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jinja2. 2024-04-23T14:28:57.560 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-23T14:28:57.598 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-23T14:28:57.918 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-23T14:28:57.929 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-3126-gd44fddd9-1jammy_all.deb ... 2024-04-23T14:28:57.968 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:57.974 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-osd. 2024-04-23T14:28:57.986 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:58.024 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-osd (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:58.297 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-23T14:28:58.308 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-23T14:28:58.346 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-23T14:28:58.632 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-routes. 2024-04-23T14:28:58.644 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-23T14:28:58.682 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-23T14:28:58.862 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph. 2024-04-23T14:28:58.874 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../068-ceph_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:58.920 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:59.019 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-23T14:28:59.031 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-3126-gd44fddd9-1jammy_all.deb ... 2024-04-23T14:28:59.069 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:59.323 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-fuse. 2024-04-23T14:28:59.336 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:59.390 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-fuse (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:28:59.810 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mds. 2024-04-23T14:28:59.823 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:28:59.869 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mds (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:00.299 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-23T14:29:00.310 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-23T14:29:00.316 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package cephadm. 2024-04-23T14:29:00.327 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../071-cephadm_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:29:00.349 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-23T14:29:00.372 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking cephadm (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:00.751 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-23T14:29:00.764 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-23T14:29:00.801 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-23T14:29:00.828 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-joblib. 2024-04-23T14:29:00.838 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-23T14:29:00.877 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-23T14:29:01.163 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-23T14:29:01.175 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-23T14:29:01.206 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-23T14:29:01.217 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-23T14:29:01.221 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-23T14:29:01.325 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-23T14:29:01.636 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-sklearn. 2024-04-23T14:29:01.647 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-23T14:29:01.678 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-23T14:29:02.210 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-babel. 2024-04-23T14:29:02.221 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-23T14:29:02.260 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-23T14:29:02.303 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-23T14:29:02.315 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-3126-gd44fddd9-1jammy_all.deb ... 2024-04-23T14:29:02.353 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:02.589 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-jinja2. 2024-04-23T14:29:02.600 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-23T14:29:02.656 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-23T14:29:03.001 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-23T14:29:03.012 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-3126-gd44fddd9-1jammy_all.deb ... 2024-04-23T14:29:03.051 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:03.265 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cachetools. 2024-04-23T14:29:03.281 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-23T14:29:03.315 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-23T14:29:03.405 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-23T14:29:03.416 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-23T14:29:03.464 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-23T14:29:03.626 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-23T14:29:03.640 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-23T14:29:03.684 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-23T14:29:03.767 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-routes. 2024-04-23T14:29:03.778 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-23T14:29:03.817 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-23T14:29:04.013 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-23T14:29:04.026 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-23T14:29:04.071 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-23T14:29:04.171 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-23T14:29:04.182 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-3126-gd44fddd9-1jammy_all.deb ... 2024-04-23T14:29:04.221 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:04.382 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rsa. 2024-04-23T14:29:04.396 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-23T14:29:04.440 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-23T14:29:04.778 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-google-auth. 2024-04-23T14:29:04.791 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-23T14:29:04.836 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-23T14:29:05.148 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-23T14:29:05.161 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-23T14:29:05.206 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-23T14:29:05.475 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-websocket. 2024-04-23T14:29:05.489 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-23T14:29:05.494 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-23T14:29:05.505 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-23T14:29:05.533 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-23T14:29:05.535 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-23T14:29:05.879 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-23T14:29:05.893 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-23T14:29:05.937 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-23T14:29:06.023 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-joblib. 2024-04-23T14:29:06.034 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-23T14:29:06.073 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-23T14:29:06.469 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-23T14:29:06.480 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-23T14:29:06.592 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-23T14:29:06.605 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-3126-gd44fddd9-1jammy_all.deb ... 2024-04-23T14:29:06.605 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-23T14:29:06.634 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:06.909 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-sklearn. 2024-04-23T14:29:06.920 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-23T14:29:06.958 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-23T14:29:06.987 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-23T14:29:06.999 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-23T14:29:07.036 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-23T14:29:07.373 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-23T14:29:07.386 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-23T14:29:07.423 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-23T14:29:07.628 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-23T14:29:07.640 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-3126-gd44fddd9-1jammy_all.deb ... 2024-04-23T14:29:07.670 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:07.684 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package jq. 2024-04-23T14:29:07.698 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-23T14:29:07.742 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-23T14:29:07.945 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package socat. 2024-04-23T14:29:07.959 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-23T14:29:08.003 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-23T14:29:08.323 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package xmlstarlet. 2024-04-23T14:29:08.337 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-23T14:29:08.381 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-23T14:29:08.707 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-cachetools. 2024-04-23T14:29:08.720 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-23T14:29:08.757 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-23T14:29:08.969 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-test. 2024-04-23T14:29:08.982 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../099-ceph-test_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:29:09.027 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-test (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:09.044 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-23T14:29:09.057 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-23T14:29:09.102 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-23T14:29:09.448 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-23T14:29:09.462 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-23T14:29:09.498 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-23T14:29:09.861 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-rsa. 2024-04-23T14:29:09.873 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-23T14:29:09.910 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-23T14:29:10.282 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-google-auth. 2024-04-23T14:29:10.294 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-23T14:29:10.340 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-23T14:29:10.744 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-23T14:29:10.756 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-23T14:29:10.802 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-23T14:29:11.007 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-volume. 2024-04-23T14:29:11.021 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-3126-gd44fddd9-1jammy_all.deb ... 2024-04-23T14:29:11.056 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-volume (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:11.181 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-websocket. 2024-04-23T14:29:11.194 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-23T14:29:11.248 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-23T14:29:11.436 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-23T14:29:11.449 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:29:11.494 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking cephfs-mirror (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:11.602 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-23T14:29:11.614 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-23T14:29:11.651 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-23T14:29:11.789 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-colorama. 2024-04-23T14:29:11.801 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-23T14:29:11.846 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-23T14:29:12.108 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package cephfs-shell. 2024-04-23T14:29:12.121 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-3126-gd44fddd9-1jammy_all.deb ... 2024-04-23T14:29:12.166 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking cephfs-shell (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:12.317 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-23T14:29:12.329 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-3126-gd44fddd9-1jammy_all.deb ... 2024-04-23T14:29:12.366 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:12.452 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package cephfs-top. 2024-04-23T14:29:12.465 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-3126-gd44fddd9-1jammy_all.deb ... 2024-04-23T14:29:12.510 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking cephfs-top (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:12.686 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-23T14:29:12.698 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-23T14:29:12.737 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-23T14:29:12.754 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-23T14:29:12.768 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:29:12.804 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libcephfs-dev (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:13.074 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-23T14:29:13.082 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-23T14:29:13.087 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-23T14:29:13.094 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-23T14:29:13.124 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-23T14:29:13.124 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-23T14:29:13.377 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package jq. 2024-04-23T14:29:13.389 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-23T14:29:13.427 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-23T14:29:13.453 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-23T14:29:13.467 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-23T14:29:13.536 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-23T14:29:13.680 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package socat. 2024-04-23T14:29:13.692 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-23T14:29:13.730 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-23T14:29:13.916 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package nvme-cli. 2024-04-23T14:29:13.928 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-23T14:29:13.965 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-23T14:29:14.114 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package xmlstarlet. 2024-04-23T14:29:14.122 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-23T14:29:14.167 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-23T14:29:14.395 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package pkg-config. 2024-04-23T14:29:14.408 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-23T14:29:14.445 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-23T14:29:14.573 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-test. 2024-04-23T14:29:14.584 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../099-ceph-test_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:29:14.631 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-test (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:14.715 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-23T14:29:14.728 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-23T14:29:14.756 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-23T14:29:15.160 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-23T14:29:15.172 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-23T14:29:15.201 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-23T14:29:15.487 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pastescript. 2024-04-23T14:29:15.500 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-23T14:29:15.537 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-23T14:29:15.883 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pluggy. 2024-04-23T14:29:15.895 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-23T14:29:15.932 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-23T14:29:16.243 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-psutil. 2024-04-23T14:29:16.256 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-23T14:29:16.293 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-23T14:29:16.630 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-py. 2024-04-23T14:29:16.642 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-23T14:29:16.679 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-23T14:29:17.028 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-volume. 2024-04-23T14:29:17.042 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pygments. 2024-04-23T14:29:17.042 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-3126-gd44fddd9-1jammy_all.deb ... 2024-04-23T14:29:17.053 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-23T14:29:17.078 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-volume (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:17.091 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-23T14:29:17.483 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-23T14:29:17.496 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:29:17.533 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking cephfs-mirror (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:17.553 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-23T14:29:17.566 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-23T14:29:17.604 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-23T14:29:17.828 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-colorama. 2024-04-23T14:29:17.841 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-23T14:29:17.886 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-23T14:29:17.931 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-toml. 2024-04-23T14:29:17.943 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-23T14:29:17.981 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-23T14:29:18.156 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package cephfs-shell. 2024-04-23T14:29:18.169 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-3126-gd44fddd9-1jammy_all.deb ... 2024-04-23T14:29:18.206 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking cephfs-shell (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:18.284 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pytest. 2024-04-23T14:29:18.296 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-23T14:29:18.325 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-23T14:29:18.510 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package cephfs-top. 2024-04-23T14:29:18.522 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-3126-gd44fddd9-1jammy_all.deb ... 2024-04-23T14:29:18.559 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking cephfs-top (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:18.654 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-simplejson. 2024-04-23T14:29:18.666 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-23T14:29:18.703 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-23T14:29:18.829 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-23T14:29:18.842 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:29:18.879 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libcephfs-dev (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:19.016 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-23T14:29:19.027 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-23T14:29:19.065 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-23T14:29:19.166 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-23T14:29:19.179 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-23T14:29:19.224 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-23T14:29:19.538 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-23T14:29:19.551 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-23T14:29:19.629 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-23T14:29:19.694 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package radosgw. 2024-04-23T14:29:19.706 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../122-radosgw_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:29:19.735 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking radosgw (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:20.026 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package nvme-cli. 2024-04-23T14:29:20.039 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-23T14:29:20.084 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-23T14:29:20.380 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package rbd-fuse. 2024-04-23T14:29:20.393 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:29:20.438 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking rbd-fuse (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:20.466 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package pkg-config. 2024-04-23T14:29:20.478 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-23T14:29:20.516 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-23T14:29:20.732 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-23T14:29:20.794 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-23T14:29:20.807 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-23T14:29:20.844 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-23T14:29:21.002 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-23T14:29:21.128 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-23T14:29:21.323 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-23T14:29:21.336 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-23T14:29:21.373 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-23T14:29:21.425 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-23T14:29:21.701 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pastescript. 2024-04-23T14:29:21.714 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-23T14:29:21.793 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-23T14:29:21.835 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-23T14:29:22.080 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pluggy. 2024-04-23T14:29:22.093 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-23T14:29:22.130 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-23T14:29:22.442 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-psutil. 2024-04-23T14:29:22.454 INFO:teuthology.orchestra.run.smithi100.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-23T14:29:22.455 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-23T14:29:22.492 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-23T14:29:22.510 INFO:teuthology.orchestra.run.smithi100.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-04-23T14:29:22.555 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up cephadm (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:22.870 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-py. 2024-04-23T14:29:22.883 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-23T14:29:22.920 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-23T14:29:23.098 INFO:teuthology.orchestra.run.smithi100.stdout:Adding system user cephadm....done 2024-04-23T14:29:23.244 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-23T14:29:23.267 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pygments. 2024-04-23T14:29:23.282 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-23T14:29:23.325 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-23T14:29:23.512 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-23T14:29:23.754 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-23T14:29:23.767 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-23T14:29:23.780 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-23T14:29:23.804 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-23T14:29:23.890 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-23T14:29:24.132 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-23T14:29:24.158 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-toml. 2024-04-23T14:29:24.172 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-23T14:29:24.216 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-23T14:29:24.503 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pytest. 2024-04-23T14:29:24.517 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-23T14:29:24.553 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-23T14:29:24.667 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-23T14:29:24.776 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-23T14:29:24.907 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-simplejson. 2024-04-23T14:29:24.921 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-23T14:29:24.965 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-23T14:29:25.078 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-23T14:29:25.277 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-23T14:29:25.291 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-23T14:29:25.327 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-23T14:29:25.446 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-23T14:29:25.706 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-23T14:29:25.966 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-23T14:29:26.199 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package radosgw. 2024-04-23T14:29:26.212 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../122-radosgw_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:29:26.250 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-23T14:29:26.265 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking radosgw (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:26.476 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-23T14:29:26.845 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-23T14:29:26.986 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package rbd-fuse. 2024-04-23T14:29:27.000 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-3126-gd44fddd9-1jammy_amd64.deb ... 2024-04-23T14:29:27.061 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking rbd-fuse (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:27.112 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-23T14:29:27.380 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-ceph-argparse (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:27.406 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-23T14:29:27.648 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-23T14:29:27.684 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-23T14:29:27.757 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-23T14:29:27.794 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-23T14:29:27.916 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-23T14:29:28.052 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-23T14:29:28.184 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-23T14:29:28.285 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-23T14:29:28.429 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-23T14:29:28.528 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-23T14:29:28.916 INFO:teuthology.orchestra.run.smithi100.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-23T14:29:28.975 INFO:teuthology.orchestra.run.smithi100.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-23T14:29:29.023 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-23T14:29:29.038 INFO:teuthology.orchestra.run.smithi151.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-23T14:29:29.079 INFO:teuthology.orchestra.run.smithi151.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-04-23T14:29:29.130 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up cephadm (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:29.140 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-23T14:29:29.509 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-23T14:29:29.617 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-23T14:29:29.869 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-23T14:29:30.059 INFO:teuthology.orchestra.run.smithi151.stdout:Adding system user cephadm....done 2024-04-23T14:29:30.136 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-23T14:29:30.230 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-23T14:29:30.446 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up zip (3.0-12build2) ... 2024-04-23T14:29:30.540 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-23T14:29:30.555 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-23T14:29:30.850 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-23T14:29:30.984 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-23T14:29:31.241 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-23T14:29:31.261 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-23T14:29:31.517 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-23T14:29:31.546 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-23T14:29:31.643 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-23T14:29:31.714 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-23T14:29:31.769 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-23T14:29:32.057 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-23T14:29:32.087 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-23T14:29:32.444 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-23T14:29:32.488 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-23T14:29:32.722 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-23T14:29:32.728 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-23T14:29:33.021 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-23T14:29:33.090 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-23T14:29:33.392 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-23T14:29:33.398 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-23T14:29:33.667 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-23T14:29:33.718 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-23T14:29:33.910 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-23T14:29:34.070 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-23T14:29:34.186 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-23T14:29:34.329 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-23T14:29:34.446 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-ceph-argparse (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:34.489 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-ceph-common (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:34.731 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-23T14:29:34.790 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-idna (3.3-1) ... 2024-04-23T14:29:34.882 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-23T14:29:35.058 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-23T14:29:35.132 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-23T14:29:35.343 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-23T14:29:35.494 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-23T14:29:35.762 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-23T14:29:36.185 INFO:teuthology.orchestra.run.smithi151.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-23T14:29:36.244 INFO:teuthology.orchestra.run.smithi151.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-23T14:29:36.318 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-23T14:29:36.355 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-23T14:29:36.469 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-23T14:29:36.623 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-23T14:29:36.758 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-23T14:29:36.854 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-23T14:29:36.980 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-23T14:29:37.034 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-23T14:29:37.160 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-23T14:29:37.272 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-23T14:29:37.285 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-23T14:29:37.557 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-23T14:29:37.570 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-23T14:29:37.847 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-23T14:29:37.891 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up zip (3.0-12build2) ... 2024-04-23T14:29:37.992 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-23T14:29:38.098 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-23T14:29:38.224 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-23T14:29:38.533 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-23T14:29:38.686 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-23T14:29:38.818 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-23T14:29:38.962 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-23T14:29:39.086 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-23T14:29:39.088 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-23T14:29:39.230 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-23T14:29:39.362 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-23T14:29:39.582 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-23T14:29:39.629 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-23T14:29:39.906 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-23T14:29:39.999 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-23T14:29:40.031 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-23T14:29:40.267 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-23T14:29:40.316 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-23T14:29:40.442 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-23T14:29:40.710 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-23T14:29:40.735 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-23T14:29:41.029 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-23T14:29:41.053 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-23T14:29:41.313 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-23T14:29:41.346 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-23T14:29:41.440 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-23T14:29:41.566 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-23T14:29:41.655 INFO:teuthology.orchestra.run.smithi100.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-23T14:29:41.706 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-23T14:29:41.893 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-23T14:29:41.973 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-23T14:29:42.044 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-23T14:29:42.082 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-ceph-common (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:42.376 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-idna (3.3-1) ... 2024-04-23T14:29:42.445 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-23T14:29:42.713 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-23T14:29:42.768 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-23T14:29:42.839 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-23T14:29:43.116 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-23T14:29:43.241 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-23T14:29:43.568 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-23T14:29:43.877 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-23T14:29:44.026 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-23T14:29:44.161 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-23T14:29:44.309 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-23T14:29:44.389 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-23T14:29:44.468 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-23T14:29:44.674 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-23T14:29:44.778 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-23T14:29:44.904 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-23T14:29:44.959 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-23T14:29:45.029 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-23T14:29:45.227 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-23T14:29:45.339 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-23T14:29:45.632 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-23T14:29:45.636 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-23T14:29:45.875 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-23T14:29:45.913 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-23T14:29:45.975 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-23T14:29:46.257 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-23T14:29:46.286 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-23T14:29:46.383 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librados2 (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:46.526 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:46.570 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-23T14:29:46.652 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-23T14:29:46.846 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-23T14:29:46.963 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libcephfs2 (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:47.089 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libradosstriper1 (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:47.155 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-23T14:29:47.215 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-23T14:29:47.439 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-23T14:29:47.525 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librbd1 (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:47.651 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-modules-core (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:47.715 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-23T14:29:47.777 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-fuse (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:47.816 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-23T14:29:47.985 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-23T14:29:47.985 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-23T14:29:48.084 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-23T14:29:48.235 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-23T14:29:48.553 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-23T14:29:48.654 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libcephfs-dev (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:48.780 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rados (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:48.872 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-23T14:29:48.889 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librgw2 (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:49.007 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up cephfs-top (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:49.133 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-23T14:29:49.138 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-23T14:29:49.289 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-23T14:29:49.440 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-23T14:29:49.545 INFO:teuthology.orchestra.run.smithi151.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-23T14:29:49.775 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-23T14:29:49.935 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-23T14:29:50.345 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-23T14:29:50.403 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rbd (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:50.512 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up rbd-fuse (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:50.629 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-23T14:29:50.630 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rgw (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:50.755 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-23T14:29:50.755 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cephfs (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:50.915 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up cephfs-shell (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:51.014 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-23T14:29:51.041 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-common (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:51.140 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-23T14:29:51.458 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-23T14:29:51.594 INFO:teuthology.orchestra.run.smithi100.stdout:Adding group ceph....done 2024-04-23T14:29:51.961 INFO:teuthology.orchestra.run.smithi100.stdout:Adding system user ceph....done 2024-04-23T14:29:52.114 INFO:teuthology.orchestra.run.smithi100.stdout:Setting system user ceph properties....done 2024-04-23T14:29:52.133 INFO:teuthology.orchestra.run.smithi100.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-23T14:29:52.167 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-23T14:29:52.272 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-23T14:29:52.469 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-23T14:29:52.684 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-23T14:29:52.713 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-23T14:29:52.998 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-23T14:29:53.290 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-23T14:29:53.401 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up cephfs-mirror (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:53.559 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-23T14:29:53.609 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-23T14:29:53.609 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-23T14:29:53.960 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-23T14:29:54.243 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-test (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:54.253 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-23T14:29:54.369 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up radosgw (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:54.605 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-23T14:29:54.756 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up librados2 (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:54.902 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-23T14:29:54.902 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-23T14:29:54.907 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:55.059 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-23T14:29:55.486 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libcephfs2 (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:55.550 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-base (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:55.654 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libradosstriper1 (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:55.780 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-23T14:29:55.902 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-23T14:29:56.081 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up librbd1 (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:56.182 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr-modules-core (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:56.308 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-fuse (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:56.500 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-23T14:29:56.500 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-23T14:29:56.558 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mds (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:56.797 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-23T14:29:56.798 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-23T14:29:57.110 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libcephfs-dev (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:57.236 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-rados (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:57.362 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up librgw2 (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:57.434 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:57.488 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up cephfs-top (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:57.630 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-23T14:29:57.663 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-23T14:29:57.663 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-23T14:29:58.277 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-osd (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:58.577 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-23T14:29:58.577 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-23T14:29:58.942 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-rbd (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:59.093 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up rbd-fuse (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:59.244 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-rgw (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:59.270 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:59.395 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-cephfs (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:59.396 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:59.530 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mon (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:59.539 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up cephfs-shell (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:59.664 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-common (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:29:59.747 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-23T14:29:59.747 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-23T14:30:00.300 INFO:teuthology.orchestra.run.smithi151.stdout:Adding group ceph....done 2024-04-23T14:30:00.382 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-cephadm (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:30:00.516 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:30:00.617 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-dashboard (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:30:00.743 INFO:teuthology.orchestra.run.smithi151.stdout:Adding system user ceph....done 2024-04-23T14:30:00.769 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-volume (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:30:00.929 INFO:teuthology.orchestra.run.smithi151.stdout:Setting system user ceph properties....done 2024-04-23T14:30:00.947 INFO:teuthology.orchestra.run.smithi151.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-23T14:30:01.093 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-23T14:30:01.113 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-23T14:30:01.213 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-23T14:30:01.366 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-23T14:30:01.494 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-23T14:30:02.181 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up cephfs-mirror (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:30:02.375 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-23T14:30:02.375 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-23T14:30:02.525 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-23T14:30:03.017 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-test (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:30:03.143 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up radosgw (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:30:03.214 INFO:teuthology.orchestra.run.smithi100.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-23T14:30:03.219 DEBUG:teuthology.orchestra.run.smithi100:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-23T14:30:03.293 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-04-23T14:30:03.468 INFO:teuthology.orchestra.run.smithi100.stdout:Building dependency tree... 2024-04-23T14:30:03.469 INFO:teuthology.orchestra.run.smithi100.stdout:Reading state information... 2024-04-23T14:30:03.642 INFO:teuthology.orchestra.run.smithi100.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-23T14:30:03.642 INFO:teuthology.orchestra.run.smithi100.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-23T14:30:03.642 INFO:teuthology.orchestra.run.smithi100.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-23T14:30:03.642 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages were automatically installed and are no longer required: 2024-04-23T14:30:03.642 INFO:teuthology.orchestra.run.smithi100.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-23T14:30:03.642 INFO:teuthology.orchestra.run.smithi100.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-23T14:30:03.642 INFO:teuthology.orchestra.run.smithi100.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-23T14:30:03.643 INFO:teuthology.orchestra.run.smithi100.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-23T14:30:03.643 INFO:teuthology.orchestra.run.smithi100.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-23T14:30:03.643 INFO:teuthology.orchestra.run.smithi100.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-23T14:30:03.644 INFO:teuthology.orchestra.run.smithi100.stdout:The following additional packages will be installed: 2024-04-23T14:30:03.644 INFO:teuthology.orchestra.run.smithi100.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-23T14:30:03.644 INFO:teuthology.orchestra.run.smithi100.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-23T14:30:03.644 INFO:teuthology.orchestra.run.smithi100.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-23T14:30:03.644 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-23T14:30:03.644 INFO:teuthology.orchestra.run.smithi100.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-23T14:30:03.645 INFO:teuthology.orchestra.run.smithi100.stdout:Suggested packages: 2024-04-23T14:30:03.645 INFO:teuthology.orchestra.run.smithi100.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-23T14:30:03.645 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-23T14:30:03.646 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-23T14:30:03.699 INFO:teuthology.orchestra.run.smithi100.stdout:The following NEW packages will be installed: 2024-04-23T14:30:03.700 INFO:teuthology.orchestra.run.smithi100.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-23T14:30:03.700 INFO:teuthology.orchestra.run.smithi100.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-23T14:30:03.700 INFO:teuthology.orchestra.run.smithi100.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-23T14:30:03.700 INFO:teuthology.orchestra.run.smithi100.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-23T14:30:03.700 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-23T14:30:03.701 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-23T14:30:03.701 INFO:teuthology.orchestra.run.smithi100.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-23T14:30:03.701 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages will be upgraded: 2024-04-23T14:30:03.702 INFO:teuthology.orchestra.run.smithi100.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-23T14:30:03.703 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-23T14:30:03.703 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-23T14:30:03.792 INFO:teuthology.orchestra.run.smithi100.stdout:4 upgraded, 39 newly installed, 0 to remove and 301 not upgraded. 2024-04-23T14:30:03.792 INFO:teuthology.orchestra.run.smithi100.stdout:Need to get 55.7 MB of archives. 2024-04-23T14:30:03.792 INFO:teuthology.orchestra.run.smithi100.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-23T14:30:03.792 INFO:teuthology.orchestra.run.smithi100.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-23T14:30:03.999 INFO:teuthology.orchestra.run.smithi100.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-23T14:30:04.004 INFO:teuthology.orchestra.run.smithi100.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-23T14:30:04.052 INFO:teuthology.orchestra.run.smithi100.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-23T14:30:04.052 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-23T14:30:04.053 INFO:teuthology.orchestra.run.smithi100.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-23T14:30:04.063 INFO:teuthology.orchestra.run.smithi100.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-23T14:30:04.063 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-23T14:30:04.065 INFO:teuthology.orchestra.run.smithi100.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-23T14:30:04.067 INFO:teuthology.orchestra.run.smithi100.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-23T14:30:04.067 INFO:teuthology.orchestra.run.smithi100.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-23T14:30:04.068 INFO:teuthology.orchestra.run.smithi100.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-23T14:30:04.084 INFO:teuthology.orchestra.run.smithi100.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-23T14:30:04.084 INFO:teuthology.orchestra.run.smithi100.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-23T14:30:04.084 INFO:teuthology.orchestra.run.smithi100.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-23T14:30:04.105 INFO:teuthology.orchestra.run.smithi100.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-23T14:30:04.361 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-base (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:30:04.475 INFO:teuthology.orchestra.run.smithi100.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-23T14:30:04.477 INFO:teuthology.orchestra.run.smithi100.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-23T14:30:04.477 INFO:teuthology.orchestra.run.smithi100.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-23T14:30:04.480 INFO:teuthology.orchestra.run.smithi100.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-23T14:30:04.480 INFO:teuthology.orchestra.run.smithi100.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-23T14:30:04.485 INFO:teuthology.orchestra.run.smithi100.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-23T14:30:04.487 INFO:teuthology.orchestra.run.smithi100.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-23T14:30:04.494 INFO:teuthology.orchestra.run.smithi100.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-23T14:30:04.494 INFO:teuthology.orchestra.run.smithi100.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-23T14:30:04.527 INFO:teuthology.orchestra.run.smithi100.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-23T14:30:04.535 INFO:teuthology.orchestra.run.smithi100.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-23T14:30:04.546 INFO:teuthology.orchestra.run.smithi100.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-23T14:30:04.573 INFO:teuthology.orchestra.run.smithi100.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-23T14:30:04.576 INFO:teuthology.orchestra.run.smithi100.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-23T14:30:04.577 INFO:teuthology.orchestra.run.smithi100.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-23T14:30:04.578 INFO:teuthology.orchestra.run.smithi100.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-23T14:30:04.578 INFO:teuthology.orchestra.run.smithi100.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-23T14:30:04.600 INFO:teuthology.orchestra.run.smithi100.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-23T14:30:04.612 INFO:teuthology.orchestra.run.smithi100.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-23T14:30:04.720 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-23T14:30:04.849 INFO:teuthology.orchestra.run.smithi100.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-23T14:30:04.850 INFO:teuthology.orchestra.run.smithi100.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-23T14:30:04.850 INFO:teuthology.orchestra.run.smithi100.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-23T14:30:04.850 INFO:teuthology.orchestra.run.smithi100.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-23T14:30:04.850 INFO:teuthology.orchestra.run.smithi100.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-23T14:30:04.850 INFO:teuthology.orchestra.run.smithi100.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-23T14:30:04.869 INFO:teuthology.orchestra.run.smithi100.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-23T14:30:04.869 INFO:teuthology.orchestra.run.smithi100.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-23T14:30:05.244 INFO:teuthology.orchestra.run.smithi100.stdout:Preconfiguring packages ... 2024-04-23T14:30:05.399 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 55.7 MB in 1s (48.1 MB/s) 2024-04-23T14:30:05.438 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mds (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:30:05.668 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-23T14:30:05.668 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-23T14:30:05.779 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140922 files and directories currently installed.) 2024-04-23T14:30:05.782 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-23T14:30:05.899 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-23T14:30:06.339 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:30:06.524 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-23T14:30:06.543 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-23T14:30:06.543 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-23T14:30:07.241 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-osd (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:30:07.567 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-23T14:30:07.567 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-23T14:30:07.608 INFO:teuthology.orchestra.run.smithi100.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-23T14:30:07.879 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package flex. 2024-04-23T14:30:07.948 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140922 files and directories currently installed.) 2024-04-23T14:30:07.952 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-23T14:30:07.994 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-23T14:30:08.201 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:30:08.327 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:30:08.470 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mon (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:30:08.506 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-23T14:30:08.678 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-23T14:30:08.679 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-23T14:30:08.684 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-23T14:30:09.055 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-23T14:30:09.329 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr-cephadm (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:30:09.472 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:30:09.581 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr-dashboard (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:30:09.623 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140995 files and directories currently installed.) 2024-04-23T14:30:09.627 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-23T14:30:09.699 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-volume (19.0.0-3126-gd44fddd9-1jammy) ... 2024-04-23T14:30:09.749 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-23T14:30:10.076 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-23T14:30:10.136 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-23T14:30:10.202 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-23T14:30:10.273 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-23T14:30:10.354 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-23T14:30:10.714 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-23T14:30:11.076 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package bison. 2024-04-23T14:30:11.144 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140995 files and directories currently installed.) 2024-04-23T14:30:11.148 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-23T14:30:11.194 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-23T14:30:11.421 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-23T14:30:11.604 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-23T14:30:11.618 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-23T14:30:11.670 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-23T14:30:12.007 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package dump. 2024-04-23T14:30:12.021 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-23T14:30:12.065 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking dump (0.4b47-3) ... 2024-04-23T14:30:12.123 INFO:teuthology.orchestra.run.smithi151.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-23T14:30:12.127 DEBUG:teuthology.orchestra.run.smithi151:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-23T14:30:12.201 INFO:teuthology.orchestra.run.smithi151.stdout:Reading package lists... 2024-04-23T14:30:12.344 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package indent. 2024-04-23T14:30:12.358 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-23T14:30:12.376 INFO:teuthology.orchestra.run.smithi151.stdout:Building dependency tree... 2024-04-23T14:30:12.377 INFO:teuthology.orchestra.run.smithi151.stdout:Reading state information... 2024-04-23T14:30:12.394 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-23T14:30:12.547 INFO:teuthology.orchestra.run.smithi151.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-23T14:30:12.547 INFO:teuthology.orchestra.run.smithi151.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-23T14:30:12.547 INFO:teuthology.orchestra.run.smithi151.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-23T14:30:12.547 INFO:teuthology.orchestra.run.smithi151.stdout:The following packages were automatically installed and are no longer required: 2024-04-23T14:30:12.547 INFO:teuthology.orchestra.run.smithi151.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-23T14:30:12.547 INFO:teuthology.orchestra.run.smithi151.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-23T14:30:12.547 INFO:teuthology.orchestra.run.smithi151.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-23T14:30:12.547 INFO:teuthology.orchestra.run.smithi151.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-23T14:30:12.547 INFO:teuthology.orchestra.run.smithi151.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-23T14:30:12.547 INFO:teuthology.orchestra.run.smithi151.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-23T14:30:12.548 INFO:teuthology.orchestra.run.smithi151.stdout:The following additional packages will be installed: 2024-04-23T14:30:12.548 INFO:teuthology.orchestra.run.smithi151.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-23T14:30:12.549 INFO:teuthology.orchestra.run.smithi151.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-23T14:30:12.549 INFO:teuthology.orchestra.run.smithi151.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-23T14:30:12.549 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-23T14:30:12.549 INFO:teuthology.orchestra.run.smithi151.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-23T14:30:12.549 INFO:teuthology.orchestra.run.smithi151.stdout:Suggested packages: 2024-04-23T14:30:12.549 INFO:teuthology.orchestra.run.smithi151.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-23T14:30:12.549 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-23T14:30:12.550 INFO:teuthology.orchestra.run.smithi151.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-23T14:30:12.603 INFO:teuthology.orchestra.run.smithi151.stdout:The following NEW packages will be installed: 2024-04-23T14:30:12.604 INFO:teuthology.orchestra.run.smithi151.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-23T14:30:12.604 INFO:teuthology.orchestra.run.smithi151.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-23T14:30:12.605 INFO:teuthology.orchestra.run.smithi151.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-23T14:30:12.605 INFO:teuthology.orchestra.run.smithi151.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-23T14:30:12.605 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-23T14:30:12.605 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-23T14:30:12.605 INFO:teuthology.orchestra.run.smithi151.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-23T14:30:12.606 INFO:teuthology.orchestra.run.smithi151.stdout:The following packages will be upgraded: 2024-04-23T14:30:12.607 INFO:teuthology.orchestra.run.smithi151.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-23T14:30:12.613 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-23T14:30:12.627 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-23T14:30:12.671 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-23T14:30:12.791 INFO:teuthology.orchestra.run.smithi151.stdout:4 upgraded, 39 newly installed, 0 to remove and 301 not upgraded. 2024-04-23T14:30:12.791 INFO:teuthology.orchestra.run.smithi151.stdout:Need to get 55.7 MB of archives. 2024-04-23T14:30:12.791 INFO:teuthology.orchestra.run.smithi151.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-23T14:30:12.791 INFO:teuthology.orchestra.run.smithi151.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-23T14:30:13.000 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-23T14:30:13.014 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-23T14:30:13.059 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-23T14:30:13.303 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-23T14:30:13.317 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-23T14:30:13.361 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-23T14:30:13.441 INFO:teuthology.orchestra.run.smithi151.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-23T14:30:13.457 INFO:teuthology.orchestra.run.smithi151.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-23T14:30:13.583 INFO:teuthology.orchestra.run.smithi151.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-23T14:30:13.583 INFO:teuthology.orchestra.run.smithi151.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-23T14:30:13.584 INFO:teuthology.orchestra.run.smithi151.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-23T14:30:13.616 INFO:teuthology.orchestra.run.smithi151.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-23T14:30:13.617 INFO:teuthology.orchestra.run.smithi151.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-23T14:30:13.620 INFO:teuthology.orchestra.run.smithi151.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-23T14:30:13.622 INFO:teuthology.orchestra.run.smithi151.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-23T14:30:13.623 INFO:teuthology.orchestra.run.smithi151.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-23T14:30:13.690 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-23T14:30:13.704 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-23T14:30:13.748 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-23T14:30:13.790 INFO:teuthology.orchestra.run.smithi151.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-23T14:30:14.009 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-23T14:30:14.023 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-23T14:30:14.068 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-23T14:30:14.082 INFO:teuthology.orchestra.run.smithi151.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-23T14:30:14.091 INFO:teuthology.orchestra.run.smithi151.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-23T14:30:14.096 INFO:teuthology.orchestra.run.smithi151.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-23T14:30:14.180 INFO:teuthology.orchestra.run.smithi151.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-23T14:30:14.295 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libjson-perl. 2024-04-23T14:30:14.309 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-23T14:30:14.354 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-23T14:30:14.657 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-23T14:30:14.671 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-23T14:30:14.715 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-23T14:30:15.148 INFO:teuthology.orchestra.run.smithi151.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-23T14:30:15.150 INFO:teuthology.orchestra.run.smithi151.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-23T14:30:15.150 INFO:teuthology.orchestra.run.smithi151.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-23T14:30:15.175 INFO:teuthology.orchestra.run.smithi151.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-23T14:30:15.176 INFO:teuthology.orchestra.run.smithi151.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-23T14:30:15.180 INFO:teuthology.orchestra.run.smithi151.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-23T14:30:15.346 INFO:teuthology.orchestra.run.smithi151.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-23T14:30:15.953 INFO:teuthology.orchestra.run.smithi151.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-23T14:30:15.954 INFO:teuthology.orchestra.run.smithi151.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-23T14:30:16.095 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-23T14:30:16.111 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-23T14:30:16.136 INFO:teuthology.orchestra.run.smithi151.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-23T14:30:16.136 INFO:teuthology.orchestra.run.smithi151.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-23T14:30:16.145 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-23T14:30:16.175 INFO:teuthology.orchestra.run.smithi151.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-23T14:30:16.237 INFO:teuthology.orchestra.run.smithi151.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-23T14:30:16.242 INFO:teuthology.orchestra.run.smithi151.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-23T14:30:16.243 INFO:teuthology.orchestra.run.smithi151.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-23T14:30:16.243 INFO:teuthology.orchestra.run.smithi151.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-23T14:30:16.244 INFO:teuthology.orchestra.run.smithi151.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-23T14:30:16.406 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-23T14:30:16.421 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-23T14:30:16.450 INFO:teuthology.orchestra.run.smithi151.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-23T14:30:16.464 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-23T14:30:16.792 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-23T14:30:16.806 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-23T14:30:16.853 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-23T14:30:16.872 INFO:teuthology.orchestra.run.smithi151.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-23T14:30:17.178 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-23T14:30:17.193 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-23T14:30:17.228 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-23T14:30:17.540 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-23T14:30:17.554 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-23T14:30:17.598 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-23T14:30:17.626 INFO:teuthology.orchestra.run.smithi151.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-23T14:30:17.627 INFO:teuthology.orchestra.run.smithi151.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-23T14:30:17.627 INFO:teuthology.orchestra.run.smithi151.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-23T14:30:17.627 INFO:teuthology.orchestra.run.smithi151.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-23T14:30:17.627 INFO:teuthology.orchestra.run.smithi151.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-23T14:30:17.628 INFO:teuthology.orchestra.run.smithi151.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-23T14:30:17.661 INFO:teuthology.orchestra.run.smithi151.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-23T14:30:17.662 INFO:teuthology.orchestra.run.smithi151.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-23T14:30:17.935 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-23T14:30:17.947 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-23T14:30:17.984 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-23T14:30:18.040 INFO:teuthology.orchestra.run.smithi151.stdout:Preconfiguring packages ... 2024-04-23T14:30:18.264 INFO:teuthology.orchestra.run.smithi151.stdout:Fetched 55.7 MB in 5s (11.0 MB/s) 2024-04-23T14:30:18.329 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-23T14:30:18.341 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-23T14:30:18.379 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-23T14:30:18.591 INFO:teuthology.orchestra.run.smithi151.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140922 files and directories currently installed.) 2024-04-23T14:30:18.594 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-23T14:30:18.656 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-23T14:30:18.672 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-23T14:30:18.715 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-23T14:30:18.763 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-23T14:30:18.959 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-23T14:30:18.972 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-23T14:30:19.017 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-23T14:30:19.412 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-23T14:30:19.471 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-23T14:30:19.484 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-23T14:30:19.521 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-23T14:30:19.874 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package modemmanager. 2024-04-23T14:30:19.887 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-23T14:30:19.932 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-23T14:30:20.514 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package network-manager. 2024-04-23T14:30:20.527 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-23T14:30:20.567 INFO:teuthology.orchestra.run.smithi151.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-23T14:30:20.579 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-23T14:30:20.858 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package flex. 2024-04-23T14:30:20.929 INFO:teuthology.orchestra.run.smithi151.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140922 files and directories currently installed.) 2024-04-23T14:30:20.932 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-23T14:30:20.981 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-23T14:30:21.240 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ppp. 2024-04-23T14:30:21.253 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-23T14:30:21.593 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-23T14:30:21.600 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-23T14:30:21.829 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-23T14:30:21.870 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package pptp-linux. 2024-04-23T14:30:21.883 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-23T14:30:21.928 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-23T14:30:22.180 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-23T14:30:22.194 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-23T14:30:22.230 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-23T14:30:22.269 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-23T14:30:22.533 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-23T14:30:22.549 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-23T14:30:22.592 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking postgresql-client-common (238) ... 2024-04-23T14:30:22.810 INFO:teuthology.orchestra.run.smithi151.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140995 files and directories currently installed.) 2024-04-23T14:30:22.812 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-23T14:30:22.895 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-23T14:30:22.910 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-23T14:30:22.945 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-23T14:30:22.978 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-23T14:30:23.425 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package postgresql-common. 2024-04-23T14:30:23.440 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-23T14:30:23.493 INFO:teuthology.orchestra.run.smithi100.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-23T14:30:23.524 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking postgresql-common (238) ... 2024-04-23T14:30:23.717 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-23T14:30:23.826 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-23T14:30:23.983 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package postgresql-14. 2024-04-23T14:30:23.997 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-23T14:30:24.038 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-23T14:30:24.168 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-23T14:30:24.568 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package bison. 2024-04-23T14:30:24.643 INFO:teuthology.orchestra.run.smithi151.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140995 files and directories currently installed.) 2024-04-23T14:30:24.647 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-23T14:30:24.704 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-23T14:30:25.127 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package postgresql. 2024-04-23T14:30:25.142 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-23T14:30:25.177 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking postgresql (14+238) ... 2024-04-23T14:30:25.246 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-23T14:30:25.261 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-23T14:30:25.312 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-23T14:30:25.421 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package postgresql-client. 2024-04-23T14:30:25.435 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-23T14:30:25.479 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking postgresql-client (14+238) ... 2024-04-23T14:30:25.599 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package dump. 2024-04-23T14:30:25.614 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-23T14:30:25.648 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking dump (0.4b47-3) ... 2024-04-23T14:30:25.723 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-23T14:30:25.738 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-23T14:30:25.782 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-23T14:30:25.944 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package indent. 2024-04-23T14:30:25.960 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-23T14:30:26.002 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-23T14:30:26.297 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-23T14:30:26.313 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-23T14:30:26.355 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-23T14:30:26.469 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package tcl8.6. 2024-04-23T14:30:26.485 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-23T14:30:26.519 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-23T14:30:26.758 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-23T14:30:26.774 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-23T14:30:26.805 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package tcl. 2024-04-23T14:30:26.820 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-23T14:30:26.825 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-23T14:30:26.868 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-23T14:30:27.145 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-23T14:30:27.161 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-23T14:30:27.211 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-23T14:30:27.224 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package wpasupplicant. 2024-04-23T14:30:27.234 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-23T14:30:27.274 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-23T14:30:27.612 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-23T14:30:27.614 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-23T14:30:27.624 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-23T14:30:27.630 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-23T14:30:27.662 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-23T14:30:27.672 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-23T14:30:27.950 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-23T14:30:27.966 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-23T14:30:27.990 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-23T14:30:28.002 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-23T14:30:28.017 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-23T14:30:28.047 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-23T14:30:28.303 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libjson-perl. 2024-04-23T14:30:28.319 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-23T14:30:28.356 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up postgresql-client-common (238) ... 2024-04-23T14:30:28.386 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-23T14:30:28.562 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-23T14:30:28.688 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-23T14:30:28.814 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-23T14:30:28.825 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-23T14:30:28.840 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-23T14:30:28.874 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-23T14:30:28.932 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-23T14:30:29.041 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-23T14:30:29.159 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-23T14:30:30.141 INFO:teuthology.orchestra.run.smithi100.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-23T14:30:30.248 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-23T14:30:30.361 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-23T14:30:30.377 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-23T14:30:30.419 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-23T14:30:30.650 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-23T14:30:30.650 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-23T14:30:30.696 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-23T14:30:30.712 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-23T14:30:30.755 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-23T14:30:31.116 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-23T14:30:31.132 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-23T14:30:31.174 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-23T14:30:31.359 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-23T14:30:31.477 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-23T14:30:31.493 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-23T14:30:31.518 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-23T14:30:31.526 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-23T14:30:31.645 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-23T14:30:31.754 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-23T14:30:31.829 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-23T14:30:31.845 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-23T14:30:31.863 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-23T14:30:31.888 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-23T14:30:31.964 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-23T14:30:32.082 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-23T14:30:32.208 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-23T14:30:32.266 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-23T14:30:32.281 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-23T14:30:32.324 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-23T14:30:32.654 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-23T14:30:32.669 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-23T14:30:32.712 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-23T14:30:32.879 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-23T14:30:32.981 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up dump (0.4b47-3) ... 2024-04-23T14:30:33.056 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-23T14:30:33.058 INFO:teuthology.orchestra.run.smithi100.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-23T14:30:33.069 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-23T14:30:33.106 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-23T14:30:33.124 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-23T14:30:33.233 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-23T14:30:33.359 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-23T14:30:33.375 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-23T14:30:33.391 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-23T14:30:33.442 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-23T14:30:33.485 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-23T14:30:33.636 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-23T14:30:33.726 INFO:teuthology.orchestra.run.smithi100.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-23T14:30:33.796 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-23T14:30:33.922 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-23T14:30:33.997 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-23T14:30:34.010 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-23T14:30:34.055 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-23T14:30:34.082 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-23T14:30:34.191 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-23T14:30:34.300 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-23T14:30:34.416 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package modemmanager. 2024-04-23T14:30:34.433 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-23T14:30:34.468 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up postgresql-client (14+238) ... 2024-04-23T14:30:34.474 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-23T14:30:34.594 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-23T14:30:35.098 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package network-manager. 2024-04-23T14:30:35.113 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-23T14:30:35.165 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-23T14:30:35.224 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:30:35.224 INFO:teuthology.orchestra.run.smithi100.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-23T14:30:35.224 INFO:teuthology.orchestra.run.smithi100.stdout:which means they are currently configured by ifupdown: 2024-04-23T14:30:35.224 INFO:teuthology.orchestra.run.smithi100.stdout:- enp3s0f1 2024-04-23T14:30:35.224 INFO:teuthology.orchestra.run.smithi100.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-23T14:30:35.225 INFO:teuthology.orchestra.run.smithi100.stdout:remove their configuration from /etc/network/interfaces. 2024-04-23T14:30:35.225 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:30:35.357 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-23T14:30:35.746 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-23T14:30:35.814 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ppp. 2024-04-23T14:30:35.828 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-23T14:30:36.170 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-23T14:30:36.204 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-23T14:30:36.568 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package pptp-linux. 2024-04-23T14:30:36.582 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-23T14:30:36.627 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-23T14:30:36.938 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-23T14:30:36.952 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-23T14:30:37.004 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-23T14:30:37.114 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-23T14:30:37.256 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-23T14:30:37.366 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-23T14:30:37.374 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-23T14:30:37.388 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-23T14:30:37.440 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking postgresql-client-common (238) ... 2024-04-23T14:30:37.467 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up postgresql-common (238) ... 2024-04-23T14:30:37.836 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-23T14:30:37.850 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-23T14:30:37.903 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-23T14:30:38.393 INFO:teuthology.orchestra.run.smithi100.stdout:Adding user postgres to group ssl-cert 2024-04-23T14:30:38.499 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package postgresql-common. 2024-04-23T14:30:38.513 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-23T14:30:38.575 INFO:teuthology.orchestra.run.smithi151.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-23T14:30:38.606 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking postgresql-common (238) ... 2024-04-23T14:30:38.653 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:30:38.653 INFO:teuthology.orchestra.run.smithi100.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-23T14:30:38.907 INFO:teuthology.orchestra.run.smithi100.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-23T14:30:38.908 INFO:teuthology.orchestra.run.smithi100.stdout:Removing obsolete dictionary files: 2024-04-23T14:30:39.098 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package postgresql-14. 2024-04-23T14:30:39.112 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-23T14:30:39.151 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-23T14:30:39.610 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-23T14:30:40.076 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-23T14:30:40.343 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package postgresql. 2024-04-23T14:30:40.357 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-23T14:30:40.401 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking postgresql (14+238) ... 2024-04-23T14:30:40.424 INFO:teuthology.orchestra.run.smithi100.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-23T14:30:40.449 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-23T14:30:40.503 INFO:teuthology.orchestra.run.smithi100.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-23T14:30:40.503 INFO:teuthology.orchestra.run.smithi100.stdout:This user must also own the server process. 2024-04-23T14:30:40.504 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:30:40.504 INFO:teuthology.orchestra.run.smithi100.stdout:The database cluster will be initialized with locale "en_US". 2024-04-23T14:30:40.504 INFO:teuthology.orchestra.run.smithi100.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-23T14:30:40.504 INFO:teuthology.orchestra.run.smithi100.stdout:The default text search configuration will be set to "english". 2024-04-23T14:30:40.504 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:30:40.504 INFO:teuthology.orchestra.run.smithi100.stdout:Data page checksums are disabled. 2024-04-23T14:30:40.504 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:30:40.504 INFO:teuthology.orchestra.run.smithi100.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-23T14:30:40.505 INFO:teuthology.orchestra.run.smithi100.stdout:creating subdirectories ... ok 2024-04-23T14:30:40.505 INFO:teuthology.orchestra.run.smithi100.stdout:selecting dynamic shared memory implementation ... posix 2024-04-23T14:30:40.528 INFO:teuthology.orchestra.run.smithi100.stdout:selecting default max_connections ... 100 2024-04-23T14:30:40.562 INFO:teuthology.orchestra.run.smithi100.stdout:selecting default shared_buffers ... 128MB 2024-04-23T14:30:40.576 INFO:teuthology.orchestra.run.smithi100.stdout:selecting default time zone ... Etc/UTC 2024-04-23T14:30:40.577 INFO:teuthology.orchestra.run.smithi100.stdout:creating configuration files ... ok 2024-04-23T14:30:40.628 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package postgresql-client. 2024-04-23T14:30:40.642 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-23T14:30:40.686 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking postgresql-client (14+238) ... 2024-04-23T14:30:40.742 INFO:teuthology.orchestra.run.smithi100.stdout:running bootstrap script ... ok 2024-04-23T14:30:40.930 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-23T14:30:40.947 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-23T14:30:41.005 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-23T14:30:41.249 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package tcl8.6. 2024-04-23T14:30:41.264 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-23T14:30:41.299 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-23T14:30:41.371 INFO:teuthology.orchestra.run.smithi100.stdout:performing post-bootstrap initialization ... ok 2024-04-23T14:30:41.585 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package tcl. 2024-04-23T14:30:41.601 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-23T14:30:41.648 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-23T14:30:41.882 INFO:teuthology.orchestra.run.smithi100.stdout:syncing data to disk ... ok 2024-04-23T14:30:42.005 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package wpasupplicant. 2024-04-23T14:30:42.021 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-23T14:30:42.088 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-23T14:30:42.411 INFO:teuthology.orchestra.run.smithi100.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-23T14:30:42.417 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-23T14:30:42.433 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-23T14:30:42.475 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-23T14:30:42.853 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-23T14:30:42.868 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-23T14:30:42.920 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-23T14:30:43.272 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up postgresql-client-common (238) ... 2024-04-23T14:30:43.459 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-23T14:30:43.577 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-23T14:30:43.703 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-23T14:30:43.838 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-23T14:30:44.381 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-23T14:30:44.507 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-23T14:30:45.453 INFO:teuthology.orchestra.run.smithi151.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-23T14:30:45.475 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-23T14:30:45.562 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-23T14:30:45.601 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-23T14:30:45.752 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up postgresql (14+238) ... 2024-04-23T14:30:45.878 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-23T14:30:45.941 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-23T14:30:45.941 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-23T14:30:46.004 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-23T14:30:46.265 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-23T14:30:46.265 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-23T14:30:46.624 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-23T14:30:46.749 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-23T14:30:46.859 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-23T14:30:46.960 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-23T14:30:47.006 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-23T14:30:47.077 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-23T14:30:47.099 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-23T14:30:47.203 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-23T14:30:47.421 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-23T14:30:47.426 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-23T14:30:47.547 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-23T14:30:47.595 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-23T14:30:48.219 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-23T14:30:48.337 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up dump (0.4b47-3) ... 2024-04-23T14:30:48.432 INFO:teuthology.orchestra.run.smithi151.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-23T14:30:48.497 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-23T14:30:48.639 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-23T14:30:48.790 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-23T14:30:48.916 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-23T14:30:48.972 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-23T14:30:49.042 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-23T14:30:49.064 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-23T14:30:49.132 INFO:teuthology.orchestra.run.smithi151.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-23T14:30:49.194 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-23T14:30:49.295 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-23T14:30:49.471 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-23T14:30:49.597 INFO:teuthology.orchestra.run.smithi100.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-23T14:30:49.597 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-23T14:30:49.604 DEBUG:teuthology.parallel:result is None 2024-04-23T14:30:49.723 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-23T14:30:49.916 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up postgresql-client (14+238) ... 2024-04-23T14:30:50.034 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-23T14:30:50.596 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:30:50.596 INFO:teuthology.orchestra.run.smithi151.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-23T14:30:50.596 INFO:teuthology.orchestra.run.smithi151.stdout:which means they are currently configured by ifupdown: 2024-04-23T14:30:50.596 INFO:teuthology.orchestra.run.smithi151.stdout:- enp3s0f1 2024-04-23T14:30:50.596 INFO:teuthology.orchestra.run.smithi151.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-23T14:30:50.596 INFO:teuthology.orchestra.run.smithi151.stdout:remove their configuration from /etc/network/interfaces. 2024-04-23T14:30:50.596 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:30:50.728 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-23T14:30:51.139 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-23T14:30:51.562 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-23T14:30:52.270 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-23T14:30:52.413 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-23T14:30:52.538 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-23T14:30:52.665 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up postgresql-common (238) ... 2024-04-23T14:30:53.630 INFO:teuthology.orchestra.run.smithi151.stdout:Adding user postgres to group ssl-cert 2024-04-23T14:30:53.880 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:30:53.880 INFO:teuthology.orchestra.run.smithi151.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-23T14:30:54.137 INFO:teuthology.orchestra.run.smithi151.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-23T14:30:54.137 INFO:teuthology.orchestra.run.smithi151.stdout:Removing obsolete dictionary files: 2024-04-23T14:30:54.867 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-23T14:30:55.391 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-23T14:30:55.721 INFO:teuthology.orchestra.run.smithi151.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-23T14:30:55.741 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-23T14:30:55.799 INFO:teuthology.orchestra.run.smithi151.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-23T14:30:55.799 INFO:teuthology.orchestra.run.smithi151.stdout:This user must also own the server process. 2024-04-23T14:30:55.799 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:30:55.799 INFO:teuthology.orchestra.run.smithi151.stdout:The database cluster will be initialized with locale "en_US". 2024-04-23T14:30:55.799 INFO:teuthology.orchestra.run.smithi151.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-23T14:30:55.799 INFO:teuthology.orchestra.run.smithi151.stdout:The default text search configuration will be set to "english". 2024-04-23T14:30:55.799 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:30:55.799 INFO:teuthology.orchestra.run.smithi151.stdout:Data page checksums are disabled. 2024-04-23T14:30:55.799 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:30:55.799 INFO:teuthology.orchestra.run.smithi151.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-23T14:30:55.800 INFO:teuthology.orchestra.run.smithi151.stdout:creating subdirectories ... ok 2024-04-23T14:30:55.800 INFO:teuthology.orchestra.run.smithi151.stdout:selecting dynamic shared memory implementation ... posix 2024-04-23T14:30:55.823 INFO:teuthology.orchestra.run.smithi151.stdout:selecting default max_connections ... 100 2024-04-23T14:30:55.857 INFO:teuthology.orchestra.run.smithi151.stdout:selecting default shared_buffers ... 128MB 2024-04-23T14:30:55.871 INFO:teuthology.orchestra.run.smithi151.stdout:selecting default time zone ... Etc/UTC 2024-04-23T14:30:55.872 INFO:teuthology.orchestra.run.smithi151.stdout:creating configuration files ... ok 2024-04-23T14:30:56.037 INFO:teuthology.orchestra.run.smithi151.stdout:running bootstrap script ... ok 2024-04-23T14:30:56.666 INFO:teuthology.orchestra.run.smithi151.stdout:performing post-bootstrap initialization ... ok 2024-04-23T14:30:57.273 INFO:teuthology.orchestra.run.smithi151.stdout:syncing data to disk ... ok 2024-04-23T14:30:57.789 INFO:teuthology.orchestra.run.smithi151.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-23T14:31:01.149 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-23T14:31:01.258 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-23T14:31:01.367 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up postgresql (14+238) ... 2024-04-23T14:31:01.493 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-23T14:31:01.619 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-23T14:31:01.887 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-23T14:31:01.888 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-23T14:31:02.655 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-23T14:31:02.755 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-23T14:31:03.116 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-23T14:31:03.267 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-23T14:31:04.670 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-23T14:31:04.771 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-23T14:31:05.304 INFO:teuthology.orchestra.run.smithi151.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-23T14:31:05.311 DEBUG:teuthology.parallel:result is None 2024-04-23T14:31:05.311 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:31:05.536 DEBUG:teuthology.orchestra.run.smithi100:> dpkg-query -W -f '${Version}' ceph 2024-04-23T14:31:05.560 INFO:teuthology.orchestra.run.smithi100.stdout:19.0.0-3126-gd44fddd9-1jammy 2024-04-23T14:31:05.560 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3126-gd44fddd9-1jammy 2024-04-23T14:31:05.560 INFO:teuthology.task.install:The correct ceph version 19.0.0-3126-gd44fddd9-1jammy is installed. 2024-04-23T14:31:05.562 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:31:05.800 DEBUG:teuthology.orchestra.run.smithi151:> dpkg-query -W -f '${Version}' ceph 2024-04-23T14:31:05.822 INFO:teuthology.orchestra.run.smithi151.stdout:19.0.0-3126-gd44fddd9-1jammy 2024-04-23T14:31:05.823 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3126-gd44fddd9-1jammy 2024-04-23T14:31:05.823 INFO:teuthology.task.install:The correct ceph version 19.0.0-3126-gd44fddd9-1jammy is installed. 2024-04-23T14:31:05.825 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-23T14:31:05.825 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:05.825 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T14:31:05.847 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:31:05.848 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T14:31:05.886 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-23T14:31:05.886 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:05.886 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T14:31:05.911 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T14:31:05.975 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:31:05.975 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T14:31:05.994 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T14:31:06.052 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-23T14:31:06.052 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:06.053 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T14:31:06.073 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T14:31:06.134 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:31:06.134 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T14:31:06.153 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T14:31:06.212 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-23T14:31:06.212 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:06.213 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T14:31:06.233 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T14:31:06.298 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:31:06.298 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T14:31:06.317 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T14:31:06.376 INFO:teuthology.run_tasks:Running task ceph... 2024-04-23T14:31:06.474 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-23T14:31:06.475 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /var/log/ceph 2024-04-23T14:31:06.478 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 777 /var/log/ceph 2024-04-23T14:31:06.497 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-23T14:31:06.497 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-23T14:31:06.545 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-23T14:31:06.564 INFO:tasks.ceph:Creating extra log directories... 2024-04-23T14:31:06.564 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-23T14:31:06.609 DEBUG:teuthology.orchestra.run.smithi151:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-23T14:31:06.637 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-23T14:31:06.637 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-23T14:31:06.637 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-23T14:31:06.638 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:06.638 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-23T14:31:06.683 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-23T14:31:06.779 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:31:06.779 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-23T14:31:06.787 DEBUG:teuthology.orchestra.run.smithi151:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-23T14:31:06.884 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-23T14:31:06.884 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-23T14:31:06.884 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669587', 'branch': 'wip-rishabh-testing-20240416.193735-3', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/pool-perm}', 'email': None, 'first_in_suite': False, 'job_id': '7669587', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240416.193735-3'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0'}}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240416.193735-3', 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7954, 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0', 'sleep_before_teardown': 0, 'subset': '364/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240416.193735-3', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi100.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE/aCshXzWt2yH2pHsDWDUnQMtkY17vqfGEhcBIgWVJVdM6pUoFiqJpdrUwnY+nyeH6VB0CdTY6TY8BHoavBEJY=', 'smithi151.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG95C7i0TtHFsiFmo24RegfJZIQu1ezh/b6jEqzBhGM/8ylKezjQ6tKnnv+fWihm7hbm1I4T7PxgWGf5ucNRTaQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_pool_perm']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-23_14:04:26', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-23T14:31:06.885 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-23T14:31:06.888 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-23T14:31:06.934 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m0777 -- /var/run/ceph 2024-04-23T14:31:06.937 DEBUG:teuthology.orchestra.run.smithi151:> sudo install -d -m0777 -- /var/run/ceph 2024-04-23T14:31:06.997 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:06.997 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T14:31:07.010 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T14:31:07.011 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2024-04-23T14:31:07.063 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T14:31:07.063 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:31:07.063 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 885 Links: 1 2024-04-23T14:31:07.063 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:31:07.063 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-04-23 14:24:12.707204947 +0000 2024-04-23T14:31:07.063 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-04-23 14:24:12.459207160 +0000 2024-04-23T14:31:07.063 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-04-23 14:24:12.459207160 +0000 2024-04-23T14:31:07.063 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-04-23 14:24:12.459207160 +0000 2024-04-23T14:31:07.064 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T14:31:07.124 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-04-23T14:31:07.125 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-04-23T14:31:07.125 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000391156 s, 1.3 MB/s 2024-04-23T14:31:07.127 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T14:31:07.185 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2024-04-23T14:31:07.235 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T14:31:07.235 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:31:07.235 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 903 Links: 1 2024-04-23T14:31:07.235 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:31:07.235 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-04-23 14:24:13.267199952 +0000 2024-04-23T14:31:07.236 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-04-23 14:24:13.011202235 +0000 2024-04-23T14:31:07.236 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-04-23 14:24:13.011202235 +0000 2024-04-23T14:31:07.236 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-04-23 14:24:13.011202235 +0000 2024-04-23T14:31:07.236 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T14:31:07.296 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-04-23T14:31:07.296 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-04-23T14:31:07.296 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000358297 s, 1.4 MB/s 2024-04-23T14:31:07.298 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T14:31:07.352 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2024-04-23T14:31:07.402 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T14:31:07.403 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:31:07.403 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 918 Links: 1 2024-04-23T14:31:07.403 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:31:07.403 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-04-23 14:24:13.843194813 +0000 2024-04-23T14:31:07.403 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-04-23 14:24:13.559197347 +0000 2024-04-23T14:31:07.403 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-04-23 14:24:13.559197347 +0000 2024-04-23T14:31:07.403 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-04-23 14:24:13.559197347 +0000 2024-04-23T14:31:07.403 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T14:31:07.464 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-04-23T14:31:07.464 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-04-23T14:31:07.464 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000351937 s, 1.5 MB/s 2024-04-23T14:31:07.466 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T14:31:07.516 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2024-04-23T14:31:07.566 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T14:31:07.567 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:31:07.567 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 938 Links: 1 2024-04-23T14:31:07.567 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:31:07.567 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-04-23 14:24:14.423189638 +0000 2024-04-23T14:31:07.567 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-04-23 14:24:14.135192208 +0000 2024-04-23T14:31:07.567 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-04-23 14:24:14.135192208 +0000 2024-04-23T14:31:07.567 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-04-23 14:24:14.135192208 +0000 2024-04-23T14:31:07.567 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T14:31:07.628 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-04-23T14:31:07.628 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-04-23T14:31:07.628 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000376454 s, 1.4 MB/s 2024-04-23T14:31:07.630 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T14:31:07.680 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-23T14:31:07.680 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:31:07.680 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T14:31:07.688 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T14:31:07.688 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_1 2024-04-23T14:31:07.738 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T14:31:07.738 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:31:07.738 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 890 Links: 1 2024-04-23T14:31:07.738 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:31:07.738 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-04-23 14:24:10.841969539 +0000 2024-04-23T14:31:07.738 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-04-23 14:24:10.577971897 +0000 2024-04-23T14:31:07.739 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-04-23 14:24:10.577971897 +0000 2024-04-23T14:31:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-04-23 14:24:10.577971897 +0000 2024-04-23T14:31:07.739 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T14:31:07.800 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-04-23T14:31:07.800 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-04-23T14:31:07.800 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.00040071 s, 1.3 MB/s 2024-04-23T14:31:07.802 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T14:31:07.852 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_2 2024-04-23T14:31:07.902 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T14:31:07.902 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:31:07.903 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 907 Links: 1 2024-04-23T14:31:07.903 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:31:07.903 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-04-23 14:24:11.393964608 +0000 2024-04-23T14:31:07.903 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-04-23 14:24:11.125967002 +0000 2024-04-23T14:31:07.903 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-04-23 14:24:11.125967002 +0000 2024-04-23T14:31:07.903 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-04-23 14:24:11.125967002 +0000 2024-04-23T14:31:07.903 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T14:31:07.964 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-04-23T14:31:07.964 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-04-23T14:31:07.964 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000368179 s, 1.4 MB/s 2024-04-23T14:31:07.966 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T14:31:08.020 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_3 2024-04-23T14:31:08.070 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T14:31:08.071 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:31:08.071 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 923 Links: 1 2024-04-23T14:31:08.071 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:31:08.071 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-04-23 14:24:11.929959821 +0000 2024-04-23T14:31:08.071 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-04-23 14:24:11.657962249 +0000 2024-04-23T14:31:08.071 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-04-23 14:24:11.657962249 +0000 2024-04-23T14:31:08.071 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-04-23 14:24:11.657962249 +0000 2024-04-23T14:31:08.072 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T14:31:08.133 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-04-23T14:31:08.133 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-04-23T14:31:08.133 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000397224 s, 1.3 MB/s 2024-04-23T14:31:08.134 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T14:31:08.189 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_4 2024-04-23T14:31:08.239 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T14:31:08.239 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:31:08.239 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 943 Links: 1 2024-04-23T14:31:08.239 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:31:08.239 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-04-23 14:24:12.545954319 +0000 2024-04-23T14:31:08.239 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-04-23 14:24:12.241957034 +0000 2024-04-23T14:31:08.240 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-04-23 14:24:12.241957034 +0000 2024-04-23T14:31:08.240 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-04-23 14:24:12.241957034 +0000 2024-04-23T14:31:08.240 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T14:31:08.301 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-04-23T14:31:08.301 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-04-23T14:31:08.301 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000404669 s, 1.3 MB/s 2024-04-23T14:31:08.303 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T14:31:08.356 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-23T14:31:08.357 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi100.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi151.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-23T14:31:08.357 INFO:tasks.ceph:Generating config... 2024-04-23T14:31:08.358 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-23T14:31:08.358 INFO:tasks.ceph:[client] debug client = 20 2024-04-23T14:31:08.358 INFO:tasks.ceph:[client] debug ms = 1 2024-04-23T14:31:08.358 INFO:tasks.ceph:[client] pid file = 2024-04-23T14:31:08.358 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-23T14:31:08.358 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-23T14:31:08.358 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-23T14:31:08.358 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-23T14:31:08.358 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-23T14:31:08.358 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-23T14:31:08.359 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-23T14:31:08.359 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-23T14:31:08.359 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-23T14:31:08.359 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-23T14:31:08.359 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-23T14:31:08.359 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-23T14:31:08.359 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-23T14:31:08.359 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-23T14:31:08.359 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-23T14:31:08.359 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-23T14:31:08.359 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-23T14:31:08.359 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-23T14:31:08.359 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-23T14:31:08.359 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-23T14:31:08.359 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-23T14:31:08.359 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-23T14:31:08.360 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-23T14:31:08.360 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-23T14:31:08.360 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-23T14:31:08.360 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-23T14:31:08.360 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-23T14:31:08.360 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-23T14:31:08.360 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-23T14:31:08.360 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-23T14:31:08.360 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-23T14:31:08.360 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-23T14:31:08.360 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-23T14:31:08.360 INFO:tasks.ceph:Setting up mon.a... 2024-04-23T14:31:08.360 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-23T14:31:08.404 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.keyring 2024-04-23T14:31:08.433 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-23T14:31:08.521 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-23T14:31:08.581 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.100'), ('mon.b', '172.21.15.151'), ('mon.c', '[v2:172.21.15.151:3301,v1:172.21.15.151:6790]')] 2024-04-23T14:31:08.582 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.100,172.21.15.151,[v2:172.21.15.151:3301,v1:172.21.15.151:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-23T14:31:08.582 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:08.583 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-23T14:31:08.630 DEBUG:teuthology.orchestra.run.smithi100:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.100 --add b 172.21.15.151 --addv c '[v2:172.21.15.151:3301,v1:172.21.15.151:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:31:08.719 INFO:teuthology.orchestra.run.smithi100.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:31:08.719 INFO:teuthology.orchestra.run.smithi100.stdout:monmaptool: generated fsid 167d2af1-01c4-4f87-8bce-0de22791f823 2024-04-23T14:31:08.719 INFO:teuthology.orchestra.run.smithi100.stdout:setting min_mon_release = quincy 2024-04-23T14:31:08.719 INFO:teuthology.orchestra.run.smithi100.stdout:epoch 0 2024-04-23T14:31:08.719 INFO:teuthology.orchestra.run.smithi100.stdout:fsid 167d2af1-01c4-4f87-8bce-0de22791f823 2024-04-23T14:31:08.720 INFO:teuthology.orchestra.run.smithi100.stdout:last_changed 2024-04-23T14:31:08.724062+0000 2024-04-23T14:31:08.720 INFO:teuthology.orchestra.run.smithi100.stdout:created 2024-04-23T14:31:08.724062+0000 2024-04-23T14:31:08.720 INFO:teuthology.orchestra.run.smithi100.stdout:min_mon_release 17 (quincy) 2024-04-23T14:31:08.720 INFO:teuthology.orchestra.run.smithi100.stdout:election_strategy: 1 2024-04-23T14:31:08.720 INFO:teuthology.orchestra.run.smithi100.stdout:0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-04-23T14:31:08.720 INFO:teuthology.orchestra.run.smithi100.stdout:1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.b 2024-04-23T14:31:08.720 INFO:teuthology.orchestra.run.smithi100.stdout:2: [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] mon.c 2024-04-23T14:31:08.720 INFO:teuthology.orchestra.run.smithi100.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-23T14:31:08.722 DEBUG:teuthology.orchestra.run.smithi100:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-23T14:31:08.730 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 167d2af1-01c4-4f87-8bce-0de22791f823... 2024-04-23T14:31:08.731 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-23T14:31:08.781 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-23T14:31:08.819 INFO:teuthology.orchestra.run.smithi100.stdout:[global] 2024-04-23T14:31:08.819 INFO:teuthology.orchestra.run.smithi100.stdout: chdir = "" 2024-04-23T14:31:08.820 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-23T14:31:08.820 INFO:teuthology.orchestra.run.smithi100.stdout: auth supported = cephx 2024-04-23T14:31:08.820 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.820 INFO:teuthology.orchestra.run.smithi100.stdout: filestore xattr use omap = true 2024-04-23T14:31:08.820 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.820 INFO:teuthology.orchestra.run.smithi100.stdout: mon clock drift allowed = 1.000 2024-04-23T14:31:08.820 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.820 INFO:teuthology.orchestra.run.smithi100.stdout: osd crush chooseleaf type = 0 2024-04-23T14:31:08.820 INFO:teuthology.orchestra.run.smithi100.stdout: auth debug = true 2024-04-23T14:31:08.820 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.820 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on old message = true 2024-04-23T14:31:08.820 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on bug = true 2024-04-23T14:31:08.820 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.821 INFO:teuthology.orchestra.run.smithi100.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-23T14:31:08.821 INFO:teuthology.orchestra.run.smithi100.stdout: mon pg warn max object skew = 0 2024-04-23T14:31:08.821 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.821 INFO:teuthology.orchestra.run.smithi100.stdout: # disable pg_autoscaler by default for new pools 2024-04-23T14:31:08.821 INFO:teuthology.orchestra.run.smithi100.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-23T14:31:08.821 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.821 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default size = 2 2024-04-23T14:31:08.821 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.821 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow primary affinity = true 2024-04-23T14:31:08.821 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow pg remap = true 2024-04-23T14:31:08.821 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on legacy crush tunables = false 2024-04-23T14:31:08.821 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on crush straw calc version zero = false 2024-04-23T14:31:08.821 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on no sortbitwise = false 2024-04-23T14:31:08.821 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on osd down out interval zero = false 2024-04-23T14:31:08.822 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on too few osds = false 2024-04-23T14:31:08.822 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-23T14:31:08.822 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-23T14:31:08.822 INFO:teuthology.orchestra.run.smithi100.stdout: mon_allow_pool_size_one = true 2024-04-23T14:31:08.822 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.822 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-23T14:31:08.822 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.822 INFO:teuthology.orchestra.run.smithi100.stdout: osd default data pool replay window = 5 2024-04-23T14:31:08.822 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.822 INFO:teuthology.orchestra.run.smithi100.stdout: mon allow pool delete = true 2024-04-23T14:31:08.822 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.822 INFO:teuthology.orchestra.run.smithi100.stdout: mon cluster log file level = debug 2024-04-23T14:31:08.822 INFO:teuthology.orchestra.run.smithi100.stdout: debug asserts on shutdown = true 2024-04-23T14:31:08.822 INFO:teuthology.orchestra.run.smithi100.stdout: mon health detail to clog = false 2024-04-23T14:31:08.823 INFO:teuthology.orchestra.run.smithi100.stdout: mon host = "172.21.15.100,172.21.15.151,[v2:172.21.15.151:3301,v1:172.21.15.151:6790]" 2024-04-23T14:31:08.823 INFO:teuthology.orchestra.run.smithi100.stdout: fsid = 167d2af1-01c4-4f87-8bce-0de22791f823 2024-04-23T14:31:08.823 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.823 INFO:teuthology.orchestra.run.smithi100.stdout:[osd] 2024-04-23T14:31:08.823 INFO:teuthology.orchestra.run.smithi100.stdout: osd journal size = 100 2024-04-23T14:31:08.823 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.823 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub load threshold = 5.0 2024-04-23T14:31:08.823 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub max interval = 600 2024-04-23T14:31:08.823 INFO:teuthology.orchestra.run.smithi100.stdout: osd mclock profile = high_recovery_ops 2024-04-23T14:31:08.823 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.823 INFO:teuthology.orchestra.run.smithi100.stdout: osd recover clone overlap = true 2024-04-23T14:31:08.823 INFO:teuthology.orchestra.run.smithi100.stdout: osd recovery max chunk = 1048576 2024-04-23T14:31:08.823 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.824 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug shutdown = true 2024-04-23T14:31:08.824 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug op order = true 2024-04-23T14:31:08.824 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug verify stray on activate = true 2024-04-23T14:31:08.824 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug trim objects = true 2024-04-23T14:31:08.824 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.824 INFO:teuthology.orchestra.run.smithi100.stdout: osd open classes on start = true 2024-04-23T14:31:08.824 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug pg log writeout = true 2024-04-23T14:31:08.824 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.824 INFO:teuthology.orchestra.run.smithi100.stdout: osd deep scrub update digest min age = 30 2024-04-23T14:31:08.824 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.824 INFO:teuthology.orchestra.run.smithi100.stdout: osd map max advance = 10 2024-04-23T14:31:08.824 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.824 INFO:teuthology.orchestra.run.smithi100.stdout: journal zero on create = true 2024-04-23T14:31:08.824 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.824 INFO:teuthology.orchestra.run.smithi100.stdout: filestore ondisk finisher threads = 3 2024-04-23T14:31:08.825 INFO:teuthology.orchestra.run.smithi100.stdout: filestore apply finisher threads = 3 2024-04-23T14:31:08.825 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.825 INFO:teuthology.orchestra.run.smithi100.stdout: bdev debug aio = true 2024-04-23T14:31:08.825 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug misdirected ops = true 2024-04-23T14:31:08.825 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore block size = 96636764160 2024-04-23T14:31:08.825 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore fsck on mount = True 2024-04-23T14:31:08.825 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluefs = 20 2024-04-23T14:31:08.825 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluestore = 20 2024-04-23T14:31:08.825 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-23T14:31:08.825 INFO:teuthology.orchestra.run.smithi100.stdout: debug osd = 20 2024-04-23T14:31:08.825 INFO:teuthology.orchestra.run.smithi100.stdout: debug rocksdb = 10 2024-04-23T14:31:08.825 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd backfillfull_ratio = 0.85 2024-04-23T14:31:08.825 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd full ratio = 0.9 2024-04-23T14:31:08.825 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd nearfull ratio = 0.8 2024-04-23T14:31:08.825 INFO:teuthology.orchestra.run.smithi100.stdout: osd failsafe full ratio = 0.95 2024-04-23T14:31:08.826 INFO:teuthology.orchestra.run.smithi100.stdout: osd objectstore = bluestore 2024-04-23T14:31:08.826 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2024-04-23T14:31:08.826 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.826 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr] 2024-04-23T14:31:08.826 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-23T14:31:08.826 INFO:teuthology.orchestra.run.smithi100.stdout: debug mgr = 20 2024-04-23T14:31:08.826 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-04-23T14:31:08.826 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-04-23T14:31:08.826 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:31:08.826 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:31:08.826 INFO:teuthology.orchestra.run.smithi100.stdout: mgr/telemetry/nag = false 2024-04-23T14:31:08.826 INFO:teuthology.orchestra.run.smithi100.stdout: debug client = 20 2024-04-23T14:31:08.826 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.826 INFO:teuthology.orchestra.run.smithi100.stdout:[mon] 2024-04-23T14:31:08.827 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-23T14:31:08.827 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-04-23T14:31:08.827 INFO:teuthology.orchestra.run.smithi100.stdout: debug paxos = 20 2024-04-23T14:31:08.827 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-04-23T14:31:08.827 INFO:teuthology.orchestra.run.smithi100.stdout: mon data avail warn = 5 2024-04-23T14:31:08.827 INFO:teuthology.orchestra.run.smithi100.stdout: mon mgr mkfs grace = 240 2024-04-23T14:31:08.827 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:31:08.827 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd reporter subtree level = osd 2024-04-23T14:31:08.827 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd prime pg temp = true 2024-04-23T14:31:08.827 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:31:08.827 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.827 INFO:teuthology.orchestra.run.smithi100.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-23T14:31:08.827 INFO:teuthology.orchestra.run.smithi100.stdout: auth mon ticket ttl = 660 # 11m 2024-04-23T14:31:08.827 INFO:teuthology.orchestra.run.smithi100.stdout: auth service ticket ttl = 240 # 4m 2024-04-23T14:31:08.828 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.828 INFO:teuthology.orchestra.run.smithi100.stdout: # don't complain about insecure global_id in the test suite 2024-04-23T14:31:08.828 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-23T14:31:08.828 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-23T14:31:08.828 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.828 INFO:teuthology.orchestra.run.smithi100.stdout: # 1m isn't quite enough 2024-04-23T14:31:08.828 INFO:teuthology.orchestra.run.smithi100.stdout: mon_down_mkfs_grace = 2m 2024-04-23T14:31:08.828 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.828 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_filestore_osds = false 2024-04-23T14:31:08.828 INFO:teuthology.orchestra.run.smithi100.stdout: mon down mkfs grace = 300 2024-04-23T14:31:08.828 INFO:teuthology.orchestra.run.smithi100.stdout: mon op complaint time = 120 2024-04-23T14:31:08.828 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:08.828 INFO:teuthology.orchestra.run.smithi100.stdout:[client] 2024-04-23T14:31:08.828 INFO:teuthology.orchestra.run.smithi100.stdout: rgw cache enabled = true 2024-04-23T14:31:08.829 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable ops log = true 2024-04-23T14:31:08.829 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable usage log = true 2024-04-23T14:31:08.829 INFO:teuthology.orchestra.run.smithi100.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-23T14:31:08.829 INFO:teuthology.orchestra.run.smithi100.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-23T14:31:08.829 INFO:teuthology.orchestra.run.smithi100.stdout: client mount timeout = 600 2024-04-23T14:31:08.829 INFO:teuthology.orchestra.run.smithi100.stdout: debug client = 20 2024-04-23T14:31:08.829 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-23T14:31:08.829 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = "" 2024-04-23T14:31:08.829 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2024-04-23T14:31:08.829 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2024-04-23T14:31:08.829 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.a] 2024-04-23T14:31:08.829 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.b] 2024-04-23T14:31:08.829 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.c] 2024-04-23T14:31:08.829 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2024-04-23T14:31:08.829 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2024-04-23T14:31:08.829 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2024-04-23T14:31:08.830 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2024-04-23T14:31:08.830 INFO:teuthology.orchestra.run.smithi100.stdout:[mds] 2024-04-23T14:31:08.830 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds = 20 2024-04-23T14:31:08.830 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds balancer = 20 2024-04-23T14:31:08.830 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-23T14:31:08.830 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug frag = True 2024-04-23T14:31:08.830 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug scatterstat = True 2024-04-23T14:31:08.830 INFO:teuthology.orchestra.run.smithi100.stdout: mds op complaint time = 180 2024-04-23T14:31:08.830 INFO:teuthology.orchestra.run.smithi100.stdout: mds verify scatter = True 2024-04-23T14:31:08.830 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2024-04-23T14:31:08.830 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2024-04-23T14:31:08.830 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2024-04-23T14:31:08.831 INFO:teuthology.orchestra.run.smithi151.stdout:[global] 2024-04-23T14:31:08.831 INFO:teuthology.orchestra.run.smithi151.stdout: chdir = "" 2024-04-23T14:31:08.831 INFO:teuthology.orchestra.run.smithi151.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-23T14:31:08.831 INFO:teuthology.orchestra.run.smithi151.stdout: auth supported = cephx 2024-04-23T14:31:08.831 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.831 INFO:teuthology.orchestra.run.smithi151.stdout: filestore xattr use omap = true 2024-04-23T14:31:08.831 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.831 INFO:teuthology.orchestra.run.smithi151.stdout: mon clock drift allowed = 1.000 2024-04-23T14:31:08.831 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.832 INFO:teuthology.orchestra.run.smithi151.stdout: osd crush chooseleaf type = 0 2024-04-23T14:31:08.832 INFO:teuthology.orchestra.run.smithi151.stdout: auth debug = true 2024-04-23T14:31:08.832 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.832 INFO:teuthology.orchestra.run.smithi151.stdout: ms die on old message = true 2024-04-23T14:31:08.832 INFO:teuthology.orchestra.run.smithi151.stdout: ms die on bug = true 2024-04-23T14:31:08.832 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.832 INFO:teuthology.orchestra.run.smithi151.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-23T14:31:08.832 INFO:teuthology.orchestra.run.smithi151.stdout: mon pg warn max object skew = 0 2024-04-23T14:31:08.832 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.832 INFO:teuthology.orchestra.run.smithi151.stdout: # disable pg_autoscaler by default for new pools 2024-04-23T14:31:08.832 INFO:teuthology.orchestra.run.smithi151.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-23T14:31:08.832 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.832 INFO:teuthology.orchestra.run.smithi151.stdout: osd pool default size = 2 2024-04-23T14:31:08.832 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.832 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd allow primary affinity = true 2024-04-23T14:31:08.833 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd allow pg remap = true 2024-04-23T14:31:08.833 INFO:teuthology.orchestra.run.smithi151.stdout: mon warn on legacy crush tunables = false 2024-04-23T14:31:08.833 INFO:teuthology.orchestra.run.smithi151.stdout: mon warn on crush straw calc version zero = false 2024-04-23T14:31:08.833 INFO:teuthology.orchestra.run.smithi151.stdout: mon warn on no sortbitwise = false 2024-04-23T14:31:08.833 INFO:teuthology.orchestra.run.smithi151.stdout: mon warn on osd down out interval zero = false 2024-04-23T14:31:08.833 INFO:teuthology.orchestra.run.smithi151.stdout: mon warn on too few osds = false 2024-04-23T14:31:08.833 INFO:teuthology.orchestra.run.smithi151.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-23T14:31:08.833 INFO:teuthology.orchestra.run.smithi151.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-23T14:31:08.833 INFO:teuthology.orchestra.run.smithi151.stdout: mon_allow_pool_size_one = true 2024-04-23T14:31:08.833 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.833 INFO:teuthology.orchestra.run.smithi151.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-23T14:31:08.833 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.833 INFO:teuthology.orchestra.run.smithi151.stdout: osd default data pool replay window = 5 2024-04-23T14:31:08.833 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.833 INFO:teuthology.orchestra.run.smithi151.stdout: mon allow pool delete = true 2024-04-23T14:31:08.834 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.834 INFO:teuthology.orchestra.run.smithi151.stdout: mon cluster log file level = debug 2024-04-23T14:31:08.834 INFO:teuthology.orchestra.run.smithi151.stdout: debug asserts on shutdown = true 2024-04-23T14:31:08.834 INFO:teuthology.orchestra.run.smithi151.stdout: mon health detail to clog = false 2024-04-23T14:31:08.834 INFO:teuthology.orchestra.run.smithi151.stdout: mon host = "172.21.15.100,172.21.15.151,[v2:172.21.15.151:3301,v1:172.21.15.151:6790]" 2024-04-23T14:31:08.834 INFO:teuthology.orchestra.run.smithi151.stdout: fsid = 167d2af1-01c4-4f87-8bce-0de22791f823 2024-04-23T14:31:08.834 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.834 INFO:teuthology.orchestra.run.smithi151.stdout:[osd] 2024-04-23T14:31:08.834 INFO:teuthology.orchestra.run.smithi151.stdout: osd journal size = 100 2024-04-23T14:31:08.834 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.834 INFO:teuthology.orchestra.run.smithi151.stdout: osd scrub load threshold = 5.0 2024-04-23T14:31:08.834 INFO:teuthology.orchestra.run.smithi151.stdout: osd scrub max interval = 600 2024-04-23T14:31:08.834 INFO:teuthology.orchestra.run.smithi151.stdout: osd mclock profile = high_recovery_ops 2024-04-23T14:31:08.834 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.834 INFO:teuthology.orchestra.run.smithi151.stdout: osd recover clone overlap = true 2024-04-23T14:31:08.835 INFO:teuthology.orchestra.run.smithi151.stdout: osd recovery max chunk = 1048576 2024-04-23T14:31:08.835 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.835 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug shutdown = true 2024-04-23T14:31:08.835 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug op order = true 2024-04-23T14:31:08.835 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug verify stray on activate = true 2024-04-23T14:31:08.835 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug trim objects = true 2024-04-23T14:31:08.835 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.835 INFO:teuthology.orchestra.run.smithi151.stdout: osd open classes on start = true 2024-04-23T14:31:08.835 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug pg log writeout = true 2024-04-23T14:31:08.835 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.835 INFO:teuthology.orchestra.run.smithi151.stdout: osd deep scrub update digest min age = 30 2024-04-23T14:31:08.835 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.835 INFO:teuthology.orchestra.run.smithi151.stdout: osd map max advance = 10 2024-04-23T14:31:08.835 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.836 INFO:teuthology.orchestra.run.smithi151.stdout: journal zero on create = true 2024-04-23T14:31:08.836 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.836 INFO:teuthology.orchestra.run.smithi151.stdout: filestore ondisk finisher threads = 3 2024-04-23T14:31:08.836 INFO:teuthology.orchestra.run.smithi151.stdout: filestore apply finisher threads = 3 2024-04-23T14:31:08.836 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.836 INFO:teuthology.orchestra.run.smithi151.stdout: bdev debug aio = true 2024-04-23T14:31:08.836 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug misdirected ops = true 2024-04-23T14:31:08.836 INFO:teuthology.orchestra.run.smithi151.stdout: bluestore block size = 96636764160 2024-04-23T14:31:08.836 INFO:teuthology.orchestra.run.smithi151.stdout: bluestore fsck on mount = True 2024-04-23T14:31:08.836 INFO:teuthology.orchestra.run.smithi151.stdout: debug bluefs = 20 2024-04-23T14:31:08.836 INFO:teuthology.orchestra.run.smithi151.stdout: debug bluestore = 20 2024-04-23T14:31:08.836 INFO:teuthology.orchestra.run.smithi151.stdout: debug ms = 1 2024-04-23T14:31:08.836 INFO:teuthology.orchestra.run.smithi151.stdout: debug osd = 20 2024-04-23T14:31:08.837 INFO:teuthology.orchestra.run.smithi151.stdout: debug rocksdb = 10 2024-04-23T14:31:08.837 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd backfillfull_ratio = 0.85 2024-04-23T14:31:08.837 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd full ratio = 0.9 2024-04-23T14:31:08.837 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd nearfull ratio = 0.8 2024-04-23T14:31:08.837 INFO:teuthology.orchestra.run.smithi151.stdout: osd failsafe full ratio = 0.95 2024-04-23T14:31:08.837 INFO:teuthology.orchestra.run.smithi151.stdout: osd objectstore = bluestore 2024-04-23T14:31:08.837 INFO:teuthology.orchestra.run.smithi151.stdout: osd op complaint time = 180 2024-04-23T14:31:08.837 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.837 INFO:teuthology.orchestra.run.smithi151.stdout:[mgr] 2024-04-23T14:31:08.837 INFO:teuthology.orchestra.run.smithi151.stdout: debug ms = 1 2024-04-23T14:31:08.837 INFO:teuthology.orchestra.run.smithi151.stdout: debug mgr = 20 2024-04-23T14:31:08.837 INFO:teuthology.orchestra.run.smithi151.stdout: debug mon = 20 2024-04-23T14:31:08.837 INFO:teuthology.orchestra.run.smithi151.stdout: debug auth = 20 2024-04-23T14:31:08.837 INFO:teuthology.orchestra.run.smithi151.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:31:08.838 INFO:teuthology.orchestra.run.smithi151.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:31:08.838 INFO:teuthology.orchestra.run.smithi151.stdout: mgr/telemetry/nag = false 2024-04-23T14:31:08.838 INFO:teuthology.orchestra.run.smithi151.stdout: debug client = 20 2024-04-23T14:31:08.838 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.838 INFO:teuthology.orchestra.run.smithi151.stdout:[mon] 2024-04-23T14:31:08.838 INFO:teuthology.orchestra.run.smithi151.stdout: debug ms = 1 2024-04-23T14:31:08.838 INFO:teuthology.orchestra.run.smithi151.stdout: debug mon = 20 2024-04-23T14:31:08.838 INFO:teuthology.orchestra.run.smithi151.stdout: debug paxos = 20 2024-04-23T14:31:08.838 INFO:teuthology.orchestra.run.smithi151.stdout: debug auth = 20 2024-04-23T14:31:08.838 INFO:teuthology.orchestra.run.smithi151.stdout: mon data avail warn = 5 2024-04-23T14:31:08.838 INFO:teuthology.orchestra.run.smithi151.stdout: mon mgr mkfs grace = 240 2024-04-23T14:31:08.838 INFO:teuthology.orchestra.run.smithi151.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:31:08.838 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd reporter subtree level = osd 2024-04-23T14:31:08.838 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd prime pg temp = true 2024-04-23T14:31:08.838 INFO:teuthology.orchestra.run.smithi151.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:31:08.839 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.839 INFO:teuthology.orchestra.run.smithi151.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-23T14:31:08.839 INFO:teuthology.orchestra.run.smithi151.stdout: auth mon ticket ttl = 660 # 11m 2024-04-23T14:31:08.839 INFO:teuthology.orchestra.run.smithi151.stdout: auth service ticket ttl = 240 # 4m 2024-04-23T14:31:08.839 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.839 INFO:teuthology.orchestra.run.smithi151.stdout: # don't complain about insecure global_id in the test suite 2024-04-23T14:31:08.839 INFO:teuthology.orchestra.run.smithi151.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-23T14:31:08.839 INFO:teuthology.orchestra.run.smithi151.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-23T14:31:08.839 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.839 INFO:teuthology.orchestra.run.smithi151.stdout: # 1m isn't quite enough 2024-04-23T14:31:08.839 INFO:teuthology.orchestra.run.smithi151.stdout: mon_down_mkfs_grace = 2m 2024-04-23T14:31:08.839 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.839 INFO:teuthology.orchestra.run.smithi151.stdout: mon_warn_on_filestore_osds = false 2024-04-23T14:31:08.839 INFO:teuthology.orchestra.run.smithi151.stdout: mon down mkfs grace = 300 2024-04-23T14:31:08.840 INFO:teuthology.orchestra.run.smithi151.stdout: mon op complaint time = 120 2024-04-23T14:31:08.840 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-23T14:31:08.840 INFO:teuthology.orchestra.run.smithi151.stdout:[client] 2024-04-23T14:31:08.840 INFO:teuthology.orchestra.run.smithi151.stdout: rgw cache enabled = true 2024-04-23T14:31:08.840 INFO:teuthology.orchestra.run.smithi151.stdout: rgw enable ops log = true 2024-04-23T14:31:08.840 INFO:teuthology.orchestra.run.smithi151.stdout: rgw enable usage log = true 2024-04-23T14:31:08.840 INFO:teuthology.orchestra.run.smithi151.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-23T14:31:08.840 INFO:teuthology.orchestra.run.smithi151.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-23T14:31:08.840 INFO:teuthology.orchestra.run.smithi151.stdout: client mount timeout = 600 2024-04-23T14:31:08.840 INFO:teuthology.orchestra.run.smithi151.stdout: debug client = 20 2024-04-23T14:31:08.840 INFO:teuthology.orchestra.run.smithi151.stdout: debug ms = 1 2024-04-23T14:31:08.840 INFO:teuthology.orchestra.run.smithi151.stdout: pid file = "" 2024-04-23T14:31:08.840 INFO:teuthology.orchestra.run.smithi151.stdout: rados mon op timeout = 900 2024-04-23T14:31:08.840 INFO:teuthology.orchestra.run.smithi151.stdout: rados osd op timeout = 900 2024-04-23T14:31:08.840 INFO:teuthology.orchestra.run.smithi151.stdout:[mon.a] 2024-04-23T14:31:08.841 INFO:teuthology.orchestra.run.smithi151.stdout:[mon.b] 2024-04-23T14:31:08.841 INFO:teuthology.orchestra.run.smithi151.stdout:[mon.c] 2024-04-23T14:31:08.841 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.a] 2024-04-23T14:31:08.841 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.c] 2024-04-23T14:31:08.841 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.b] 2024-04-23T14:31:08.841 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.d] 2024-04-23T14:31:08.841 INFO:teuthology.orchestra.run.smithi151.stdout:[mds] 2024-04-23T14:31:08.841 INFO:teuthology.orchestra.run.smithi151.stdout: debug mds = 20 2024-04-23T14:31:08.841 INFO:teuthology.orchestra.run.smithi151.stdout: debug mds balancer = 20 2024-04-23T14:31:08.841 INFO:teuthology.orchestra.run.smithi151.stdout: debug ms = 1 2024-04-23T14:31:08.841 INFO:teuthology.orchestra.run.smithi151.stdout: mds debug frag = True 2024-04-23T14:31:08.841 INFO:teuthology.orchestra.run.smithi151.stdout: mds debug scatterstat = True 2024-04-23T14:31:08.841 INFO:teuthology.orchestra.run.smithi151.stdout: mds op complaint time = 180 2024-04-23T14:31:08.841 INFO:teuthology.orchestra.run.smithi151.stdout: mds verify scatter = True 2024-04-23T14:31:08.841 INFO:teuthology.orchestra.run.smithi151.stdout: osd op complaint time = 180 2024-04-23T14:31:08.841 INFO:teuthology.orchestra.run.smithi151.stdout: rados mon op timeout = 900 2024-04-23T14:31:08.842 INFO:teuthology.orchestra.run.smithi151.stdout: rados osd op timeout = 900 2024-04-23T14:31:08.843 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-23T14:31:08.843 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-23T14:31:08.945 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-23T14:31:08.945 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:08.946 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-23T14:31:08.954 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:08.954 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-23T14:31:09.010 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:31:09.010 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:09.010 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-23T14:31:09.010 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-23T14:31:09.085 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:09.085 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:31:09.138 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi151.front.sepia.ceph.com 2024-04-23T14:31:09.138 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:31:09.138 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-23T14:31:09.138 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-23T14:31:09.173 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:31:09.173 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:31:09.226 INFO:tasks.ceph:Setting up mon nodes... 2024-04-23T14:31:09.226 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-23T14:31:09.227 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-23T14:31:09.279 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-23T14:31:09.285 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-23T14:31:09.343 INFO:teuthology.orchestra.run.smithi151.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-23T14:31:09.360 INFO:tasks.ceph:Setting up mds nodes... 2024-04-23T14:31:09.361 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-23T14:31:09.414 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-23T14:31:09.418 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-23T14:31:09.438 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-23T14:31:09.531 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-23T14:31:09.535 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-23T14:31:09.554 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-23T14:31:09.612 INFO:teuthology.orchestra.run.smithi151.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-23T14:31:09.617 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-23T14:31:09.638 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-23T14:31:09.737 INFO:teuthology.orchestra.run.smithi151.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-23T14:31:09.741 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-23T14:31:09.761 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-23T14:31:09.761 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-23T14:31:09.802 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-23T14:31:09.819 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-23T14:31:09.903 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-23T14:31:09.920 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-23T14:31:09.961 INFO:teuthology.orchestra.run.smithi151.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-23T14:31:09.978 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-23T14:31:10.059 INFO:teuthology.orchestra.run.smithi151.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-23T14:31:10.076 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-23T14:31:10.076 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi100.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi151.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-23T14:31:10.076 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-23T14:31:10.095 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-23T14:31:10.095 INFO:tasks.ceph:role: osd.0 2024-04-23T14:31:10.095 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:31:10.095 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-23T14:31:10.168 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T14:31:10.168 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T14:31:10.168 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T14:31:10.168 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-23T14:31:10.169 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T14:31:10.169 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-04-23T14:31:10.169 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T14:31:10.169 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-23T14:31:10.169 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T14:31:10.169 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T14:31:10.552 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-04-23T14:31:10.555 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-04-23T14:31:10.555 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-23T14:31:10.653 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-23T14:31:10.669 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: /sbin/restorecon: command not found 2024-04-23T14:31:10.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:31:10.670 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-23T14:31:10.729 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-23T14:31:10.729 INFO:tasks.ceph:role: osd.1 2024-04-23T14:31:10.729 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:31:10.729 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-23T14:31:10.798 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T14:31:10.798 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T14:31:10.798 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T14:31:10.798 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-23T14:31:10.798 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T14:31:10.798 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-04-23T14:31:10.798 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T14:31:10.798 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-23T14:31:10.799 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T14:31:10.799 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T14:31:11.197 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-04-23T14:31:11.200 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-04-23T14:31:11.201 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-23T14:31:11.321 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-23T14:31:11.337 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: /sbin/restorecon: command not found 2024-04-23T14:31:11.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:31:11.338 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-23T14:31:11.397 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-23T14:31:11.397 INFO:tasks.ceph:role: osd.2 2024-04-23T14:31:11.397 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:31:11.397 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-23T14:31:11.469 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T14:31:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T14:31:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T14:31:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-23T14:31:11.470 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T14:31:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-04-23T14:31:11.470 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T14:31:11.470 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-23T14:31:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T14:31:11.470 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T14:31:11.876 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-04-23T14:31:11.879 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-04-23T14:31:11.879 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-23T14:31:12.020 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-23T14:31:12.037 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: /sbin/restorecon: command not found 2024-04-23T14:31:12.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:31:12.038 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-23T14:31:12.103 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-23T14:31:12.103 INFO:tasks.ceph:role: osd.3 2024-04-23T14:31:12.103 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:31:12.104 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-23T14:31:12.175 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T14:31:12.176 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T14:31:12.176 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T14:31:12.176 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-23T14:31:12.176 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T14:31:12.176 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-04-23T14:31:12.176 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T14:31:12.176 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-23T14:31:12.176 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T14:31:12.176 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T14:31:12.533 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-04-23T14:31:12.537 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-04-23T14:31:12.537 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-23T14:31:12.615 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-23T14:31:12.679 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: /sbin/restorecon: command not found 2024-04-23T14:31:12.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:31:12.680 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:31:12.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:12.804+0000 7f0086bab7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-23T14:31:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:12.804+0000 7f0086bab7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-23T14:31:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:12.804+0000 7f0086bab7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-23T14:31:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:12.804+0000 7f0086bab7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-23T14:31:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:12.804+0000 7f0086bab7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-23T14:31:12.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:12.812+0000 7f0086bab7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-23T14:31:15.854 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-23T14:31:15.874 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:31:15.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:15.968+0000 7f52279d67c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-23T14:31:15.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:15.968+0000 7f52279d67c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-23T14:31:15.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:15.968+0000 7f52279d67c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-23T14:31:15.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:15.968+0000 7f52279d67c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-23T14:31:15.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:15.968+0000 7f52279d67c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-23T14:31:15.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:15.972+0000 7f52279d67c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-23T14:31:18.991 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-23T14:31:19.012 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:31:19.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:19.107+0000 7f98d3dd67c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-23T14:31:19.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:19.107+0000 7f98d3dd67c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-23T14:31:19.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:19.107+0000 7f98d3dd67c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-23T14:31:19.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:19.107+0000 7f98d3dd67c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-23T14:31:19.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:19.107+0000 7f98d3dd67c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-23T14:31:19.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:19.111+0000 7f98d3dd67c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-23T14:31:22.157 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-23T14:31:22.178 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:31:22.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:22.279+0000 7f9ec3fe17c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-23T14:31:22.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:22.279+0000 7f9ec3fe17c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-23T14:31:22.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:22.279+0000 7f9ec3fe17c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-23T14:31:22.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:22.279+0000 7f9ec3fe17c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-23T14:31:22.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:22.279+0000 7f9ec3fe17c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-23T14:31:22.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:22.279+0000 7f9ec3fe17c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-23T14:31:25.330 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-23T14:31:25.351 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-23T14:31:25.374 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-23T14:31:25.374 INFO:tasks.ceph:role: osd.4 2024-04-23T14:31:25.374 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi151.front.sepia.ceph.com 2024-04-23T14:31:25.374 DEBUG:teuthology.orchestra.run.smithi151:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-23T14:31:25.447 INFO:teuthology.orchestra.run.smithi151.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T14:31:25.447 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T14:31:25.447 INFO:teuthology.orchestra.run.smithi151.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T14:31:25.447 INFO:teuthology.orchestra.run.smithi151.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-23T14:31:25.447 INFO:teuthology.orchestra.run.smithi151.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T14:31:25.447 INFO:teuthology.orchestra.run.smithi151.stdout: = sunit=0 swidth=0 blks 2024-04-23T14:31:25.447 INFO:teuthology.orchestra.run.smithi151.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T14:31:25.447 INFO:teuthology.orchestra.run.smithi151.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-23T14:31:25.447 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T14:31:25.447 INFO:teuthology.orchestra.run.smithi151.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T14:31:25.852 INFO:teuthology.orchestra.run.smithi151.stdout:Discarding blocks...Done. 2024-04-23T14:31:25.855 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi151.front.sepia.ceph.com -o noatime 2024-04-23T14:31:25.855 DEBUG:teuthology.orchestra.run.smithi151:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-23T14:31:25.954 DEBUG:teuthology.orchestra.run.smithi151:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-23T14:31:25.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:31:25.972 INFO:teuthology.orchestra.run.smithi151.stderr:sudo: /sbin/restorecon: command not found 2024-04-23T14:31:25.973 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-23T14:31:26.035 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-23T14:31:26.035 INFO:tasks.ceph:role: osd.5 2024-04-23T14:31:26.035 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi151.front.sepia.ceph.com 2024-04-23T14:31:26.035 DEBUG:teuthology.orchestra.run.smithi151:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-23T14:31:26.107 INFO:teuthology.orchestra.run.smithi151.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T14:31:26.108 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T14:31:26.108 INFO:teuthology.orchestra.run.smithi151.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T14:31:26.108 INFO:teuthology.orchestra.run.smithi151.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-23T14:31:26.108 INFO:teuthology.orchestra.run.smithi151.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T14:31:26.108 INFO:teuthology.orchestra.run.smithi151.stdout: = sunit=0 swidth=0 blks 2024-04-23T14:31:26.108 INFO:teuthology.orchestra.run.smithi151.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T14:31:26.109 INFO:teuthology.orchestra.run.smithi151.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-23T14:31:26.109 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T14:31:26.109 INFO:teuthology.orchestra.run.smithi151.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T14:31:26.513 INFO:teuthology.orchestra.run.smithi151.stdout:Discarding blocks...Done. 2024-04-23T14:31:26.516 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi151.front.sepia.ceph.com -o noatime 2024-04-23T14:31:26.516 DEBUG:teuthology.orchestra.run.smithi151:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-23T14:31:26.641 DEBUG:teuthology.orchestra.run.smithi151:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-23T14:31:26.657 INFO:teuthology.orchestra.run.smithi151.stderr:sudo: /sbin/restorecon: command not found 2024-04-23T14:31:26.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:31:26.658 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-23T14:31:26.727 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-23T14:31:26.727 INFO:tasks.ceph:role: osd.6 2024-04-23T14:31:26.727 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi151.front.sepia.ceph.com 2024-04-23T14:31:26.727 DEBUG:teuthology.orchestra.run.smithi151:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-23T14:31:26.802 INFO:teuthology.orchestra.run.smithi151.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T14:31:26.802 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T14:31:26.803 INFO:teuthology.orchestra.run.smithi151.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T14:31:26.803 INFO:teuthology.orchestra.run.smithi151.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-23T14:31:26.803 INFO:teuthology.orchestra.run.smithi151.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T14:31:26.803 INFO:teuthology.orchestra.run.smithi151.stdout: = sunit=0 swidth=0 blks 2024-04-23T14:31:26.803 INFO:teuthology.orchestra.run.smithi151.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T14:31:26.803 INFO:teuthology.orchestra.run.smithi151.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-23T14:31:26.803 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T14:31:26.803 INFO:teuthology.orchestra.run.smithi151.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T14:31:27.220 INFO:teuthology.orchestra.run.smithi151.stdout:Discarding blocks...Done. 2024-04-23T14:31:27.223 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi151.front.sepia.ceph.com -o noatime 2024-04-23T14:31:27.223 DEBUG:teuthology.orchestra.run.smithi151:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-23T14:31:27.347 DEBUG:teuthology.orchestra.run.smithi151:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-23T14:31:27.364 INFO:teuthology.orchestra.run.smithi151.stderr:sudo: /sbin/restorecon: command not found 2024-04-23T14:31:27.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:31:27.365 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-23T14:31:27.431 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-23T14:31:27.431 INFO:tasks.ceph:role: osd.7 2024-04-23T14:31:27.431 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi151.front.sepia.ceph.com 2024-04-23T14:31:27.431 DEBUG:teuthology.orchestra.run.smithi151:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-23T14:31:27.502 INFO:teuthology.orchestra.run.smithi151.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T14:31:27.502 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T14:31:27.502 INFO:teuthology.orchestra.run.smithi151.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T14:31:27.503 INFO:teuthology.orchestra.run.smithi151.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-23T14:31:27.503 INFO:teuthology.orchestra.run.smithi151.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T14:31:27.503 INFO:teuthology.orchestra.run.smithi151.stdout: = sunit=0 swidth=0 blks 2024-04-23T14:31:27.503 INFO:teuthology.orchestra.run.smithi151.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T14:31:27.503 INFO:teuthology.orchestra.run.smithi151.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-23T14:31:27.503 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T14:31:27.503 INFO:teuthology.orchestra.run.smithi151.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T14:31:27.870 INFO:teuthology.orchestra.run.smithi151.stdout:Discarding blocks...Done. 2024-04-23T14:31:27.873 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi151.front.sepia.ceph.com -o noatime 2024-04-23T14:31:27.873 DEBUG:teuthology.orchestra.run.smithi151:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-23T14:31:27.948 DEBUG:teuthology.orchestra.run.smithi151:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-23T14:31:28.010 INFO:teuthology.orchestra.run.smithi151.stderr:sudo: /sbin/restorecon: command not found 2024-04-23T14:31:28.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:31:28.012 DEBUG:teuthology.orchestra.run.smithi151:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:31:28.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:28.144+0000 7ff2627677c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-23T14:31:28.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:28.148+0000 7ff2627677c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-23T14:31:28.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:28.148+0000 7ff2627677c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-23T14:31:28.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:28.148+0000 7ff2627677c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-23T14:31:28.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:28.148+0000 7ff2627677c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-23T14:31:28.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:28.148+0000 7ff2627677c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-23T14:31:31.190 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-23T14:31:31.211 DEBUG:teuthology.orchestra.run.smithi151:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:31:31.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:31.308+0000 7f49790e07c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-23T14:31:31.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:31.312+0000 7f49790e07c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-23T14:31:31.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:31.312+0000 7f49790e07c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-23T14:31:31.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:31.312+0000 7f49790e07c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-23T14:31:31.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:31.312+0000 7f49790e07c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-23T14:31:31.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:31.312+0000 7f49790e07c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-23T14:31:34.362 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-23T14:31:34.383 DEBUG:teuthology.orchestra.run.smithi151:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:31:34.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:34.484+0000 7f759322b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-23T14:31:34.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:34.484+0000 7f759322b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-23T14:31:34.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:34.484+0000 7f759322b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-23T14:31:34.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:34.484+0000 7f759322b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-23T14:31:34.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:34.484+0000 7f759322b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-23T14:31:34.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:34.484+0000 7f759322b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-23T14:31:36.887 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:31:36.891 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:31:37.538 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-23T14:31:37.559 DEBUG:teuthology.orchestra.run.smithi151:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:31:37.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:37.660+0000 7fc7768397c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-23T14:31:37.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:37.660+0000 7fc7768397c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-23T14:31:37.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:37.660+0000 7fc7768397c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-23T14:31:37.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:37.660+0000 7fc7768397c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-23T14:31:37.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:37.660+0000 7fc7768397c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-23T14:31:37.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:37.660+0000 7fc7768397c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-23T14:31:40.703 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-23T14:31:40.723 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-23T14:31:40.723 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:40.723 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-23T14:31:40.745 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:40.745 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-23T14:31:40.812 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:40.813 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-23T14:31:40.875 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:40.875 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-23T14:31:40.939 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:40.939 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-23T14:31:41.004 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:41.004 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-23T14:31:41.071 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:41.072 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-23T14:31:41.132 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:31:41.132 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-23T14:31:41.153 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:31:41.153 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-23T14:31:41.218 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:31:41.218 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-23T14:31:41.282 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:31:41.282 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-23T14:31:41.342 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:31:41.342 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-23T14:31:41.406 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:31:41.406 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-23T14:31:41.466 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:31:41.466 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-23T14:31:41.525 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:41.526 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-23T14:31:41.534 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:41.534 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-23T14:31:41.588 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:31:41.588 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-23T14:31:41.597 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:31:41.597 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-23T14:31:41.650 INFO:tasks.ceph:Adding keys to all mons... 2024-04-23T14:31:41.650 DEBUG:teuthology.orchestra.run.smithi100:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-23T14:31:41.654 DEBUG:teuthology.orchestra.run.smithi151:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-23T14:31:41.702 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr.y] 2024-04-23T14:31:41.702 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAtxidmNTL4EBAAj88L1GzandAkaqg9AxEQqg== 2024-04-23T14:31:41.702 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2024-04-23T14:31:41.703 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAtxidmcgr8GBAA944tuhsKzhnsmZrdxOKbAA== 2024-04-23T14:31:41.703 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2024-04-23T14:31:41.703 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAtxidmrYjvHxAABsf7ouy0ULqAFxwPrLCMsg== 2024-04-23T14:31:41.703 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.0] 2024-04-23T14:31:41.703 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAwxidmtJJBMBAAktb3nILtB6ywDU5lca43OQ== 2024-04-23T14:31:41.703 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.1] 2024-04-23T14:31:41.703 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAzxidmUgz/ORAASX3vikkabjnWpzY1efthig== 2024-04-23T14:31:41.703 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.2] 2024-04-23T14:31:41.703 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQA3xidmUzrNBhAAI8ymZvKTD5dm+gLYh42WMA== 2024-04-23T14:31:41.703 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.3] 2024-04-23T14:31:41.703 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQA6xidmfEC8EBAAiv84aaTlpMGaG6ZFxKp2ng== 2024-04-23T14:31:41.703 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr.x] 2024-04-23T14:31:41.703 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAtxidmKdCRFBAA4h8/t4aNmcR58fKgXGUB4Q== 2024-04-23T14:31:41.703 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2024-04-23T14:31:41.703 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAtxidmJcqnJBAA95zJpqTJW92I+2Nnh7SpXg== 2024-04-23T14:31:41.704 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2024-04-23T14:31:41.704 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAtxidmmN4JLBAANawA9RtKrB31GqCz/wuz3Q== 2024-04-23T14:31:41.704 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.4] 2024-04-23T14:31:41.704 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBAxidmWoujCBAAclplSDUC/Hwhh4a4NWgH/Q== 2024-04-23T14:31:41.704 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.5] 2024-04-23T14:31:41.704 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBDxidmRO+eEhAADfwa1qucvA1FGwuIY0A62w== 2024-04-23T14:31:41.704 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.6] 2024-04-23T14:31:41.704 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBGxidmMujsHBAAKNjjM4nlyaUVesLu1CixhA== 2024-04-23T14:31:41.704 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.7] 2024-04-23T14:31:41.704 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBJxidmHxlnJxAAk69hiCwS8GrIPMcLV+iVpg== 2024-04-23T14:31:41.704 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-04-23T14:31:41.704 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAtxidmyhImMBAAa49ApQEF2Lng6rY94F1HAQ== 2024-04-23T14:31:41.704 INFO:teuthology.orchestra.run.smithi100.stdout:[client.2] 2024-04-23T14:31:41.704 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAtxidm34okNhAAkagR1bwlKj7wxsY/E/nRGg== 2024-04-23T14:31:41.705 INFO:teuthology.orchestra.run.smithi100.stdout:[client.1] 2024-04-23T14:31:41.705 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAtxidmpLhsORAAFLucAGu/GbEJTmkxy0LY8Q== 2024-04-23T14:31:41.705 INFO:teuthology.orchestra.run.smithi100.stdout:[client.3] 2024-04-23T14:31:41.705 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAuxidmtJWrAxAAYBGSngF85+O1YO5on41OMw== 2024-04-23T14:31:41.716 INFO:teuthology.orchestra.run.smithi151.stdout:[mgr.y] 2024-04-23T14:31:41.716 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQAtxidmNTL4EBAAj88L1GzandAkaqg9AxEQqg== 2024-04-23T14:31:41.716 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.a] 2024-04-23T14:31:41.716 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQAtxidmcgr8GBAA944tuhsKzhnsmZrdxOKbAA== 2024-04-23T14:31:41.716 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.c] 2024-04-23T14:31:41.716 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQAtxidmrYjvHxAABsf7ouy0ULqAFxwPrLCMsg== 2024-04-23T14:31:41.716 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.0] 2024-04-23T14:31:41.716 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQAwxidmtJJBMBAAktb3nILtB6ywDU5lca43OQ== 2024-04-23T14:31:41.716 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.1] 2024-04-23T14:31:41.716 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQAzxidmUgz/ORAASX3vikkabjnWpzY1efthig== 2024-04-23T14:31:41.716 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.2] 2024-04-23T14:31:41.717 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQA3xidmUzrNBhAAI8ymZvKTD5dm+gLYh42WMA== 2024-04-23T14:31:41.717 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.3] 2024-04-23T14:31:41.717 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQA6xidmfEC8EBAAiv84aaTlpMGaG6ZFxKp2ng== 2024-04-23T14:31:41.717 INFO:teuthology.orchestra.run.smithi151.stdout:[mgr.x] 2024-04-23T14:31:41.717 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQAtxidmKdCRFBAA4h8/t4aNmcR58fKgXGUB4Q== 2024-04-23T14:31:41.717 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.b] 2024-04-23T14:31:41.717 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQAtxidmJcqnJBAA95zJpqTJW92I+2Nnh7SpXg== 2024-04-23T14:31:41.717 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.d] 2024-04-23T14:31:41.717 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQAtxidmmN4JLBAANawA9RtKrB31GqCz/wuz3Q== 2024-04-23T14:31:41.717 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.4] 2024-04-23T14:31:41.717 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQBAxidmWoujCBAAclplSDUC/Hwhh4a4NWgH/Q== 2024-04-23T14:31:41.717 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.5] 2024-04-23T14:31:41.718 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQBDxidmRO+eEhAADfwa1qucvA1FGwuIY0A62w== 2024-04-23T14:31:41.718 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.6] 2024-04-23T14:31:41.718 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQBGxidmMujsHBAAKNjjM4nlyaUVesLu1CixhA== 2024-04-23T14:31:41.718 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.7] 2024-04-23T14:31:41.718 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQBJxidmHxlnJxAAk69hiCwS8GrIPMcLV+iVpg== 2024-04-23T14:31:41.718 INFO:teuthology.orchestra.run.smithi151.stdout:[client.0] 2024-04-23T14:31:41.718 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQAtxidmyhImMBAAa49ApQEF2Lng6rY94F1HAQ== 2024-04-23T14:31:41.718 INFO:teuthology.orchestra.run.smithi151.stdout:[client.2] 2024-04-23T14:31:41.718 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQAtxidm34okNhAAkagR1bwlKj7wxsY/E/nRGg== 2024-04-23T14:31:41.718 INFO:teuthology.orchestra.run.smithi151.stdout:[client.1] 2024-04-23T14:31:41.718 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQAtxidmpLhsORAAFLucAGu/GbEJTmkxy0LY8Q== 2024-04-23T14:31:41.718 INFO:teuthology.orchestra.run.smithi151.stdout:[client.3] 2024-04-23T14:31:41.719 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQAuxidmtJWrAxAAYBGSngF85+O1YO5on41OMw== 2024-04-23T14:31:41.719 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-23T14:31:41.751 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-23T14:31:41.821 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-23T14:31:41.825 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-23T14:31:41.881 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-23T14:31:41.884 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-23T14:31:41.940 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:31:41.943 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:31:41.996 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:31:41.999 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:31:42.052 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:31:42.054 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:31:42.108 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:31:42.111 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:31:42.164 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-23T14:31:42.167 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-23T14:31:42.221 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-23T14:31:42.224 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-23T14:31:42.281 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-23T14:31:42.284 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-23T14:31:42.340 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:31:42.343 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:31:42.396 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:31:42.400 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:31:42.453 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:31:42.457 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:31:42.510 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:31:42.513 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:31:42.565 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T14:31:42.568 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T14:31:42.623 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T14:31:42.626 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T14:31:42.682 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T14:31:42.685 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T14:31:42.740 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T14:31:42.743 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T14:31:42.798 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-23T14:31:42.798 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-23T14:31:42.817 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-23T14:31:42.931 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-23T14:31:42.951 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-23T14:31:42.970 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-23T14:31:43.077 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-23T14:31:43.096 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-23T14:31:43.161 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-23T14:31:43.273 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-23T14:31:43.293 DEBUG:teuthology.orchestra.run.smithi100:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:31:43.296 DEBUG:teuthology.orchestra.run.smithi151:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:31:43.345 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-23T14:31:43.346 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-23T14:31:43.347 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-23T14:31:43.350 INFO:tasks.ceph.mon.a:Started 2024-04-23T14:31:43.350 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-23T14:31:43.350 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-23T14:31:43.389 INFO:tasks.ceph.mon.b:Started 2024-04-23T14:31:43.389 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-23T14:31:43.389 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-23T14:31:43.392 INFO:tasks.ceph.mon.c:Started 2024-04-23T14:31:43.392 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-23T14:31:43.392 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-23T14:31:43.392 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-23T14:31:43.395 INFO:tasks.ceph.mgr.y:Started 2024-04-23T14:31:43.395 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-23T14:31:43.395 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-23T14:31:43.398 INFO:tasks.ceph.mgr.x:Started 2024-04-23T14:31:43.399 INFO:tasks.ceph:Setting crush tunables to default 2024-04-23T14:31:43.399 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd crush tunables default 2024-04-23T14:31:43.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.534+0000 7f0289200640 1 Processor -- start 2024-04-23T14:31:43.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.534+0000 7f0289200640 1 -- start start 2024-04-23T14:31:43.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.538+0000 7f0289200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f0284146090 0x7f02840cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.538+0000 7f0289200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f028415d250 0x7f028406c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.538+0000 7f0289200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02840d0380 con 0x7f028406f870 2024-04-23T14:31:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.538+0000 7f0289200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f02840d0660 con 0x7f028415d250 2024-04-23T14:31:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.538+0000 7f0289200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f02840d0940 con 0x7f0284146090 2024-04-23T14:31:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.538+0000 7f0282800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f028415d250 0x7f028406c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.538+0000 7f0282800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f028415d250 0x7f028406c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48250/0 (socket says 172.21.15.100:48250) 2024-04-23T14:31:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.538+0000 7f0282800640 1 -- 172.21.15.100:0/1041738385 learned_addr learned my addr 172.21.15.100:0/1041738385 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:31:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.538+0000 7f0282800640 1 -- 172.21.15.100:0/1041738385 >> v2:172.21.15.100:3300/0 conn(0x7f028415d250 msgr2=0x7f028406c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-23T14:31:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.538+0000 7f0282800640 1 -- 172.21.15.100:0/1041738385 >> v2:172.21.15.100:3300/0 conn(0x7f028415d250 msgr2=0x7f028406c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-23T14:31:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.538+0000 7f0282800640 1 --2- 172.21.15.100:0/1041738385 >> v2:172.21.15.100:3300/0 conn(0x7f028415d250 0x7f028406c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-23T14:31:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.538+0000 7f0281e00640 1 --2- 172.21.15.100:0/1041738385 >> v2:172.21.15.151:3300/0 conn(0x7f0284146090 0x7f02840cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.538+0000 7f0282800640 1 --2- 172.21.15.100:0/1041738385 >> v2:172.21.15.100:3300/0 conn(0x7f028415d250 0x7f028406c4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T14:31:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.538+0000 7f0281e00640 1 -- 172.21.15.100:0/1041738385 >> v2:172.21.15.151:3300/0 conn(0x7f0284146090 msgr2=0x7f02840cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-23T14:31:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.538+0000 7f0281e00640 1 -- 172.21.15.100:0/1041738385 >> v2:172.21.15.151:3300/0 conn(0x7f0284146090 msgr2=0x7f02840cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-23T14:31:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.538+0000 7f0281e00640 1 --2- 172.21.15.100:0/1041738385 >> v2:172.21.15.151:3300/0 conn(0x7f0284146090 0x7f02840cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-23T14:31:43.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.538+0000 7f0281e00640 1 --2- 172.21.15.100:0/1041738385 >> v2:172.21.15.151:3300/0 conn(0x7f0284146090 0x7f02840cfda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T14:31:43.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.690+0000 7f0280a00640 1 -- 172.21.15.100:0/1041738385 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1340898972 0 0) 0x7f02840d0380 con 0x7f028406f870 2024-04-23T14:31:43.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.690+0000 7f0280a00640 1 -- 172.21.15.100:0/1041738385 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0268003530 con 0x7f028406f870 2024-04-23T14:31:43.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.694+0000 7f0280a00640 1 -- 172.21.15.100:0/1041738385 <== mon.1 v1:172.21.15.151:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1651932293 0 0) 0x7f0268003530 con 0x7f028406f870 2024-04-23T14:31:43.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.694+0000 7f0280a00640 1 -- 172.21.15.100:0/1041738385 >> v2:172.21.15.151:3300/0 conn(0x7f0284146090 msgr2=0x7f02840cfda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:31:43.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.694+0000 7f0280a00640 1 --2- 172.21.15.100:0/1041738385 >> v2:172.21.15.151:3300/0 conn(0x7f0284146090 0x7f02840cfda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.694+0000 7f0280a00640 1 -- 172.21.15.100:0/1041738385 >> v2:172.21.15.100:3300/0 conn(0x7f028415d250 msgr2=0x7f028406c4a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:31:43.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.694+0000 7f0280a00640 1 --2- 172.21.15.100:0/1041738385 >> v2:172.21.15.100:3300/0 conn(0x7f028415d250 0x7f028406c4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.694+0000 7f0280a00640 1 -- 172.21.15.100:0/1041738385 --> v1:172.21.15.151:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02840d0c20 con 0x7f028406f870 2024-04-23T14:31:43.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.694+0000 7f0280a00640 1 -- 172.21.15.100:0/1041738385 <== mon.1 v1:172.21.15.151:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f0270001680 con 0x7f028406f870 2024-04-23T14:31:43.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.694+0000 7f0280a00640 1 -- 172.21.15.100:0/1041738385 >> v1:172.21.15.151:6789/0 conn(0x7f028406f870 legacy=0x7f028415cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:43.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.698+0000 7f0280a00640 1 --2- 172.21.15.100:0/1041738385 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02680039c0 0x7f0268003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.698+0000 7f0280a00640 1 --2- 172.21.15.100:0/1041738385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0268004810 0x7f0268004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.698+0000 7f0280a00640 1 --2- 172.21.15.100:0/1041738385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02680051d0 0x7f026800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.698+0000 7f0280a00640 1 -- 172.21.15.100:0/1041738385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f02840d0660 con 0x7f0268004810 2024-04-23T14:31:43.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.698+0000 7f0280a00640 1 -- 172.21.15.100:0/1041738385 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f02840d0940 con 0x7f02680051d0 2024-04-23T14:31:43.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.698+0000 7f0280a00640 1 -- 172.21.15.100:0/1041738385 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f026800c0d0 con 0x7f02680039c0 2024-04-23T14:31:43.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.698+0000 7f0281e00640 1 --2- 172.21.15.100:0/1041738385 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02680039c0 0x7f0268003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.698+0000 7f0281400640 1 --2- 172.21.15.100:0/1041738385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0268004810 0x7f0268004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.698+0000 7f0282800640 1 --2- 172.21.15.100:0/1041738385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02680051d0 0x7f026800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.698+0000 7f0281e00640 1 -- 172.21.15.100:0/1041738385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02680051d0 msgr2=0x7f026800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:43.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.698+0000 7f0281e00640 1 --2- 172.21.15.100:0/1041738385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02680051d0 0x7f026800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.698+0000 7f0281e00640 1 -- 172.21.15.100:0/1041738385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0268004810 msgr2=0x7f0268004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:43.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.698+0000 7f0281e00640 1 --2- 172.21.15.100:0/1041738385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0268004810 0x7f0268004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.698+0000 7f0281e00640 1 -- 172.21.15.100:0/1041738385 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0268003530 con 0x7f02680039c0 2024-04-23T14:31:43.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.698+0000 7f0282800640 1 -- 172.21.15.100:0/1041738385 reap_dead start 2024-04-23T14:31:43.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.702+0000 7f0281e00640 1 --2- 172.21.15.100:0/1041738385 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02680039c0 0x7f0268003de0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0274009fd0 tx=0x7f027401ce20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b81e2d10a9e39ef server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:43.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.702+0000 7f0280a00640 1 -- 172.21.15.100:0/1041738385 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02740b2070 con 0x7f02680039c0 2024-04-23T14:31:43.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.702+0000 7f0280a00640 1 -- 172.21.15.100:0/1041738385 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02740a5c10 con 0x7f02680039c0 2024-04-23T14:31:43.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.702+0000 7f0280a00640 1 -- 172.21.15.100:0/1041738385 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274009bd0 con 0x7f02680039c0 2024-04-23T14:31:43.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.702+0000 7f0289200640 1 -- 172.21.15.100:0/1041738385 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02680039c0 msgr2=0x7f0268003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:43.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.702+0000 7f0289200640 1 --2- 172.21.15.100:0/1041738385 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02680039c0 0x7f0268003de0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0274009fd0 tx=0x7f027401ce20 comp rx=0 tx=0).stop 2024-04-23T14:31:43.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.702+0000 7f0289200640 1 -- 172.21.15.100:0/1041738385 shutdown_connections 2024-04-23T14:31:43.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.702+0000 7f0289200640 1 --2- 172.21.15.100:0/1041738385 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02680039c0 0x7f0268003de0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.702+0000 7f0289200640 1 -- 172.21.15.100:0/1041738385 >> 172.21.15.100:0/1041738385 conn(0x7f0284070670 msgr2=0x7f0284070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:43.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.702+0000 7f0289200640 1 -- 172.21.15.100:0/1041738385 shutdown_connections 2024-04-23T14:31:43.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.702+0000 7f0289200640 1 -- 172.21.15.100:0/1041738385 wait complete. 2024-04-23T14:31:43.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.702+0000 7f0289200640 1 Processor -- start 2024-04-23T14:31:43.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.702+0000 7f0289200640 1 -- start start 2024-04-23T14:31:43.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f0289200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02680039c0 0x7f028415c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f0289200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02841f4e00 0x7f02841446e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f0289200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0284142810 0x7f0284142c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f0289200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f02840cfda0 con 0x7f02680039c0 2024-04-23T14:31:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f0289200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f028406fd00 con 0x7f0284142810 2024-04-23T14:31:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f0289200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f02840cff20 con 0x7f02841f4e00 2024-04-23T14:31:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f0281e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02680039c0 0x7f028415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f0281e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02680039c0 0x7f028415c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48302/0 (socket says 172.21.15.100:48302) 2024-04-23T14:31:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f0281e00640 1 -- 172.21.15.100:0/1041111560 learned_addr learned my addr 172.21.15.100:0/1041111560 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:31:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f0281e00640 1 -- 172.21.15.100:0/1041111560 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02841f4e00 msgr2=0x7f02841446e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f0281400640 1 --2- 172.21.15.100:0/1041111560 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02841f4e00 0x7f02841446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f0281e00640 1 --2- 172.21.15.100:0/1041111560 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02841f4e00 0x7f02841446e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f0281e00640 1 -- 172.21.15.100:0/1041111560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0284142810 msgr2=0x7f0284142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f0281e00640 1 --2- 172.21.15.100:0/1041111560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0284142810 0x7f0284142c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f0281e00640 1 -- 172.21.15.100:0/1041111560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0284145030 con 0x7f02680039c0 2024-04-23T14:31:43.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f0281e00640 1 --2- 172.21.15.100:0/1041111560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02680039c0 0x7f028415c030 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f027401cc50 tx=0x7f0274019bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:43.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f027a000640 1 -- 172.21.15.100:0/1041111560 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02740b2040 con 0x7f02680039c0 2024-04-23T14:31:43.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f027a000640 1 -- 172.21.15.100:0/1041111560 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02740a5a10 con 0x7f02680039c0 2024-04-23T14:31:43.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f0289200640 1 -- 172.21.15.100:0/1041111560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0284145310 con 0x7f02680039c0 2024-04-23T14:31:43.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f027a000640 1 -- 172.21.15.100:0/1041111560 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02740add60 con 0x7f02680039c0 2024-04-23T14:31:43.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f0289200640 1 -- 172.21.15.100:0/1041111560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f028415c570 con 0x7f02680039c0 2024-04-23T14:31:43.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f027a000640 1 -- 172.21.15.100:0/1041111560 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f02740c0020 con 0x7f02680039c0 2024-04-23T14:31:43.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f027a000640 1 -- 172.21.15.100:0/1041111560 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f02740c2530 con 0x7f02680039c0 2024-04-23T14:31:43.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.706+0000 7f0289200640 1 -- 172.21.15.100:0/1041111560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0284068ad0 con 0x7f02680039c0 2024-04-23T14:31:43.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.718+0000 7f027a000640 1 -- 172.21.15.100:0/1041111560 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f02740c7020 con 0x7f02680039c0 2024-04-23T14:31:43.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.778+0000 7f0289200640 1 -- 172.21.15.100:0/1041111560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f0284071540 con 0x7f02680039c0 2024-04-23T14:31:43.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.782+0000 7f027a000640 1 -- 172.21.15.100:0/1041111560 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f0274016b30 con 0x7f02680039c0 2024-04-23T14:31:43.783 INFO:teuthology.orchestra.run.smithi100.stderr:adjusted tunables profile to default 2024-04-23T14:31:43.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.782+0000 7f0289200640 1 -- 172.21.15.100:0/1041111560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02680039c0 msgr2=0x7f028415c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:43.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.782+0000 7f0289200640 1 --2- 172.21.15.100:0/1041111560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02680039c0 0x7f028415c030 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f027401cc50 tx=0x7f0274019bc0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.782+0000 7f0289200640 1 -- 172.21.15.100:0/1041111560 shutdown_connections 2024-04-23T14:31:43.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.782+0000 7f0289200640 1 --2- 172.21.15.100:0/1041111560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0284142810 0x7f0284142c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.782+0000 7f0289200640 1 --2- 172.21.15.100:0/1041111560 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02841f4e00 0x7f02841446e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.782+0000 7f0289200640 1 --2- 172.21.15.100:0/1041111560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02680039c0 0x7f028415c030 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.782+0000 7f0289200640 1 -- 172.21.15.100:0/1041111560 >> 172.21.15.100:0/1041111560 conn(0x7f0284070670 msgr2=0x7f02840b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:43.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.782+0000 7f0289200640 1 -- 172.21.15.100:0/1041111560 shutdown_connections 2024-04-23T14:31:43.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:43.782+0000 7f0289200640 1 -- 172.21.15.100:0/1041111560 wait complete. 2024-04-23T14:31:43.800 INFO:tasks.ceph:check_enable_crimson: False 2024-04-23T14:31:43.800 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-23T14:31:43.800 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:43.800 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-23T14:31:43.820 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:43.820 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-23T14:31:43.883 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:43.883 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-23T14:31:43.912 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:43.912+0000 7f78bb08d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T14:31:43.916 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:43.914+0000 7f94c028b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T14:31:43.934 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:31:43.934 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-23T14:31:43.995 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:31:43.996 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-23T14:31:44.016 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:31:44.016 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-23T14:31:44.077 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:31:44.077 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-23T14:31:44.137 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:31:44.137 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-23T14:31:44.197 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new ff0dca4f-928b-498f-aae5-8909e2f05df8 0 2024-04-23T14:31:44.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.372+0000 7f6432800640 1 Processor -- start 2024-04-23T14:31:44.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.372+0000 7f6432800640 1 -- start start 2024-04-23T14:31:44.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.372+0000 7f6432800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f642c0cca50 0x7f642c0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.372+0000 7f6432800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f642c15d600 0x7f642c160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.372+0000 7f6432800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f642c0cd370 con 0x7f642c15c7e0 2024-04-23T14:31:44.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.372+0000 7f6432800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f642c0d2270 con 0x7f642c0cca50 2024-04-23T14:31:44.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.372+0000 7f6432800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f642c15db00 con 0x7f642c15d600 2024-04-23T14:31:44.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.372+0000 7f6423e00640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f642c15c7e0 0x7f642c160170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.151:52880/0 (socket says 172.21.15.151:52880) 2024-04-23T14:31:44.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.372+0000 7f6423e00640 1 -- 172.21.15.151:0/3252852931 learned_addr learned my addr 172.21.15.151:0/3252852931 (peer_addr_for_me v1:172.21.15.151:0/0) 2024-04-23T14:31:44.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.372+0000 7f642be00640 1 --2- 172.21.15.151:0/3252852931 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f642c15d600 0x7f642c160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:44.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.372+0000 7f642b400640 1 --2- 172.21.15.151:0/3252852931 >> v2:172.21.15.100:3300/0 conn(0x7f642c0cca50 0x7f642c0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:44.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.376+0000 7f642be00640 1 -- 172.21.15.151:0/3252852931 >> v2:172.21.15.100:3300/0 conn(0x7f642c0cca50 msgr2=0x7f642c0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:44.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.376+0000 7f642be00640 1 --2- 172.21.15.151:0/3252852931 >> v2:172.21.15.100:3300/0 conn(0x7f642c0cca50 0x7f642c0cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.376+0000 7f642be00640 1 -- 172.21.15.151:0/3252852931 >> v1:172.21.15.151:6789/0 conn(0x7f642c15c7e0 legacy=0x7f642c160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:44.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.376+0000 7f642be00640 1 -- 172.21.15.151:0/3252852931 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642c0d04a0 con 0x7f642c15d600 2024-04-23T14:31:44.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.376+0000 7f642aa00640 1 -- 172.21.15.151:0/3252852931 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1336182120 0 0) 0x7f642c0cd370 con 0x7f642c15c7e0 2024-04-23T14:31:44.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.376+0000 7f642be00640 1 --2- 172.21.15.151:0/3252852931 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f642c15d600 0x7f642c160880 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6424009d90 tx=0x7f642401c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecc79a7515c049d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:44.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.376+0000 7f642aa00640 1 -- 172.21.15.151:0/3252852931 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64240b2070 con 0x7f642c15d600 2024-04-23T14:31:44.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.376+0000 7f642aa00640 1 -- 172.21.15.151:0/3252852931 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64240a5970 con 0x7f642c15d600 2024-04-23T14:31:44.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.376+0000 7f642aa00640 1 -- 172.21.15.151:0/3252852931 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64240a5d40 con 0x7f642c15d600 2024-04-23T14:31:44.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.376+0000 7f6432800640 1 -- 172.21.15.151:0/3252852931 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f642c15d600 msgr2=0x7f642c160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:44.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.376+0000 7f6432800640 1 --2- 172.21.15.151:0/3252852931 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f642c15d600 0x7f642c160880 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6424009d90 tx=0x7f642401c8b0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f6432800640 1 -- 172.21.15.151:0/3252852931 shutdown_connections 2024-04-23T14:31:44.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f6432800640 1 --2- 172.21.15.151:0/3252852931 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f642c15d600 0x7f642c160880 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f6432800640 1 --2- 172.21.15.151:0/3252852931 >> v2:172.21.15.100:3300/0 conn(0x7f642c0cca50 0x7f642c0cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f6432800640 1 -- 172.21.15.151:0/3252852931 >> 172.21.15.151:0/3252852931 conn(0x7f642c0cee80 msgr2=0x7f642c0cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:44.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f6432800640 1 -- 172.21.15.151:0/3252852931 shutdown_connections 2024-04-23T14:31:44.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f6432800640 1 -- 172.21.15.151:0/3252852931 wait complete. 2024-04-23T14:31:44.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f6432800640 1 Processor -- start 2024-04-23T14:31:44.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f6432800640 1 -- start start 2024-04-23T14:31:44.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f6432800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f642c0cca50 0x7f642c155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f6432800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f642c15c7e0 0x7f642c0ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f6432800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f642c15d600 0x7f642c0ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f6432800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f642c0b5430 con 0x7f642c15c7e0 2024-04-23T14:31:44.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f6432800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f642c0b52b0 con 0x7f642c0cca50 2024-04-23T14:31:44.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f6432800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f642c0b3bf0 con 0x7f642c15d600 2024-04-23T14:31:44.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f642b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f642c0cca50 0x7f642c155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:44.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f642be00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f642c15d600 0x7f642c0ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:44.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f642be00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f642c15d600 0x7f642c0ca5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.151:56698/0 (socket says 172.21.15.151:56698) 2024-04-23T14:31:44.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f642be00640 1 -- 172.21.15.151:0/781127111 learned_addr learned my addr 172.21.15.151:0/781127111 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-23T14:31:44.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f642be00640 1 -- 172.21.15.151:0/781127111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f642c0cca50 msgr2=0x7f642c155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:44.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f642be00640 1 --2- 172.21.15.151:0/781127111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f642c0cca50 0x7f642c155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f642be00640 1 -- 172.21.15.151:0/781127111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f642c15c7e0 msgr2=0x7f642c0ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:44.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f642be00640 1 --2- 172.21.15.151:0/781127111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f642c15c7e0 0x7f642c0ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f642be00640 1 -- 172.21.15.151:0/781127111 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642c0cb8e0 con 0x7f642c15d600 2024-04-23T14:31:44.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.380+0000 7f642be00640 1 --2- 172.21.15.151:0/781127111 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f642c15d600 0x7f642c0ca5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f64240b0040 tx=0x7f6424019650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:44.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.384+0000 7f6423400640 1 -- 172.21.15.151:0/781127111 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64240b2050 con 0x7f642c15d600 2024-04-23T14:31:44.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.384+0000 7f6423400640 1 -- 172.21.15.151:0/781127111 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64240be070 con 0x7f642c15d600 2024-04-23T14:31:44.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.384+0000 7f6432800640 1 -- 172.21.15.151:0/781127111 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f642c0cbbc0 con 0x7f642c15d600 2024-04-23T14:31:44.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.384+0000 7f6423400640 1 -- 172.21.15.151:0/781127111 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64240ada90 con 0x7f642c15d600 2024-04-23T14:31:44.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.384+0000 7f6432800640 1 -- 172.21.15.151:0/781127111 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f642c0cc180 con 0x7f642c15d600 2024-04-23T14:31:44.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.384+0000 7f6423400640 1 -- 172.21.15.151:0/781127111 <== mon.2 v2:172.21.15.151:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f64240adc30 con 0x7f642c15d600 2024-04-23T14:31:44.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.384+0000 7f6423400640 1 -- 172.21.15.151:0/781127111 <== mon.2 v2:172.21.15.151:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f64240c3d50 con 0x7f642c15d600 2024-04-23T14:31:44.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.384+0000 7f6432800640 1 -- 172.21.15.151:0/781127111 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f642c068ad0 con 0x7f642c15d600 2024-04-23T14:31:44.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.400+0000 7f6423400640 1 -- 172.21.15.151:0/781127111 <== mon.2 v2:172.21.15.151:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6424016560 con 0x7f642c15d600 2024-04-23T14:31:44.402 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:44.402+0000 7f94c028b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T14:31:44.404 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:44.404+0000 7f78bb08d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T14:31:44.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.464+0000 7f6432800640 1 -- 172.21.15.151:0/781127111 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ff0dca4f-928b-498f-aae5-8909e2f05df8", "id": 0} v 0) -- 0x7f642c15a160 con 0x7f642c15d600 2024-04-23T14:31:44.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.468+0000 7f6423400640 1 -- 172.21.15.151:0/781127111 <== mon.2 v2:172.21.15.151:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ff0dca4f-928b-498f-aae5-8909e2f05df8", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f64240a5930 con 0x7f642c15d600 2024-04-23T14:31:44.471 INFO:teuthology.orchestra.run.smithi151.stdout:0 2024-04-23T14:31:44.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.472+0000 7f6432800640 1 -- 172.21.15.151:0/781127111 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f642c15d600 msgr2=0x7f642c0ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:44.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.472+0000 7f6432800640 1 --2- 172.21.15.151:0/781127111 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f642c15d600 0x7f642c0ca5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f64240b0040 tx=0x7f6424019650 comp rx=0 tx=0).stop 2024-04-23T14:31:44.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.472+0000 7f6432800640 1 -- 172.21.15.151:0/781127111 shutdown_connections 2024-04-23T14:31:44.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.472+0000 7f6432800640 1 --2- 172.21.15.151:0/781127111 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f642c15d600 0x7f642c0ca5b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.472+0000 7f6432800640 1 --2- 172.21.15.151:0/781127111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f642c15c7e0 0x7f642c0ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.472+0000 7f6432800640 1 --2- 172.21.15.151:0/781127111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f642c0cca50 0x7f642c155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.472+0000 7f6432800640 1 -- 172.21.15.151:0/781127111 >> 172.21.15.151:0/781127111 conn(0x7f642c0cee80 msgr2=0x7f642c0b5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:44.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.472+0000 7f6432800640 1 -- 172.21.15.151:0/781127111 shutdown_connections 2024-04-23T14:31:44.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.472+0000 7f6432800640 1 -- 172.21.15.151:0/781127111 wait complete. 2024-04-23T14:31:44.490 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new c313da00-b031-49e3-822b-727da06be7e9 1 2024-04-23T14:31:44.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.624+0000 7f0235800640 1 Processor -- start 2024-04-23T14:31:44.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.624+0000 7f0235800640 1 -- start start 2024-04-23T14:31:44.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.628+0000 7f0235800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f023006d450 0x7f023006d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.628+0000 7f0235800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02300cd460 0x7f02300b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.628+0000 7f0235800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f023006dd70 con 0x7f02300cea60 2024-04-23T14:31:44.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.628+0000 7f0235800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f02300b1f90 con 0x7f023006d450 2024-04-23T14:31:44.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.628+0000 7f0235800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f02300b2270 con 0x7f02300cd460 2024-04-23T14:31:44.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.628+0000 7f022ea00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f023006d450 0x7f023006d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:44.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.628+0000 7f022f400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02300cd460 0x7f02300b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:44.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.628+0000 7f022ea00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f023006d450 0x7f023006d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33664/0 (socket says 172.21.15.151:33664) 2024-04-23T14:31:44.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.628+0000 7f022ea00640 1 -- 172.21.15.151:0/1363001625 learned_addr learned my addr 172.21.15.151:0/1363001625 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-23T14:31:44.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.628+0000 7f022f400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02300cd460 0x7f02300b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.151:56712/0 (socket says 172.21.15.151:56712) 2024-04-23T14:31:44.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.628+0000 7f022f400640 1 -- 172.21.15.151:0/1363001625 >> v2:172.21.15.151:3300/0 conn(0x7f023006d450 msgr2=0x7f023006d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:44.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.628+0000 7f022f400640 1 --2- 172.21.15.151:0/1363001625 >> v2:172.21.15.151:3300/0 conn(0x7f023006d450 0x7f023006d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.628+0000 7f022f400640 1 -- 172.21.15.151:0/1363001625 >> v1:172.21.15.100:6789/0 conn(0x7f02300cea60 legacy=0x7f02300d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:44.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.628+0000 7f022f400640 1 -- 172.21.15.151:0/1363001625 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02300b2550 con 0x7f02300cd460 2024-04-23T14:31:44.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.628+0000 7f022ea00640 1 --2- 172.21.15.151:0/1363001625 >> v2:172.21.15.151:3300/0 conn(0x7f023006d450 0x7f023006d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:31:44.630 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:44.628+0000 7f78bb08d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T14:31:44.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.632+0000 7f022f400640 1 --2- 172.21.15.151:0/1363001625 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02300cd460 0x7f02300b2d50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0228009d90 tx=0x7f022801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1d03550517b12e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:44.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.632+0000 7f022e000640 1 -- 172.21.15.151:0/1363001625 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02280b2070 con 0x7f02300cd460 2024-04-23T14:31:44.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.632+0000 7f022e000640 1 -- 172.21.15.151:0/1363001625 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02280a57a0 con 0x7f02300cd460 2024-04-23T14:31:44.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.632+0000 7f022e000640 1 -- 172.21.15.151:0/1363001625 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02280a5b70 con 0x7f02300cd460 2024-04-23T14:31:44.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.632+0000 7f0235800640 1 -- 172.21.15.151:0/1363001625 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02300cd460 msgr2=0x7f02300b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:44.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.632+0000 7f0235800640 1 --2- 172.21.15.151:0/1363001625 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02300cd460 0x7f02300b2d50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0228009d90 tx=0x7f022801c8e0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.632+0000 7f0235800640 1 -- 172.21.15.151:0/1363001625 shutdown_connections 2024-04-23T14:31:44.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.632+0000 7f0235800640 1 --2- 172.21.15.151:0/1363001625 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02300cd460 0x7f02300b2d50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.632+0000 7f0235800640 1 --2- 172.21.15.151:0/1363001625 >> v2:172.21.15.151:3300/0 conn(0x7f023006d450 0x7f023006d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.632+0000 7f0235800640 1 -- 172.21.15.151:0/1363001625 >> 172.21.15.151:0/1363001625 conn(0x7f02300cee80 msgr2=0x7f02300cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:44.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.632+0000 7f0235800640 1 -- 172.21.15.151:0/1363001625 shutdown_connections 2024-04-23T14:31:44.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.632+0000 7f0235800640 1 -- 172.21.15.151:0/1363001625 wait complete. 2024-04-23T14:31:44.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.632+0000 7f0235800640 1 Processor -- start 2024-04-23T14:31:44.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.632+0000 7f0235800640 1 -- start start 2024-04-23T14:31:44.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.632+0000 7f0235800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f023006d450 0x7f0230071af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.632+0000 7f0235800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02300cd460 0x7f0230160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.632+0000 7f0235800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02300cea60 0x7f0230153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.632+0000 7f0235800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f023016b740 con 0x7f023006d450 2024-04-23T14:31:44.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.632+0000 7f0235800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f023016b5c0 con 0x7f02300cea60 2024-04-23T14:31:44.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.632+0000 7f0235800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f023016b8c0 con 0x7f02300cd460 2024-04-23T14:31:44.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.636+0000 7f0227e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02300cd460 0x7f0230160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:44.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.636+0000 7f022f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02300cea60 0x7f0230153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:44.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.636+0000 7f022ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f023006d450 0x7f0230071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:44.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.636+0000 7f022f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02300cea60 0x7f0230153d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33678/0 (socket says 172.21.15.151:33678) 2024-04-23T14:31:44.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.636+0000 7f022f400640 1 -- 172.21.15.151:0/2454453603 learned_addr learned my addr 172.21.15.151:0/2454453603 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-23T14:31:44.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.636+0000 7f022f400640 1 -- 172.21.15.151:0/2454453603 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02300cd460 msgr2=0x7f0230160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:44.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.636+0000 7f022f400640 1 --2- 172.21.15.151:0/2454453603 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02300cd460 0x7f0230160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.636+0000 7f022f400640 1 -- 172.21.15.151:0/2454453603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f023006d450 msgr2=0x7f0230071af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:44.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.636+0000 7f022f400640 1 --2- 172.21.15.151:0/2454453603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f023006d450 0x7f0230071af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.636+0000 7f022f400640 1 -- 172.21.15.151:0/2454453603 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02300cacd0 con 0x7f02300cea60 2024-04-23T14:31:44.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.636+0000 7f0227e00640 1 --2- 172.21.15.151:0/2454453603 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02300cd460 0x7f0230160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:31:44.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.636+0000 7f022f400640 1 --2- 172.21.15.151:0/2454453603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02300cea60 0x7f0230153d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f02280b0040 tx=0x7f0228019680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:44.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.636+0000 7f0226a00640 1 -- 172.21.15.151:0/2454453603 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02280b2050 con 0x7f02300cea60 2024-04-23T14:31:44.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.636+0000 7f0226a00640 1 -- 172.21.15.151:0/2454453603 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02280be070 con 0x7f02300cea60 2024-04-23T14:31:44.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.636+0000 7f0226a00640 1 -- 172.21.15.151:0/2454453603 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02280ad890 con 0x7f02300cea60 2024-04-23T14:31:44.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.636+0000 7f0235800640 1 -- 172.21.15.151:0/2454453603 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02300cafb0 con 0x7f02300cea60 2024-04-23T14:31:44.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.636+0000 7f0235800640 1 -- 172.21.15.151:0/2454453603 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02300cb570 con 0x7f02300cea60 2024-04-23T14:31:44.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.636+0000 7f0226a00640 1 -- 172.21.15.151:0/2454453603 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f02280ada30 con 0x7f02300cea60 2024-04-23T14:31:44.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.636+0000 7f0226a00640 1 -- 172.21.15.151:0/2454453603 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f02280044a0 con 0x7f02300cea60 2024-04-23T14:31:44.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.636+0000 7f0235800640 1 -- 172.21.15.151:0/2454453603 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0230068ad0 con 0x7f02300cea60 2024-04-23T14:31:44.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.640+0000 7f0226a00640 1 -- 172.21.15.151:0/2454453603 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f02280c7020 con 0x7f02300cea60 2024-04-23T14:31:44.649 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:44.650+0000 7f94c028b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T14:31:44.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.708+0000 7f0235800640 1 -- 172.21.15.151:0/2454453603 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c313da00-b031-49e3-822b-727da06be7e9", "id": 1} v 0) -- 0x7f02300b2cf0 con 0x7f02300cea60 2024-04-23T14:31:44.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.712+0000 7f0226a00640 1 -- 172.21.15.151:0/2454453603 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c313da00-b031-49e3-822b-727da06be7e9", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f0228016560 con 0x7f02300cea60 2024-04-23T14:31:44.713 INFO:teuthology.orchestra.run.smithi151.stdout:1 2024-04-23T14:31:44.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.712+0000 7f0235800640 1 -- 172.21.15.151:0/2454453603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02300cea60 msgr2=0x7f0230153d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:44.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.712+0000 7f0235800640 1 --2- 172.21.15.151:0/2454453603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02300cea60 0x7f0230153d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f02280b0040 tx=0x7f0228019680 comp rx=0 tx=0).stop 2024-04-23T14:31:44.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.712+0000 7f0235800640 1 -- 172.21.15.151:0/2454453603 shutdown_connections 2024-04-23T14:31:44.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.712+0000 7f0235800640 1 --2- 172.21.15.151:0/2454453603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02300cea60 0x7f0230153d00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.712+0000 7f0235800640 1 --2- 172.21.15.151:0/2454453603 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02300cd460 0x7f0230160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.712+0000 7f0235800640 1 --2- 172.21.15.151:0/2454453603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f023006d450 0x7f0230071af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.712+0000 7f0235800640 1 -- 172.21.15.151:0/2454453603 >> 172.21.15.151:0/2454453603 conn(0x7f02300cee80 msgr2=0x7f02300ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:44.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.712+0000 7f0235800640 1 -- 172.21.15.151:0/2454453603 shutdown_connections 2024-04-23T14:31:44.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.712+0000 7f0235800640 1 -- 172.21.15.151:0/2454453603 wait complete. 2024-04-23T14:31:44.733 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new ec06612c-20c6-4c09-9f2b-2380186182f5 2 2024-04-23T14:31:44.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.864+0000 7f3ba5200640 1 Processor -- start 2024-04-23T14:31:44.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.864+0000 7f3ba5200640 1 -- start start 2024-04-23T14:31:44.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.868+0000 7f3ba5200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3ba00cca50 0x7f3ba00cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.868+0000 7f3ba5200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f3ba015d600 0x7f3ba0160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.868+0000 7f3ba5200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ba00cd370 con 0x7f3ba015c7e0 2024-04-23T14:31:44.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.868+0000 7f3ba5200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f3ba00d2270 con 0x7f3ba00cca50 2024-04-23T14:31:44.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.868+0000 7f3ba5200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f3ba015db00 con 0x7f3ba015d600 2024-04-23T14:31:44.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.868+0000 7f3b95400640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f3ba015c7e0 0x7f3ba0160170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.151:52882/0 (socket says 172.21.15.151:52882) 2024-04-23T14:31:44.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.868+0000 7f3b95400640 1 -- 172.21.15.151:0/1566111940 learned_addr learned my addr 172.21.15.151:0/1566111940 (peer_addr_for_me v1:172.21.15.151:0/0) 2024-04-23T14:31:44.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.868+0000 7f3b9e800640 1 --2- 172.21.15.151:0/1566111940 >> v2:172.21.15.151:3300/0 conn(0x7f3ba015d600 0x7f3ba0160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:44.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.868+0000 7f3b9de00640 1 --2- 172.21.15.151:0/1566111940 >> v2:172.21.15.100:3300/0 conn(0x7f3ba00cca50 0x7f3ba00cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:44.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.868+0000 7f3b9e800640 1 -- 172.21.15.151:0/1566111940 >> v2:172.21.15.100:3300/0 conn(0x7f3ba00cca50 msgr2=0x7f3ba00cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:44.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.868+0000 7f3b9e800640 1 --2- 172.21.15.151:0/1566111940 >> v2:172.21.15.100:3300/0 conn(0x7f3ba00cca50 0x7f3ba00cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.868+0000 7f3b9e800640 1 -- 172.21.15.151:0/1566111940 >> v1:172.21.15.151:6789/0 conn(0x7f3ba015c7e0 legacy=0x7f3ba0160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:44.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.868+0000 7f3b9e800640 1 -- 172.21.15.151:0/1566111940 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba00d04a0 con 0x7f3ba015d600 2024-04-23T14:31:44.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.872+0000 7f3b9e800640 1 --2- 172.21.15.151:0/1566111940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba015d600 0x7f3ba0160880 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00cfa10 tx=0x7f3b9801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81393b29a3d3a697 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:44.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.872+0000 7f3b9d400640 1 -- 172.21.15.151:0/1566111940 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b980b1070 con 0x7f3ba015d600 2024-04-23T14:31:44.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.872+0000 7f3b9d400640 1 -- 172.21.15.151:0/1566111940 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b980ac620 con 0x7f3ba015d600 2024-04-23T14:31:44.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.872+0000 7f3b9d400640 1 -- 172.21.15.151:0/1566111940 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b980ac9f0 con 0x7f3ba015d600 2024-04-23T14:31:44.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.872+0000 7f3ba5200640 1 -- 172.21.15.151:0/1566111940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba015d600 msgr2=0x7f3ba0160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:44.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.872+0000 7f3ba5200640 1 --2- 172.21.15.151:0/1566111940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba015d600 0x7f3ba0160880 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00cfa10 tx=0x7f3b9801c3c0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.872+0000 7f3ba5200640 1 -- 172.21.15.151:0/1566111940 shutdown_connections 2024-04-23T14:31:44.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.872+0000 7f3ba5200640 1 --2- 172.21.15.151:0/1566111940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba015d600 0x7f3ba0160880 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.872+0000 7f3ba5200640 1 --2- 172.21.15.151:0/1566111940 >> v2:172.21.15.100:3300/0 conn(0x7f3ba00cca50 0x7f3ba00cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.872+0000 7f3ba5200640 1 -- 172.21.15.151:0/1566111940 >> 172.21.15.151:0/1566111940 conn(0x7f3ba00cee80 msgr2=0x7f3ba00cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:44.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.872+0000 7f3ba5200640 1 -- 172.21.15.151:0/1566111940 shutdown_connections 2024-04-23T14:31:44.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.872+0000 7f3ba5200640 1 -- 172.21.15.151:0/1566111940 wait complete. 2024-04-23T14:31:44.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3ba5200640 1 Processor -- start 2024-04-23T14:31:44.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3ba5200640 1 -- start start 2024-04-23T14:31:44.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3ba5200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba00cca50 0x7f3ba006de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3ba5200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3ba015c7e0 0x7f3ba0156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3ba5200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba015d600 0x7f3ba01466a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3ba5200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba00b5430 con 0x7f3ba00cca50 2024-04-23T14:31:44.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3ba5200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba00b52b0 con 0x7f3ba015d600 2024-04-23T14:31:44.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3ba5200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f3ba00b3bf0 con 0x7f3ba015c7e0 2024-04-23T14:31:44.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3b95400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3ba015c7e0 0x7f3ba0156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:44.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3b95400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3ba015c7e0 0x7f3ba0156040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.151:56734/0 (socket says 172.21.15.151:56734) 2024-04-23T14:31:44.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3b9e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba015d600 0x7f3ba01466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:44.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3b95400640 1 -- 172.21.15.151:0/2777279602 learned_addr learned my addr 172.21.15.151:0/2777279602 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-23T14:31:44.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3b9de00640 1 --2- 172.21.15.151:0/2777279602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba00cca50 0x7f3ba006de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:44.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3b9e800640 1 -- 172.21.15.151:0/2777279602 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3ba015c7e0 msgr2=0x7f3ba0156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:44.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3b9e800640 1 --2- 172.21.15.151:0/2777279602 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3ba015c7e0 0x7f3ba0156040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3b9e800640 1 -- 172.21.15.151:0/2777279602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba00cca50 msgr2=0x7f3ba006de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:44.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3b9e800640 1 --2- 172.21.15.151:0/2777279602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba00cca50 0x7f3ba006de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3b9e800640 1 -- 172.21.15.151:0/2777279602 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba00b79e0 con 0x7f3ba015d600 2024-04-23T14:31:44.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3b95400640 1 --2- 172.21.15.151:0/2777279602 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3ba015c7e0 0x7f3ba0156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:31:44.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3b9e800640 1 --2- 172.21.15.151:0/2777279602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba015d600 0x7f3ba01466a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3b980ac000 tx=0x7f3b98019160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:44.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3b96a00640 1 -- 172.21.15.151:0/2777279602 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b980b9020 con 0x7f3ba015d600 2024-04-23T14:31:44.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3b96a00640 1 -- 172.21.15.151:0/2777279602 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b980b5410 con 0x7f3ba015d600 2024-04-23T14:31:44.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3ba5200640 1 -- 172.21.15.151:0/2777279602 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba00cfc20 con 0x7f3ba015d600 2024-04-23T14:31:44.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3b96a00640 1 -- 172.21.15.151:0/2777279602 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b980b1040 con 0x7f3ba015d600 2024-04-23T14:31:44.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.876+0000 7f3ba5200640 1 -- 172.21.15.151:0/2777279602 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba01558c0 con 0x7f3ba015d600 2024-04-23T14:31:44.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.880+0000 7f3b96a00640 1 -- 172.21.15.151:0/2777279602 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3b980bc020 con 0x7f3ba015d600 2024-04-23T14:31:44.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.880+0000 7f3b96a00640 1 -- 172.21.15.151:0/2777279602 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f3b980be810 con 0x7f3ba015d600 2024-04-23T14:31:44.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.880+0000 7f3ba5200640 1 -- 172.21.15.151:0/2777279602 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ba006ec50 con 0x7f3ba015d600 2024-04-23T14:31:44.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.880+0000 7f3b96a00640 1 -- 172.21.15.151:0/2777279602 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3b980c3030 con 0x7f3ba015d600 2024-04-23T14:31:44.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.948+0000 7f3ba5200640 1 -- 172.21.15.151:0/2777279602 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ec06612c-20c6-4c09-9f2b-2380186182f5", "id": 2} v 0) -- 0x7f3ba0068ad0 con 0x7f3ba015d600 2024-04-23T14:31:44.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.952+0000 7f3b96a00640 1 -- 172.21.15.151:0/2777279602 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ec06612c-20c6-4c09-9f2b-2380186182f5", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f3b98016070 con 0x7f3ba015d600 2024-04-23T14:31:44.954 INFO:teuthology.orchestra.run.smithi151.stdout:2 2024-04-23T14:31:44.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.956+0000 7f3ba5200640 1 -- 172.21.15.151:0/2777279602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba015d600 msgr2=0x7f3ba01466a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:44.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.956+0000 7f3ba5200640 1 --2- 172.21.15.151:0/2777279602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba015d600 0x7f3ba01466a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3b980ac000 tx=0x7f3b98019160 comp rx=0 tx=0).stop 2024-04-23T14:31:44.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.956+0000 7f3ba5200640 1 -- 172.21.15.151:0/2777279602 shutdown_connections 2024-04-23T14:31:44.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.956+0000 7f3ba5200640 1 --2- 172.21.15.151:0/2777279602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba015d600 0x7f3ba01466a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.956+0000 7f3ba5200640 1 --2- 172.21.15.151:0/2777279602 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3ba015c7e0 0x7f3ba0156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.956+0000 7f3ba5200640 1 --2- 172.21.15.151:0/2777279602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba00cca50 0x7f3ba006de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.956+0000 7f3ba5200640 1 -- 172.21.15.151:0/2777279602 >> 172.21.15.151:0/2777279602 conn(0x7f3ba00cee80 msgr2=0x7f3ba00b6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:44.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.956+0000 7f3ba5200640 1 -- 172.21.15.151:0/2777279602 shutdown_connections 2024-04-23T14:31:44.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:44.956+0000 7f3ba5200640 1 -- 172.21.15.151:0/2777279602 wait complete. 2024-04-23T14:31:44.974 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new 6066e9dc-0909-4b36-8a4c-d03c2c2e08e7 3 2024-04-23T14:31:45.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.108+0000 7f1b9a400640 1 Processor -- start 2024-04-23T14:31:45.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.108+0000 7f1b9a400640 1 -- start start 2024-04-23T14:31:45.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.108+0000 7f1b9a400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b940d00c0 0x7f1b941693b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:45.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.108+0000 7f1b9a400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b940ce460 con 0x7f1b940b0810 2024-04-23T14:31:45.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.108+0000 7f1b9a400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b940ce710 con 0x7f1b9415edf0 2024-04-23T14:31:45.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.108+0000 7f1b9a400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1b940b4580 con 0x7f1b940d00c0 2024-04-23T14:31:45.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.108+0000 7f1b92a00640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f1b9415edf0 0x7f1b940cf9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.151:52888/0 (socket says 172.21.15.151:52888) 2024-04-23T14:31:45.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.108+0000 7f1b93e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b940d00c0 0x7f1b941693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:45.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.108+0000 7f1b92a00640 1 -- 172.21.15.151:0/651157059 learned_addr learned my addr 172.21.15.151:0/651157059 (peer_addr_for_me v1:172.21.15.151:0/0) 2024-04-23T14:31:45.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.108+0000 7f1b93e00640 1 -- 172.21.15.151:0/651157059 >> v1:172.21.15.151:6789/0 conn(0x7f1b9415edf0 legacy=0x7f1b940cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:45.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.108+0000 7f1b93e00640 1 -- 172.21.15.151:0/651157059 >> v1:172.21.15.100:6789/0 conn(0x7f1b940b0810 legacy=0x7f1b940cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:45.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.108+0000 7f1b93e00640 1 -- 172.21.15.151:0/651157059 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b940ce9f0 con 0x7f1b940d00c0 2024-04-23T14:31:45.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.112+0000 7f1b93e00640 1 --2- 172.21.15.151:0/651157059 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b940d00c0 0x7f1b941693b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009dc0 tx=0x7f1b8c01c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaa1759171d82cc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:45.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.112+0000 7f1b92000640 1 -- 172.21.15.151:0/651157059 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c0b2070 con 0x7f1b940d00c0 2024-04-23T14:31:45.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.112+0000 7f1b92000640 1 -- 172.21.15.151:0/651157059 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8c0a5920 con 0x7f1b940d00c0 2024-04-23T14:31:45.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.112+0000 7f1b92000640 1 -- 172.21.15.151:0/651157059 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c0a5cf0 con 0x7f1b940d00c0 2024-04-23T14:31:45.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.112+0000 7f1b9a400640 1 -- 172.21.15.151:0/651157059 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b940d00c0 msgr2=0x7f1b941693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:45.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.112+0000 7f1b9a400640 1 --2- 172.21.15.151:0/651157059 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b940d00c0 0x7f1b941693b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009dc0 tx=0x7f1b8c01c990 comp rx=0 tx=0).stop 2024-04-23T14:31:45.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.112+0000 7f1b9a400640 1 -- 172.21.15.151:0/651157059 shutdown_connections 2024-04-23T14:31:45.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.112+0000 7f1b9a400640 1 --2- 172.21.15.151:0/651157059 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b940d00c0 0x7f1b941693b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.112+0000 7f1b9a400640 1 -- 172.21.15.151:0/651157059 >> 172.21.15.151:0/651157059 conn(0x7f1b94070600 msgr2=0x7f1b94070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:45.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.112+0000 7f1b9a400640 1 -- 172.21.15.151:0/651157059 shutdown_connections 2024-04-23T14:31:45.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.112+0000 7f1b9a400640 1 -- 172.21.15.151:0/651157059 wait complete. 2024-04-23T14:31:45.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b9a400640 1 Processor -- start 2024-04-23T14:31:45.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b9a400640 1 -- start start 2024-04-23T14:31:45.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b9a400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b940b0810 0x7f1b94161030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:45.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b9a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b940d00c0 0x7f1b94162810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:45.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b9a400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b9415edf0 0x7f1b9414d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:45.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b9a400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9416c8b0 con 0x7f1b940d00c0 2024-04-23T14:31:45.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b9a400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9416c730 con 0x7f1b9415edf0 2024-04-23T14:31:45.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b9a400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1b9416ca30 con 0x7f1b940b0810 2024-04-23T14:31:45.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b93400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b940b0810 0x7f1b94161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:45.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b93400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b940b0810 0x7f1b94161030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.151:56756/0 (socket says 172.21.15.151:56756) 2024-04-23T14:31:45.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b93400640 1 -- 172.21.15.151:0/2358901910 learned_addr learned my addr 172.21.15.151:0/2358901910 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-23T14:31:45.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b92a00640 1 --2- 172.21.15.151:0/2358901910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b940d00c0 0x7f1b94162810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:45.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b93e00640 1 --2- 172.21.15.151:0/2358901910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b9415edf0 0x7f1b9414d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:45.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b93400640 1 -- 172.21.15.151:0/2358901910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b9415edf0 msgr2=0x7f1b9414d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:45.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b93400640 1 --2- 172.21.15.151:0/2358901910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b9415edf0 0x7f1b9414d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b93400640 1 -- 172.21.15.151:0/2358901910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b940d00c0 msgr2=0x7f1b94162810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:45.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b93400640 1 --2- 172.21.15.151:0/2358901910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b940d00c0 0x7f1b94162810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b93400640 1 -- 172.21.15.151:0/2358901910 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9414d830 con 0x7f1b940b0810 2024-04-23T14:31:45.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b93400640 1 --2- 172.21.15.151:0/2358901910 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b940b0810 0x7f1b94161030 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400c450 tx=0x7f1b8400c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:45.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b8b400640 1 -- 172.21.15.151:0/2358901910 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b84016020 con 0x7f1b940b0810 2024-04-23T14:31:45.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b8b400640 1 -- 172.21.15.151:0/2358901910 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b840052d0 con 0x7f1b940b0810 2024-04-23T14:31:45.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b8b400640 1 -- 172.21.15.151:0/2358901910 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b84007260 con 0x7f1b940b0810 2024-04-23T14:31:45.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b9a400640 1 -- 172.21.15.151:0/2358901910 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b941fb4c0 con 0x7f1b940b0810 2024-04-23T14:31:45.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b9a400640 1 -- 172.21.15.151:0/2358901910 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b941fba80 con 0x7f1b940b0810 2024-04-23T14:31:45.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b8b400640 1 -- 172.21.15.151:0/2358901910 <== mon.2 v2:172.21.15.151:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1b840067e0 con 0x7f1b940b0810 2024-04-23T14:31:45.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.116+0000 7f1b8b400640 1 -- 172.21.15.151:0/2358901910 <== mon.2 v2:172.21.15.151:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f1b84012bd0 con 0x7f1b940b0810 2024-04-23T14:31:45.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.120+0000 7f1b9a400640 1 -- 172.21.15.151:0/2358901910 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b940691e0 con 0x7f1b940b0810 2024-04-23T14:31:45.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.120+0000 7f1b8b400640 1 -- 172.21.15.151:0/2358901910 <== mon.2 v2:172.21.15.151:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f1b84020020 con 0x7f1b940b0810 2024-04-23T14:31:45.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.188+0000 7f1b9a400640 1 -- 172.21.15.151:0/2358901910 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6066e9dc-0909-4b36-8a4c-d03c2c2e08e7", "id": 3} v 0) -- 0x7f1b940cfc90 con 0x7f1b940b0810 2024-04-23T14:31:45.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.192+0000 7f1b8b400640 1 -- 172.21.15.151:0/2358901910 <== mon.2 v2:172.21.15.151:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6066e9dc-0909-4b36-8a4c-d03c2c2e08e7", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f1b84005480 con 0x7f1b940b0810 2024-04-23T14:31:45.194 INFO:teuthology.orchestra.run.smithi151.stdout:3 2024-04-23T14:31:45.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.192+0000 7f1b9a400640 1 -- 172.21.15.151:0/2358901910 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b940b0810 msgr2=0x7f1b94161030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:45.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.192+0000 7f1b9a400640 1 --2- 172.21.15.151:0/2358901910 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b940b0810 0x7f1b94161030 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400c450 tx=0x7f1b8400c910 comp rx=0 tx=0).stop 2024-04-23T14:31:45.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.192+0000 7f1b9a400640 1 -- 172.21.15.151:0/2358901910 shutdown_connections 2024-04-23T14:31:45.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.192+0000 7f1b9a400640 1 --2- 172.21.15.151:0/2358901910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b9415edf0 0x7f1b9414d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.192+0000 7f1b9a400640 1 --2- 172.21.15.151:0/2358901910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b940d00c0 0x7f1b94162810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.192+0000 7f1b9a400640 1 --2- 172.21.15.151:0/2358901910 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b940b0810 0x7f1b94161030 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.192+0000 7f1b9a400640 1 -- 172.21.15.151:0/2358901910 >> 172.21.15.151:0/2358901910 conn(0x7f1b94070600 msgr2=0x7f1b940afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:45.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.192+0000 7f1b9a400640 1 -- 172.21.15.151:0/2358901910 shutdown_connections 2024-04-23T14:31:45.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.192+0000 7f1b9a400640 1 -- 172.21.15.151:0/2358901910 wait complete. 2024-04-23T14:31:45.213 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new 0eb3f089-4f39-449f-8598-c48b0dee081b 4 2024-04-23T14:31:45.270 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:45.270+0000 7f94c028b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T14:31:45.280 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:45.280+0000 7f78bb08d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T14:31:45.332 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:45.330+0000 7f94c028b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T14:31:45.344 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:45.344+0000 7f78bb08d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T14:31:45.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.348+0000 7fbfa7200640 1 Processor -- start 2024-04-23T14:31:45.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.348+0000 7fbfa7200640 1 -- start start 2024-04-23T14:31:45.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.348+0000 7fbfa7200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbfa015e660 0x7fbfa00b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:45.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.348+0000 7fbfa7200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fbfa0073590 0x7fbfa00d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:45.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.348+0000 7fbfa7200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfa00b5b00 con 0x7fbfa015ea40 2024-04-23T14:31:45.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.348+0000 7fbfa7200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fbfa00cf340 con 0x7fbfa015e660 2024-04-23T14:31:45.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.348+0000 7fbfa7200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fbfa00cf620 con 0x7fbfa0073590 2024-04-23T14:31:45.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.348+0000 7fbfa5200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fbfa0073590 0x7fbfa00d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:45.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.348+0000 7fbf9fe00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbfa015e660 0x7fbfa00b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:45.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.348+0000 7fbfa5200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fbfa0073590 0x7fbfa00d0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33702/0 (socket says 172.21.15.151:33702) 2024-04-23T14:31:45.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.348+0000 7fbfa5200640 1 -- 172.21.15.151:0/1094427940 learned_addr learned my addr 172.21.15.151:0/1094427940 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-23T14:31:45.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.348+0000 7fbf9f400640 1 --1- 172.21.15.151:0/1094427940 >> v1:172.21.15.100:6789/0 conn(0x7fbfa015ea40 0x7fbfa0072e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.151:47604/0 (socket says 172.21.15.151:47604) 2024-04-23T14:31:45.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.348+0000 7fbf9fe00640 1 -- 172.21.15.151:0/1094427940 >> v2:172.21.15.151:3300/0 conn(0x7fbfa0073590 msgr2=0x7fbfa00d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:45.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.348+0000 7fbf9fe00640 1 --2- 172.21.15.151:0/1094427940 >> v2:172.21.15.151:3300/0 conn(0x7fbfa0073590 0x7fbfa00d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.348+0000 7fbf9fe00640 1 -- 172.21.15.151:0/1094427940 >> v1:172.21.15.100:6789/0 conn(0x7fbfa015ea40 legacy=0x7fbfa0072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:45.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.348+0000 7fbf9fe00640 1 -- 172.21.15.151:0/1094427940 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa00cf900 con 0x7fbfa015e660 2024-04-23T14:31:45.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.348+0000 7fbfa5200640 1 --2- 172.21.15.151:0/1094427940 >> v2:172.21.15.151:3300/0 conn(0x7fbfa0073590 0x7fbfa00d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:31:45.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.348+0000 7fbf9ea00640 1 -- 172.21.15.151:0/1094427940 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3066925561 0 0) 0x7fbfa00b5b00 con 0x7fbfa015ea40 2024-04-23T14:31:45.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.352+0000 7fbf9fe00640 1 --2- 172.21.15.151:0/1094427940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfa015e660 0x7fbfa00b55c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00b0fd0 tx=0x7fbf9001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ffd1bdc24953d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:45.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.352+0000 7fbf9ea00640 1 -- 172.21.15.151:0/1094427940 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf900b0070 con 0x7fbfa015e660 2024-04-23T14:31:45.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.352+0000 7fbf9ea00640 1 -- 172.21.15.151:0/1094427940 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf900ab600 con 0x7fbfa015e660 2024-04-23T14:31:45.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.352+0000 7fbf9ea00640 1 -- 172.21.15.151:0/1094427940 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf900ab9d0 con 0x7fbfa015e660 2024-04-23T14:31:45.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.352+0000 7fbfa7200640 1 -- 172.21.15.151:0/1094427940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfa015e660 msgr2=0x7fbfa00b55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:45.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.352+0000 7fbfa7200640 1 --2- 172.21.15.151:0/1094427940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfa015e660 0x7fbfa00b55c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00b0fd0 tx=0x7fbf9001c3f0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.352+0000 7fbfa7200640 1 -- 172.21.15.151:0/1094427940 shutdown_connections 2024-04-23T14:31:45.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.352+0000 7fbfa7200640 1 --2- 172.21.15.151:0/1094427940 >> v2:172.21.15.151:3300/0 conn(0x7fbfa0073590 0x7fbfa00d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.352+0000 7fbfa7200640 1 --2- 172.21.15.151:0/1094427940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfa015e660 0x7fbfa00b55c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.352+0000 7fbfa7200640 1 -- 172.21.15.151:0/1094427940 >> 172.21.15.151:0/1094427940 conn(0x7fbfa00cde10 msgr2=0x7fbfa00ce220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:45.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.352+0000 7fbfa7200640 1 -- 172.21.15.151:0/1094427940 shutdown_connections 2024-04-23T14:31:45.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.352+0000 7fbfa7200640 1 -- 172.21.15.151:0/1094427940 wait complete. 2024-04-23T14:31:45.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbfa7200640 1 Processor -- start 2024-04-23T14:31:45.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbfa7200640 1 -- start start 2024-04-23T14:31:45.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbfa7200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfa0073590 0x7fbfa014c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:45.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbfa7200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbfa015e660 0x7fbfa006dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:45.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbfa7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfa015ea40 0x7fbfa01566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:45.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbfa7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa00d0e50 con 0x7fbfa015ea40 2024-04-23T14:31:45.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbfa7200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa00cb330 con 0x7fbfa0073590 2024-04-23T14:31:45.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbfa7200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fbfa00d0fd0 con 0x7fbfa015e660 2024-04-23T14:31:45.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbf9f400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbfa015e660 0x7fbfa006dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:45.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbf9fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfa0073590 0x7fbfa014c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:45.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbf9f400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbfa015e660 0x7fbfa006dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.151:56760/0 (socket says 172.21.15.151:56760) 2024-04-23T14:31:45.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbf9f400640 1 -- 172.21.15.151:0/2053274677 learned_addr learned my addr 172.21.15.151:0/2053274677 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-23T14:31:45.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbfa5200640 1 --2- 172.21.15.151:0/2053274677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfa015ea40 0x7fbfa01566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:45.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbf9fe00640 1 -- 172.21.15.151:0/2053274677 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbfa015e660 msgr2=0x7fbfa006dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:45.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbf9fe00640 1 --2- 172.21.15.151:0/2053274677 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbfa015e660 0x7fbfa006dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbf9fe00640 1 -- 172.21.15.151:0/2053274677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfa015ea40 msgr2=0x7fbfa01566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:45.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbf9fe00640 1 --2- 172.21.15.151:0/2053274677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfa015ea40 0x7fbfa01566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbf9fe00640 1 -- 172.21.15.151:0/2053274677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa014d110 con 0x7fbfa0073590 2024-04-23T14:31:45.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbfa5200640 1 --2- 172.21.15.151:0/2053274677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfa015ea40 0x7fbfa01566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:31:45.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbf9fe00640 1 --2- 172.21.15.151:0/2053274677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfa0073590 0x7fbfa014c500 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbf900ab000 tx=0x7fbf9001ac30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:45.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbf97e00640 1 -- 172.21.15.151:0/2053274677 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf900b0050 con 0x7fbfa0073590 2024-04-23T14:31:45.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbf97e00640 1 -- 172.21.15.151:0/2053274677 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf900bc070 con 0x7fbfa0073590 2024-04-23T14:31:45.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbf97e00640 1 -- 172.21.15.151:0/2053274677 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf90006e20 con 0x7fbfa0073590 2024-04-23T14:31:45.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbfa7200640 1 -- 172.21.15.151:0/2053274677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa014d3f0 con 0x7fbfa0073590 2024-04-23T14:31:45.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbfa7200640 1 -- 172.21.15.151:0/2053274677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa014d9b0 con 0x7fbfa0073590 2024-04-23T14:31:45.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbf97e00640 1 -- 172.21.15.151:0/2053274677 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbf900b5870 con 0x7fbfa0073590 2024-04-23T14:31:45.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.356+0000 7fbf97e00640 1 -- 172.21.15.151:0/2053274677 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fbf900c1460 con 0x7fbfa0073590 2024-04-23T14:31:45.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.360+0000 7fbfa7200640 1 -- 172.21.15.151:0/2053274677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa00691e0 con 0x7fbfa0073590 2024-04-23T14:31:45.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.360+0000 7fbf97e00640 1 -- 172.21.15.151:0/2053274677 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fbf90016070 con 0x7fbfa0073590 2024-04-23T14:31:45.394 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:45.394+0000 7f94c028b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T14:31:45.409 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:45.408+0000 7f78bb08d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T14:31:45.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.428+0000 7fbfa7200640 1 -- 172.21.15.151:0/2053274677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0eb3f089-4f39-449f-8598-c48b0dee081b", "id": 4} v 0) -- 0x7fbfa006fd80 con 0x7fbfa0073590 2024-04-23T14:31:45.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.432+0000 7fbf97e00640 1 -- 172.21.15.151:0/2053274677 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0eb3f089-4f39-449f-8598-c48b0dee081b", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fbf900b5b60 con 0x7fbfa0073590 2024-04-23T14:31:45.432 INFO:teuthology.orchestra.run.smithi151.stdout:4 2024-04-23T14:31:45.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.432+0000 7fbfa7200640 1 -- 172.21.15.151:0/2053274677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfa0073590 msgr2=0x7fbfa014c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:45.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.432+0000 7fbfa7200640 1 --2- 172.21.15.151:0/2053274677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfa0073590 0x7fbfa014c500 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbf900ab000 tx=0x7fbf9001ac30 comp rx=0 tx=0).stop 2024-04-23T14:31:45.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.432+0000 7fbfa7200640 1 -- 172.21.15.151:0/2053274677 shutdown_connections 2024-04-23T14:31:45.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.432+0000 7fbfa7200640 1 --2- 172.21.15.151:0/2053274677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfa015ea40 0x7fbfa01566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.432+0000 7fbfa7200640 1 --2- 172.21.15.151:0/2053274677 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbfa015e660 0x7fbfa006dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.432+0000 7fbfa7200640 1 --2- 172.21.15.151:0/2053274677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfa0073590 0x7fbfa014c500 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.432+0000 7fbfa7200640 1 -- 172.21.15.151:0/2053274677 >> 172.21.15.151:0/2053274677 conn(0x7fbfa00cde10 msgr2=0x7fbfa00ccaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:45.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.432+0000 7fbfa7200640 1 -- 172.21.15.151:0/2053274677 shutdown_connections 2024-04-23T14:31:45.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.432+0000 7fbfa7200640 1 -- 172.21.15.151:0/2053274677 wait complete. 2024-04-23T14:31:45.451 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new 8fd37274-87b3-464b-b6d2-d45a98d50b63 5 2024-04-23T14:31:45.483 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:45.482+0000 7f94c028b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T14:31:45.503 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:45.504+0000 7f78bb08d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T14:31:45.543 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:45.542+0000 7f94c028b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T14:31:45.565 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:45.564+0000 7f78bb08d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T14:31:45.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.588+0000 7f9112800640 1 Processor -- start 2024-04-23T14:31:45.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.588+0000 7f9112800640 1 -- start start 2024-04-23T14:31:45.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.588+0000 7f9112800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f910c15c7e0 0x7f910c0d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:45.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.588+0000 7f9112800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f910c15d600 0x7f910c0d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:45.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.588+0000 7f9112800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f910c0d04a0 con 0x7f910c0cca50 2024-04-23T14:31:45.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.588+0000 7f9112800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f910c15db00 con 0x7f910c15d600 2024-04-23T14:31:45.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.588+0000 7f9112800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f910c0d0780 con 0x7f910c15c7e0 2024-04-23T14:31:45.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.588+0000 7f9103e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f910c15c7e0 0x7f910c0d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:45.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.588+0000 7f9103e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f910c15c7e0 0x7f910c0d1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33720/0 (socket says 172.21.15.151:33720) 2024-04-23T14:31:45.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.588+0000 7f9103e00640 1 -- 172.21.15.151:0/2774619407 learned_addr learned my addr 172.21.15.151:0/2774619407 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-23T14:31:45.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.588+0000 7f910be00640 1 --2- 172.21.15.151:0/2774619407 >> v2:172.21.15.100:3300/0 conn(0x7f910c15d600 0x7f910c0d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:45.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.588+0000 7f9103e00640 1 -- 172.21.15.151:0/2774619407 >> v2:172.21.15.100:3300/0 conn(0x7f910c15d600 msgr2=0x7f910c0d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:45.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.588+0000 7f9103e00640 1 --2- 172.21.15.151:0/2774619407 >> v2:172.21.15.100:3300/0 conn(0x7f910c15d600 0x7f910c0d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.588+0000 7f9103e00640 1 -- 172.21.15.151:0/2774619407 >> v1:172.21.15.100:6789/0 conn(0x7f910c0cca50 legacy=0x7f910c0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:45.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.588+0000 7f9103e00640 1 -- 172.21.15.151:0/2774619407 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910c0d0a60 con 0x7f910c15c7e0 2024-04-23T14:31:45.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.592+0000 7f9103e00640 1 --2- 172.21.15.151:0/2774619407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f910c15c7e0 0x7f910c0d1260 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f910c074250 tx=0x7f90f401c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=410e76ed350b743 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:45.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.592+0000 7f910aa00640 1 -- 172.21.15.151:0/2774619407 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f40ab680 con 0x7f910c15c7e0 2024-04-23T14:31:45.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.592+0000 7f910aa00640 1 -- 172.21.15.151:0/2774619407 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f40b0070 con 0x7f910c15c7e0 2024-04-23T14:31:45.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.592+0000 7f910aa00640 1 -- 172.21.15.151:0/2774619407 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f40aba40 con 0x7f910c15c7e0 2024-04-23T14:31:45.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.592+0000 7f9112800640 1 -- 172.21.15.151:0/2774619407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f910c15c7e0 msgr2=0x7f910c0d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:45.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.592+0000 7f9112800640 1 --2- 172.21.15.151:0/2774619407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f910c15c7e0 0x7f910c0d1260 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f910c074250 tx=0x7f90f401c460 comp rx=0 tx=0).stop 2024-04-23T14:31:45.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.592+0000 7f9112800640 1 -- 172.21.15.151:0/2774619407 shutdown_connections 2024-04-23T14:31:45.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.592+0000 7f9112800640 1 --2- 172.21.15.151:0/2774619407 >> v2:172.21.15.100:3300/0 conn(0x7f910c15d600 0x7f910c0d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.592+0000 7f9112800640 1 --2- 172.21.15.151:0/2774619407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f910c15c7e0 0x7f910c0d1260 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.592+0000 7f9112800640 1 -- 172.21.15.151:0/2774619407 >> 172.21.15.151:0/2774619407 conn(0x7f910c0cee80 msgr2=0x7f910c0cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:45.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.592+0000 7f9112800640 1 -- 172.21.15.151:0/2774619407 shutdown_connections 2024-04-23T14:31:45.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.592+0000 7f9112800640 1 -- 172.21.15.151:0/2774619407 wait complete. 2024-04-23T14:31:45.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f9112800640 1 Processor -- start 2024-04-23T14:31:45.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f9112800640 1 -- start start 2024-04-23T14:31:45.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f9112800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f910c0cca50 0x7f910c150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:45.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f9112800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f910c15c7e0 0x7f910c0c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:45.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f9112800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f910c15d600 0x7f910c165690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:45.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f9112800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f910c0b3d70 con 0x7f910c0cca50 2024-04-23T14:31:45.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f9112800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f910c0b3bf0 con 0x7f910c15c7e0 2024-04-23T14:31:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f9112800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f910c169ad0 con 0x7f910c15d600 2024-04-23T14:31:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f9103e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f910c15c7e0 0x7f910c0c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f910be00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f910c15d600 0x7f910c165690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f910be00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f910c15d600 0x7f910c165690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.151:56768/0 (socket says 172.21.15.151:56768) 2024-04-23T14:31:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f910be00640 1 -- 172.21.15.151:0/4139410391 learned_addr learned my addr 172.21.15.151:0/4139410391 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-23T14:31:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f910b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f910c0cca50 0x7f910c150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f9103e00640 1 -- 172.21.15.151:0/4139410391 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f910c15d600 msgr2=0x7f910c165690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f9103e00640 1 --2- 172.21.15.151:0/4139410391 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f910c15d600 0x7f910c165690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f9103e00640 1 -- 172.21.15.151:0/4139410391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f910c0cca50 msgr2=0x7f910c150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f9103e00640 1 --2- 172.21.15.151:0/4139410391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f910c0cca50 0x7f910c150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f9103e00640 1 -- 172.21.15.151:0/4139410391 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910c1f7ac0 con 0x7f910c15c7e0 2024-04-23T14:31:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f910b400640 1 --2- 172.21.15.151:0/4139410391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f910c0cca50 0x7f910c150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:31:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f9103e00640 1 --2- 172.21.15.151:0/4139410391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f910c15c7e0 0x7f910c0c8560 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f90f40ac000 tx=0x7f90f401aca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:45.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f9103400640 1 -- 172.21.15.151:0/4139410391 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f40be5f0 con 0x7f910c15c7e0 2024-04-23T14:31:45.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f9103400640 1 -- 172.21.15.151:0/4139410391 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f40a7070 con 0x7f910c15c7e0 2024-04-23T14:31:45.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f9112800640 1 -- 172.21.15.151:0/4139410391 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f910c1f7da0 con 0x7f910c15c7e0 2024-04-23T14:31:45.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f9103400640 1 -- 172.21.15.151:0/4139410391 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f40be9f0 con 0x7f910c15c7e0 2024-04-23T14:31:45.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f9112800640 1 -- 172.21.15.151:0/4139410391 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f910c1f8360 con 0x7f910c15c7e0 2024-04-23T14:31:45.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f9103400640 1 -- 172.21.15.151:0/4139410391 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f90f40c7440 con 0x7f910c15c7e0 2024-04-23T14:31:45.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.596+0000 7f9103400640 1 -- 172.21.15.151:0/4139410391 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f90f40b4ab0 con 0x7f910c15c7e0 2024-04-23T14:31:45.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.600+0000 7f9112800640 1 -- 172.21.15.151:0/4139410391 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f910c068ad0 con 0x7f910c15c7e0 2024-04-23T14:31:45.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.604+0000 7f9103400640 1 -- 172.21.15.151:0/4139410391 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f90f40dde50 con 0x7f910c15c7e0 2024-04-23T14:31:45.631 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:45.630+0000 7f94c028b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T14:31:45.659 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:45.660+0000 7f78bb08d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T14:31:45.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.668+0000 7f9112800640 1 -- 172.21.15.151:0/4139410391 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8fd37274-87b3-464b-b6d2-d45a98d50b63", "id": 5} v 0) -- 0x7f910c074f20 con 0x7f910c15c7e0 2024-04-23T14:31:45.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.676+0000 7f9103400640 1 -- 172.21.15.151:0/4139410391 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8fd37274-87b3-464b-b6d2-d45a98d50b63", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f90f40cc030 con 0x7f910c15c7e0 2024-04-23T14:31:45.675 INFO:teuthology.orchestra.run.smithi151.stdout:5 2024-04-23T14:31:45.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.676+0000 7f9112800640 1 -- 172.21.15.151:0/4139410391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f910c15c7e0 msgr2=0x7f910c0c8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:45.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.676+0000 7f9112800640 1 --2- 172.21.15.151:0/4139410391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f910c15c7e0 0x7f910c0c8560 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f90f40ac000 tx=0x7f90f401aca0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.676+0000 7f9112800640 1 -- 172.21.15.151:0/4139410391 shutdown_connections 2024-04-23T14:31:45.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.676+0000 7f9112800640 1 --2- 172.21.15.151:0/4139410391 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f910c15d600 0x7f910c165690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.676+0000 7f9112800640 1 --2- 172.21.15.151:0/4139410391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f910c15c7e0 0x7f910c0c8560 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.676+0000 7f9112800640 1 --2- 172.21.15.151:0/4139410391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f910c0cca50 0x7f910c150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.676+0000 7f9112800640 1 -- 172.21.15.151:0/4139410391 >> 172.21.15.151:0/4139410391 conn(0x7f910c0cee80 msgr2=0x7f910c0d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:45.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.676+0000 7f9112800640 1 -- 172.21.15.151:0/4139410391 shutdown_connections 2024-04-23T14:31:45.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.676+0000 7f9112800640 1 -- 172.21.15.151:0/4139410391 wait complete. 2024-04-23T14:31:45.695 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd 6 2024-04-23T14:31:45.696 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:45.698+0000 7f94c028b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T14:31:45.729 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:45.728+0000 7f78bb08d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T14:31:45.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.828+0000 7f4686800640 1 Processor -- start 2024-04-23T14:31:45.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.828+0000 7f4686800640 1 -- start start 2024-04-23T14:31:45.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.828+0000 7f4686800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f468006d450 0x7f468006d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:45.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.828+0000 7f4686800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f468006dd70 con 0x7f46800cea60 2024-04-23T14:31:45.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.828+0000 7f4686800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46800b1f90 con 0x7f46800cd460 2024-04-23T14:31:45.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.828+0000 7f4686800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f46800b2270 con 0x7f468006d450 2024-04-23T14:31:45.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.832+0000 7f467f400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f468006d450 0x7f468006d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:45.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.832+0000 7f467f400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f468006d450 0x7f468006d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33736/0 (socket says 172.21.15.151:33736) 2024-04-23T14:31:45.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.832+0000 7f467f400640 1 -- 172.21.15.151:0/4054355667 learned_addr learned my addr 172.21.15.151:0/4054355667 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-23T14:31:45.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.832+0000 7f467fe00640 1 --1- 172.21.15.151:0/4054355667 >> v1:172.21.15.151:6789/0 conn(0x7f46800cd460 0x7f46800b2d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.151:52896/0 (socket says 172.21.15.151:52896) 2024-04-23T14:31:45.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.832+0000 7f467f400640 1 -- 172.21.15.151:0/4054355667 >> v1:172.21.15.151:6789/0 conn(0x7f46800cd460 legacy=0x7f46800b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:45.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.832+0000 7f467f400640 1 -- 172.21.15.151:0/4054355667 >> v1:172.21.15.100:6789/0 conn(0x7f46800cea60 legacy=0x7f46800d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:45.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.832+0000 7f467f400640 1 -- 172.21.15.151:0/4054355667 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46800b2550 con 0x7f468006d450 2024-04-23T14:31:45.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.832+0000 7f467f400640 1 --2- 172.21.15.151:0/4054355667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f468006d450 0x7f468006d830 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4670009870 tx=0x7f467001c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7ebda07dac1e829 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:45.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.832+0000 7f467e000640 1 -- 172.21.15.151:0/4054355667 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46700b0070 con 0x7f468006d450 2024-04-23T14:31:45.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.832+0000 7f467e000640 1 -- 172.21.15.151:0/4054355667 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46700ab600 con 0x7f468006d450 2024-04-23T14:31:45.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.832+0000 7f467e000640 1 -- 172.21.15.151:0/4054355667 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46700ab9d0 con 0x7f468006d450 2024-04-23T14:31:45.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.836+0000 7f4686800640 1 -- 172.21.15.151:0/4054355667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f468006d450 msgr2=0x7f468006d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:45.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.836+0000 7f4686800640 1 --2- 172.21.15.151:0/4054355667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f468006d450 0x7f468006d830 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4670009870 tx=0x7f467001c4d0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.836+0000 7f4686800640 1 -- 172.21.15.151:0/4054355667 shutdown_connections 2024-04-23T14:31:45.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.836+0000 7f4686800640 1 --2- 172.21.15.151:0/4054355667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f468006d450 0x7f468006d830 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.836+0000 7f4686800640 1 -- 172.21.15.151:0/4054355667 >> 172.21.15.151:0/4054355667 conn(0x7f46800cee80 msgr2=0x7f46800cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:45.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.836+0000 7f4686800640 1 -- 172.21.15.151:0/4054355667 shutdown_connections 2024-04-23T14:31:45.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.836+0000 7f4686800640 1 -- 172.21.15.151:0/4054355667 wait complete. 2024-04-23T14:31:45.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.836+0000 7f4686800640 1 Processor -- start 2024-04-23T14:31:45.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.836+0000 7f4686800640 1 -- start start 2024-04-23T14:31:45.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.836+0000 7f4686800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f468006d450 0x7f468015cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:45.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.836+0000 7f4686800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46800cd460 0x7f46801596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:45.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.836+0000 7f4686800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46800cea60 0x7f468015d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:45.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.836+0000 7f4686800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f468016cd50 con 0x7f46800cea60 2024-04-23T14:31:45.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.836+0000 7f4686800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f468016cbd0 con 0x7f46800cd460 2024-04-23T14:31:45.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.836+0000 7f4686800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f468016ced0 con 0x7f468006d450 2024-04-23T14:31:45.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.836+0000 7f467f400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f468006d450 0x7f468015cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:45.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.836+0000 7f467ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46800cd460 0x7f46801596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:45.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.836+0000 7f467f400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f468006d450 0x7f468015cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.151:56784/0 (socket says 172.21.15.151:56784) 2024-04-23T14:31:45.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.836+0000 7f467ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46800cd460 0x7f46801596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33740/0 (socket says 172.21.15.151:33740) 2024-04-23T14:31:45.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.836+0000 7f467f400640 1 -- 172.21.15.151:0/3585625312 learned_addr learned my addr 172.21.15.151:0/3585625312 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-23T14:31:45.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.836+0000 7f467fe00640 1 --2- 172.21.15.151:0/3585625312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46800cea60 0x7f468015d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:45.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.840+0000 7f467ea00640 1 -- 172.21.15.151:0/3585625312 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f468006d450 msgr2=0x7f468015cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:45.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.840+0000 7f467ea00640 1 --2- 172.21.15.151:0/3585625312 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f468006d450 0x7f468015cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.840+0000 7f467ea00640 1 -- 172.21.15.151:0/3585625312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46800cea60 msgr2=0x7f468015d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:45.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.840+0000 7f467ea00640 1 --2- 172.21.15.151:0/3585625312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46800cea60 0x7f468015d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.840+0000 7f467ea00640 1 -- 172.21.15.151:0/3585625312 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4680159e60 con 0x7f46800cd460 2024-04-23T14:31:45.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.840+0000 7f467f400640 1 --2- 172.21.15.151:0/3585625312 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f468006d450 0x7f468015cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:31:45.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.840+0000 7f467ea00640 1 --2- 172.21.15.151:0/3585625312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46800cd460 0x7f46801596f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f466c00c390 tx=0x7f466c00c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:45.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.840+0000 7f4677400640 1 -- 172.21.15.151:0/3585625312 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c0052d0 con 0x7f46800cd460 2024-04-23T14:31:45.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.840+0000 7f4677400640 1 -- 172.21.15.151:0/3585625312 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f466c005470 con 0x7f46800cd460 2024-04-23T14:31:45.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.840+0000 7f4677400640 1 -- 172.21.15.151:0/3585625312 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c007260 con 0x7f46800cd460 2024-04-23T14:31:45.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.840+0000 7f4686800640 1 -- 172.21.15.151:0/3585625312 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f468015a140 con 0x7f46800cd460 2024-04-23T14:31:45.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.840+0000 7f4686800640 1 -- 172.21.15.151:0/3585625312 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f468015a700 con 0x7f46800cd460 2024-04-23T14:31:45.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.840+0000 7f4677400640 1 -- 172.21.15.151:0/3585625312 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f466c007400 con 0x7f46800cd460 2024-04-23T14:31:45.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.840+0000 7f4677400640 1 -- 172.21.15.151:0/3585625312 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f466c0067e0 con 0x7f46800cd460 2024-04-23T14:31:45.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.840+0000 7f4686800640 1 -- 172.21.15.151:0/3585625312 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4680068ad0 con 0x7f46800cd460 2024-04-23T14:31:45.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.844+0000 7f4677400640 1 -- 172.21.15.151:0/3585625312 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f466c003600 con 0x7f46800cd460 2024-04-23T14:31:45.863 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:45.862+0000 7f94c028b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T14:31:45.901 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:45.900+0000 7f78bb08d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T14:31:45.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.908+0000 7f4686800640 1 -- 172.21.15.151:0/3585625312 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd", "id": 6} v 0) -- 0x7f4680072610 con 0x7f46800cd460 2024-04-23T14:31:45.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.916+0000 7f4677400640 1 -- 172.21.15.151:0/3585625312 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f466c006b00 con 0x7f46800cd460 2024-04-23T14:31:45.917 INFO:teuthology.orchestra.run.smithi151.stdout:6 2024-04-23T14:31:45.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.916+0000 7f4686800640 1 -- 172.21.15.151:0/3585625312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46800cd460 msgr2=0x7f46801596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:45.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.916+0000 7f4686800640 1 --2- 172.21.15.151:0/3585625312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46800cd460 0x7f46801596f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f466c00c390 tx=0x7f466c00c850 comp rx=0 tx=0).stop 2024-04-23T14:31:45.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.916+0000 7f4686800640 1 -- 172.21.15.151:0/3585625312 shutdown_connections 2024-04-23T14:31:45.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.916+0000 7f4686800640 1 --2- 172.21.15.151:0/3585625312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46800cea60 0x7f468015d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.916+0000 7f4686800640 1 --2- 172.21.15.151:0/3585625312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46800cd460 0x7f46801596f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.916+0000 7f4686800640 1 --2- 172.21.15.151:0/3585625312 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f468006d450 0x7f468015cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:45.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.916+0000 7f4686800640 1 -- 172.21.15.151:0/3585625312 >> 172.21.15.151:0/3585625312 conn(0x7f46800cee80 msgr2=0x7f46800ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:45.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.916+0000 7f4686800640 1 -- 172.21.15.151:0/3585625312 shutdown_connections 2024-04-23T14:31:45.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:45.916+0000 7f4686800640 1 -- 172.21.15.151:0/3585625312 wait complete. 2024-04-23T14:31:45.934 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:45.934+0000 7f94c028b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T14:31:45.936 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new 015597f2-4f97-492d-b821-56111164ed6c 7 2024-04-23T14:31:45.976 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:45.976+0000 7f78bb08d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T14:31:46.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.108+0000 7fdb55800640 1 Processor -- start 2024-04-23T14:31:46.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.108+0000 7fdb55800640 1 -- start start 2024-04-23T14:31:46.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.112+0000 7fdb55800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdb50072000 0x7fdb50160180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:46.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.112+0000 7fdb55800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb500b7e70 0x7fdb500d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:46.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.112+0000 7fdb55800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb500d03a0 con 0x7fdb5015c700 2024-04-23T14:31:46.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.112+0000 7fdb55800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fdb50160760 con 0x7fdb50072000 2024-04-23T14:31:46.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.112+0000 7fdb55800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fdb50160a40 con 0x7fdb500b7e70 2024-04-23T14:31:46.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.112+0000 7fdb4ea00640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7fdb5015c700 0x7fdb500718f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.151:52910/0 (socket says 172.21.15.151:52910) 2024-04-23T14:31:46.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.112+0000 7fdb4ea00640 1 -- 172.21.15.151:0/1216128295 learned_addr learned my addr 172.21.15.151:0/1216128295 (peer_addr_for_me v1:172.21.15.151:0/0) 2024-04-23T14:31:46.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.112+0000 7fdb4f400640 1 --2- 172.21.15.151:0/1216128295 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb500b7e70 0x7fdb500d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:46.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.112+0000 7fdb47e00640 1 --2- 172.21.15.151:0/1216128295 >> v2:172.21.15.100:3300/0 conn(0x7fdb50072000 0x7fdb50160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:46.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.112+0000 7fdb4e000640 1 -- 172.21.15.151:0/1216128295 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 65213426 0 0) 0x7fdb500d03a0 con 0x7fdb5015c700 2024-04-23T14:31:46.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.112+0000 7fdb4f400640 1 -- 172.21.15.151:0/1216128295 >> v2:172.21.15.100:3300/0 conn(0x7fdb50072000 msgr2=0x7fdb50160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:46.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.112+0000 7fdb4f400640 1 --2- 172.21.15.151:0/1216128295 >> v2:172.21.15.100:3300/0 conn(0x7fdb50072000 0x7fdb50160180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:46.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.112+0000 7fdb4f400640 1 -- 172.21.15.151:0/1216128295 >> v1:172.21.15.151:6789/0 conn(0x7fdb5015c700 legacy=0x7fdb500718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:46.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.112+0000 7fdb4f400640 1 -- 172.21.15.151:0/1216128295 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb50160d20 con 0x7fdb500b7e70 2024-04-23T14:31:46.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.112+0000 7fdb47e00640 1 --2- 172.21.15.151:0/1216128295 >> v2:172.21.15.100:3300/0 conn(0x7fdb50072000 0x7fdb50160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:31:46.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.116+0000 7fdb4f400640 1 --2- 172.21.15.151:0/1216128295 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb500b7e70 0x7fdb500d0850 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009d90 tx=0x7fdb4801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be4d146473ba22f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:46.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.116+0000 7fdb4e000640 1 -- 172.21.15.151:0/1216128295 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb480b2070 con 0x7fdb500b7e70 2024-04-23T14:31:46.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.116+0000 7fdb4e000640 1 -- 172.21.15.151:0/1216128295 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb480a5940 con 0x7fdb500b7e70 2024-04-23T14:31:46.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.116+0000 7fdb4e000640 1 -- 172.21.15.151:0/1216128295 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb480a5d10 con 0x7fdb500b7e70 2024-04-23T14:31:46.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.116+0000 7fdb55800640 1 -- 172.21.15.151:0/1216128295 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb500b7e70 msgr2=0x7fdb500d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:46.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.116+0000 7fdb55800640 1 --2- 172.21.15.151:0/1216128295 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb500b7e70 0x7fdb500d0850 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009d90 tx=0x7fdb4801c970 comp rx=0 tx=0).stop 2024-04-23T14:31:46.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.116+0000 7fdb55800640 1 -- 172.21.15.151:0/1216128295 shutdown_connections 2024-04-23T14:31:46.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.116+0000 7fdb55800640 1 --2- 172.21.15.151:0/1216128295 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb500b7e70 0x7fdb500d0850 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:46.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.116+0000 7fdb55800640 1 --2- 172.21.15.151:0/1216128295 >> v2:172.21.15.100:3300/0 conn(0x7fdb50072000 0x7fdb50160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:46.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.116+0000 7fdb55800640 1 -- 172.21.15.151:0/1216128295 >> 172.21.15.151:0/1216128295 conn(0x7fdb500d2400 msgr2=0x7fdb500d2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:46.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.116+0000 7fdb55800640 1 -- 172.21.15.151:0/1216128295 shutdown_connections 2024-04-23T14:31:46.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.116+0000 7fdb55800640 1 -- 172.21.15.151:0/1216128295 wait complete. 2024-04-23T14:31:46.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb55800640 1 Processor -- start 2024-04-23T14:31:46.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb55800640 1 -- start start 2024-04-23T14:31:46.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb55800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb50072000 0x7fdb5015ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:46.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb55800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb500b7e70 0x7fdb501540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:46.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb55800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb5015c700 0x7fdb501599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:46.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb55800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdb500d1930 con 0x7fdb500b7e70 2024-04-23T14:31:46.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb55800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdb500d17b0 con 0x7fdb50072000 2024-04-23T14:31:46.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb55800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fdb500d1ab0 con 0x7fdb5015c700 2024-04-23T14:31:46.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb4f400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb5015c700 0x7fdb501599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:46.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb4ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb50072000 0x7fdb5015ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:46.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb4f400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb5015c700 0x7fdb501599f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.151:56814/0 (socket says 172.21.15.151:56814) 2024-04-23T14:31:46.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb4f400640 1 -- 172.21.15.151:0/3570751573 learned_addr learned my addr 172.21.15.151:0/3570751573 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-23T14:31:46.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb47e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb500b7e70 0x7fdb501540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:46.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb4f400640 1 -- 172.21.15.151:0/3570751573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb50072000 msgr2=0x7fdb5015ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:46.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb4f400640 1 --2- 172.21.15.151:0/3570751573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb50072000 0x7fdb5015ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:46.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb4f400640 1 -- 172.21.15.151:0/3570751573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb500b7e70 msgr2=0x7fdb501540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:46.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb4f400640 1 --2- 172.21.15.151:0/3570751573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb500b7e70 0x7fdb501540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:46.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb4f400640 1 -- 172.21.15.151:0/3570751573 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb50153030 con 0x7fdb5015c700 2024-04-23T14:31:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb4f400640 1 --2- 172.21.15.151:0/3570751573 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb5015c700 0x7fdb501599f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdb480b0040 tx=0x7fdb48019710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb46a00640 1 -- 172.21.15.151:0/3570751573 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb480b2040 con 0x7fdb5015c700 2024-04-23T14:31:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb46a00640 1 -- 172.21.15.151:0/3570751573 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb480bf070 con 0x7fdb5015c700 2024-04-23T14:31:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb55800640 1 -- 172.21.15.151:0/3570751573 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb50153310 con 0x7fdb5015c700 2024-04-23T14:31:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb46a00640 1 -- 172.21.15.151:0/3570751573 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb480b8670 con 0x7fdb5015c700 2024-04-23T14:31:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb55800640 1 -- 172.21.15.151:0/3570751573 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb501538d0 con 0x7fdb5015c700 2024-04-23T14:31:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb46a00640 1 -- 172.21.15.151:0/3570751573 <== mon.2 v2:172.21.15.151:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdb480b8810 con 0x7fdb5015c700 2024-04-23T14:31:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb46a00640 1 -- 172.21.15.151:0/3570751573 <== mon.2 v2:172.21.15.151:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fdb48003d30 con 0x7fdb5015c700 2024-04-23T14:31:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.120+0000 7fdb55800640 1 -- 172.21.15.151:0/3570751573 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb50068ad0 con 0x7fdb5015c700 2024-04-23T14:31:46.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.124+0000 7fdb46a00640 1 -- 172.21.15.151:0/3570751573 <== mon.2 v2:172.21.15.151:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fdb48016560 con 0x7fdb5015c700 2024-04-23T14:31:46.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.192+0000 7fdb55800640 1 -- 172.21.15.151:0/3570751573 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "osd new", "uuid": "015597f2-4f97-492d-b821-56111164ed6c", "id": 7} v 0) -- 0x7fdb50159310 con 0x7fdb5015c700 2024-04-23T14:31:46.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.200+0000 7fdb46a00640 1 -- 172.21.15.151:0/3570751573 <== mon.2 v2:172.21.15.151:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "015597f2-4f97-492d-b821-56111164ed6c", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fdb480b8b00 con 0x7fdb5015c700 2024-04-23T14:31:46.200 INFO:teuthology.orchestra.run.smithi151.stdout:7 2024-04-23T14:31:46.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.200+0000 7fdb55800640 1 -- 172.21.15.151:0/3570751573 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb5015c700 msgr2=0x7fdb501599f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:46.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.200+0000 7fdb55800640 1 --2- 172.21.15.151:0/3570751573 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb5015c700 0x7fdb501599f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdb480b0040 tx=0x7fdb48019710 comp rx=0 tx=0).stop 2024-04-23T14:31:46.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.200+0000 7fdb55800640 1 -- 172.21.15.151:0/3570751573 shutdown_connections 2024-04-23T14:31:46.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.200+0000 7fdb55800640 1 --2- 172.21.15.151:0/3570751573 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb5015c700 0x7fdb501599f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:46.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.200+0000 7fdb55800640 1 --2- 172.21.15.151:0/3570751573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb500b7e70 0x7fdb501540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:46.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.200+0000 7fdb55800640 1 --2- 172.21.15.151:0/3570751573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb50072000 0x7fdb5015ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:46.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.200+0000 7fdb55800640 1 -- 172.21.15.151:0/3570751573 >> 172.21.15.151:0/3570751573 conn(0x7fdb500d2400 msgr2=0x7fdb500d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:46.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.200+0000 7fdb55800640 1 -- 172.21.15.151:0/3570751573 shutdown_connections 2024-04-23T14:31:46.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:31:46.200+0000 7fdb55800640 1 -- 172.21.15.151:0/3570751573 wait complete. 2024-04-23T14:31:46.219 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-23T14:31:46.219 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-23T14:31:46.222 INFO:tasks.ceph.osd.0:Started 2024-04-23T14:31:46.222 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-23T14:31:46.222 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-23T14:31:46.225 INFO:tasks.ceph.osd.1:Started 2024-04-23T14:31:46.225 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-23T14:31:46.225 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-23T14:31:46.228 INFO:tasks.ceph.osd.2:Started 2024-04-23T14:31:46.228 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-23T14:31:46.228 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-23T14:31:46.231 INFO:tasks.ceph.osd.3:Started 2024-04-23T14:31:46.231 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-23T14:31:46.232 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-23T14:31:46.234 INFO:tasks.ceph.osd.4:Started 2024-04-23T14:31:46.235 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-23T14:31:46.235 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-23T14:31:46.237 INFO:tasks.ceph.osd.5:Started 2024-04-23T14:31:46.238 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-23T14:31:46.238 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-23T14:31:46.241 INFO:tasks.ceph.osd.6:Started 2024-04-23T14:31:46.241 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-23T14:31:46.241 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-23T14:31:46.247 INFO:tasks.ceph.osd.7:Started 2024-04-23T14:31:46.247 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:31:46.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.466+0000 7f052be00640 1 Processor -- start 2024-04-23T14:31:46.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.466+0000 7f052be00640 1 -- start start 2024-04-23T14:31:46.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.466+0000 7f052be00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0524073590 0x7f052406f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:46.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.466+0000 7f052be00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f052406f800 con 0x7f052406ec60 2024-04-23T14:31:46.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.466+0000 7f052be00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f052406fae0 con 0x7f052415c580 2024-04-23T14:31:46.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.466+0000 7f052be00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f05240cfe70 con 0x7f0524073590 2024-04-23T14:31:46.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.466+0000 7f0528a00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f052406ec60 0x7f0524072e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46638/0 (socket says 172.21.15.100:46638) 2024-04-23T14:31:46.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.466+0000 7f0528a00640 1 -- 172.21.15.100:0/2316633114 learned_addr learned my addr 172.21.15.100:0/2316633114 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:31:46.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.466+0000 7f0529e00640 1 --2- 172.21.15.100:0/2316633114 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0524073590 0x7f052406f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:46.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.466+0000 7f0523e00640 1 -- 172.21.15.100:0/2316633114 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2915236027 0 0) 0x7f052406f800 con 0x7f052406ec60 2024-04-23T14:31:46.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.466+0000 7f0523e00640 1 -- 172.21.15.100:0/2316633114 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f050c003530 con 0x7f052406ec60 2024-04-23T14:31:46.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.466+0000 7f0523e00640 1 -- 172.21.15.100:0/2316633114 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 894118025 0 0) 0x7f052406fae0 con 0x7f052415c580 2024-04-23T14:31:46.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.466+0000 7f0523e00640 1 -- 172.21.15.100:0/2316633114 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f052406f800 con 0x7f052415c580 2024-04-23T14:31:46.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.466+0000 7f0529e00640 1 -- 172.21.15.100:0/2316633114 >> v1:172.21.15.151:6789/0 conn(0x7f052415c580 legacy=0x7f05240cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:46.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.466+0000 7f0529e00640 1 -- 172.21.15.100:0/2316633114 >> v1:172.21.15.100:6789/0 conn(0x7f052406ec60 legacy=0x7f0524072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:46.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.466+0000 7f0529e00640 1 -- 172.21.15.100:0/2316633114 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052406c470 con 0x7f0524073590 2024-04-23T14:31:46.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.470+0000 7f0529e00640 1 --2- 172.21.15.100:0/2316633114 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0524073590 0x7f052406f0c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f051c009dc0 tx=0x7f051c01c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86683aa53025b81b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:46.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.470+0000 7f0523e00640 1 -- 172.21.15.100:0/2316633114 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c0b2070 con 0x7f0524073590 2024-04-23T14:31:46.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.470+0000 7f0523e00640 1 -- 172.21.15.100:0/2316633114 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f051c0a57c0 con 0x7f0524073590 2024-04-23T14:31:46.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.470+0000 7f0523e00640 1 -- 172.21.15.100:0/2316633114 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c0a5b90 con 0x7f0524073590 2024-04-23T14:31:46.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.470+0000 7f052be00640 1 -- 172.21.15.100:0/2316633114 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0524073590 msgr2=0x7f052406f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:46.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.470+0000 7f052be00640 1 --2- 172.21.15.100:0/2316633114 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0524073590 0x7f052406f0c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f051c009dc0 tx=0x7f051c01c900 comp rx=0 tx=0).stop 2024-04-23T14:31:46.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.470+0000 7f052be00640 1 -- 172.21.15.100:0/2316633114 shutdown_connections 2024-04-23T14:31:46.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.470+0000 7f052be00640 1 --2- 172.21.15.100:0/2316633114 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0524073590 0x7f052406f0c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:46.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.470+0000 7f052be00640 1 -- 172.21.15.100:0/2316633114 >> 172.21.15.100:0/2316633114 conn(0x7f05240b3050 msgr2=0x7f05240b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:46.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.470+0000 7f052be00640 1 -- 172.21.15.100:0/2316633114 shutdown_connections 2024-04-23T14:31:46.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.470+0000 7f052be00640 1 -- 172.21.15.100:0/2316633114 wait complete. 2024-04-23T14:31:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.470+0000 7f052be00640 1 Processor -- start 2024-04-23T14:31:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f052be00640 1 -- start start 2024-04-23T14:31:46.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f052be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f052406ec60 0x7f0524146cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:46.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f052be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0524073590 0x7f052414e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:46.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f052be00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f052415c580 0x7f052414ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:46.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f052be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f052406c7e0 con 0x7f0524073590 2024-04-23T14:31:46.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f052be00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f052406d730 con 0x7f052406ec60 2024-04-23T14:31:46.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f052be00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f052416a4f0 con 0x7f052415c580 2024-04-23T14:31:46.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f0528a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0524073590 0x7f052414e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:46.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f0528a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0524073590 0x7f052414e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48338/0 (socket says 172.21.15.100:48338) 2024-04-23T14:31:46.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f0528a00640 1 -- 172.21.15.100:0/1103721179 learned_addr learned my addr 172.21.15.100:0/1103721179 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:31:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f0529e00640 1 --2- 172.21.15.100:0/1103721179 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f052415c580 0x7f052414ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f0529400640 1 --2- 172.21.15.100:0/1103721179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f052406ec60 0x7f0524146cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f0528a00640 1 -- 172.21.15.100:0/1103721179 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f052415c580 msgr2=0x7f052414ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f0528a00640 1 --2- 172.21.15.100:0/1103721179 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f052415c580 0x7f052414ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f0528a00640 1 -- 172.21.15.100:0/1103721179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f052406ec60 msgr2=0x7f0524146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f0528a00640 1 --2- 172.21.15.100:0/1103721179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f052406ec60 0x7f0524146cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f0528a00640 1 -- 172.21.15.100:0/1103721179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05241478f0 con 0x7f0524073590 2024-04-23T14:31:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f0528a00640 1 --2- 172.21.15.100:0/1103721179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0524073590 0x7f052414e800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f051000c420 tx=0x7f051000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f0521600640 1 -- 172.21.15.100:0/1103721179 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0510017070 con 0x7f0524073590 2024-04-23T14:31:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f0521600640 1 -- 172.21.15.100:0/1103721179 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0510005cf0 con 0x7f0524073590 2024-04-23T14:31:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f052be00640 1 -- 172.21.15.100:0/1103721179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0524147bd0 con 0x7f0524073590 2024-04-23T14:31:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f0521600640 1 -- 172.21.15.100:0/1103721179 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05100072d0 con 0x7f0524073590 2024-04-23T14:31:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f052be00640 1 -- 172.21.15.100:0/1103721179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05241442e0 con 0x7f0524073590 2024-04-23T14:31:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f0521600640 1 -- 172.21.15.100:0/1103721179 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f051001a070 con 0x7f0524073590 2024-04-23T14:31:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f0521600640 1 -- 172.21.15.100:0/1103721179 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f051001cc60 con 0x7f0524073590 2024-04-23T14:31:46.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.474+0000 7f052be00640 1 -- 172.21.15.100:0/1103721179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0524068ad0 con 0x7f0524073590 2024-04-23T14:31:46.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.478+0000 7f0521600640 1 -- 172.21.15.100:0/1103721179 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0510007470 con 0x7f0524073590 2024-04-23T14:31:46.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.534+0000 7f052be00640 1 -- 172.21.15.100:0/1103721179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05240cf310 con 0x7f0524073590 2024-04-23T14:31:46.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.534+0000 7f0521600640 1 -- 172.21.15.100:0/1103721179 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f0510007650 con 0x7f0524073590 2024-04-23T14:31:46.535 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:31:46.535 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":10,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:31:46.195181+0000","last_up_change":"0.000000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:31:46.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.534+0000 7f052be00640 1 -- 172.21.15.100:0/1103721179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0524073590 msgr2=0x7f052414e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:46.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.534+0000 7f052be00640 1 --2- 172.21.15.100:0/1103721179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0524073590 0x7f052414e800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f051000c420 tx=0x7f051000c8e0 comp rx=0 tx=0).stop 2024-04-23T14:31:46.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.534+0000 7f052be00640 1 -- 172.21.15.100:0/1103721179 shutdown_connections 2024-04-23T14:31:46.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.534+0000 7f052be00640 1 --2- 172.21.15.100:0/1103721179 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f052415c580 0x7f052414ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:46.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.534+0000 7f052be00640 1 --2- 172.21.15.100:0/1103721179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0524073590 0x7f052414e800 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:46.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.534+0000 7f052be00640 1 --2- 172.21.15.100:0/1103721179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f052406ec60 0x7f0524146cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:46.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.534+0000 7f052be00640 1 -- 172.21.15.100:0/1103721179 >> 172.21.15.100:0/1103721179 conn(0x7f05240b3050 msgr2=0x7f052415e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:46.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.534+0000 7f052be00640 1 -- 172.21.15.100:0/1103721179 shutdown_connections 2024-04-23T14:31:46.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:31:46.534+0000 7f052be00640 1 -- 172.21.15.100:0/1103721179 wait complete. 2024-04-23T14:31:46.554 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-23T14:31:46.554 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-23T14:31:46.607 INFO:tasks.ceph.mgr.y.smithi100.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-23T14:31:46.607 INFO:tasks.ceph.mgr.y.smithi100.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-23T14:31:46.607 INFO:tasks.ceph.mgr.y.smithi100.stderr: from numpy import show_config as show_numpy_config 2024-04-23T14:31:46.653 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:46.654+0000 7f94c028b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T14:31:46.656 INFO:tasks.ceph.mgr.x.smithi151.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-23T14:31:46.656 INFO:tasks.ceph.mgr.x.smithi151.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-23T14:31:46.656 INFO:tasks.ceph.mgr.x.smithi151.stderr: from numpy import show_config as show_numpy_config 2024-04-23T14:31:46.693 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:46.692+0000 7f78bb08d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T14:31:46.802 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:46.802+0000 7f94c028b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:31:46.842 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:46.840+0000 7f78bb08d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:31:46.898 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-04-23T14:31:46.898+0000 7f53e792b7c0 -1 Falling back to public interface 2024-04-23T14:31:46.913 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-04-23T14:31:46.914+0000 7f66d8e4a7c0 -1 Falling back to public interface 2024-04-23T14:31:46.928 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-23T14:31:46.928+0000 7fd85893a7c0 -1 Falling back to public interface 2024-04-23T14:31:46.936 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-23T14:31:46.936+0000 7f04514e67c0 -1 Falling back to public interface 2024-04-23T14:31:46.947 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-23T14:31:46.948+0000 7f65cfe227c0 -1 Falling back to public interface 2024-04-23T14:31:46.949 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-04-23T14:31:46.950+0000 7f092fb617c0 -1 Falling back to public interface 2024-04-23T14:31:46.972 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-23T14:31:46.972+0000 7fb85acf07c0 -1 Falling back to public interface 2024-04-23T14:31:46.973 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-04-23T14:31:46.974+0000 7f018c8957c0 -1 Falling back to public interface 2024-04-23T14:31:47.007 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:47.006+0000 7f94c028b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T14:31:47.029 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:47.028+0000 7f78bb08d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T14:31:47.080 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:47.078+0000 7f94c028b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T14:31:47.101 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:47.100+0000 7f78bb08d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T14:31:47.230 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:47.230+0000 7f94c028b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:31:47.251 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:47.252+0000 7f78bb08d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:31:47.359 DEBUG:teuthology.orchestra.run.smithi100:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-23T14:31:47.485 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.486+0000 7f0fafa00640 1 Processor -- start 2024-04-23T14:31:47.485 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.486+0000 7f0fafa00640 1 -- start start 2024-04-23T14:31:47.487 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.486+0000 7f0fafa00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fa815c320 0x7f0fa815ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:47.487 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.486+0000 7f0fafa00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f0fa806da80 0x7f0fa806ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:47.487 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.486+0000 7f0fafa00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fa815f4a0 con 0x7f0fa80b3bb0 2024-04-23T14:31:47.487 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.486+0000 7f0fafa00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f0fa806c1c0 con 0x7f0fa806da80 2024-04-23T14:31:47.487 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.486+0000 7f0fafa00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0fa815fd40 con 0x7f0fa815c320 2024-04-23T14:31:47.488 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.486+0000 7f0fad000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fa815c320 0x7f0fa815ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:47.488 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.486+0000 7f0fada00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f0fa806da80 0x7f0fa806ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:47.488 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.486+0000 7f0fad000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fa815c320 0x7f0fa815ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:53072/0 (socket says 172.21.15.100:53072) 2024-04-23T14:31:47.489 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.486+0000 7f0fad000640 1 -- 172.21.15.100:0/1889071776 learned_addr learned my addr 172.21.15.100:0/1889071776 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:31:47.489 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.486+0000 7f0fa7400640 1 -- 172.21.15.100:0/1889071776 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3703064558 0 0) 0x7f0fa815f4a0 con 0x7f0fa80b3bb0 2024-04-23T14:31:47.489 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.486+0000 7f0fa7400640 1 -- 172.21.15.100:0/1889071776 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f90003530 con 0x7f0fa80b3bb0 2024-04-23T14:31:47.489 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.486+0000 7f0fad000640 1 -- 172.21.15.100:0/1889071776 >> v2:172.21.15.151:3300/0 conn(0x7f0fa806da80 msgr2=0x7f0fa806ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:47.489 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.486+0000 7f0fad000640 1 --2- 172.21.15.100:0/1889071776 >> v2:172.21.15.151:3300/0 conn(0x7f0fa806da80 0x7f0fa806ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:47.489 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.486+0000 7f0fad000640 1 -- 172.21.15.100:0/1889071776 >> v1:172.21.15.151:6789/0 conn(0x7f0fa80b3bb0 legacy=0x7f0fa806d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:47.489 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.486+0000 7f0fad000640 1 -- 172.21.15.100:0/1889071776 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa815f780 con 0x7f0fa815c320 2024-04-23T14:31:47.489 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.486+0000 7f0fada00640 1 --2- 172.21.15.100:0/1889071776 >> v2:172.21.15.151:3300/0 conn(0x7f0fa806da80 0x7f0fa806ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:31:47.491 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.490+0000 7f0fad000640 1 --2- 172.21.15.100:0/1889071776 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fa815c320 0x7f0fa815ed50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009870 tx=0x7f0f9c01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=462fb14fb1863e12 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:47.492 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.490+0000 7f0fa7400640 1 -- 172.21.15.100:0/1889071776 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c0b0070 con 0x7f0fa815c320 2024-04-23T14:31:47.492 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.490+0000 7f0fa7400640 1 -- 172.21.15.100:0/1889071776 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f9c0ab600 con 0x7f0fa815c320 2024-04-23T14:31:47.492 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.490+0000 7f0fa7400640 1 -- 172.21.15.100:0/1889071776 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c0ab9d0 con 0x7f0fa815c320 2024-04-23T14:31:47.492 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.490+0000 7f0fafa00640 1 -- 172.21.15.100:0/1889071776 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fa815c320 msgr2=0x7f0fa815ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:47.492 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.490+0000 7f0fafa00640 1 --2- 172.21.15.100:0/1889071776 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fa815c320 0x7f0fa815ed50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009870 tx=0x7f0f9c01c4d0 comp rx=0 tx=0).stop 2024-04-23T14:31:47.492 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.490+0000 7f0fafa00640 1 -- 172.21.15.100:0/1889071776 shutdown_connections 2024-04-23T14:31:47.492 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.490+0000 7f0fafa00640 1 --2- 172.21.15.100:0/1889071776 >> v2:172.21.15.151:3300/0 conn(0x7f0fa806da80 0x7f0fa806ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:47.492 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.490+0000 7f0fafa00640 1 --2- 172.21.15.100:0/1889071776 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fa815c320 0x7f0fa815ed50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:47.493 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.490+0000 7f0fafa00640 1 -- 172.21.15.100:0/1889071776 >> 172.21.15.100:0/1889071776 conn(0x7f0fa80725b0 msgr2=0x7f0fa80729c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:47.493 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.490+0000 7f0fafa00640 1 -- 172.21.15.100:0/1889071776 shutdown_connections 2024-04-23T14:31:47.493 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.490+0000 7f0fafa00640 1 -- 172.21.15.100:0/1889071776 wait complete. 2024-04-23T14:31:47.493 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fafa00640 1 Processor -- start 2024-04-23T14:31:47.493 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fafa00640 1 -- start start 2024-04-23T14:31:47.495 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fafa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fa806da80 0x7f0fa81545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:47.495 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fafa00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fa80b3bb0 0x7f0fa813e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:47.495 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fafa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fa815c320 0x7f0fa80b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:47.495 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fafa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa806fd40 con 0x7f0fa815c320 2024-04-23T14:31:47.495 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fafa00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa806fbc0 con 0x7f0fa806da80 2024-04-23T14:31:47.495 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fafa00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0fa806fec0 con 0x7f0fa80b3bb0 2024-04-23T14:31:47.495 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fada00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fa815c320 0x7f0fa80b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:47.495 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fad000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fa806da80 0x7f0fa81545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:47.495 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fada00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fa815c320 0x7f0fa80b13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48354/0 (socket says 172.21.15.100:48354) 2024-04-23T14:31:47.495 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fada00640 1 -- 172.21.15.100:0/3623250897 learned_addr learned my addr 172.21.15.100:0/3623250897 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:31:47.495 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fa7e00640 1 --2- 172.21.15.100:0/3623250897 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fa80b3bb0 0x7f0fa813e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:47.496 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fada00640 1 -- 172.21.15.100:0/3623250897 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fa80b3bb0 msgr2=0x7f0fa813e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:47.496 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fada00640 1 --2- 172.21.15.100:0/3623250897 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fa80b3bb0 0x7f0fa813e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:47.496 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fada00640 1 -- 172.21.15.100:0/3623250897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fa806da80 msgr2=0x7f0fa81545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:47.496 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fada00640 1 --2- 172.21.15.100:0/3623250897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fa806da80 0x7f0fa81545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:47.496 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fada00640 1 -- 172.21.15.100:0/3623250897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa8160440 con 0x7f0fa815c320 2024-04-23T14:31:47.496 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fada00640 1 --2- 172.21.15.100:0/3623250897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fa815c320 0x7f0fa80b13e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000efc0 tx=0x7f0fa000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:47.497 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fa4c00640 1 -- 172.21.15.100:0/3623250897 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0018030 con 0x7f0fa815c320 2024-04-23T14:31:47.497 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fa4c00640 1 -- 172.21.15.100:0/3623250897 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa000cdf0 con 0x7f0fa815c320 2024-04-23T14:31:47.497 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fafa00640 1 -- 172.21.15.100:0/3623250897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa8160720 con 0x7f0fa815c320 2024-04-23T14:31:47.497 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fa4c00640 1 -- 172.21.15.100:0/3623250897 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0003bc0 con 0x7f0fa815c320 2024-04-23T14:31:47.497 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fafa00640 1 -- 172.21.15.100:0/3623250897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa81542c0 con 0x7f0fa815c320 2024-04-23T14:31:47.497 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fa4c00640 1 -- 172.21.15.100:0/3623250897 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0fa001b070 con 0x7f0fa815c320 2024-04-23T14:31:47.497 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fa4c00640 1 -- 172.21.15.100:0/3623250897 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f0fa0014b50 con 0x7f0fa815c320 2024-04-23T14:31:47.498 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.494+0000 7f0fafa00640 1 -- 172.21.15.100:0/3623250897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa8069400 con 0x7f0fa815c320 2024-04-23T14:31:47.498 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.498+0000 7f0fa4c00640 1 -- 172.21.15.100:0/3623250897 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0fa0010040 con 0x7f0fa815c320 2024-04-23T14:31:47.556 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.554+0000 7f0fafa00640 1 -- 172.21.15.100:0/3623250897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0fa81573b0 con 0x7f0fa815c320 2024-04-23T14:31:47.556 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.558+0000 7f0fa4c00640 1 -- 172.21.15.100:0/3623250897 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f0fa0014480 con 0x7f0fa815c320 2024-04-23T14:31:47.557 INFO:teuthology.misc.health.smithi100.stdout: 2024-04-23T14:31:47.557 INFO:teuthology.misc.health.smithi100.stdout:{"epoch":10,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:31:46.195181+0000","last_up_change":"0.000000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:31:47.558 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.558+0000 7f0fafa00640 1 -- 172.21.15.100:0/3623250897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fa815c320 msgr2=0x7f0fa80b13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:47.558 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.558+0000 7f0fafa00640 1 --2- 172.21.15.100:0/3623250897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fa815c320 0x7f0fa80b13e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000efc0 tx=0x7f0fa000c490 comp rx=0 tx=0).stop 2024-04-23T14:31:47.558 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.558+0000 7f0fafa00640 1 -- 172.21.15.100:0/3623250897 shutdown_connections 2024-04-23T14:31:47.558 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.558+0000 7f0fafa00640 1 --2- 172.21.15.100:0/3623250897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fa815c320 0x7f0fa80b13e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:47.558 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.558+0000 7f0fafa00640 1 --2- 172.21.15.100:0/3623250897 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fa80b3bb0 0x7f0fa813e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:47.558 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.558+0000 7f0fafa00640 1 --2- 172.21.15.100:0/3623250897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fa806da80 0x7f0fa81545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:47.558 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.558+0000 7f0fafa00640 1 -- 172.21.15.100:0/3623250897 >> 172.21.15.100:0/3623250897 conn(0x7f0fa80725b0 msgr2=0x7f0fa80aeef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:47.558 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.558+0000 7f0fafa00640 1 -- 172.21.15.100:0/3623250897 shutdown_connections 2024-04-23T14:31:47.558 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:47.558+0000 7f0fafa00640 1 -- 172.21.15.100:0/3623250897 wait complete. 2024-04-23T14:31:47.574 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-23T14:31:47.944 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:47.944+0000 7f78bb08d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T14:31:47.946 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:47.946+0000 7f94c028b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T14:31:48.031 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:48.028+0000 7f78bb08d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T14:31:48.032 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:48.030+0000 7f94c028b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T14:31:48.234 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:48.232+0000 7f78bb08d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T14:31:48.235 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:48.234+0000 7f94c028b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T14:31:48.319 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:48.320+0000 7f78bb08d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T14:31:48.320 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:48.318+0000 7f94c028b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T14:31:48.648 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:48.646+0000 7f94c028b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T14:31:48.648 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:48.648+0000 7f78bb08d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T14:31:48.718 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:48.718+0000 7f94c028b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T14:31:48.719 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-04-23T14:31:48.720+0000 7f78bb08d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T14:31:49.738 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:49.737+0000 7f9466400640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-23T14:31:49.738 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-04-23T14:31:49.737+0000 7f9466400640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-23T14:31:50.776 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-23T14:31:50.776+0000 7f04514e67c0 -1 osd.5 0 log_to_monitors true 2024-04-23T14:31:50.795 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-04-23T14:31:50.793+0000 7f66d8e4a7c0 -1 osd.1 0 log_to_monitors true 2024-04-23T14:31:50.802 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-04-23T14:31:50.801+0000 7f53e792b7c0 -1 osd.0 0 log_to_monitors true 2024-04-23T14:31:50.880 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-23T14:31:50.880+0000 7f65cfe227c0 -1 osd.7 0 log_to_monitors true 2024-04-23T14:31:50.938 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-23T14:31:50.936+0000 7fd85893a7c0 -1 osd.4 0 log_to_monitors true 2024-04-23T14:31:50.968 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-23T14:31:50.968+0000 7fb85acf07c0 -1 osd.6 0 log_to_monitors true 2024-04-23T14:31:51.042 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-04-23T14:31:51.041+0000 7f092fb617c0 -1 osd.2 0 log_to_monitors true 2024-04-23T14:31:51.042 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-04-23T14:31:51.041+0000 7f018c8957c0 -1 osd.3 0 log_to_monitors true 2024-04-23T14:31:54.380 DEBUG:teuthology.orchestra.run.smithi100:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-23T14:31:54.587 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.585+0000 7fa418e00640 1 Processor -- start 2024-04-23T14:31:54.590 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.585+0000 7fa418e00640 1 -- start start 2024-04-23T14:31:54.590 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.589+0000 7fa418e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa41415c7c0 0x7fa41406f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:54.590 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.589+0000 7fa418e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4140d10c0 0x7fa4140d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:54.590 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.589+0000 7fa418e00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4140ce9a0 con 0x7fa4140cdd50 2024-04-23T14:31:54.590 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.589+0000 7fa418e00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fa4140d1a60 con 0x7fa41415c7c0 2024-04-23T14:31:54.590 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.589+0000 7fa418e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa41415cba0 con 0x7fa4140d10c0 2024-04-23T14:31:54.591 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.589+0000 7fa411e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa41415c7c0 0x7fa41406f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:54.591 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.589+0000 7fa411400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4140d10c0 0x7fa4140d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:54.591 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.589+0000 7fa411e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa41415c7c0 0x7fa41406f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47504/0 (socket says 172.21.15.100:47504) 2024-04-23T14:31:54.591 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.589+0000 7fa411400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4140d10c0 0x7fa4140d1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:51702/0 (socket says 172.21.15.100:51702) 2024-04-23T14:31:54.591 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.589+0000 7fa411e00640 1 -- 172.21.15.100:0/2940105449 learned_addr learned my addr 172.21.15.100:0/2940105449 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:31:54.591 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.589+0000 7fa411e00640 1 -- 172.21.15.100:0/2940105449 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4140d10c0 msgr2=0x7fa4140d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:54.591 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.589+0000 7fa411e00640 1 --2- 172.21.15.100:0/2940105449 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4140d10c0 0x7fa4140d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:54.592 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.589+0000 7fa411e00640 1 -- 172.21.15.100:0/2940105449 >> v1:172.21.15.151:6789/0 conn(0x7fa4140cdd50 legacy=0x7fa4140ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:54.595 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.589+0000 7fa411e00640 1 -- 172.21.15.100:0/2940105449 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4140cec80 con 0x7fa41415c7c0 2024-04-23T14:31:54.596 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.593+0000 7fa411e00640 1 --2- 172.21.15.100:0/2940105449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa41415c7c0 0x7fa41406f680 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa40400dc90 tx=0x7fa404023020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585620bf1ba8956b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:54.596 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.593+0000 7fa410a00640 1 -- 172.21.15.100:0/2940105449 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40400d2d0 con 0x7fa41415c7c0 2024-04-23T14:31:54.596 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.593+0000 7fa410a00640 1 -- 172.21.15.100:0/2940105449 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4040acbf0 con 0x7fa41415c7c0 2024-04-23T14:31:54.597 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.593+0000 7fa410a00640 1 -- 172.21.15.100:0/2940105449 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40400b450 con 0x7fa41415c7c0 2024-04-23T14:31:54.598 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.597+0000 7fa418e00640 1 -- 172.21.15.100:0/2940105449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa41415c7c0 msgr2=0x7fa41406f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:54.598 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.597+0000 7fa418e00640 1 --2- 172.21.15.100:0/2940105449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa41415c7c0 0x7fa41406f680 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa40400dc90 tx=0x7fa404023020 comp rx=0 tx=0).stop 2024-04-23T14:31:54.598 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.597+0000 7fa418e00640 1 -- 172.21.15.100:0/2940105449 shutdown_connections 2024-04-23T14:31:54.598 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.597+0000 7fa418e00640 1 --2- 172.21.15.100:0/2940105449 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4140d10c0 0x7fa4140d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:54.599 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.597+0000 7fa418e00640 1 --2- 172.21.15.100:0/2940105449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa41415c7c0 0x7fa41406f680 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:54.599 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.597+0000 7fa418e00640 1 -- 172.21.15.100:0/2940105449 >> 172.21.15.100:0/2940105449 conn(0x7fa4140b3fd0 msgr2=0x7fa4140b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:54.599 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.597+0000 7fa418e00640 1 -- 172.21.15.100:0/2940105449 shutdown_connections 2024-04-23T14:31:54.599 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.597+0000 7fa418e00640 1 -- 172.21.15.100:0/2940105449 wait complete. 2024-04-23T14:31:54.600 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.597+0000 7fa418e00640 1 Processor -- start 2024-04-23T14:31:54.601 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.601+0000 7fa418e00640 1 -- start start 2024-04-23T14:31:54.602 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.601+0000 7fa418e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4140cdd50 0x7fa41415aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:54.602 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.601+0000 7fa418e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4140d10c0 0x7fa4141603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:54.602 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.601+0000 7fa418e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa41415c7c0 0x7fa414160900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:54.603 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.601+0000 7fa418e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa4140b5db0 con 0x7fa4140d10c0 2024-04-23T14:31:54.603 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.601+0000 7fa418e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa4140b5ad0 con 0x7fa41415c7c0 2024-04-23T14:31:54.603 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.601+0000 7fa418e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa41406fbc0 con 0x7fa4140cdd50 2024-04-23T14:31:54.603 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.601+0000 7fa412800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa41415c7c0 0x7fa414160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:54.604 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.601+0000 7fa412800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa41415c7c0 0x7fa414160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:59710/0 (socket says 172.21.15.100:59710) 2024-04-23T14:31:54.604 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.601+0000 7fa412800640 1 -- 172.21.15.100:0/2172725106 learned_addr learned my addr 172.21.15.100:0/2172725106 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:31:54.604 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.601+0000 7fa411400640 1 --2- 172.21.15.100:0/2172725106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4140d10c0 0x7fa4141603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:54.604 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.601+0000 7fa411e00640 1 --2- 172.21.15.100:0/2172725106 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4140cdd50 0x7fa41415aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:54.604 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.601+0000 7fa412800640 1 -- 172.21.15.100:0/2172725106 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4140cdd50 msgr2=0x7fa41415aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:54.604 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.601+0000 7fa412800640 1 --2- 172.21.15.100:0/2172725106 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4140cdd50 0x7fa41415aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:54.604 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.601+0000 7fa412800640 1 -- 172.21.15.100:0/2172725106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4140d10c0 msgr2=0x7fa4141603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:54.604 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.601+0000 7fa412800640 1 --2- 172.21.15.100:0/2172725106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4140d10c0 0x7fa4141603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:54.604 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.601+0000 7fa412800640 1 -- 172.21.15.100:0/2172725106 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41415d8d0 con 0x7fa41415c7c0 2024-04-23T14:31:54.604 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.601+0000 7fa412800640 1 --2- 172.21.15.100:0/2172725106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa41415c7c0 0x7fa414160900 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00b990 tx=0x7fa40c00be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:54.605 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.601+0000 7fa40a000640 1 -- 172.21.15.100:0/2172725106 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c019070 con 0x7fa41415c7c0 2024-04-23T14:31:54.605 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.601+0000 7fa418e00640 1 -- 172.21.15.100:0/2172725106 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41415dbb0 con 0x7fa41415c7c0 2024-04-23T14:31:54.605 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.605+0000 7fa418e00640 1 -- 172.21.15.100:0/2172725106 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa41415e170 con 0x7fa41415c7c0 2024-04-23T14:31:54.605 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.605+0000 7fa40a000640 1 -- 172.21.15.100:0/2172725106 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40c00cba0 con 0x7fa41415c7c0 2024-04-23T14:31:54.605 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.605+0000 7fa40a000640 1 -- 172.21.15.100:0/2172725106 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c014440 con 0x7fa41415c7c0 2024-04-23T14:31:54.606 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.605+0000 7fa418e00640 1 -- 172.21.15.100:0/2172725106 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3ec0052a0 con 0x7fa41415c7c0 2024-04-23T14:31:54.608 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.605+0000 7fa40a000640 1 -- 172.21.15.100:0/2172725106 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 4) ==== 87931+0+0 (secure 0 0 0) 0x7fa40c01c070 con 0x7fa41415c7c0 2024-04-23T14:31:54.608 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.605+0000 7fa40a000640 1 --2- 172.21.15.100:0/2172725106 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa3f40682f0 0x7fa3f406a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:54.608 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.605+0000 7fa40a000640 1 -- 172.21.15.100:0/2172725106 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7fa40c085810 con 0x7fa41415c7c0 2024-04-23T14:31:54.608 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.605+0000 7fa411e00640 1 --2- 172.21.15.100:0/2172725106 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa3f40682f0 0x7fa3f406a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:54.609 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.609+0000 7fa411e00640 1 --2- 172.21.15.100:0/2172725106 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa3f40682f0 0x7fa3f406a7b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa4040b5000 tx=0x7fa404008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:54.619 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.617+0000 7fa40a000640 1 -- 172.21.15.100:0/2172725106 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa40c056660 con 0x7fa41415c7c0 2024-04-23T14:31:54.917 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.917+0000 7fa418e00640 1 -- 172.21.15.100:0/2172725106 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3ec005860 con 0x7fa41415c7c0 2024-04-23T14:31:54.921 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.921+0000 7fa40a000640 1 -- 172.21.15.100:0/2172725106 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7fa40c05a960 con 0x7fa41415c7c0 2024-04-23T14:31:54.923 INFO:teuthology.misc.health.smithi100.stdout: 2024-04-23T14:31:54.923 INFO:teuthology.misc.health.smithi100.stdout:{"epoch":12,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:31:52.748057+0000","last_up_change":"0.000000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:31:54.927 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.925+0000 7fa418e00640 1 -- 172.21.15.100:0/2172725106 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa3f40682f0 msgr2=0x7fa3f406a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:54.927 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.925+0000 7fa418e00640 1 --2- 172.21.15.100:0/2172725106 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa3f40682f0 0x7fa3f406a7b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa4040b5000 tx=0x7fa404008040 comp rx=0 tx=0).stop 2024-04-23T14:31:54.927 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.925+0000 7fa418e00640 1 -- 172.21.15.100:0/2172725106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa41415c7c0 msgr2=0x7fa414160900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:54.927 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.925+0000 7fa418e00640 1 --2- 172.21.15.100:0/2172725106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa41415c7c0 0x7fa414160900 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00b990 tx=0x7fa40c00be50 comp rx=0 tx=0).stop 2024-04-23T14:31:54.927 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.925+0000 7fa418e00640 1 -- 172.21.15.100:0/2172725106 shutdown_connections 2024-04-23T14:31:54.927 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.925+0000 7fa418e00640 1 --2- 172.21.15.100:0/2172725106 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa3f40682f0 0x7fa3f406a7b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:54.927 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.925+0000 7fa418e00640 1 --2- 172.21.15.100:0/2172725106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa41415c7c0 0x7fa414160900 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:54.927 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.925+0000 7fa418e00640 1 --2- 172.21.15.100:0/2172725106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4140d10c0 0x7fa4141603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:54.927 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.925+0000 7fa418e00640 1 --2- 172.21.15.100:0/2172725106 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4140cdd50 0x7fa41415aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:54.927 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.925+0000 7fa418e00640 1 -- 172.21.15.100:0/2172725106 >> 172.21.15.100:0/2172725106 conn(0x7fa4140b3fd0 msgr2=0x7fa4140b3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:54.928 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.925+0000 7fa418e00640 1 -- 172.21.15.100:0/2172725106 shutdown_connections 2024-04-23T14:31:54.929 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:31:54.925+0000 7fa418e00640 1 -- 172.21.15.100:0/2172725106 wait complete. 2024-04-23T14:31:54.963 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-23T14:31:55.096 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-23T14:31:55.092+0000 7f044c400640 -1 osd.5 0 waiting for initial osdmap 2024-04-23T14:31:55.128 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-23T14:31:55.128+0000 7f65cac00640 -1 osd.7 0 waiting for initial osdmap 2024-04-23T14:31:55.133 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-04-23T14:31:55.132+0000 7f0442400640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:31:55.155 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-23T14:31:55.156+0000 7fb855c00640 -1 osd.6 0 waiting for initial osdmap 2024-04-23T14:31:55.160 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-04-23T14:31:55.160+0000 7f65c0c00640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:31:55.164 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-23T14:31:55.164+0000 7fd853800640 -1 osd.4 0 waiting for initial osdmap 2024-04-23T14:31:55.192 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-04-23T14:31:55.192+0000 7fb84bc00640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:31:55.192 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-04-23T14:31:55.192+0000 7fd849800640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:31:55.236 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-04-23T14:31:55.233+0000 7f0188200640 -1 osd.3 0 waiting for initial osdmap 2024-04-23T14:31:55.253 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-04-23T14:31:55.253+0000 7f092aa00640 -1 osd.2 0 waiting for initial osdmap 2024-04-23T14:31:55.261 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-04-23T14:31:55.261+0000 7f66d3e00640 -1 osd.1 0 waiting for initial osdmap 2024-04-23T14:31:55.270 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-04-23T14:31:55.269+0000 7f017d800640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:31:55.286 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-04-23T14:31:55.285+0000 7f0920a00640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:31:55.302 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-04-23T14:31:55.301+0000 7f66c9e00640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:31:55.327 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-04-23T14:31:55.325+0000 7f53e2800640 -1 osd.0 0 waiting for initial osdmap 2024-04-23T14:31:55.366 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-04-23T14:31:55.365+0000 7f53d8800640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:32:01.769 DEBUG:teuthology.orchestra.run.smithi100:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:01.899 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.897+0000 7f173ce00640 1 Processor -- start 2024-04-23T14:32:01.900 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.897+0000 7f173ce00640 1 -- start start 2024-04-23T14:32:01.901 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.901+0000 7f173ce00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f173815c320 0x7f173815ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:01.901 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.901+0000 7f173ce00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f173806da80 0x7f173806ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:01.901 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.901+0000 7f173ce00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f173815f4a0 con 0x7f17380b3bb0 2024-04-23T14:32:01.901 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.901+0000 7f173ce00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f173806c1c0 con 0x7f173815c320 2024-04-23T14:32:01.901 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.901+0000 7f173ce00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f173815fd40 con 0x7f173806da80 2024-04-23T14:32:01.901 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.901+0000 7f1735e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f173815c320 0x7f173815ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:01.901 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.901+0000 7f1735e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f173815c320 0x7f173815ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46000/0 (socket says 172.21.15.100:46000) 2024-04-23T14:32:01.901 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.901+0000 7f1735e00640 1 -- 172.21.15.100:0/2807471266 learned_addr learned my addr 172.21.15.100:0/2807471266 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:01.902 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.901+0000 7f1736800640 1 --2- 172.21.15.100:0/2807471266 >> v2:172.21.15.151:3300/0 conn(0x7f173806da80 0x7f173806ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:01.902 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.901+0000 7f1735e00640 1 -- 172.21.15.100:0/2807471266 >> v2:172.21.15.151:3300/0 conn(0x7f173806da80 msgr2=0x7f173806ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:01.902 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.901+0000 7f1735e00640 1 --2- 172.21.15.100:0/2807471266 >> v2:172.21.15.151:3300/0 conn(0x7f173806da80 0x7f173806ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:01.902 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.901+0000 7f1735e00640 1 -- 172.21.15.100:0/2807471266 >> v1:172.21.15.151:6789/0 conn(0x7f17380b3bb0 legacy=0x7f173806d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:01.902 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.901+0000 7f1735e00640 1 -- 172.21.15.100:0/2807471266 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173815f780 con 0x7f173815c320 2024-04-23T14:32:01.904 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.901+0000 7f1735e00640 1 --2- 172.21.15.100:0/2807471266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f173815c320 0x7f173815ed50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f17380b4040 tx=0x7f172801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ede27379a4a6619 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:01.905 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f1734a00640 1 -- 172.21.15.100:0/2807471266 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17280b0070 con 0x7f173815c320 2024-04-23T14:32:01.905 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f1734a00640 1 -- 172.21.15.100:0/2807471266 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17280ab600 con 0x7f173815c320 2024-04-23T14:32:01.905 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f1734a00640 1 -- 172.21.15.100:0/2807471266 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17280ab9d0 con 0x7f173815c320 2024-04-23T14:32:01.906 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f173ce00640 1 -- 172.21.15.100:0/2807471266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f173815c320 msgr2=0x7f173815ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:01.906 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f173ce00640 1 --2- 172.21.15.100:0/2807471266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f173815c320 0x7f173815ed50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f17380b4040 tx=0x7f172801c490 comp rx=0 tx=0).stop 2024-04-23T14:32:01.906 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f173ce00640 1 -- 172.21.15.100:0/2807471266 shutdown_connections 2024-04-23T14:32:01.906 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f173ce00640 1 --2- 172.21.15.100:0/2807471266 >> v2:172.21.15.151:3300/0 conn(0x7f173806da80 0x7f173806ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:01.906 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f173ce00640 1 --2- 172.21.15.100:0/2807471266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f173815c320 0x7f173815ed50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:01.906 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f173ce00640 1 -- 172.21.15.100:0/2807471266 >> 172.21.15.100:0/2807471266 conn(0x7f17380725b0 msgr2=0x7f17380729c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:01.906 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f173ce00640 1 -- 172.21.15.100:0/2807471266 shutdown_connections 2024-04-23T14:32:01.906 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f173ce00640 1 -- 172.21.15.100:0/2807471266 wait complete. 2024-04-23T14:32:01.907 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f173ce00640 1 Processor -- start 2024-04-23T14:32:01.907 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f173ce00640 1 -- start start 2024-04-23T14:32:01.909 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f173ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f173806da80 0x7f1738157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:01.909 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f173ce00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f17380b3bb0 0x7f1738152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:01.909 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f173ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f173815c320 0x7f1738152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:01.910 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f173ce00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f173806fd80 con 0x7f173815c320 2024-04-23T14:32:01.910 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f173ce00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f173806fc00 con 0x7f173806da80 2024-04-23T14:32:01.910 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f173ce00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f173806ff00 con 0x7f17380b3bb0 2024-04-23T14:32:01.910 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f1736800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f173815c320 0x7f1738152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:01.910 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f1736800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f173815c320 0x7f1738152bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46010/0 (socket says 172.21.15.100:46010) 2024-04-23T14:32:01.910 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f1736800640 1 -- 172.21.15.100:0/1375303665 learned_addr learned my addr 172.21.15.100:0/1375303665 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:01.910 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f1735e00640 1 --2- 172.21.15.100:0/1375303665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f173806da80 0x7f1738157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:01.910 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.905+0000 7f1735400640 1 --2- 172.21.15.100:0/1375303665 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f17380b3bb0 0x7f1738152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:01.910 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.909+0000 7f1736800640 1 -- 172.21.15.100:0/1375303665 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f17380b3bb0 msgr2=0x7f1738152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:01.910 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.909+0000 7f1736800640 1 --2- 172.21.15.100:0/1375303665 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f17380b3bb0 0x7f1738152690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:01.910 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.909+0000 7f1736800640 1 -- 172.21.15.100:0/1375303665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f173806da80 msgr2=0x7f1738157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:01.910 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.909+0000 7f1736800640 1 --2- 172.21.15.100:0/1375303665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f173806da80 0x7f1738157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:01.910 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.909+0000 7f1736800640 1 -- 172.21.15.100:0/1375303665 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1738153110 con 0x7f173815c320 2024-04-23T14:32:01.910 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.909+0000 7f1736800640 1 --2- 172.21.15.100:0/1375303665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f173815c320 0x7f1738152bd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f173000efc0 tx=0x7f173000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:01.911 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.909+0000 7f172e000640 1 -- 172.21.15.100:0/1375303665 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1730018020 con 0x7f173815c320 2024-04-23T14:32:01.911 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.909+0000 7f172e000640 1 -- 172.21.15.100:0/1375303665 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1730014480 con 0x7f173815c320 2024-04-23T14:32:01.911 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.909+0000 7f173ce00640 1 -- 172.21.15.100:0/1375303665 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f173815a530 con 0x7f173815c320 2024-04-23T14:32:01.911 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.909+0000 7f172e000640 1 -- 172.21.15.100:0/1375303665 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1730014870 con 0x7f173815c320 2024-04-23T14:32:01.911 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.909+0000 7f173ce00640 1 -- 172.21.15.100:0/1375303665 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17381fb250 con 0x7f173815c320 2024-04-23T14:32:01.912 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.909+0000 7f173ce00640 1 -- 172.21.15.100:0/1375303665 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17381fb740 con 0x7f173815c320 2024-04-23T14:32:01.912 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.909+0000 7f172e000640 1 -- 172.21.15.100:0/1375303665 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f173001b020 con 0x7f173815c320 2024-04-23T14:32:01.912 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.909+0000 7f172e000640 1 --2- 172.21.15.100:0/1375303665 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1718068380 0x7f171806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:01.912 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.909+0000 7f172e000640 1 -- 172.21.15.100:0/1375303665 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f1730086fc0 con 0x7f173815c320 2024-04-23T14:32:01.912 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.909+0000 7f1735e00640 1 --2- 172.21.15.100:0/1375303665 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1718068380 0x7f171806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:01.912 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.913+0000 7f1735e00640 1 --2- 172.21.15.100:0/1375303665 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1718068380 0x7f171806a840 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f17280ad000 tx=0x7f17280a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:01.917 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:01.917+0000 7f172e000640 1 -- 172.21.15.100:0/1375303665 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1730022070 con 0x7f173815c320 2024-04-23T14:32:02.075 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:02.073+0000 7f173ce00640 1 -- 172.21.15.100:0/1375303665 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f17380b5470 con 0x7f173815c320 2024-04-23T14:32:02.076 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:02.073+0000 7f172e000640 1 -- 172.21.15.100:0/1375303665 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12896 (secure 0 0 0) 0x7f1738153110 con 0x7f173815c320 2024-04-23T14:32:02.076 INFO:teuthology.misc.health.smithi100.stdout: 2024-04-23T14:32:02.076 INFO:teuthology.misc.health.smithi100.stdout:{"epoch":16,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:31:59.150908+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:02.078 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:02.077+0000 7f173ce00640 1 -- 172.21.15.100:0/1375303665 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1718068380 msgr2=0x7f171806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.078 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:02.077+0000 7f173ce00640 1 --2- 172.21.15.100:0/1375303665 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1718068380 0x7f171806a840 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f17280ad000 tx=0x7f17280a6000 comp rx=0 tx=0).stop 2024-04-23T14:32:02.078 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:02.077+0000 7f173ce00640 1 -- 172.21.15.100:0/1375303665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f173815c320 msgr2=0x7f1738152bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.079 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:02.077+0000 7f173ce00640 1 --2- 172.21.15.100:0/1375303665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f173815c320 0x7f1738152bd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f173000efc0 tx=0x7f173000c490 comp rx=0 tx=0).stop 2024-04-23T14:32:02.079 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:02.077+0000 7f173ce00640 1 -- 172.21.15.100:0/1375303665 shutdown_connections 2024-04-23T14:32:02.079 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:02.077+0000 7f173ce00640 1 --2- 172.21.15.100:0/1375303665 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1718068380 0x7f171806a840 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.079 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:02.077+0000 7f173ce00640 1 --2- 172.21.15.100:0/1375303665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f173815c320 0x7f1738152bd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.079 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:02.077+0000 7f173ce00640 1 --2- 172.21.15.100:0/1375303665 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f17380b3bb0 0x7f1738152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.079 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:02.077+0000 7f173ce00640 1 --2- 172.21.15.100:0/1375303665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f173806da80 0x7f1738157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.079 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:02.077+0000 7f173ce00640 1 -- 172.21.15.100:0/1375303665 >> 172.21.15.100:0/1375303665 conn(0x7f17380725b0 msgr2=0x7f17380b0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:02.079 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:02.077+0000 7f173ce00640 1 -- 172.21.15.100:0/1375303665 shutdown_connections 2024-04-23T14:32:02.079 INFO:teuthology.misc.health.smithi100.stderr:2024-04-23T14:32:02.077+0000 7f173ce00640 1 -- 172.21.15.100:0/1375303665 wait complete. 2024-04-23T14:32:02.095 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-23T14:32:02.095 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-23T14:32:02.095 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-23T14:32:02.095 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-23T14:32:02.098 INFO:tasks.ceph.mds.a:Started 2024-04-23T14:32:02.098 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-23T14:32:02.099 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-23T14:32:02.101 INFO:tasks.ceph.mds.c:Started 2024-04-23T14:32:02.101 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-23T14:32:02.102 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-23T14:32:02.105 INFO:tasks.ceph.mds.b:Started 2024-04-23T14:32:02.105 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-23T14:32:02.105 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-23T14:32:02.108 INFO:tasks.ceph.mds.d:Started 2024-04-23T14:32:02.109 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-23T14:32:02.109 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:02.196 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-04-23T14:32:02.196 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-04-23T14:32:02.211 INFO:tasks.ceph.mds.b.smithi151.stdout:starting mds.b at 2024-04-23T14:32:02.212 INFO:tasks.ceph.mds.d.smithi151.stdout:starting mds.d at 2024-04-23T14:32:02.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.253+0000 7f6674e00640 1 Processor -- start 2024-04-23T14:32:02.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.253+0000 7f6674e00640 1 -- start start 2024-04-23T14:32:02.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.253+0000 7f6674e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f667015c7e0 0x7f66700d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.253+0000 7f6674e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f667015d600 0x7f66700d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.253+0000 7f6674e00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66700d04a0 con 0x7f66700cca50 2024-04-23T14:32:02.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.253+0000 7f6674e00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f667015db00 con 0x7f667015c7e0 2024-04-23T14:32:02.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.253+0000 7f6674e00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f66700d0780 con 0x7f667015d600 2024-04-23T14:32:02.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.253+0000 7f666d400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f667015c7e0 0x7f66700d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.253+0000 7f666d400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f667015c7e0 0x7f66700d1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46030/0 (socket says 172.21.15.100:46030) 2024-04-23T14:32:02.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.253+0000 7f666d400640 1 -- 172.21.15.100:0/2939311284 learned_addr learned my addr 172.21.15.100:0/2939311284 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:02.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.253+0000 7f666de00640 1 --1- 172.21.15.100:0/2939311284 >> v1:172.21.15.100:6789/0 conn(0x7f66700cca50 0x7f66700cce30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:39374/0 (socket says 172.21.15.100:39374) 2024-04-23T14:32:02.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.253+0000 7f666e800640 1 --2- 172.21.15.100:0/2939311284 >> v2:172.21.15.151:3300/0 conn(0x7f667015d600 0x7f66700d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.253+0000 7f666d400640 1 -- 172.21.15.100:0/2939311284 >> v2:172.21.15.151:3300/0 conn(0x7f667015d600 msgr2=0x7f66700d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:02.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.253+0000 7f666d400640 1 --2- 172.21.15.100:0/2939311284 >> v2:172.21.15.151:3300/0 conn(0x7f667015d600 0x7f66700d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.253+0000 7f666d400640 1 -- 172.21.15.100:0/2939311284 >> v1:172.21.15.100:6789/0 conn(0x7f66700cca50 legacy=0x7f66700cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.253+0000 7f666d400640 1 -- 172.21.15.100:0/2939311284 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66700d0a60 con 0x7f667015c7e0 2024-04-23T14:32:02.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.257+0000 7f666d400640 1 --2- 172.21.15.100:0/2939311284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f667015c7e0 0x7f66700d1260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f66700b8550 tx=0x7f665c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b6ad40c178bce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:02.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.257+0000 7f666ca00640 1 -- 172.21.15.100:0/2939311284 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0ab680 con 0x7f667015c7e0 2024-04-23T14:32:02.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.257+0000 7f666ca00640 1 -- 172.21.15.100:0/2939311284 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f665c0b0070 con 0x7f667015c7e0 2024-04-23T14:32:02.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.257+0000 7f666ca00640 1 -- 172.21.15.100:0/2939311284 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0aba40 con 0x7f667015c7e0 2024-04-23T14:32:02.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.257+0000 7f6674e00640 1 -- 172.21.15.100:0/2939311284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f667015c7e0 msgr2=0x7f66700d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.257+0000 7f6674e00640 1 --2- 172.21.15.100:0/2939311284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f667015c7e0 0x7f66700d1260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f66700b8550 tx=0x7f665c01c460 comp rx=0 tx=0).stop 2024-04-23T14:32:02.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.257+0000 7f6674e00640 1 -- 172.21.15.100:0/2939311284 shutdown_connections 2024-04-23T14:32:02.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.257+0000 7f6674e00640 1 --2- 172.21.15.100:0/2939311284 >> v2:172.21.15.151:3300/0 conn(0x7f667015d600 0x7f66700d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.257+0000 7f6674e00640 1 --2- 172.21.15.100:0/2939311284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f667015c7e0 0x7f66700d1260 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.257+0000 7f6674e00640 1 -- 172.21.15.100:0/2939311284 >> 172.21.15.100:0/2939311284 conn(0x7f66700cee80 msgr2=0x7f66700cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.257+0000 7f6674e00640 1 -- 172.21.15.100:0/2939311284 shutdown_connections 2024-04-23T14:32:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.257+0000 7f6674e00640 1 -- 172.21.15.100:0/2939311284 wait complete. 2024-04-23T14:32:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.257+0000 7f6674e00640 1 Processor -- start 2024-04-23T14:32:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f6674e00640 1 -- start start 2024-04-23T14:32:02.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f6674e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f66700cca50 0x7f66701476b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f6674e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f667015c7e0 0x7f6670165690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f6674e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f667015d600 0x7f6670165bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f6674e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f66700b3d70 con 0x7f667015d600 2024-04-23T14:32:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f6674e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f66700b3bf0 con 0x7f667015c7e0 2024-04-23T14:32:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f6674e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f6670169ad0 con 0x7f66700cca50 2024-04-23T14:32:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f666e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f667015d600 0x7f6670165bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f666e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f667015d600 0x7f6670165bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46040/0 (socket says 172.21.15.100:46040) 2024-04-23T14:32:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f666e800640 1 -- 172.21.15.100:0/1002141017 learned_addr learned my addr 172.21.15.100:0/1002141017 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f666de00640 1 --2- 172.21.15.100:0/1002141017 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f66700cca50 0x7f66701476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f666d400640 1 --2- 172.21.15.100:0/1002141017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f667015c7e0 0x7f6670165690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f666e800640 1 -- 172.21.15.100:0/1002141017 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f66700cca50 msgr2=0x7f66701476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f666e800640 1 --2- 172.21.15.100:0/1002141017 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f66700cca50 0x7f66701476b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f666e800640 1 -- 172.21.15.100:0/1002141017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f667015c7e0 msgr2=0x7f6670165690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f666e800640 1 --2- 172.21.15.100:0/1002141017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f667015c7e0 0x7f6670165690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f666e800640 1 -- 172.21.15.100:0/1002141017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6670159e80 con 0x7f667015d600 2024-04-23T14:32:02.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f666e800640 1 --2- 172.21.15.100:0/1002141017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f667015d600 0x7f6670165bd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f666800e7f0 tx=0x7f666800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:02.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f6666000640 1 -- 172.21.15.100:0/1002141017 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668017070 con 0x7f667015d600 2024-04-23T14:32:02.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f6666000640 1 -- 172.21.15.100:0/1002141017 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f666800c940 con 0x7f667015d600 2024-04-23T14:32:02.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f6666000640 1 -- 172.21.15.100:0/1002141017 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666800eea0 con 0x7f667015d600 2024-04-23T14:32:02.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f6674e00640 1 -- 172.21.15.100:0/1002141017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667015a160 con 0x7f667015d600 2024-04-23T14:32:02.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f6674e00640 1 -- 172.21.15.100:0/1002141017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f667006ea80 con 0x7f667015d600 2024-04-23T14:32:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.261+0000 7f6674e00640 1 -- 172.21.15.100:0/1002141017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6670068ad0 con 0x7f667015d600 2024-04-23T14:32:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.265+0000 7f6666000640 1 -- 172.21.15.100:0/1002141017 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f666801a070 con 0x7f667015d600 2024-04-23T14:32:02.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.265+0000 7f6666000640 1 --2- 172.21.15.100:0/1002141017 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6650068380 0x7f665006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.265+0000 7f6666000640 1 -- 172.21.15.100:0/1002141017 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f6668086800 con 0x7f667015d600 2024-04-23T14:32:02.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.265+0000 7f666de00640 1 --2- 172.21.15.100:0/1002141017 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6650068380 0x7f665006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.265+0000 7f666de00640 1 --2- 172.21.15.100:0/1002141017 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6650068380 0x7f665006a840 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f66600012f0 tx=0x7f66600016c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:02.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.269+0000 7f6666000640 1 -- 172.21.15.100:0/1002141017 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6668056ac0 con 0x7f667015d600 2024-04-23T14:32:02.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.425+0000 7f6674e00640 1 -- 172.21.15.100:0/1002141017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f667006fae0 con 0x7f667015d600 2024-04-23T14:32:02.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.429+0000 7f6666000640 1 -- 172.21.15.100:0/1002141017 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12896 (secure 0 0 0) 0x7f666805adc0 con 0x7f667015d600 2024-04-23T14:32:02.429 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:02.429 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":16,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:31:59.150908+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:02.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.429+0000 7f6674e00640 1 -- 172.21.15.100:0/1002141017 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6650068380 msgr2=0x7f665006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.429+0000 7f6674e00640 1 --2- 172.21.15.100:0/1002141017 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6650068380 0x7f665006a840 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f66600012f0 tx=0x7f66600016c0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.429+0000 7f6674e00640 1 -- 172.21.15.100:0/1002141017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f667015d600 msgr2=0x7f6670165bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.429+0000 7f6674e00640 1 --2- 172.21.15.100:0/1002141017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f667015d600 0x7f6670165bd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f666800e7f0 tx=0x7f666800ecb0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.429+0000 7f6674e00640 1 -- 172.21.15.100:0/1002141017 shutdown_connections 2024-04-23T14:32:02.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.429+0000 7f6674e00640 1 --2- 172.21.15.100:0/1002141017 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6650068380 0x7f665006a840 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.429+0000 7f6674e00640 1 --2- 172.21.15.100:0/1002141017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f667015d600 0x7f6670165bd0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.429+0000 7f6674e00640 1 --2- 172.21.15.100:0/1002141017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f667015c7e0 0x7f6670165690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.429+0000 7f6674e00640 1 --2- 172.21.15.100:0/1002141017 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f66700cca50 0x7f66701476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.429+0000 7f6674e00640 1 -- 172.21.15.100:0/1002141017 >> 172.21.15.100:0/1002141017 conn(0x7f66700cee80 msgr2=0x7f66700d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:02.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.429+0000 7f6674e00640 1 -- 172.21.15.100:0/1002141017 shutdown_connections 2024-04-23T14:32:02.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.429+0000 7f6674e00640 1 -- 172.21.15.100:0/1002141017 wait complete. 2024-04-23T14:32:02.449 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:32:02.449 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:02.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.585+0000 7f769e000640 1 Processor -- start 2024-04-23T14:32:02.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.585+0000 7f769e000640 1 -- start start 2024-04-23T14:32:02.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.589+0000 7f769e000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f769815ed80 0x7f76980d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.589+0000 7f769e000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f769806d450 0x7f769806d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.589+0000 7f769e000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76980cefc0 con 0x7f769815f890 2024-04-23T14:32:02.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.589+0000 7f769e000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f769806de00 con 0x7f769815ed80 2024-04-23T14:32:02.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.589+0000 7f769e000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f76980cf2a0 con 0x7f769806d450 2024-04-23T14:32:02.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.589+0000 7f768ea00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f769815ed80 0x7f76980d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.589+0000 7f768ea00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f769815ed80 0x7f76980d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:37772/0 (socket says 172.21.15.100:37772) 2024-04-23T14:32:02.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.589+0000 7f768ea00640 1 -- 172.21.15.100:0/554635860 learned_addr learned my addr 172.21.15.100:0/554635860 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:02.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.589+0000 7f7697e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f769806d450 0x7f769806d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.589+0000 7f7697400640 1 --1- 172.21.15.100:0/554635860 >> v1:172.21.15.151:6789/0 conn(0x7f769815f890 0x7f76980ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:53290/0 (socket says 172.21.15.100:53290) 2024-04-23T14:32:02.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.589+0000 7f768ea00640 1 -- 172.21.15.100:0/554635860 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f769806d450 msgr2=0x7f769806d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:02.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.589+0000 7f768ea00640 1 --2- 172.21.15.100:0/554635860 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f769806d450 0x7f769806d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.589+0000 7f768ea00640 1 -- 172.21.15.100:0/554635860 >> v1:172.21.15.151:6789/0 conn(0x7f769815f890 legacy=0x7f76980ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.589+0000 7f768ea00640 1 -- 172.21.15.100:0/554635860 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76980cf580 con 0x7f769815ed80 2024-04-23T14:32:02.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.589+0000 7f7697e00640 1 --2- 172.21.15.100:0/554635860 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f769806d450 0x7f769806d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:02.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.589+0000 7f7696a00640 1 -- 172.21.15.100:0/554635860 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 532299278 0 0) 0x7f76980cefc0 con 0x7f769815f890 2024-04-23T14:32:02.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.593+0000 7f768ea00640 1 --2- 172.21.15.100:0/554635860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f769815ed80 0x7f76980d0300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7680009870 tx=0x7f768001c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9647fafcc64069d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:02.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.593+0000 7f7696a00640 1 -- 172.21.15.100:0/554635860 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76800ab5f0 con 0x7f769815ed80 2024-04-23T14:32:02.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.593+0000 7f7696a00640 1 -- 172.21.15.100:0/554635860 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76800b0070 con 0x7f769815ed80 2024-04-23T14:32:02.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.593+0000 7f7696a00640 1 -- 172.21.15.100:0/554635860 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76800ab9b0 con 0x7f769815ed80 2024-04-23T14:32:02.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.593+0000 7f769e000640 1 -- 172.21.15.100:0/554635860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f769815ed80 msgr2=0x7f76980d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.593+0000 7f769e000640 1 --2- 172.21.15.100:0/554635860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f769815ed80 0x7f76980d0300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7680009870 tx=0x7f768001c560 comp rx=0 tx=0).stop 2024-04-23T14:32:02.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.593+0000 7f769e000640 1 -- 172.21.15.100:0/554635860 shutdown_connections 2024-04-23T14:32:02.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.593+0000 7f769e000640 1 --2- 172.21.15.100:0/554635860 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f769806d450 0x7f769806d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.593+0000 7f769e000640 1 --2- 172.21.15.100:0/554635860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f769815ed80 0x7f76980d0300 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.593+0000 7f769e000640 1 -- 172.21.15.100:0/554635860 >> 172.21.15.100:0/554635860 conn(0x7f76980719d0 msgr2=0x7f7698071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:02.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.593+0000 7f769e000640 1 -- 172.21.15.100:0/554635860 shutdown_connections 2024-04-23T14:32:02.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.593+0000 7f769e000640 1 -- 172.21.15.100:0/554635860 wait complete. 2024-04-23T14:32:02.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.593+0000 7f769e000640 1 Processor -- start 2024-04-23T14:32:02.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.593+0000 7f769e000640 1 -- start start 2024-04-23T14:32:02.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f769e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f769806d450 0x7f76980d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f769e000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f769815ed80 0x7f7698148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f769e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f769815f890 0x7f7698149460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f769e000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f76980cdaa0 con 0x7f769806d450 2024-04-23T14:32:02.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f769e000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f76980cd920 con 0x7f769815f890 2024-04-23T14:32:02.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f769e000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f76980cdc20 con 0x7f769815ed80 2024-04-23T14:32:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f7697400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f769806d450 0x7f76980d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f7697400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f769806d450 0x7f76980d0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46046/0 (socket says 172.21.15.100:46046) 2024-04-23T14:32:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f7697400640 1 -- 172.21.15.100:0/2496637458 learned_addr learned my addr 172.21.15.100:0/2496637458 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f768ea00640 1 --2- 172.21.15.100:0/2496637458 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f769815ed80 0x7f7698148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f7697e00640 1 --2- 172.21.15.100:0/2496637458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f769815f890 0x7f7698149460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f7697400640 1 -- 172.21.15.100:0/2496637458 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f769815ed80 msgr2=0x7f7698148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f7697400640 1 --2- 172.21.15.100:0/2496637458 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f769815ed80 0x7f7698148f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f7697400640 1 -- 172.21.15.100:0/2496637458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f769815f890 msgr2=0x7f7698149460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f7697400640 1 --2- 172.21.15.100:0/2496637458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f769815f890 0x7f7698149460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f7697400640 1 -- 172.21.15.100:0/2496637458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76980c7b60 con 0x7f769806d450 2024-04-23T14:32:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f768ea00640 1 --2- 172.21.15.100:0/2496637458 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f769815ed80 0x7f7698148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f7697400640 1 --2- 172.21.15.100:0/2496637458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f769806d450 0x7f76980d0db0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f768800c4e0 tx=0x7f768800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f768fe00640 1 -- 172.21.15.100:0/2496637458 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76880074e0 con 0x7f769806d450 2024-04-23T14:32:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f768fe00640 1 -- 172.21.15.100:0/2496637458 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7688007680 con 0x7f769806d450 2024-04-23T14:32:02.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f769e000640 1 -- 172.21.15.100:0/2496637458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7698159810 con 0x7f769806d450 2024-04-23T14:32:02.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f768fe00640 1 -- 172.21.15.100:0/2496637458 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7688004a30 con 0x7f769806d450 2024-04-23T14:32:02.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f769e000640 1 -- 172.21.15.100:0/2496637458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76981555d0 con 0x7f769806d450 2024-04-23T14:32:02.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f769e000640 1 -- 172.21.15.100:0/2496637458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7698068ad0 con 0x7f769806d450 2024-04-23T14:32:02.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.597+0000 7f768fe00640 1 -- 172.21.15.100:0/2496637458 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f768801a020 con 0x7f769806d450 2024-04-23T14:32:02.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.601+0000 7f768fe00640 1 --2- 172.21.15.100:0/2496637458 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7674068380 0x7f767406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.601+0000 7f768fe00640 1 -- 172.21.15.100:0/2496637458 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f7688085ea0 con 0x7f769806d450 2024-04-23T14:32:02.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.601+0000 7f768ea00640 1 --2- 172.21.15.100:0/2496637458 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7674068380 0x7f767406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.601+0000 7f768ea00640 1 --2- 172.21.15.100:0/2496637458 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7674068380 0x7f767406a840 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f76800af000 tx=0x7f76800a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:02.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.605+0000 7f768fe00640 1 -- 172.21.15.100:0/2496637458 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f76880567d0 con 0x7f769806d450 2024-04-23T14:32:02.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.761+0000 7f769e000640 1 -- 172.21.15.100:0/2496637458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f76980d1fa0 con 0x7f769806d450 2024-04-23T14:32:02.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.761+0000 7f768fe00640 1 -- 172.21.15.100:0/2496637458 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f76880560d0 con 0x7f769806d450 2024-04-23T14:32:02.764 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:32:02.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.765+0000 7f769e000640 1 -- 172.21.15.100:0/2496637458 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7674068380 msgr2=0x7f767406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.765+0000 7f769e000640 1 --2- 172.21.15.100:0/2496637458 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7674068380 0x7f767406a840 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f76800af000 tx=0x7f76800a7000 comp rx=0 tx=0).stop 2024-04-23T14:32:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.765+0000 7f769e000640 1 -- 172.21.15.100:0/2496637458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f769806d450 msgr2=0x7f76980d0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.765+0000 7f769e000640 1 --2- 172.21.15.100:0/2496637458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f769806d450 0x7f76980d0db0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f768800c4e0 tx=0x7f768800c8e0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.765+0000 7f769e000640 1 -- 172.21.15.100:0/2496637458 shutdown_connections 2024-04-23T14:32:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.765+0000 7f769e000640 1 --2- 172.21.15.100:0/2496637458 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7674068380 0x7f767406a840 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.765+0000 7f769e000640 1 --2- 172.21.15.100:0/2496637458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f769815f890 0x7f7698149460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.765+0000 7f769e000640 1 --2- 172.21.15.100:0/2496637458 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f769815ed80 0x7f7698148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.765+0000 7f769e000640 1 --2- 172.21.15.100:0/2496637458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f769806d450 0x7f76980d0db0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.765+0000 7f769e000640 1 -- 172.21.15.100:0/2496637458 >> 172.21.15.100:0/2496637458 conn(0x7f76980719d0 msgr2=0x7f7698160090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.765+0000 7f769e000640 1 -- 172.21.15.100:0/2496637458 shutdown_connections 2024-04-23T14:32:02.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.765+0000 7f769e000640 1 -- 172.21.15.100:0/2496637458 wait complete. 2024-04-23T14:32:02.785 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:32:02.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.917+0000 7f7574e00640 1 Processor -- start 2024-04-23T14:32:02.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.917+0000 7f7574e00640 1 -- start start 2024-04-23T14:32:02.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.917+0000 7f7574e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75700b2a10 0x7f75700adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.917+0000 7f7574e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f7570073610 0x7f757006df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.917+0000 7f7574e00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75700af080 con 0x7f757015b640 2024-04-23T14:32:02.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.917+0000 7f7574e00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f75700afbc0 con 0x7f7570073610 2024-04-23T14:32:02.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.917+0000 7f7574e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f757015ba20 con 0x7f75700b2a10 2024-04-23T14:32:02.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.917+0000 7f7565400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75700b2a10 0x7f75700adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.917+0000 7f756e800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f7570073610 0x7f757006df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.917+0000 7f7565400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75700b2a10 0x7f75700adfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:41168/0 (socket says 172.21.15.100:41168) 2024-04-23T14:32:02.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.917+0000 7f7565400640 1 -- 172.21.15.100:0/3677054200 learned_addr learned my addr 172.21.15.100:0/3677054200 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:02.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.921+0000 7f7565400640 1 -- 172.21.15.100:0/3677054200 >> v2:172.21.15.151:3300/0 conn(0x7f7570073610 msgr2=0x7f757006df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:02.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.921+0000 7f7565400640 1 --2- 172.21.15.100:0/3677054200 >> v2:172.21.15.151:3300/0 conn(0x7f7570073610 0x7f757006df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.921+0000 7f7565400640 1 -- 172.21.15.100:0/3677054200 >> v1:172.21.15.151:6789/0 conn(0x7f757015b640 legacy=0x7f7570072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.921+0000 7f7565400640 1 -- 172.21.15.100:0/3677054200 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75700af320 con 0x7f75700b2a10 2024-04-23T14:32:02.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.921+0000 7f7565400640 1 --2- 172.21.15.100:0/3677054200 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75700b2a10 0x7f75700adfc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7558009870 tx=0x7f755801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61474fcc1c7420ff server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:02.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.921+0000 7f756d400640 1 -- 172.21.15.100:0/3677054200 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75580a95f0 con 0x7f75700b2a10 2024-04-23T14:32:02.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.921+0000 7f756d400640 1 -- 172.21.15.100:0/3677054200 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75580a9790 con 0x7f75700b2a10 2024-04-23T14:32:02.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.921+0000 7f756d400640 1 -- 172.21.15.100:0/3677054200 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75580a9b60 con 0x7f75700b2a10 2024-04-23T14:32:02.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.921+0000 7f7574e00640 1 -- 172.21.15.100:0/3677054200 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75700b2a10 msgr2=0x7f75700adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.921+0000 7f7574e00640 1 --2- 172.21.15.100:0/3677054200 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75700b2a10 0x7f75700adfc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7558009870 tx=0x7f755801c530 comp rx=0 tx=0).stop 2024-04-23T14:32:02.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.921+0000 7f7574e00640 1 -- 172.21.15.100:0/3677054200 shutdown_connections 2024-04-23T14:32:02.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.921+0000 7f7574e00640 1 --2- 172.21.15.100:0/3677054200 >> v2:172.21.15.151:3300/0 conn(0x7f7570073610 0x7f757006df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.921+0000 7f7574e00640 1 --2- 172.21.15.100:0/3677054200 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75700b2a10 0x7f75700adfc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.921+0000 7f7574e00640 1 -- 172.21.15.100:0/3677054200 >> 172.21.15.100:0/3677054200 conn(0x7f75700b1e80 msgr2=0x7f75700b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:02.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.921+0000 7f7574e00640 1 -- 172.21.15.100:0/3677054200 shutdown_connections 2024-04-23T14:32:02.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.921+0000 7f7574e00640 1 -- 172.21.15.100:0/3677054200 wait complete. 2024-04-23T14:32:02.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f7574e00640 1 Processor -- start 2024-04-23T14:32:02.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f7574e00640 1 -- start start 2024-04-23T14:32:02.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f7574e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7570073610 0x7f7570155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f7574e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75700b2a10 0x7f75701488f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f7574e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f757015b640 0x7f7570151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f7574e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f757006f0d0 con 0x7f75700b2a10 2024-04-23T14:32:02.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f7574e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f757006ef50 con 0x7f757015b640 2024-04-23T14:32:02.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f7574e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f757006f250 con 0x7f7570073610 2024-04-23T14:32:02.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f7565400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75700b2a10 0x7f75701488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f756de00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7570073610 0x7f7570155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f7565400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75700b2a10 0x7f75701488f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46048/0 (socket says 172.21.15.100:46048) 2024-04-23T14:32:02.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f7565400640 1 -- 172.21.15.100:0/2545885495 learned_addr learned my addr 172.21.15.100:0/2545885495 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:02.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f756e800640 1 --2- 172.21.15.100:0/2545885495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f757015b640 0x7f7570151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f7565400640 1 -- 172.21.15.100:0/2545885495 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7570073610 msgr2=0x7f7570155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f7565400640 1 --2- 172.21.15.100:0/2545885495 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7570073610 0x7f7570155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f7565400640 1 -- 172.21.15.100:0/2545885495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f757015b640 msgr2=0x7f7570151230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f7565400640 1 --2- 172.21.15.100:0/2545885495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f757015b640 0x7f7570151230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f7565400640 1 -- 172.21.15.100:0/2545885495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7570150190 con 0x7f75700b2a10 2024-04-23T14:32:02.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f7565400640 1 --2- 172.21.15.100:0/2545885495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75700b2a10 0x7f75701488f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f75580ab040 tx=0x7f755801ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:02.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f7566a00640 1 -- 172.21.15.100:0/2545885495 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75580b1600 con 0x7f75700b2a10 2024-04-23T14:32:02.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f7566a00640 1 -- 172.21.15.100:0/2545885495 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75580b17a0 con 0x7f75700b2a10 2024-04-23T14:32:02.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f7574e00640 1 -- 172.21.15.100:0/2545885495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7570150470 con 0x7f75700b2a10 2024-04-23T14:32:02.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f7566a00640 1 -- 172.21.15.100:0/2545885495 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75580b1b90 con 0x7f75700b2a10 2024-04-23T14:32:02.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.925+0000 7f7574e00640 1 -- 172.21.15.100:0/2545885495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7570150a30 con 0x7f75700b2a10 2024-04-23T14:32:02.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.929+0000 7f7574e00640 1 -- 172.21.15.100:0/2545885495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7570068ad0 con 0x7f75700b2a10 2024-04-23T14:32:02.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.929+0000 7f7566a00640 1 -- 172.21.15.100:0/2545885495 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f75580bc450 con 0x7f75700b2a10 2024-04-23T14:32:02.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.929+0000 7f7566a00640 1 --2- 172.21.15.100:0/2545885495 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f754c0683b0 0x7f754c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.929+0000 7f7566a00640 1 -- 172.21.15.100:0/2545885495 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f7558120ef0 con 0x7f75700b2a10 2024-04-23T14:32:02.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.929+0000 7f756de00640 1 --2- 172.21.15.100:0/2545885495 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f754c0683b0 0x7f754c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.929+0000 7f756de00640 1 --2- 172.21.15.100:0/2545885495 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f754c0683b0 0x7f754c06a870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7560001e00 tx=0x7f7560009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:02.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:02.933+0000 7f7566a00640 1 -- 172.21.15.100:0/2545885495 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f75580ad020 con 0x7f75700b2a10 2024-04-23T14:32:03.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:03.121+0000 7f7574e00640 1 -- 172.21.15.100:0/2545885495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75700720c0 con 0x7f75700b2a10 2024-04-23T14:32:03.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:03.121+0000 7f7566a00640 1 -- 172.21.15.100:0/2545885495 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7f7558016150 con 0x7f75700b2a10 2024-04-23T14:32:03.124 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:03.124 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1456958549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1456958549},{"type":"v1","addr":"172.21.15.100:6836","nonce":1456958549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-23T14:32:03.124 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 2 2024-04-23T14:32:03.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:03.125+0000 7f7574e00640 1 -- 172.21.15.100:0/2545885495 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f754c0683b0 msgr2=0x7f754c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:03.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:03.125+0000 7f7574e00640 1 --2- 172.21.15.100:0/2545885495 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f754c0683b0 0x7f754c06a870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7560001e00 tx=0x7f7560009040 comp rx=0 tx=0).stop 2024-04-23T14:32:03.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:03.125+0000 7f7574e00640 1 -- 172.21.15.100:0/2545885495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75700b2a10 msgr2=0x7f75701488f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:03.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:03.125+0000 7f7574e00640 1 --2- 172.21.15.100:0/2545885495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75700b2a10 0x7f75701488f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f75580ab040 tx=0x7f755801ad70 comp rx=0 tx=0).stop 2024-04-23T14:32:03.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:03.125+0000 7f7574e00640 1 -- 172.21.15.100:0/2545885495 shutdown_connections 2024-04-23T14:32:03.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:03.125+0000 7f7574e00640 1 --2- 172.21.15.100:0/2545885495 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f754c0683b0 0x7f754c06a870 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:03.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:03.125+0000 7f7574e00640 1 --2- 172.21.15.100:0/2545885495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f757015b640 0x7f7570151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:03.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:03.125+0000 7f7574e00640 1 --2- 172.21.15.100:0/2545885495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75700b2a10 0x7f75701488f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:03.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:03.125+0000 7f7574e00640 1 --2- 172.21.15.100:0/2545885495 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7570073610 0x7f7570155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:03.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:03.125+0000 7f7574e00640 1 -- 172.21.15.100:0/2545885495 >> 172.21.15.100:0/2545885495 conn(0x7f75700b1e80 msgr2=0x7f7570072350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:03.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:03.125+0000 7f7574e00640 1 -- 172.21.15.100:0/2545885495 shutdown_connections 2024-04-23T14:32:03.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:03.125+0000 7f7574e00640 1 -- 172.21.15.100:0/2545885495 wait complete. 2024-04-23T14:32:05.144 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:32:05.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.289+0000 7fa5e9800640 1 Processor -- start 2024-04-23T14:32:05.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.289+0000 7fa5e9800640 1 -- start start 2024-04-23T14:32:05.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.289+0000 7fa5e9800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5e406d450 0x7fa5e406d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.289+0000 7fa5e9800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fa5e40cd460 0x7fa5e40b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.289+0000 7fa5e9800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5e406dd70 con 0x7fa5e40cea60 2024-04-23T14:32:05.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.289+0000 7fa5e9800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fa5e40b1f90 con 0x7fa5e40cd460 2024-04-23T14:32:05.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.289+0000 7fa5e9800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e40b2270 con 0x7fa5e406d450 2024-04-23T14:32:05.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.289+0000 7fa5e3400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fa5e40cd460 0x7fa5e40b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.289+0000 7fa5e2a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5e406d450 0x7fa5e406d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.289+0000 7fa5e3400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fa5e40cd460 0x7fa5e40b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:37820/0 (socket says 172.21.15.100:37820) 2024-04-23T14:32:05.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.289+0000 7fa5e3400640 1 -- 172.21.15.100:0/3320214017 learned_addr learned my addr 172.21.15.100:0/3320214017 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:05.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.289+0000 7fa5e2000640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7fa5e40cea60 0x7fa5e40d0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:53306/0 (socket says 172.21.15.100:53306) 2024-04-23T14:32:05.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.289+0000 7fa5e2a00640 1 -- 172.21.15.100:0/3320214017 >> v2:172.21.15.151:3300/0 conn(0x7fa5e40cd460 msgr2=0x7fa5e40b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:05.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.289+0000 7fa5e2a00640 1 --2- 172.21.15.100:0/3320214017 >> v2:172.21.15.151:3300/0 conn(0x7fa5e40cd460 0x7fa5e40b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.289+0000 7fa5e2a00640 1 -- 172.21.15.100:0/3320214017 >> v1:172.21.15.151:6789/0 conn(0x7fa5e40cea60 legacy=0x7fa5e40d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.289+0000 7fa5e2a00640 1 -- 172.21.15.100:0/3320214017 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e40b2550 con 0x7fa5e406d450 2024-04-23T14:32:05.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.289+0000 7fa5e3400640 1 --2- 172.21.15.100:0/3320214017 >> v2:172.21.15.151:3300/0 conn(0x7fa5e40cd460 0x7fa5e40b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:05.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.289+0000 7fa5e1600640 1 -- 172.21.15.100:0/3320214017 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 340255733 0 0) 0x7fa5e406dd70 con 0x7fa5e40cea60 2024-04-23T14:32:05.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.293+0000 7fa5e2a00640 1 --2- 172.21.15.100:0/3320214017 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5e406d450 0x7fa5e406d830 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009d90 tx=0x7fa5d401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e956b602d237a5b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:05.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.293+0000 7fa5e1600640 1 -- 172.21.15.100:0/3320214017 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d40b2070 con 0x7fa5e406d450 2024-04-23T14:32:05.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.293+0000 7fa5e1600640 1 -- 172.21.15.100:0/3320214017 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5d40a57a0 con 0x7fa5e406d450 2024-04-23T14:32:05.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.293+0000 7fa5e1600640 1 -- 172.21.15.100:0/3320214017 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d40a5b70 con 0x7fa5e406d450 2024-04-23T14:32:05.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.293+0000 7fa5e9800640 1 -- 172.21.15.100:0/3320214017 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5e406d450 msgr2=0x7fa5e406d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.293+0000 7fa5e9800640 1 --2- 172.21.15.100:0/3320214017 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5e406d450 0x7fa5e406d830 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009d90 tx=0x7fa5d401c8e0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.293+0000 7fa5e9800640 1 -- 172.21.15.100:0/3320214017 shutdown_connections 2024-04-23T14:32:05.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.293+0000 7fa5e9800640 1 --2- 172.21.15.100:0/3320214017 >> v2:172.21.15.151:3300/0 conn(0x7fa5e40cd460 0x7fa5e40b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.293+0000 7fa5e9800640 1 --2- 172.21.15.100:0/3320214017 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5e406d450 0x7fa5e406d830 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.293+0000 7fa5e9800640 1 -- 172.21.15.100:0/3320214017 >> 172.21.15.100:0/3320214017 conn(0x7fa5e40cee80 msgr2=0x7fa5e40cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:05.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.293+0000 7fa5e9800640 1 -- 172.21.15.100:0/3320214017 shutdown_connections 2024-04-23T14:32:05.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.293+0000 7fa5e9800640 1 -- 172.21.15.100:0/3320214017 wait complete. 2024-04-23T14:32:05.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5e9800640 1 Processor -- start 2024-04-23T14:32:05.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5e9800640 1 -- start start 2024-04-23T14:32:05.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5e9800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5e406d450 0x7fa5e4159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5e9800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5e40cd460 0x7fa5e4156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5e9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5e40cea60 0x7fa5e41561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5e9800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e416b740 con 0x7fa5e40cea60 2024-04-23T14:32:05.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5e9800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e416b5c0 con 0x7fa5e40cd460 2024-04-23T14:32:05.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5e9800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e416b8c0 con 0x7fa5e406d450 2024-04-23T14:32:05.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5e3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5e40cea60 0x7fa5e41561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5e3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5e40cea60 0x7fa5e41561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46050/0 (socket says 172.21.15.100:46050) 2024-04-23T14:32:05.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5e3400640 1 -- 172.21.15.100:0/1584135758 learned_addr learned my addr 172.21.15.100:0/1584135758 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:05.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5e2a00640 1 --2- 172.21.15.100:0/1584135758 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5e406d450 0x7fa5e4159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5e2000640 1 --2- 172.21.15.100:0/1584135758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5e40cd460 0x7fa5e4156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5e3400640 1 -- 172.21.15.100:0/1584135758 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5e406d450 msgr2=0x7fa5e4159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5e3400640 1 --2- 172.21.15.100:0/1584135758 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5e406d450 0x7fa5e4159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5e3400640 1 -- 172.21.15.100:0/1584135758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5e40cd460 msgr2=0x7fa5e4156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5e3400640 1 --2- 172.21.15.100:0/1584135758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5e40cd460 0x7fa5e4156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5e3400640 1 -- 172.21.15.100:0/1584135758 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e41582b0 con 0x7fa5e40cea60 2024-04-23T14:32:05.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5e3400640 1 --2- 172.21.15.100:0/1584135758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5e40cea60 0x7fa5e41561c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00e950 tx=0x7fa5dc00ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:05.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5daa00640 1 -- 172.21.15.100:0/1584135758 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc017070 con 0x7fa5e40cea60 2024-04-23T14:32:05.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5daa00640 1 -- 172.21.15.100:0/1584135758 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5dc00ca00 con 0x7fa5e40cea60 2024-04-23T14:32:05.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5e9800640 1 -- 172.21.15.100:0/1584135758 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e41570e0 con 0x7fa5e40cea60 2024-04-23T14:32:05.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5daa00640 1 -- 172.21.15.100:0/1584135758 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc00cdd0 con 0x7fa5e40cea60 2024-04-23T14:32:05.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.297+0000 7fa5e9800640 1 -- 172.21.15.100:0/1584135758 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e41576a0 con 0x7fa5e40cea60 2024-04-23T14:32:05.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.301+0000 7fa5e9800640 1 -- 172.21.15.100:0/1584135758 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e4068ad0 con 0x7fa5e40cea60 2024-04-23T14:32:05.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.301+0000 7fa5daa00640 1 -- 172.21.15.100:0/1584135758 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa5dc013470 con 0x7fa5e40cea60 2024-04-23T14:32:05.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.301+0000 7fa5daa00640 1 --2- 172.21.15.100:0/1584135758 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa5c4068380 0x7fa5c406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.301+0000 7fa5daa00640 1 -- 172.21.15.100:0/1584135758 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa5dc086190 con 0x7fa5e40cea60 2024-04-23T14:32:05.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.301+0000 7fa5e2a00640 1 --2- 172.21.15.100:0/1584135758 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa5c4068380 0x7fa5c406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.301+0000 7fa5e2a00640 1 --2- 172.21.15.100:0/1584135758 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa5c4068380 0x7fa5c406a840 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40af000 tx=0x7fa5d40a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:05.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.305+0000 7fa5daa00640 1 -- 172.21.15.100:0/1584135758 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa5e41582b0 con 0x7fa5e40cea60 2024-04-23T14:32:05.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.493+0000 7fa5e9800640 1 -- 172.21.15.100:0/1584135758 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5e4070230 con 0x7fa5e40cea60 2024-04-23T14:32:05.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.497+0000 7fa5daa00640 1 -- 172.21.15.100:0/1584135758 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3876 (secure 0 0 0) 0x7fa5dc0563b0 con 0x7fa5e40cea60 2024-04-23T14:32:05.497 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:05.497 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1456958549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1456958549},{"type":"v1","addr":"172.21.15.100:6836","nonce":1456958549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3655609551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3655609551},{"type":"v1","addr":"172.21.15.100:6837","nonce":3655609551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/4242180972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":4242180972},{"type":"v1","addr":"172.21.15.151:6834","nonce":4242180972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1533322173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1533322173},{"type":"v1","addr":"172.21.15.151:6835","nonce":1533322173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-23T14:32:05.497 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 3 2024-04-23T14:32:05.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.497+0000 7fa5e9800640 1 -- 172.21.15.100:0/1584135758 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa5c4068380 msgr2=0x7fa5c406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.497+0000 7fa5e9800640 1 --2- 172.21.15.100:0/1584135758 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa5c4068380 0x7fa5c406a840 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40af000 tx=0x7fa5d40a7070 comp rx=0 tx=0).stop 2024-04-23T14:32:05.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.497+0000 7fa5e9800640 1 -- 172.21.15.100:0/1584135758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5e40cea60 msgr2=0x7fa5e41561c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.497+0000 7fa5e9800640 1 --2- 172.21.15.100:0/1584135758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5e40cea60 0x7fa5e41561c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00e950 tx=0x7fa5dc00ee10 comp rx=0 tx=0).stop 2024-04-23T14:32:05.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.497+0000 7fa5e9800640 1 -- 172.21.15.100:0/1584135758 shutdown_connections 2024-04-23T14:32:05.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.497+0000 7fa5e9800640 1 --2- 172.21.15.100:0/1584135758 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa5c4068380 0x7fa5c406a840 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.497+0000 7fa5e9800640 1 --2- 172.21.15.100:0/1584135758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5e40cea60 0x7fa5e41561c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.497+0000 7fa5e9800640 1 --2- 172.21.15.100:0/1584135758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5e40cd460 0x7fa5e4156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.497+0000 7fa5e9800640 1 --2- 172.21.15.100:0/1584135758 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5e406d450 0x7fa5e4159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.497+0000 7fa5e9800640 1 -- 172.21.15.100:0/1584135758 >> 172.21.15.100:0/1584135758 conn(0x7fa5e40cee80 msgr2=0x7fa5e40ce1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:05.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.497+0000 7fa5e9800640 1 -- 172.21.15.100:0/1584135758 shutdown_connections 2024-04-23T14:32:05.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.497+0000 7fa5e9800640 1 -- 172.21.15.100:0/1584135758 wait complete. 2024-04-23T14:32:05.518 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:05.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.653+0000 7f9c09400640 1 Processor -- start 2024-04-23T14:32:05.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.653+0000 7f9c09400640 1 -- start start 2024-04-23T14:32:05.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.653+0000 7f9c09400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9c0415c7e0 0x7f9c040d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.653+0000 7f9c09400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c040d04a0 con 0x7f9c0415d600 2024-04-23T14:32:05.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.653+0000 7f9c09400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c040d0780 con 0x7f9c040cca50 2024-04-23T14:32:05.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.653+0000 7f9c09400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f9c0415db00 con 0x7f9c0415c7e0 2024-04-23T14:32:05.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.653+0000 7f9c01e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9c0415c7e0 0x7f9c040d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.653+0000 7f9c03200640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f9c0415d600 0x7f9c040d17a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:39382/0 (socket says 172.21.15.100:39382) 2024-04-23T14:32:05.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.653+0000 7f9c03200640 1 -- 172.21.15.100:0/606981594 learned_addr learned my addr 172.21.15.100:0/606981594 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:05.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.653+0000 7f9c01e00640 1 -- 172.21.15.100:0/606981594 >> v1:172.21.15.151:6789/0 conn(0x7f9c040cca50 legacy=0x7f9c040cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.653+0000 7f9c01e00640 1 -- 172.21.15.100:0/606981594 >> v1:172.21.15.100:6789/0 conn(0x7f9c0415d600 legacy=0x7f9c040d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.653+0000 7f9c01e00640 1 -- 172.21.15.100:0/606981594 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c040d0a60 con 0x7f9c0415c7e0 2024-04-23T14:32:05.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.653+0000 7f9c01400640 1 -- 172.21.15.100:0/606981594 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3610565223 0 0) 0x7f9c040d04a0 con 0x7f9c0415d600 2024-04-23T14:32:05.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.657+0000 7f9c01e00640 1 --2- 172.21.15.100:0/606981594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c0415c7e0 0x7f9c040d1260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00097c0 tx=0x7f9bf001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c113b986e7aa206 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:05.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.657+0000 7f9c01400640 1 -- 172.21.15.100:0/606981594 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00ab600 con 0x7f9c0415c7e0 2024-04-23T14:32:05.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.657+0000 7f9c01400640 1 -- 172.21.15.100:0/606981594 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bf00b0070 con 0x7f9c0415c7e0 2024-04-23T14:32:05.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.657+0000 7f9c01400640 1 -- 172.21.15.100:0/606981594 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00ab9c0 con 0x7f9c0415c7e0 2024-04-23T14:32:05.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.657+0000 7f9c09400640 1 -- 172.21.15.100:0/606981594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c0415c7e0 msgr2=0x7f9c040d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.657+0000 7f9c09400640 1 --2- 172.21.15.100:0/606981594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c0415c7e0 0x7f9c040d1260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00097c0 tx=0x7f9bf001c3e0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.657+0000 7f9c09400640 1 -- 172.21.15.100:0/606981594 shutdown_connections 2024-04-23T14:32:05.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.657+0000 7f9c09400640 1 --2- 172.21.15.100:0/606981594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c0415c7e0 0x7f9c040d1260 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.657+0000 7f9c09400640 1 -- 172.21.15.100:0/606981594 >> 172.21.15.100:0/606981594 conn(0x7f9c040cee80 msgr2=0x7f9c040cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:05.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.657+0000 7f9c09400640 1 -- 172.21.15.100:0/606981594 shutdown_connections 2024-04-23T14:32:05.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.657+0000 7f9c09400640 1 -- 172.21.15.100:0/606981594 wait complete. 2024-04-23T14:32:05.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c09400640 1 Processor -- start 2024-04-23T14:32:05.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c09400640 1 -- start start 2024-04-23T14:32:05.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c09400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c040cca50 0x7f9c04148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c09400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9c0415c7e0 0x7f9c04149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c09400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c0415d600 0x7f9c0414c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c09400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0416ac60 con 0x7f9c040cca50 2024-04-23T14:32:05.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c09400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0416aae0 con 0x7f9c0415d600 2024-04-23T14:32:05.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c09400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9c0416ade0 con 0x7f9c0415c7e0 2024-04-23T14:32:05.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c02800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c040cca50 0x7f9c04148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c02800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c040cca50 0x7f9c04148cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46070/0 (socket says 172.21.15.100:46070) 2024-04-23T14:32:05.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c02800640 1 -- 172.21.15.100:0/2936024431 learned_addr learned my addr 172.21.15.100:0/2936024431 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:05.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c03200640 1 --2- 172.21.15.100:0/2936024431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c0415d600 0x7f9c0414c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c01e00640 1 --2- 172.21.15.100:0/2936024431 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9c0415c7e0 0x7f9c04149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c02800640 1 -- 172.21.15.100:0/2936024431 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9c0415c7e0 msgr2=0x7f9c04149210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c02800640 1 --2- 172.21.15.100:0/2936024431 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9c0415c7e0 0x7f9c04149210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c02800640 1 -- 172.21.15.100:0/2936024431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c0415d600 msgr2=0x7f9c0414c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c02800640 1 --2- 172.21.15.100:0/2936024431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c0415d600 0x7f9c0414c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c02800640 1 -- 172.21.15.100:0/2936024431 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0414b030 con 0x7f9c040cca50 2024-04-23T14:32:05.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c03200640 1 --2- 172.21.15.100:0/2936024431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c0415d600 0x7f9c0414c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:05.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c01e00640 1 --2- 172.21.15.100:0/2936024431 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9c0415c7e0 0x7f9c04149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:05.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c02800640 1 --2- 172.21.15.100:0/2936024431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c040cca50 0x7f9c04148cd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400c530 tx=0x7f9bf400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:05.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9bfaa00640 1 -- 172.21.15.100:0/2936024431 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4016020 con 0x7f9c040cca50 2024-04-23T14:32:05.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9bfaa00640 1 -- 172.21.15.100:0/2936024431 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bf4005220 con 0x7f9c040cca50 2024-04-23T14:32:05.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c09400640 1 -- 172.21.15.100:0/2936024431 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0414b310 con 0x7f9c040cca50 2024-04-23T14:32:05.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9bfaa00640 1 -- 172.21.15.100:0/2936024431 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4007030 con 0x7f9c040cca50 2024-04-23T14:32:05.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c09400640 1 -- 172.21.15.100:0/2936024431 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0414c0a0 con 0x7f9c040cca50 2024-04-23T14:32:05.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.661+0000 7f9c09400640 1 -- 172.21.15.100:0/2936024431 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c0414c2a0 con 0x7f9c040cca50 2024-04-23T14:32:05.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.665+0000 7f9bfaa00640 1 -- 172.21.15.100:0/2936024431 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9bf40071d0 con 0x7f9c040cca50 2024-04-23T14:32:05.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.665+0000 7f9bfaa00640 1 --2- 172.21.15.100:0/2936024431 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9be40683b0 0x7f9be406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.665+0000 7f9bfaa00640 1 -- 172.21.15.100:0/2936024431 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f9bf4086150 con 0x7f9c040cca50 2024-04-23T14:32:05.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.665+0000 7f9c01e00640 1 --2- 172.21.15.100:0/2936024431 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9be40683b0 0x7f9be406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.665+0000 7f9c01e00640 1 --2- 172.21.15.100:0/2936024431 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9be40683b0 0x7f9be406a870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00af000 tx=0x7f9bf00a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:05.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.669+0000 7f9bfaa00640 1 -- 172.21.15.100:0/2936024431 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9c0414c2a0 con 0x7f9c040cca50 2024-04-23T14:32:05.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.825+0000 7f9c09400640 1 -- 172.21.15.100:0/2936024431 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c0414bbd0 con 0x7f9c040cca50 2024-04-23T14:32:05.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.825+0000 7f9bfaa00640 1 -- 172.21.15.100:0/2936024431 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12896 (secure 0 0 0) 0x7f9bf4020020 con 0x7f9c040cca50 2024-04-23T14:32:05.828 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:05.829 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":16,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:31:59.150908+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:05.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.829+0000 7f9c09400640 1 -- 172.21.15.100:0/2936024431 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9be40683b0 msgr2=0x7f9be406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.829+0000 7f9c09400640 1 --2- 172.21.15.100:0/2936024431 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9be40683b0 0x7f9be406a870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00af000 tx=0x7f9bf00a7000 comp rx=0 tx=0).stop 2024-04-23T14:32:05.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.829+0000 7f9c09400640 1 -- 172.21.15.100:0/2936024431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c040cca50 msgr2=0x7f9c04148cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.829+0000 7f9c09400640 1 --2- 172.21.15.100:0/2936024431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c040cca50 0x7f9c04148cd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400c530 tx=0x7f9bf400c9f0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.829+0000 7f9c09400640 1 -- 172.21.15.100:0/2936024431 shutdown_connections 2024-04-23T14:32:05.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.829+0000 7f9c09400640 1 --2- 172.21.15.100:0/2936024431 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9be40683b0 0x7f9be406a870 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.829+0000 7f9c09400640 1 --2- 172.21.15.100:0/2936024431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c0415d600 0x7f9c0414c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.829+0000 7f9c09400640 1 --2- 172.21.15.100:0/2936024431 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9c0415c7e0 0x7f9c04149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.829+0000 7f9c09400640 1 --2- 172.21.15.100:0/2936024431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c040cca50 0x7f9c04148cd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.829+0000 7f9c09400640 1 -- 172.21.15.100:0/2936024431 >> 172.21.15.100:0/2936024431 conn(0x7f9c040cee80 msgr2=0x7f9c040cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:05.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.829+0000 7f9c09400640 1 -- 172.21.15.100:0/2936024431 shutdown_connections 2024-04-23T14:32:05.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.829+0000 7f9c09400640 1 -- 172.21.15.100:0/2936024431 wait complete. 2024-04-23T14:32:05.849 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:32:05.849 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:05.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.985+0000 7f2e69c00640 1 Processor -- start 2024-04-23T14:32:05.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.985+0000 7f2e69c00640 1 -- start start 2024-04-23T14:32:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.985+0000 7f2e69c00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f2e6406d190 0x7f2e640cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.985+0000 7f2e69c00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e640cdab0 con 0x7f2e6406cd50 2024-04-23T14:32:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.985+0000 7f2e69c00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e64146270 con 0x7f2e64145e90 2024-04-23T14:32:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.985+0000 7f2e69c00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f2e640cdd50 con 0x7f2e6406d190 2024-04-23T14:32:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.985+0000 7f2e5a000640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f2e6406cd50 0x7f2e640cedf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:39394/0 (socket says 172.21.15.100:39394) 2024-04-23T14:32:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.985+0000 7f2e5a000640 1 -- 172.21.15.100:0/3635969091 learned_addr learned my addr 172.21.15.100:0/3635969091 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.985+0000 7f2e63400640 1 --2- 172.21.15.100:0/3635969091 >> v2:172.21.15.151:3300/0 conn(0x7f2e6406d190 0x7f2e640cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.985+0000 7f2e62000640 1 -- 172.21.15.100:0/3635969091 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2441170683 0 0) 0x7f2e640cdab0 con 0x7f2e6406cd50 2024-04-23T14:32:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.985+0000 7f2e62000640 1 -- 172.21.15.100:0/3635969091 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e48003530 con 0x7f2e6406cd50 2024-04-23T14:32:05.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.985+0000 7f2e62000640 1 -- 172.21.15.100:0/3635969091 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 657029560 0 0) 0x7f2e48003530 con 0x7f2e6406cd50 2024-04-23T14:32:05.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.985+0000 7f2e62000640 1 -- 172.21.15.100:0/3635969091 >> v2:172.21.15.151:3300/0 conn(0x7f2e6406d190 msgr2=0x7f2e640cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:05.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.985+0000 7f2e62000640 1 --2- 172.21.15.100:0/3635969091 >> v2:172.21.15.151:3300/0 conn(0x7f2e6406d190 0x7f2e640cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.985+0000 7f2e62000640 1 -- 172.21.15.100:0/3635969091 >> v1:172.21.15.151:6789/0 conn(0x7f2e64145e90 legacy=0x7f2e6406c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.985+0000 7f2e62000640 1 -- 172.21.15.100:0/3635969091 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e640ce030 con 0x7f2e6406cd50 2024-04-23T14:32:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.985+0000 7f2e62000640 1 -- 172.21.15.100:0/3635969091 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f2e4c0016a0 con 0x7f2e6406cd50 2024-04-23T14:32:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.985+0000 7f2e63400640 1 --2- 172.21.15.100:0/3635969091 >> v2:172.21.15.151:3300/0 conn(0x7f2e6406d190 0x7f2e640cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.985+0000 7f2e62000640 1 -- 172.21.15.100:0/3635969091 >> v1:172.21.15.100:6789/0 conn(0x7f2e6406cd50 legacy=0x7f2e640cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.985+0000 7f2e62000640 1 --2- 172.21.15.100:0/3635969091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e480039c0 0x7f2e48003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.985+0000 7f2e62000640 1 --2- 172.21.15.100:0/3635969091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e48004810 0x7f2e48004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.985+0000 7f2e62000640 1 --2- 172.21.15.100:0/3635969091 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2e480051d0 0x7f2e4800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.989+0000 7f2e62000640 1 -- 172.21.15.100:0/3635969091 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2e640cdd50 con 0x7f2e480039c0 2024-04-23T14:32:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.989+0000 7f2e62000640 1 -- 172.21.15.100:0/3635969091 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4800c170 con 0x7f2e48004810 2024-04-23T14:32:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.989+0000 7f2e62000640 1 -- 172.21.15.100:0/3635969091 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f2e4800c370 con 0x7f2e480051d0 2024-04-23T14:32:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.989+0000 7f2e62a00640 1 --2- 172.21.15.100:0/3635969091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e480039c0 0x7f2e48003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.989+0000 7f2e63400640 1 --2- 172.21.15.100:0/3635969091 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2e480051d0 0x7f2e4800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.989+0000 7f2e5a000640 1 --2- 172.21.15.100:0/3635969091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e48004810 0x7f2e48004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.989+0000 7f2e62a00640 1 -- 172.21.15.100:0/3635969091 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2e480051d0 msgr2=0x7f2e4800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.989+0000 7f2e62a00640 1 --2- 172.21.15.100:0/3635969091 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2e480051d0 0x7f2e4800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.989+0000 7f2e62a00640 1 -- 172.21.15.100:0/3635969091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e48004810 msgr2=0x7f2e48004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.989+0000 7f2e62a00640 1 --2- 172.21.15.100:0/3635969091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e48004810 0x7f2e48004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.989+0000 7f2e62a00640 1 -- 172.21.15.100:0/3635969091 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e48003530 con 0x7f2e480039c0 2024-04-23T14:32:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.989+0000 7f2e5a000640 1 --2- 172.21.15.100:0/3635969091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e48004810 0x7f2e48004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.989+0000 7f2e63400640 1 -- 172.21.15.100:0/3635969091 reap_dead start 2024-04-23T14:32:05.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.989+0000 7f2e62a00640 1 --2- 172.21.15.100:0/3635969091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e480039c0 0x7f2e48003de0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2e54001780 tx=0x7f2e5401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17d920f100daf2e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:05.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.989+0000 7f2e62000640 1 -- 172.21.15.100:0/3635969091 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e540013a0 con 0x7f2e480039c0 2024-04-23T14:32:05.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.989+0000 7f2e62000640 1 -- 172.21.15.100:0/3635969091 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e540052f0 con 0x7f2e480039c0 2024-04-23T14:32:05.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.989+0000 7f2e62000640 1 -- 172.21.15.100:0/3635969091 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e540a4e60 con 0x7f2e480039c0 2024-04-23T14:32:05.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.989+0000 7f2e69c00640 1 -- 172.21.15.100:0/3635969091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e480039c0 msgr2=0x7f2e48003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.989+0000 7f2e69c00640 1 --2- 172.21.15.100:0/3635969091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e480039c0 0x7f2e48003de0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2e54001780 tx=0x7f2e5401c420 comp rx=0 tx=0).stop 2024-04-23T14:32:05.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e69c00640 1 -- 172.21.15.100:0/3635969091 shutdown_connections 2024-04-23T14:32:05.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e69c00640 1 --2- 172.21.15.100:0/3635969091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e480039c0 0x7f2e48003de0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e69c00640 1 -- 172.21.15.100:0/3635969091 >> 172.21.15.100:0/3635969091 conn(0x7f2e640704d0 msgr2=0x7f2e640708e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:05.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e69c00640 1 -- 172.21.15.100:0/3635969091 shutdown_connections 2024-04-23T14:32:05.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e69c00640 1 -- 172.21.15.100:0/3635969091 wait complete. 2024-04-23T14:32:05.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e69c00640 1 Processor -- start 2024-04-23T14:32:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e69c00640 1 -- start start 2024-04-23T14:32:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e69c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2e480039c0 0x7f2e6413bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e69c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e641415f0 0x7f2e6413e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e69c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e64141a70 0x7f2e6413eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e69c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6406d570 con 0x7f2e641415f0 2024-04-23T14:32:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e69c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2e641600b0 con 0x7f2e64141a70 2024-04-23T14:32:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e69c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f2e6415f060 con 0x7f2e480039c0 2024-04-23T14:32:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e5a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e641415f0 0x7f2e6413e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e5a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e641415f0 0x7f2e6413e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46094/0 (socket says 172.21.15.100:46094) 2024-04-23T14:32:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e5a000640 1 -- 172.21.15.100:0/492678116 learned_addr learned my addr 172.21.15.100:0/492678116 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e62a00640 1 --2- 172.21.15.100:0/492678116 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2e480039c0 0x7f2e6413bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e63400640 1 --2- 172.21.15.100:0/492678116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e64141a70 0x7f2e6413eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e5a000640 1 -- 172.21.15.100:0/492678116 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2e480039c0 msgr2=0x7f2e6413bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e5a000640 1 --2- 172.21.15.100:0/492678116 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2e480039c0 0x7f2e6413bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e5a000640 1 -- 172.21.15.100:0/492678116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e64141a70 msgr2=0x7f2e6413eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e5a000640 1 --2- 172.21.15.100:0/492678116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e64141a70 0x7f2e6413eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e5a000640 1 -- 172.21.15.100:0/492678116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e641f5980 con 0x7f2e641415f0 2024-04-23T14:32:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e62a00640 1 --2- 172.21.15.100:0/492678116 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2e480039c0 0x7f2e6413bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e63400640 1 --2- 172.21.15.100:0/492678116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e64141a70 0x7f2e6413eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e5a000640 1 --2- 172.21.15.100:0/492678116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e641415f0 0x7f2e6413e9a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c007fd0 tx=0x7f2e4c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.993+0000 7f2e5b400640 1 -- 172.21.15.100:0/492678116 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c019070 con 0x7f2e641415f0 2024-04-23T14:32:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.997+0000 7f2e5b400640 1 -- 172.21.15.100:0/492678116 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e4c0072d0 con 0x7f2e641415f0 2024-04-23T14:32:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.997+0000 7f2e5b400640 1 -- 172.21.15.100:0/492678116 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c0047f0 con 0x7f2e641415f0 2024-04-23T14:32:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.997+0000 7f2e69c00640 1 -- 172.21.15.100:0/492678116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e641f5c60 con 0x7f2e641415f0 2024-04-23T14:32:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.997+0000 7f2e69c00640 1 -- 172.21.15.100:0/492678116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e6415cf30 con 0x7f2e641415f0 2024-04-23T14:32:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.997+0000 7f2e69c00640 1 -- 172.21.15.100:0/492678116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e64068ad0 con 0x7f2e641415f0 2024-04-23T14:32:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.997+0000 7f2e5b400640 1 -- 172.21.15.100:0/492678116 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2e4c004990 con 0x7f2e641415f0 2024-04-23T14:32:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.997+0000 7f2e5b400640 1 --2- 172.21.15.100:0/492678116 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2e400683b0 0x7f2e4006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.997+0000 7f2e5b400640 1 -- 172.21.15.100:0/492678116 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f2e4c085eb0 con 0x7f2e641415f0 2024-04-23T14:32:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.997+0000 7f2e62a00640 1 --2- 172.21.15.100:0/492678116 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2e400683b0 0x7f2e4006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:05.997+0000 7f2e62a00640 1 --2- 172.21.15.100:0/492678116 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2e400683b0 0x7f2e4006a870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2e54001750 tx=0x7f2e540011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:06.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.001+0000 7f2e5b400640 1 -- 172.21.15.100:0/492678116 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2e4c0567b0 con 0x7f2e641415f0 2024-04-23T14:32:06.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.157+0000 7f2e69c00640 1 -- 172.21.15.100:0/492678116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2e64071520 con 0x7f2e641415f0 2024-04-23T14:32:06.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.157+0000 7f2e5b400640 1 -- 172.21.15.100:0/492678116 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f2e4c0560b0 con 0x7f2e641415f0 2024-04-23T14:32:06.158 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:32:06.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.157+0000 7f2e69c00640 1 -- 172.21.15.100:0/492678116 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2e400683b0 msgr2=0x7f2e4006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.157+0000 7f2e69c00640 1 --2- 172.21.15.100:0/492678116 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2e400683b0 0x7f2e4006a870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2e54001750 tx=0x7f2e540011a0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.157+0000 7f2e69c00640 1 -- 172.21.15.100:0/492678116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e641415f0 msgr2=0x7f2e6413e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.157+0000 7f2e69c00640 1 --2- 172.21.15.100:0/492678116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e641415f0 0x7f2e6413e9a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c007fd0 tx=0x7f2e4c00dc00 comp rx=0 tx=0).stop 2024-04-23T14:32:06.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.157+0000 7f2e69c00640 1 -- 172.21.15.100:0/492678116 shutdown_connections 2024-04-23T14:32:06.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.157+0000 7f2e69c00640 1 --2- 172.21.15.100:0/492678116 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2e400683b0 0x7f2e4006a870 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.157+0000 7f2e69c00640 1 --2- 172.21.15.100:0/492678116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e64141a70 0x7f2e6413eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.157+0000 7f2e69c00640 1 --2- 172.21.15.100:0/492678116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e641415f0 0x7f2e6413e9a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.157+0000 7f2e69c00640 1 --2- 172.21.15.100:0/492678116 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2e480039c0 0x7f2e6413bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.157+0000 7f2e69c00640 1 -- 172.21.15.100:0/492678116 >> 172.21.15.100:0/492678116 conn(0x7f2e640704d0 msgr2=0x7f2e640cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:06.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.157+0000 7f2e69c00640 1 -- 172.21.15.100:0/492678116 shutdown_connections 2024-04-23T14:32:06.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.157+0000 7f2e69c00640 1 -- 172.21.15.100:0/492678116 wait complete. 2024-04-23T14:32:06.178 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-23T14:32:06.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.313+0000 7fb608400640 1 Processor -- start 2024-04-23T14:32:06.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.313+0000 7fb608400640 1 -- start start 2024-04-23T14:32:06.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.313+0000 7fb608400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6000cea60 0x7fb6000d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.313+0000 7fb608400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb6000cd460 0x7fb6000b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.313+0000 7fb608400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6000b1f90 con 0x7fb60006d450 2024-04-23T14:32:06.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.313+0000 7fb608400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fb6000b2270 con 0x7fb6000cd460 2024-04-23T14:32:06.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.313+0000 7fb608400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb6000b2550 con 0x7fb6000cea60 2024-04-23T14:32:06.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.313+0000 7fb606400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb6000cd460 0x7fb6000b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.313+0000 7fb606400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb6000cd460 0x7fb6000b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46104/0 (socket says 172.21.15.100:46104) 2024-04-23T14:32:06.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.313+0000 7fb606400640 1 -- 172.21.15.100:0/2802186609 learned_addr learned my addr 172.21.15.100:0/2802186609 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:06.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.313+0000 7fb605000640 1 --2- 172.21.15.100:0/2802186609 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6000cea60 0x7fb6000d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.313+0000 7fb606400640 1 -- 172.21.15.100:0/2802186609 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6000cea60 msgr2=0x7fb6000d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:06.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.313+0000 7fb606400640 1 --2- 172.21.15.100:0/2802186609 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6000cea60 0x7fb6000d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.313+0000 7fb606400640 1 -- 172.21.15.100:0/2802186609 >> v1:172.21.15.151:6789/0 conn(0x7fb60006d450 legacy=0x7fb60006d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.313+0000 7fb606400640 1 -- 172.21.15.100:0/2802186609 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6000b2830 con 0x7fb6000cd460 2024-04-23T14:32:06.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.317+0000 7fb606400640 1 --2- 172.21.15.100:0/2802186609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6000cd460 0x7fb6000b2d50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009870 tx=0x7fb5f801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e139fad3c62074cc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:06.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.317+0000 7fb5ffe00640 1 -- 172.21.15.100:0/2802186609 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f80b0070 con 0x7fb6000cd460 2024-04-23T14:32:06.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.317+0000 7fb5ffe00640 1 -- 172.21.15.100:0/2802186609 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f80ab600 con 0x7fb6000cd460 2024-04-23T14:32:06.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.317+0000 7fb5ffe00640 1 -- 172.21.15.100:0/2802186609 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f80ab9d0 con 0x7fb6000cd460 2024-04-23T14:32:06.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.317+0000 7fb608400640 1 -- 172.21.15.100:0/2802186609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6000cd460 msgr2=0x7fb6000b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.317+0000 7fb608400640 1 --2- 172.21.15.100:0/2802186609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6000cd460 0x7fb6000b2d50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009870 tx=0x7fb5f801c4d0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.317+0000 7fb608400640 1 -- 172.21.15.100:0/2802186609 shutdown_connections 2024-04-23T14:32:06.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.317+0000 7fb608400640 1 --2- 172.21.15.100:0/2802186609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6000cd460 0x7fb6000b2d50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.317+0000 7fb608400640 1 --2- 172.21.15.100:0/2802186609 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6000cea60 0x7fb6000d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.317+0000 7fb608400640 1 -- 172.21.15.100:0/2802186609 >> 172.21.15.100:0/2802186609 conn(0x7fb6000cee80 msgr2=0x7fb6000cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:06.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.317+0000 7fb608400640 1 -- 172.21.15.100:0/2802186609 shutdown_connections 2024-04-23T14:32:06.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.317+0000 7fb608400640 1 -- 172.21.15.100:0/2802186609 wait complete. 2024-04-23T14:32:06.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.317+0000 7fb608400640 1 Processor -- start 2024-04-23T14:32:06.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb608400640 1 -- start start 2024-04-23T14:32:06.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb608400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb60006d450 0x7fb6000c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb608400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6000cd460 0x7fb600152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb608400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6000cea60 0x7fb600157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb608400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb60016b740 con 0x7fb6000cea60 2024-04-23T14:32:06.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb608400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb60016b5c0 con 0x7fb60006d450 2024-04-23T14:32:06.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb608400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb60016b8c0 con 0x7fb6000cd460 2024-04-23T14:32:06.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb606400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6000cea60 0x7fb600157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb606400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6000cea60 0x7fb600157810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46114/0 (socket says 172.21.15.100:46114) 2024-04-23T14:32:06.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb606400640 1 -- 172.21.15.100:0/2694695429 learned_addr learned my addr 172.21.15.100:0/2694695429 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:06.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb605000640 1 --2- 172.21.15.100:0/2694695429 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6000cd460 0x7fb600152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb605a00640 1 --2- 172.21.15.100:0/2694695429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb60006d450 0x7fb6000c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb606400640 1 -- 172.21.15.100:0/2694695429 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6000cd460 msgr2=0x7fb600152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb606400640 1 --2- 172.21.15.100:0/2694695429 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6000cd460 0x7fb600152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb606400640 1 -- 172.21.15.100:0/2694695429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb60006d450 msgr2=0x7fb6000c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb606400640 1 --2- 172.21.15.100:0/2694695429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb60006d450 0x7fb6000c5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb606400640 1 -- 172.21.15.100:0/2694695429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6001fb0f0 con 0x7fb6000cea60 2024-04-23T14:32:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb605a00640 1 --2- 172.21.15.100:0/2694695429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb60006d450 0x7fb6000c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb606400640 1 --2- 172.21.15.100:0/2694695429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6000cea60 0x7fb600157810 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009870 tx=0x7fb5f8019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb5fd600640 1 -- 172.21.15.100:0/2694695429 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f80b0040 con 0x7fb6000cea60 2024-04-23T14:32:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb5fd600640 1 -- 172.21.15.100:0/2694695429 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f80b4420 con 0x7fb6000cea60 2024-04-23T14:32:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb608400640 1 -- 172.21.15.100:0/2694695429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6001fb3d0 con 0x7fb6000cea60 2024-04-23T14:32:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb5fd600640 1 -- 172.21.15.100:0/2694695429 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f80b47d0 con 0x7fb6000cea60 2024-04-23T14:32:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb608400640 1 -- 172.21.15.100:0/2694695429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6000c6450 con 0x7fb6000cea60 2024-04-23T14:32:06.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.321+0000 7fb608400640 1 -- 172.21.15.100:0/2694695429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb600068ad0 con 0x7fb6000cea60 2024-04-23T14:32:06.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.325+0000 7fb5fd600640 1 -- 172.21.15.100:0/2694695429 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb5f80b4970 con 0x7fb6000cea60 2024-04-23T14:32:06.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.325+0000 7fb5fd600640 1 --2- 172.21.15.100:0/2694695429 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb5e0068380 0x7fb5e006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.325+0000 7fb5fd600640 1 -- 172.21.15.100:0/2694695429 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fb5f8120250 con 0x7fb6000cea60 2024-04-23T14:32:06.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.325+0000 7fb605a00640 1 --2- 172.21.15.100:0/2694695429 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb5e0068380 0x7fb5e006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.325+0000 7fb605a00640 1 --2- 172.21.15.100:0/2694695429 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb5e0068380 0x7fb5e006a840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb6001584b0 tx=0x7fb5f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:06.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.329+0000 7fb5fd600640 1 -- 172.21.15.100:0/2694695429 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb5f8016150 con 0x7fb6000cea60 2024-04-23T14:32:06.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.485+0000 7fb608400640 1 -- 172.21.15.100:0/2694695429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb6000727d0 con 0x7fb6000cea60 2024-04-23T14:32:06.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.485+0000 7fb5fd600640 1 -- 172.21.15.100:0/2694695429 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7fb5f80f0b90 con 0x7fb6000cea60 2024-04-23T14:32:06.488 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:06.488 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-04-23T14:32:06.488 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:32:06.488 INFO:teuthology.orchestra.run.smithi100.stdout: "poolnum": 1, 2024-04-23T14:32:06.488 INFO:teuthology.orchestra.run.smithi100.stdout: "poolname": ".mgr" 2024-04-23T14:32:06.488 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:32:06.488 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-04-23T14:32:06.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.489+0000 7fb608400640 1 -- 172.21.15.100:0/2694695429 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb5e0068380 msgr2=0x7fb5e006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.489+0000 7fb608400640 1 --2- 172.21.15.100:0/2694695429 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb5e0068380 0x7fb5e006a840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb6001584b0 tx=0x7fb5f4009040 comp rx=0 tx=0).stop 2024-04-23T14:32:06.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.489+0000 7fb608400640 1 -- 172.21.15.100:0/2694695429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6000cea60 msgr2=0x7fb600157810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.489+0000 7fb608400640 1 --2- 172.21.15.100:0/2694695429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6000cea60 0x7fb600157810 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009870 tx=0x7fb5f8019270 comp rx=0 tx=0).stop 2024-04-23T14:32:06.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.489+0000 7fb608400640 1 -- 172.21.15.100:0/2694695429 shutdown_connections 2024-04-23T14:32:06.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.489+0000 7fb608400640 1 --2- 172.21.15.100:0/2694695429 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb5e0068380 0x7fb5e006a840 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.489+0000 7fb608400640 1 --2- 172.21.15.100:0/2694695429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6000cea60 0x7fb600157810 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.489+0000 7fb608400640 1 --2- 172.21.15.100:0/2694695429 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6000cd460 0x7fb600152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.489+0000 7fb608400640 1 --2- 172.21.15.100:0/2694695429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb60006d450 0x7fb6000c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.489+0000 7fb608400640 1 -- 172.21.15.100:0/2694695429 >> 172.21.15.100:0/2694695429 conn(0x7fb6000cee80 msgr2=0x7fb60006f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:06.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.489+0000 7fb608400640 1 -- 172.21.15.100:0/2694695429 shutdown_connections 2024-04-23T14:32:06.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.489+0000 7fb608400640 1 -- 172.21.15.100:0/2694695429 wait complete. 2024-04-23T14:32:06.508 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-23T14:32:06.508 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-23T14:32:06.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.645+0000 7f2888000640 1 Processor -- start 2024-04-23T14:32:06.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.645+0000 7f2888000640 1 -- start start 2024-04-23T14:32:06.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.645+0000 7f2888000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f288006c6c0 0x7f28800ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.645+0000 7f2888000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f288006dd60 con 0x7f28800ce620 2024-04-23T14:32:06.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.645+0000 7f2888000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28800d09c0 con 0x7f288006f8c0 2024-04-23T14:32:06.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.645+0000 7f2888000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f28800ad050 con 0x7f288006c6c0 2024-04-23T14:32:06.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.645+0000 7f2886000640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f28800ce620 0x7f28800add80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:39408/0 (socket says 172.21.15.100:39408) 2024-04-23T14:32:06.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.645+0000 7f2886000640 1 -- 172.21.15.100:0/1278800536 learned_addr learned my addr 172.21.15.100:0/1278800536 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:06.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.645+0000 7f2884c00640 1 --2- 172.21.15.100:0/1278800536 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f288006c6c0 0x7f28800ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.645+0000 7f287fe00640 1 -- 172.21.15.100:0/1278800536 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3109508558 0 0) 0x7f28780014a0 con 0x7f28800ce620 2024-04-23T14:32:06.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.649+0000 7f287fe00640 1 -- 172.21.15.100:0/1278800536 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2868003530 con 0x7f28800ce620 2024-04-23T14:32:06.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.649+0000 7f2884c00640 1 -- 172.21.15.100:0/1278800536 >> v1:172.21.15.151:6789/0 conn(0x7f288006f8c0 legacy=0x7f28800d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.649+0000 7f287fe00640 1 -- 172.21.15.100:0/1278800536 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1152827931 0 0) 0x7f28800d09c0 con 0x7f288006f8c0 2024-04-23T14:32:06.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.649+0000 7f2884c00640 1 -- 172.21.15.100:0/1278800536 >> v1:172.21.15.100:6789/0 conn(0x7f28800ce620 legacy=0x7f28800add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.649+0000 7f2884c00640 1 -- 172.21.15.100:0/1278800536 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28800ad2a0 con 0x7f288006c6c0 2024-04-23T14:32:06.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.649+0000 7f2884c00640 1 --2- 172.21.15.100:0/1278800536 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f288006c6c0 0x7f28800ce0e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f286c0097f0 tx=0x7f286c01c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b38dbdc96f212df server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:06.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.649+0000 7f287fe00640 1 -- 172.21.15.100:0/1278800536 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c0b0070 con 0x7f288006c6c0 2024-04-23T14:32:06.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.649+0000 7f287fe00640 1 -- 172.21.15.100:0/1278800536 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f286c0ab600 con 0x7f288006c6c0 2024-04-23T14:32:06.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.649+0000 7f287fe00640 1 -- 172.21.15.100:0/1278800536 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c0ab9d0 con 0x7f288006c6c0 2024-04-23T14:32:06.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.649+0000 7f2888000640 1 -- 172.21.15.100:0/1278800536 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f288006c6c0 msgr2=0x7f28800ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.649+0000 7f2888000640 1 --2- 172.21.15.100:0/1278800536 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f288006c6c0 0x7f28800ce0e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f286c0097f0 tx=0x7f286c01c4e0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2888000640 1 -- 172.21.15.100:0/1278800536 shutdown_connections 2024-04-23T14:32:06.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2888000640 1 --2- 172.21.15.100:0/1278800536 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f288006c6c0 0x7f28800ce0e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2888000640 1 -- 172.21.15.100:0/1278800536 >> 172.21.15.100:0/1278800536 conn(0x7f28800b5200 msgr2=0x7f28800b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:06.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2888000640 1 -- 172.21.15.100:0/1278800536 shutdown_connections 2024-04-23T14:32:06.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2888000640 1 -- 172.21.15.100:0/1278800536 wait complete. 2024-04-23T14:32:06.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2888000640 1 Processor -- start 2024-04-23T14:32:06.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2888000640 1 -- start start 2024-04-23T14:32:06.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2888000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f288006c6c0 0x7f28800ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2888000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f288006f8c0 0x7f28800cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2888000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28800ce620 0x7f28800c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2888000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2880169e30 con 0x7f288006c6c0 2024-04-23T14:32:06.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2888000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f28800cea30 con 0x7f288006f8c0 2024-04-23T14:32:06.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2888000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f2880169fb0 con 0x7f28800ce620 2024-04-23T14:32:06.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2885600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f288006c6c0 0x7f28800ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2885600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f288006c6c0 0x7f28800ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46126/0 (socket says 172.21.15.100:46126) 2024-04-23T14:32:06.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2885600640 1 -- 172.21.15.100:0/2267484830 learned_addr learned my addr 172.21.15.100:0/2267484830 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:06.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2886000640 1 --2- 172.21.15.100:0/2267484830 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28800ce620 0x7f28800c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2884c00640 1 --2- 172.21.15.100:0/2267484830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f288006f8c0 0x7f28800cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2885600640 1 -- 172.21.15.100:0/2267484830 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28800ce620 msgr2=0x7f28800c3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2885600640 1 --2- 172.21.15.100:0/2267484830 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28800ce620 0x7f28800c3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2885600640 1 -- 172.21.15.100:0/2267484830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f288006f8c0 msgr2=0x7f28800cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2885600640 1 --2- 172.21.15.100:0/2267484830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f288006f8c0 0x7f28800cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2885600640 1 -- 172.21.15.100:0/2267484830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28800c49d0 con 0x7f288006c6c0 2024-04-23T14:32:06.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2885600640 1 --2- 172.21.15.100:0/2267484830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f288006c6c0 0x7f28800ccc60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f287400c4d0 tx=0x7f287400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:06.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f287d600640 1 -- 172.21.15.100:0/2267484830 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874017070 con 0x7f288006c6c0 2024-04-23T14:32:06.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f287d600640 1 -- 172.21.15.100:0/2267484830 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2874005480 con 0x7f288006c6c0 2024-04-23T14:32:06.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2888000640 1 -- 172.21.15.100:0/2267484830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28800c4cb0 con 0x7f288006c6c0 2024-04-23T14:32:06.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f287d600640 1 -- 172.21.15.100:0/2267484830 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874003840 con 0x7f288006c6c0 2024-04-23T14:32:06.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.653+0000 7f2888000640 1 -- 172.21.15.100:0/2267484830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28800c5270 con 0x7f288006c6c0 2024-04-23T14:32:06.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.657+0000 7f2888000640 1 -- 172.21.15.100:0/2267484830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2880068ad0 con 0x7f288006c6c0 2024-04-23T14:32:06.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.657+0000 7f287d600640 1 -- 172.21.15.100:0/2267484830 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f287401a070 con 0x7f288006c6c0 2024-04-23T14:32:06.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.657+0000 7f287d600640 1 --2- 172.21.15.100:0/2267484830 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2860068380 0x7f286006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.657+0000 7f287d600640 1 -- 172.21.15.100:0/2267484830 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f2874085a70 con 0x7f288006c6c0 2024-04-23T14:32:06.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.657+0000 7f2884c00640 1 --2- 172.21.15.100:0/2267484830 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2860068380 0x7f286006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.657+0000 7f2884c00640 1 --2- 172.21.15.100:0/2267484830 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2860068380 0x7f286006a840 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f286c0ad000 tx=0x7f286c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:06.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.661+0000 7f287d600640 1 -- 172.21.15.100:0/2267484830 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2874056430 con 0x7f288006c6c0 2024-04-23T14:32:06.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:06.821+0000 7f2888000640 1 -- 172.21.15.100:0/2267484830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f288006cc00 con 0x7f288006c6c0 2024-04-23T14:32:06.958 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:32:06.962 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:32:07.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.569+0000 7f287d600640 1 -- 172.21.15.100:0/2267484830 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f2874055d30 con 0x7f288006c6c0 2024-04-23T14:32:07.573 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' created 2024-04-23T14:32:07.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.573+0000 7f2888000640 1 -- 172.21.15.100:0/2267484830 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2860068380 msgr2=0x7f286006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:07.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.573+0000 7f2888000640 1 --2- 172.21.15.100:0/2267484830 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2860068380 0x7f286006a840 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f286c0ad000 tx=0x7f286c0a6000 comp rx=0 tx=0).stop 2024-04-23T14:32:07.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.573+0000 7f2888000640 1 -- 172.21.15.100:0/2267484830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f288006c6c0 msgr2=0x7f28800ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:07.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.573+0000 7f2888000640 1 --2- 172.21.15.100:0/2267484830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f288006c6c0 0x7f28800ccc60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f287400c4d0 tx=0x7f287400c990 comp rx=0 tx=0).stop 2024-04-23T14:32:07.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.573+0000 7f2888000640 1 -- 172.21.15.100:0/2267484830 shutdown_connections 2024-04-23T14:32:07.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.573+0000 7f2888000640 1 --2- 172.21.15.100:0/2267484830 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2860068380 0x7f286006a840 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f286c0ad000 tx=0x7f286c0a6000 comp rx=0 tx=0).stop 2024-04-23T14:32:07.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.573+0000 7f2888000640 1 --2- 172.21.15.100:0/2267484830 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28800ce620 0x7f28800c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.573+0000 7f2888000640 1 --2- 172.21.15.100:0/2267484830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f288006f8c0 0x7f28800cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.573+0000 7f2888000640 1 --2- 172.21.15.100:0/2267484830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f288006c6c0 0x7f28800ccc60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.573+0000 7f2888000640 1 -- 172.21.15.100:0/2267484830 >> 172.21.15.100:0/2267484830 conn(0x7f28800b5200 msgr2=0x7f28800add80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:07.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.573+0000 7f2888000640 1 -- 172.21.15.100:0/2267484830 shutdown_connections 2024-04-23T14:32:07.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.573+0000 7f2888000640 1 -- 172.21.15.100:0/2267484830 wait complete. 2024-04-23T14:32:07.601 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-23T14:32:07.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.741+0000 7fcb2f000640 1 Processor -- start 2024-04-23T14:32:07.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.741+0000 7fcb2f000640 1 -- start start 2024-04-23T14:32:07.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.745+0000 7fcb2f000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcb280afa80 0x7fcb2806c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:07.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.745+0000 7fcb2f000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fcb2806cab0 0x7fcb2806cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:07.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.745+0000 7fcb2f000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb2806f7e0 con 0x7fcb2815c330 2024-04-23T14:32:07.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.745+0000 7fcb2f000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fcb28073300 con 0x7fcb280afa80 2024-04-23T14:32:07.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.745+0000 7fcb2f000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fcb2806d460 con 0x7fcb2806cab0 2024-04-23T14:32:07.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.745+0000 7fcb2d000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fcb2806cab0 0x7fcb2806cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:07.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.745+0000 7fcb27400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcb280afa80 0x7fcb2806c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:07.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.745+0000 7fcb2d000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fcb2806cab0 0x7fcb2806cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:37860/0 (socket says 172.21.15.100:37860) 2024-04-23T14:32:07.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.745+0000 7fcb2d000640 1 -- 172.21.15.100:0/3230111133 learned_addr learned my addr 172.21.15.100:0/3230111133 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:07.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.745+0000 7fcb27400640 1 -- 172.21.15.100:0/3230111133 >> v2:172.21.15.151:3300/0 conn(0x7fcb2806cab0 msgr2=0x7fcb2806cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:07.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.745+0000 7fcb27400640 1 --2- 172.21.15.100:0/3230111133 >> v2:172.21.15.151:3300/0 conn(0x7fcb2806cab0 0x7fcb2806cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.745+0000 7fcb27400640 1 -- 172.21.15.100:0/3230111133 >> v1:172.21.15.151:6789/0 conn(0x7fcb2815c330 legacy=0x7fcb2815c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:07.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.745+0000 7fcb27400640 1 -- 172.21.15.100:0/3230111133 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2806bab0 con 0x7fcb280afa80 2024-04-23T14:32:07.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.749+0000 7fcb27400640 1 --2- 172.21.15.100:0/3230111133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb280afa80 0x7fcb2806c570 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcb280cebf0 tx=0x7fcb1401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94dd093c23d8695 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:07.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.749+0000 7fcb26a00640 1 -- 172.21.15.100:0/3230111133 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb140b0070 con 0x7fcb280afa80 2024-04-23T14:32:07.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.749+0000 7fcb26a00640 1 -- 172.21.15.100:0/3230111133 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb14004230 con 0x7fcb280afa80 2024-04-23T14:32:07.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.749+0000 7fcb26a00640 1 -- 172.21.15.100:0/3230111133 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14003620 con 0x7fcb280afa80 2024-04-23T14:32:07.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.749+0000 7fcb2f000640 1 -- 172.21.15.100:0/3230111133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb280afa80 msgr2=0x7fcb2806c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:07.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.749+0000 7fcb2f000640 1 --2- 172.21.15.100:0/3230111133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb280afa80 0x7fcb2806c570 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcb280cebf0 tx=0x7fcb1401c450 comp rx=0 tx=0).stop 2024-04-23T14:32:07.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.749+0000 7fcb2f000640 1 -- 172.21.15.100:0/3230111133 shutdown_connections 2024-04-23T14:32:07.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.749+0000 7fcb2f000640 1 --2- 172.21.15.100:0/3230111133 >> v2:172.21.15.151:3300/0 conn(0x7fcb2806cab0 0x7fcb2806cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.749+0000 7fcb2f000640 1 --2- 172.21.15.100:0/3230111133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb280afa80 0x7fcb2806c570 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.749+0000 7fcb2f000640 1 -- 172.21.15.100:0/3230111133 >> 172.21.15.100:0/3230111133 conn(0x7fcb280d0860 msgr2=0x7fcb280d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:07.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.749+0000 7fcb2f000640 1 -- 172.21.15.100:0/3230111133 shutdown_connections 2024-04-23T14:32:07.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.749+0000 7fcb2f000640 1 -- 172.21.15.100:0/3230111133 wait complete. 2024-04-23T14:32:07.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.749+0000 7fcb2f000640 1 Processor -- start 2024-04-23T14:32:07.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.749+0000 7fcb2f000640 1 -- start start 2024-04-23T14:32:07.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb2f000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcb2806cab0 0x7fcb2814fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:07.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb2f000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb280afa80 0x7fcb281546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:07.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb2f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb2815c330 0x7fcb280ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:07.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb2f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcb280cfb70 con 0x7fcb2815c330 2024-04-23T14:32:07.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb2f000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2806c050 con 0x7fcb280afa80 2024-04-23T14:32:07.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb2f000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fcb280cfcf0 con 0x7fcb2806cab0 2024-04-23T14:32:07.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb2d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb2815c330 0x7fcb280ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:07.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb2d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb2815c330 0x7fcb280ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46132/0 (socket says 172.21.15.100:46132) 2024-04-23T14:32:07.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb2d000640 1 -- 172.21.15.100:0/3908797357 learned_addr learned my addr 172.21.15.100:0/3908797357 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:07.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb27400640 1 --2- 172.21.15.100:0/3908797357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb280afa80 0x7fcb281546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:07.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb27e00640 1 --2- 172.21.15.100:0/3908797357 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcb2806cab0 0x7fcb2814fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:07.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb27400640 1 -- 172.21.15.100:0/3908797357 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcb2806cab0 msgr2=0x7fcb2814fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:07.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb27400640 1 --2- 172.21.15.100:0/3908797357 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcb2806cab0 0x7fcb2814fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb27400640 1 -- 172.21.15.100:0/3908797357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb2815c330 msgr2=0x7fcb280ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:07.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb27400640 1 --2- 172.21.15.100:0/3908797357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb2815c330 0x7fcb280ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb27400640 1 -- 172.21.15.100:0/3908797357 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb281573a0 con 0x7fcb280afa80 2024-04-23T14:32:07.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb2d000640 1 --2- 172.21.15.100:0/3908797357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb2815c330 0x7fcb280ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:07.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb27400640 1 --2- 172.21.15.100:0/3908797357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb280afa80 0x7fcb281546b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcb280b5c80 tx=0x7fcb140191f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:07.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb1fe00640 1 -- 172.21.15.100:0/3908797357 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb140b0040 con 0x7fcb280afa80 2024-04-23T14:32:07.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb1fe00640 1 -- 172.21.15.100:0/3908797357 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb140bb070 con 0x7fcb280afa80 2024-04-23T14:32:07.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb2f000640 1 -- 172.21.15.100:0/3908797357 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb28155130 con 0x7fcb280afa80 2024-04-23T14:32:07.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb1fe00640 1 -- 172.21.15.100:0/3908797357 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14003e30 con 0x7fcb280afa80 2024-04-23T14:32:07.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb2f000640 1 -- 172.21.15.100:0/3908797357 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb28157940 con 0x7fcb280afa80 2024-04-23T14:32:07.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb2f000640 1 -- 172.21.15.100:0/3908797357 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb28068ad0 con 0x7fcb280afa80 2024-04-23T14:32:07.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.753+0000 7fcb1fe00640 1 -- 172.21.15.100:0/3908797357 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcb140b3070 con 0x7fcb280afa80 2024-04-23T14:32:07.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.757+0000 7fcb1fe00640 1 --2- 172.21.15.100:0/3908797357 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcb08068380 0x7fcb0806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:07.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.757+0000 7fcb1fe00640 1 -- 172.21.15.100:0/3908797357 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7fcb141204e0 con 0x7fcb280afa80 2024-04-23T14:32:07.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.757+0000 7fcb27e00640 1 --2- 172.21.15.100:0/3908797357 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcb08068380 0x7fcb0806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:07.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.757+0000 7fcb27e00640 1 --2- 172.21.15.100:0/3908797357 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcb08068380 0x7fcb0806a840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcb18001770 tx=0x7fcb18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:07.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.761+0000 7fcb1fe00640 1 -- 172.21.15.100:0/3908797357 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcb14016070 con 0x7fcb280afa80 2024-04-23T14:32:07.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:07.917+0000 7fcb2f000640 1 -- 172.21.15.100:0/3908797357 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fcb2806e690 con 0x7fcb280afa80 2024-04-23T14:32:08.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.581+0000 7fcb1fe00640 1 -- 172.21.15.100:0/3908797357 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7fcb140f0500 con 0x7fcb280afa80 2024-04-23T14:32:08.583 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' created 2024-04-23T14:32:08.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.589+0000 7fcb2f000640 1 -- 172.21.15.100:0/3908797357 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcb08068380 msgr2=0x7fcb0806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:08.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.589+0000 7fcb2f000640 1 --2- 172.21.15.100:0/3908797357 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcb08068380 0x7fcb0806a840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcb18001770 tx=0x7fcb18009040 comp rx=0 tx=0).stop 2024-04-23T14:32:08.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.589+0000 7fcb2f000640 1 -- 172.21.15.100:0/3908797357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb280afa80 msgr2=0x7fcb281546b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:08.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.589+0000 7fcb2f000640 1 --2- 172.21.15.100:0/3908797357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb280afa80 0x7fcb281546b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcb280b5c80 tx=0x7fcb140191f0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.589+0000 7fcb2f000640 1 -- 172.21.15.100:0/3908797357 shutdown_connections 2024-04-23T14:32:08.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.589+0000 7fcb2f000640 1 --2- 172.21.15.100:0/3908797357 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcb08068380 0x7fcb0806a840 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.589+0000 7fcb2f000640 1 --2- 172.21.15.100:0/3908797357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb2815c330 0x7fcb280ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.589+0000 7fcb2f000640 1 --2- 172.21.15.100:0/3908797357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb280afa80 0x7fcb281546b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.589+0000 7fcb2f000640 1 --2- 172.21.15.100:0/3908797357 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcb2806cab0 0x7fcb2814fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.589+0000 7fcb2f000640 1 -- 172.21.15.100:0/3908797357 >> 172.21.15.100:0/3908797357 conn(0x7fcb280d0860 msgr2=0x7fcb2815c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:08.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.593+0000 7fcb2f000640 1 -- 172.21.15.100:0/3908797357 shutdown_connections 2024-04-23T14:32:08.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.593+0000 7fcb2f000640 1 -- 172.21.15.100:0/3908797357 wait complete. 2024-04-23T14:32:08.626 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-23T14:32:08.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.765+0000 7fd4d5400640 1 Processor -- start 2024-04-23T14:32:08.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.765+0000 7fd4d5400640 1 -- start start 2024-04-23T14:32:08.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.765+0000 7fd4d5400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd4d015c7a0 0x7fd4d00af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:08.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.765+0000 7fd4d5400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd4d006bb20 0x7fd4d00ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:08.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.765+0000 7fd4d5400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4d006cbb0 con 0x7fd4d00cd130 2024-04-23T14:32:08.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.765+0000 7fd4d5400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fd4d00cd5a0 con 0x7fd4d015c7a0 2024-04-23T14:32:08.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.765+0000 7fd4d5400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d006d450 con 0x7fd4d006bb20 2024-04-23T14:32:08.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.765+0000 7fd4ce800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd4d015c7a0 0x7fd4d00af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:08.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.765+0000 7fd4c7e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd4d006bb20 0x7fd4d00ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:08.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.765+0000 7fd4cf200640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7fd4d00cd130 0x7fd4d006e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:35620/0 (socket says 172.21.15.100:35620) 2024-04-23T14:32:08.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.765+0000 7fd4cf200640 1 -- 172.21.15.100:0/1120588115 learned_addr learned my addr 172.21.15.100:0/1120588115 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:08.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.769+0000 7fd4ce800640 1 -- 172.21.15.100:0/1120588115 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd4d006bb20 msgr2=0x7fd4d00ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:08.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.769+0000 7fd4ce800640 1 --2- 172.21.15.100:0/1120588115 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd4d006bb20 0x7fd4d00ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.769+0000 7fd4ce800640 1 -- 172.21.15.100:0/1120588115 >> v1:172.21.15.151:6789/0 conn(0x7fd4d00cd130 legacy=0x7fd4d006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:08.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.769+0000 7fd4ce800640 1 -- 172.21.15.100:0/1120588115 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d006ce90 con 0x7fd4d015c7a0 2024-04-23T14:32:08.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.769+0000 7fd4c7e00640 1 --2- 172.21.15.100:0/1120588115 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd4d006bb20 0x7fd4d00ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:08.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.769+0000 7fd4ce800640 1 --2- 172.21.15.100:0/1120588115 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4d015c7a0 0x7fd4d00af190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00097c0 tx=0x7fd4c001c2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e418f2feb392352c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:08.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.769+0000 7fd4cde00640 1 -- 172.21.15.100:0/1120588115 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c00b0070 con 0x7fd4d015c7a0 2024-04-23T14:32:08.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.769+0000 7fd4cde00640 1 -- 172.21.15.100:0/1120588115 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c00ab600 con 0x7fd4d015c7a0 2024-04-23T14:32:08.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.769+0000 7fd4cde00640 1 -- 172.21.15.100:0/1120588115 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c00ab9d0 con 0x7fd4d015c7a0 2024-04-23T14:32:08.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4d5400640 1 -- 172.21.15.100:0/1120588115 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4d015c7a0 msgr2=0x7fd4d00af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:08.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4d5400640 1 --2- 172.21.15.100:0/1120588115 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4d015c7a0 0x7fd4d00af190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00097c0 tx=0x7fd4c001c2a0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4d5400640 1 -- 172.21.15.100:0/1120588115 shutdown_connections 2024-04-23T14:32:08.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4d5400640 1 --2- 172.21.15.100:0/1120588115 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd4d006bb20 0x7fd4d00ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4d5400640 1 --2- 172.21.15.100:0/1120588115 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4d015c7a0 0x7fd4d00af190 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4d5400640 1 -- 172.21.15.100:0/1120588115 >> 172.21.15.100:0/1120588115 conn(0x7fd4d00b5200 msgr2=0x7fd4d00b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:08.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4d5400640 1 -- 172.21.15.100:0/1120588115 shutdown_connections 2024-04-23T14:32:08.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4d5400640 1 -- 172.21.15.100:0/1120588115 wait complete. 2024-04-23T14:32:08.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4d5400640 1 Processor -- start 2024-04-23T14:32:08.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4d5400640 1 -- start start 2024-04-23T14:32:08.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4d5400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4d006bb20 0x7fd4d00cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:08.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4d5400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd4d00cd130 0x7fd4d00b2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:08.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4d5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4d015c7a0 0x7fd4d00b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:08.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4d5400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d00af850 con 0x7fd4d015c7a0 2024-04-23T14:32:08.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4d5400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d00af6d0 con 0x7fd4d006bb20 2024-04-23T14:32:08.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4d5400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d00ae440 con 0x7fd4d00cd130 2024-04-23T14:32:08.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4cf200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4d015c7a0 0x7fd4d00b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:08.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4cf200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4d015c7a0 0x7fd4d00b2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48584/0 (socket says 172.21.15.100:48584) 2024-04-23T14:32:08.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4cf200640 1 -- 172.21.15.100:0/2943859505 learned_addr learned my addr 172.21.15.100:0/2943859505 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:08.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4ce800640 1 --2- 172.21.15.100:0/2943859505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4d006bb20 0x7fd4d00cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:08.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4c7e00640 1 --2- 172.21.15.100:0/2943859505 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd4d00cd130 0x7fd4d00b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:08.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4cf200640 1 -- 172.21.15.100:0/2943859505 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd4d00cd130 msgr2=0x7fd4d00b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:08.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4cf200640 1 --2- 172.21.15.100:0/2943859505 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd4d00cd130 0x7fd4d00b2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4cf200640 1 -- 172.21.15.100:0/2943859505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4d006bb20 msgr2=0x7fd4d00cb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:08.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4cf200640 1 --2- 172.21.15.100:0/2943859505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4d006bb20 0x7fd4d00cb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.773+0000 7fd4cf200640 1 -- 172.21.15.100:0/2943859505 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d00afd60 con 0x7fd4d015c7a0 2024-04-23T14:32:08.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.777+0000 7fd4cf200640 1 --2- 172.21.15.100:0/2943859505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4d015c7a0 0x7fd4d00b2770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd4c800e7b0 tx=0x7fd4c800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:08.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.777+0000 7fd4c6a00640 1 -- 172.21.15.100:0/2943859505 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c800f930 con 0x7fd4d015c7a0 2024-04-23T14:32:08.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.777+0000 7fd4c6a00640 1 -- 172.21.15.100:0/2943859505 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c8013070 con 0x7fd4d015c7a0 2024-04-23T14:32:08.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.777+0000 7fd4c6a00640 1 -- 172.21.15.100:0/2943859505 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c800fcd0 con 0x7fd4d015c7a0 2024-04-23T14:32:08.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.777+0000 7fd4d5400640 1 -- 172.21.15.100:0/2943859505 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d00b06d0 con 0x7fd4d015c7a0 2024-04-23T14:32:08.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.777+0000 7fd4d5400640 1 -- 172.21.15.100:0/2943859505 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d00cc770 con 0x7fd4d015c7a0 2024-04-23T14:32:08.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.777+0000 7fd4d5400640 1 -- 172.21.15.100:0/2943859505 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d01590a0 con 0x7fd4d015c7a0 2024-04-23T14:32:08.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.777+0000 7fd4c6a00640 1 -- 172.21.15.100:0/2943859505 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd4c80090d0 con 0x7fd4d015c7a0 2024-04-23T14:32:08.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.777+0000 7fd4c6a00640 1 --2- 172.21.15.100:0/2943859505 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd4ac068380 0x7fd4ac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:08.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.777+0000 7fd4c6a00640 1 -- 172.21.15.100:0/2943859505 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7fd4c8089780 con 0x7fd4d015c7a0 2024-04-23T14:32:08.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.777+0000 7fd4ce800640 1 --2- 172.21.15.100:0/2943859505 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd4ac068380 0x7fd4ac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:08.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.781+0000 7fd4ce800640 1 --2- 172.21.15.100:0/2943859505 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd4ac068380 0x7fd4ac06a840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00ad000 tx=0x7fd4c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:08.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.781+0000 7fd4c6a00640 1 -- 172.21.15.100:0/2943859505 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd4c8027080 con 0x7fd4d015c7a0 2024-04-23T14:32:08.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:08.973+0000 7fd4d5400640 1 -- 172.21.15.100:0/2943859505 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd4d0068ad0 con 0x7fd4d015c7a0 2024-04-23T14:32:09.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.601+0000 7fd4c6a00640 1 -- 172.21.15.100:0/2943859505 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fd4c80598c0 con 0x7fd4d015c7a0 2024-04-23T14:32:09.603 INFO:teuthology.orchestra.run.smithi100.stderr:new fs with metadata pool 2 and data pool 3 2024-04-23T14:32:09.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.609+0000 7fd4d5400640 1 -- 172.21.15.100:0/2943859505 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd4ac068380 msgr2=0x7fd4ac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:09.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.609+0000 7fd4d5400640 1 --2- 172.21.15.100:0/2943859505 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd4ac068380 0x7fd4ac06a840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00ad000 tx=0x7fd4c0008040 comp rx=0 tx=0).stop 2024-04-23T14:32:09.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.609+0000 7fd4d5400640 1 -- 172.21.15.100:0/2943859505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4d015c7a0 msgr2=0x7fd4d00b2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:09.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.609+0000 7fd4d5400640 1 --2- 172.21.15.100:0/2943859505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4d015c7a0 0x7fd4d00b2770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd4c800e7b0 tx=0x7fd4c800ec70 comp rx=0 tx=0).stop 2024-04-23T14:32:09.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.609+0000 7fd4d5400640 1 -- 172.21.15.100:0/2943859505 shutdown_connections 2024-04-23T14:32:09.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.609+0000 7fd4d5400640 1 --2- 172.21.15.100:0/2943859505 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd4ac068380 0x7fd4ac06a840 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.609+0000 7fd4d5400640 1 --2- 172.21.15.100:0/2943859505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4d015c7a0 0x7fd4d00b2770 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.609+0000 7fd4d5400640 1 --2- 172.21.15.100:0/2943859505 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd4d00cd130 0x7fd4d00b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.609+0000 7fd4d5400640 1 --2- 172.21.15.100:0/2943859505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4d006bb20 0x7fd4d00cb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.609+0000 7fd4d5400640 1 -- 172.21.15.100:0/2943859505 >> 172.21.15.100:0/2943859505 conn(0x7fd4d00b5200 msgr2=0x7fd4d00d06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:09.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.617+0000 7fd4d5400640 1 -- 172.21.15.100:0/2943859505 shutdown_connections 2024-04-23T14:32:09.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.617+0000 7fd4d5400640 1 -- 172.21.15.100:0/2943859505 wait complete. 2024-04-23T14:32:09.648 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-23T14:32:09.648 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-23T14:32:09.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.785+0000 7f3bd0c00640 1 Processor -- start 2024-04-23T14:32:09.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.785+0000 7f3bd0c00640 1 -- start start 2024-04-23T14:32:09.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.789+0000 7f3bd0c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bcc15e860 0x7f3bcc0b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:09.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.789+0000 7f3bd0c00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f3bcc0adb90 0x7f3bcc0adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:09.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.789+0000 7f3bd0c00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bcc06e5f0 con 0x7f3bcc06f950 2024-04-23T14:32:09.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.789+0000 7f3bd0c00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f3bcc06e8b0 con 0x7f3bcc0adb90 2024-04-23T14:32:09.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.789+0000 7f3bd0c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f3bcc06eb90 con 0x7f3bcc15e860 2024-04-23T14:32:09.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.789+0000 7f3bca800640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f3bcc06f950 0x7f3bcc06d350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:34128/0 (socket says 172.21.15.100:34128) 2024-04-23T14:32:09.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.789+0000 7f3bca800640 1 -- 172.21.15.100:0/3246488860 learned_addr learned my addr 172.21.15.100:0/3246488860 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:09.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.789+0000 7f3bc1400640 1 --2- 172.21.15.100:0/3246488860 >> v2:172.21.15.151:3300/0 conn(0x7f3bcc0adb90 0x7f3bcc0adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:09.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.789+0000 7f3bc9e00640 1 --2- 172.21.15.100:0/3246488860 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bcc15e860 0x7f3bcc0b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:09.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.789+0000 7f3bc9400640 1 -- 172.21.15.100:0/3246488860 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1661252506 0 0) 0x7f3bcc06e5f0 con 0x7f3bcc06f950 2024-04-23T14:32:09.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.789+0000 7f3bc9400640 1 -- 172.21.15.100:0/3246488860 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3bb0003530 con 0x7f3bcc06f950 2024-04-23T14:32:09.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.789+0000 7f3bc9400640 1 -- 172.21.15.100:0/3246488860 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 270575767 0 0) 0x7f3bb0003530 con 0x7f3bcc06f950 2024-04-23T14:32:09.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.789+0000 7f3bc1400640 1 -- 172.21.15.100:0/3246488860 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bcc15e860 msgr2=0x7f3bcc0b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:09.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.789+0000 7f3bc1400640 1 --2- 172.21.15.100:0/3246488860 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bcc15e860 0x7f3bcc0b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.789+0000 7f3bc1400640 1 -- 172.21.15.100:0/3246488860 >> v1:172.21.15.100:6789/0 conn(0x7f3bcc06f950 legacy=0x7f3bcc06d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:09.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.789+0000 7f3bc1400640 1 -- 172.21.15.100:0/3246488860 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bcc06ee70 con 0x7f3bcc0adb90 2024-04-23T14:32:09.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.793+0000 7f3bc1400640 1 --2- 172.21.15.100:0/3246488860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bcc0adb90 0x7f3bcc0adfb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40097c0 tx=0x7f3bb401c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f3454d10c887d31 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:09.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.793+0000 7f3bc9400640 1 -- 172.21.15.100:0/3246488860 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb40b0070 con 0x7f3bcc0adb90 2024-04-23T14:32:09.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.793+0000 7f3bc9400640 1 -- 172.21.15.100:0/3246488860 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bb40ab400 con 0x7f3bcc0adb90 2024-04-23T14:32:09.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.793+0000 7f3bc9400640 1 -- 172.21.15.100:0/3246488860 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4001680 con 0x7f3bcc0adb90 2024-04-23T14:32:09.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.793+0000 7f3bd0c00640 1 -- 172.21.15.100:0/3246488860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bcc0adb90 msgr2=0x7f3bcc0adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:09.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.793+0000 7f3bd0c00640 1 --2- 172.21.15.100:0/3246488860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bcc0adb90 0x7f3bcc0adfb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40097c0 tx=0x7f3bb401c390 comp rx=0 tx=0).stop 2024-04-23T14:32:09.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.793+0000 7f3bd0c00640 1 -- 172.21.15.100:0/3246488860 shutdown_connections 2024-04-23T14:32:09.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.793+0000 7f3bd0c00640 1 --2- 172.21.15.100:0/3246488860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bcc0adb90 0x7f3bcc0adfb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.793+0000 7f3bd0c00640 1 --2- 172.21.15.100:0/3246488860 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bcc15e860 0x7f3bcc0b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.793+0000 7f3bd0c00640 1 -- 172.21.15.100:0/3246488860 >> 172.21.15.100:0/3246488860 conn(0x7f3bcc072590 msgr2=0x7f3bcc0729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:09.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.793+0000 7f3bd0c00640 1 -- 172.21.15.100:0/3246488860 shutdown_connections 2024-04-23T14:32:09.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.793+0000 7f3bd0c00640 1 -- 172.21.15.100:0/3246488860 wait complete. 2024-04-23T14:32:09.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.793+0000 7f3bd0c00640 1 Processor -- start 2024-04-23T14:32:09.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.793+0000 7f3bd0c00640 1 -- start start 2024-04-23T14:32:09.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bd0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bcc06f950 0x7f3bcc1400c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:09.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bd0c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bcc0adb90 0x7f3bcc151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:09.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bd0c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bcc15e860 0x7f3bcc151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:09.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bd0c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc15ec40 con 0x7f3bcc06f950 2024-04-23T14:32:09.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bd0c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc073790 con 0x7f3bcc0adb90 2024-04-23T14:32:09.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bd0c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f3bcc06da60 con 0x7f3bcc15e860 2024-04-23T14:32:09.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bc9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bcc06f950 0x7f3bcc1400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:09.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bc9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bcc06f950 0x7f3bcc1400c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48596/0 (socket says 172.21.15.100:48596) 2024-04-23T14:32:09.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bc9e00640 1 -- 172.21.15.100:0/2234381216 learned_addr learned my addr 172.21.15.100:0/2234381216 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bca800640 1 --2- 172.21.15.100:0/2234381216 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bcc15e860 0x7f3bcc151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bc9e00640 1 -- 172.21.15.100:0/2234381216 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bcc15e860 msgr2=0x7f3bcc151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bc1400640 1 --2- 172.21.15.100:0/2234381216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bcc0adb90 0x7f3bcc151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bc9e00640 1 --2- 172.21.15.100:0/2234381216 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bcc15e860 0x7f3bcc151750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bc9e00640 1 -- 172.21.15.100:0/2234381216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bcc0adb90 msgr2=0x7f3bcc151210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bc9e00640 1 --2- 172.21.15.100:0/2234381216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bcc0adb90 0x7f3bcc151210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bc9e00640 1 -- 172.21.15.100:0/2234381216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bcc1faaa0 con 0x7f3bcc06f950 2024-04-23T14:32:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bc1400640 1 --2- 172.21.15.100:0/2234381216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bcc0adb90 0x7f3bcc151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bc9e00640 1 --2- 172.21.15.100:0/2234381216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bcc06f950 0x7f3bcc1400c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00efc0 tx=0x7f3bbc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bc2a00640 1 -- 172.21.15.100:0/2234381216 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc018020 con 0x7f3bcc06f950 2024-04-23T14:32:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bc2a00640 1 -- 172.21.15.100:0/2234381216 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bbc014480 con 0x7f3bcc06f950 2024-04-23T14:32:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bc2a00640 1 -- 172.21.15.100:0/2234381216 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc014850 con 0x7f3bcc06f950 2024-04-23T14:32:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bd0c00640 1 -- 172.21.15.100:0/2234381216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bcc1fad60 con 0x7f3bcc06f950 2024-04-23T14:32:09.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bd0c00640 1 -- 172.21.15.100:0/2234381216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bcc1fb320 con 0x7f3bcc06f950 2024-04-23T14:32:09.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bd0c00640 1 -- 172.21.15.100:0/2234381216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bcc068ad0 con 0x7f3bcc06f950 2024-04-23T14:32:09.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.797+0000 7f3bc2a00640 1 -- 172.21.15.100:0/2234381216 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3bbc01b020 con 0x7f3bcc06f950 2024-04-23T14:32:09.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.801+0000 7f3bc2a00640 1 --2- 172.21.15.100:0/2234381216 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3ba8068380 0x7f3ba806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:09.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.801+0000 7f3bc2a00640 1 -- 172.21.15.100:0/2234381216 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f3bbc0874b0 con 0x7f3bcc06f950 2024-04-23T14:32:09.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.801+0000 7f3bc1400640 1 --2- 172.21.15.100:0/2234381216 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3ba8068380 0x7f3ba806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:09.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.801+0000 7f3bc1400640 1 --2- 172.21.15.100:0/2234381216 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3ba8068380 0x7f3ba806a840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40ad000 tx=0x7f3bb4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:09.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.805+0000 7f3bc2a00640 1 -- 172.21.15.100:0/2234381216 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3bbc022070 con 0x7f3bcc06f950 2024-04-23T14:32:09.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:09.969+0000 7f3bd0c00640 1 -- 172.21.15.100:0/2234381216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f3bcc1621e0 con 0x7f3bcc06f950 2024-04-23T14:32:10.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.605+0000 7f3bc2a00640 1 -- 172.21.15.100:0/2234381216 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7f3bbc010040 con 0x7f3bcc06f950 2024-04-23T14:32:10.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.609+0000 7f3bd0c00640 1 -- 172.21.15.100:0/2234381216 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3ba8068380 msgr2=0x7f3ba806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:10.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.609+0000 7f3bd0c00640 1 --2- 172.21.15.100:0/2234381216 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3ba8068380 0x7f3ba806a840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40ad000 tx=0x7f3bb4008040 comp rx=0 tx=0).stop 2024-04-23T14:32:10.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.609+0000 7f3bd0c00640 1 -- 172.21.15.100:0/2234381216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bcc06f950 msgr2=0x7f3bcc1400c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:10.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.609+0000 7f3bd0c00640 1 --2- 172.21.15.100:0/2234381216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bcc06f950 0x7f3bcc1400c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00efc0 tx=0x7f3bbc00c490 comp rx=0 tx=0).stop 2024-04-23T14:32:10.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.609+0000 7f3bd0c00640 1 -- 172.21.15.100:0/2234381216 shutdown_connections 2024-04-23T14:32:10.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.609+0000 7f3bd0c00640 1 --2- 172.21.15.100:0/2234381216 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3ba8068380 0x7f3ba806a840 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.609+0000 7f3bd0c00640 1 --2- 172.21.15.100:0/2234381216 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bcc15e860 0x7f3bcc151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.609+0000 7f3bd0c00640 1 --2- 172.21.15.100:0/2234381216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bcc0adb90 0x7f3bcc151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.609+0000 7f3bd0c00640 1 --2- 172.21.15.100:0/2234381216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bcc06f950 0x7f3bcc1400c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.609+0000 7f3bd0c00640 1 -- 172.21.15.100:0/2234381216 >> 172.21.15.100:0/2234381216 conn(0x7f3bcc072590 msgr2=0x7f3bcc15ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:10.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.609+0000 7f3bd0c00640 1 -- 172.21.15.100:0/2234381216 shutdown_connections 2024-04-23T14:32:10.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.609+0000 7f3bd0c00640 1 -- 172.21.15.100:0/2234381216 wait complete. 2024-04-23T14:32:10.633 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-23T14:32:10.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.777+0000 7f6896400640 1 Processor -- start 2024-04-23T14:32:10.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.777+0000 7f6896400640 1 -- start start 2024-04-23T14:32:10.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.777+0000 7f6896400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f689006d450 0x7f689006d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:10.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.777+0000 7f6896400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f68900cea60 0x7f68900d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:10.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.777+0000 7f6896400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f689006dd70 con 0x7f68900cd460 2024-04-23T14:32:10.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.777+0000 7f6896400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f68900b1f90 con 0x7f68900cea60 2024-04-23T14:32:10.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.777+0000 7f6896400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f68900b2270 con 0x7f689006d450 2024-04-23T14:32:10.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.777+0000 7f688ea00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f68900cea60 0x7f68900d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:10.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.777+0000 7f688ea00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f68900cea60 0x7f68900d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48608/0 (socket says 172.21.15.100:48608) 2024-04-23T14:32:10.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.777+0000 7f688ea00640 1 -- 172.21.15.100:0/822361366 learned_addr learned my addr 172.21.15.100:0/822361366 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:10.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.777+0000 7f688f400640 1 --2- 172.21.15.100:0/822361366 >> v2:172.21.15.151:3300/0 conn(0x7f689006d450 0x7f689006d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:10.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.777+0000 7f688ea00640 1 -- 172.21.15.100:0/822361366 >> v2:172.21.15.151:3300/0 conn(0x7f689006d450 msgr2=0x7f689006d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:10.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.777+0000 7f688ea00640 1 --2- 172.21.15.100:0/822361366 >> v2:172.21.15.151:3300/0 conn(0x7f689006d450 0x7f689006d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.777+0000 7f688ea00640 1 -- 172.21.15.100:0/822361366 >> v1:172.21.15.151:6789/0 conn(0x7f68900cd460 legacy=0x7f68900b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:10.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.777+0000 7f688ea00640 1 -- 172.21.15.100:0/822361366 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68900b2550 con 0x7f68900cea60 2024-04-23T14:32:10.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.781+0000 7f688ea00640 1 --2- 172.21.15.100:0/822361366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68900cea60 0x7f68900d0300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6890068600 tx=0x7f688001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b43a8fd010abaf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:10.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.781+0000 7f688e000640 1 -- 172.21.15.100:0/822361366 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68800b0070 con 0x7f68900cea60 2024-04-23T14:32:10.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.781+0000 7f688e000640 1 -- 172.21.15.100:0/822361366 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68800ab600 con 0x7f68900cea60 2024-04-23T14:32:10.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.781+0000 7f688e000640 1 -- 172.21.15.100:0/822361366 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68800ab9d0 con 0x7f68900cea60 2024-04-23T14:32:10.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.781+0000 7f6896400640 1 -- 172.21.15.100:0/822361366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68900cea60 msgr2=0x7f68900d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:10.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.781+0000 7f6896400640 1 --2- 172.21.15.100:0/822361366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68900cea60 0x7f68900d0300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6890068600 tx=0x7f688001c450 comp rx=0 tx=0).stop 2024-04-23T14:32:10.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.781+0000 7f6896400640 1 -- 172.21.15.100:0/822361366 shutdown_connections 2024-04-23T14:32:10.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.781+0000 7f6896400640 1 --2- 172.21.15.100:0/822361366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68900cea60 0x7f68900d0300 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.781+0000 7f6896400640 1 --2- 172.21.15.100:0/822361366 >> v2:172.21.15.151:3300/0 conn(0x7f689006d450 0x7f689006d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.781+0000 7f6896400640 1 -- 172.21.15.100:0/822361366 >> 172.21.15.100:0/822361366 conn(0x7f68900cee80 msgr2=0x7f68900cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:10.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.781+0000 7f6896400640 1 -- 172.21.15.100:0/822361366 shutdown_connections 2024-04-23T14:32:10.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.781+0000 7f6896400640 1 -- 172.21.15.100:0/822361366 wait complete. 2024-04-23T14:32:10.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f6896400640 1 Processor -- start 2024-04-23T14:32:10.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f6896400640 1 -- start start 2024-04-23T14:32:10.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f6896400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f689006d450 0x7f689015a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:10.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f6896400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f68900cd460 0x7f68900c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:10.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f6896400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68900cea60 0x7f6890160330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:10.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f6896400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f689015fc70 con 0x7f68900cea60 2024-04-23T14:32:10.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f6896400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f689015faf0 con 0x7f689006d450 2024-04-23T14:32:10.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f6896400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f689016bbc0 con 0x7f68900cd460 2024-04-23T14:32:10.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f688fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68900cea60 0x7f6890160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:10.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f688fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68900cea60 0x7f6890160330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48618/0 (socket says 172.21.15.100:48618) 2024-04-23T14:32:10.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f688fe00640 1 -- 172.21.15.100:0/250675216 learned_addr learned my addr 172.21.15.100:0/250675216 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:10.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f688f400640 1 --2- 172.21.15.100:0/250675216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f689006d450 0x7f689015a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:10.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f688ea00640 1 --2- 172.21.15.100:0/250675216 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f68900cd460 0x7f68900c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:10.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f688fe00640 1 -- 172.21.15.100:0/250675216 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f68900cd460 msgr2=0x7f68900c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:10.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f688fe00640 1 --2- 172.21.15.100:0/250675216 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f68900cd460 0x7f68900c7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f688fe00640 1 -- 172.21.15.100:0/250675216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f689006d450 msgr2=0x7f689015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:10.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f688fe00640 1 --2- 172.21.15.100:0/250675216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f689006d450 0x7f689015a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f688fe00640 1 -- 172.21.15.100:0/250675216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6890155de0 con 0x7f68900cea60 2024-04-23T14:32:10.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f688fe00640 1 --2- 172.21.15.100:0/250675216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68900cea60 0x7f6890160330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f688800c780 tx=0x7f688800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:10.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f6887400640 1 -- 172.21.15.100:0/250675216 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888004b70 con 0x7f68900cea60 2024-04-23T14:32:10.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f6887400640 1 -- 172.21.15.100:0/250675216 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6888004d10 con 0x7f68900cea60 2024-04-23T14:32:10.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f6887400640 1 -- 172.21.15.100:0/250675216 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688800aa90 con 0x7f68900cea60 2024-04-23T14:32:10.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f6896400640 1 -- 172.21.15.100:0/250675216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68901560c0 con 0x7f68900cea60 2024-04-23T14:32:10.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f6896400640 1 -- 172.21.15.100:0/250675216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6890156680 con 0x7f68900cea60 2024-04-23T14:32:10.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.785+0000 7f6896400640 1 -- 172.21.15.100:0/250675216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6890068ad0 con 0x7f68900cea60 2024-04-23T14:32:10.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.789+0000 7f6887400640 1 -- 172.21.15.100:0/250675216 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f688800ac30 con 0x7f68900cea60 2024-04-23T14:32:10.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.789+0000 7f6887400640 1 --2- 172.21.15.100:0/250675216 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f686c068380 0x7f686c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:10.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.789+0000 7f6887400640 1 -- 172.21.15.100:0/250675216 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7f6888086cf0 con 0x7f68900cea60 2024-04-23T14:32:10.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.789+0000 7f688f400640 1 --2- 172.21.15.100:0/250675216 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f686c068380 0x7f686c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:10.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.789+0000 7f688f400640 1 --2- 172.21.15.100:0/250675216 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f686c068380 0x7f686c06a840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f68780079c0 tx=0x7f6878008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:10.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.793+0000 7f6887400640 1 -- 172.21.15.100:0/250675216 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6888020070 con 0x7f68900cea60 2024-04-23T14:32:10.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:10.949+0000 7f6896400640 1 -- 172.21.15.100:0/250675216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f689015dc30 con 0x7f68900cea60 2024-04-23T14:32:12.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.625+0000 7f6887400640 1 -- 172.21.15.100:0/250675216 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7f6888056a90 con 0x7f68900cea60 2024-04-23T14:32:12.626 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data_ec' created 2024-04-23T14:32:12.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.629+0000 7f6896400640 1 -- 172.21.15.100:0/250675216 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f686c068380 msgr2=0x7f686c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:12.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.629+0000 7f6896400640 1 --2- 172.21.15.100:0/250675216 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f686c068380 0x7f686c06a840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f68780079c0 tx=0x7f6878008040 comp rx=0 tx=0).stop 2024-04-23T14:32:12.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.629+0000 7f6896400640 1 -- 172.21.15.100:0/250675216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68900cea60 msgr2=0x7f6890160330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:12.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.629+0000 7f6896400640 1 --2- 172.21.15.100:0/250675216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68900cea60 0x7f6890160330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f688800c780 tx=0x7f688800cc40 comp rx=0 tx=0).stop 2024-04-23T14:32:12.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.629+0000 7f6896400640 1 -- 172.21.15.100:0/250675216 shutdown_connections 2024-04-23T14:32:12.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.629+0000 7f6896400640 1 --2- 172.21.15.100:0/250675216 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f686c068380 0x7f686c06a840 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.629+0000 7f6896400640 1 --2- 172.21.15.100:0/250675216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68900cea60 0x7f6890160330 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.629+0000 7f6896400640 1 --2- 172.21.15.100:0/250675216 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f68900cd460 0x7f68900c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.629+0000 7f6896400640 1 --2- 172.21.15.100:0/250675216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f689006d450 0x7f689015a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.629+0000 7f6896400640 1 -- 172.21.15.100:0/250675216 >> 172.21.15.100:0/250675216 conn(0x7f68900cee80 msgr2=0x7f689015d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:12.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.629+0000 7f6896400640 1 -- 172.21.15.100:0/250675216 shutdown_connections 2024-04-23T14:32:12.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.629+0000 7f6896400640 1 -- 172.21.15.100:0/250675216 wait complete. 2024-04-23T14:32:12.664 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-23T14:32:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.805+0000 7f807de00640 1 Processor -- start 2024-04-23T14:32:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.805+0000 7f807de00640 1 -- start start 2024-04-23T14:32:12.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.805+0000 7f807de00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f807815e7a0 0x7f80780aea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:12.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.805+0000 7f807de00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f80780cee40 0x7f80780cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:12.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.805+0000 7f807de00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80780ada30 con 0x7f80780cf7c0 2024-04-23T14:32:12.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.805+0000 7f807de00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f80780aefb0 con 0x7f80780cee40 2024-04-23T14:32:12.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.805+0000 7f807de00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f80780adcb0 con 0x7f807815e7a0 2024-04-23T14:32:12.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.805+0000 7f806e000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f80780cee40 0x7f80780cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:12.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.805+0000 7f806e000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f80780cee40 0x7f80780cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48634/0 (socket says 172.21.15.100:48634) 2024-04-23T14:32:12.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.805+0000 7f806e000640 1 -- 172.21.15.100:0/8747079 learned_addr learned my addr 172.21.15.100:0/8747079 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:12.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.805+0000 7f8076a00640 1 --2- 172.21.15.100:0/8747079 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f807815e7a0 0x7f80780aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:12.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.809+0000 7f806e000640 1 -- 172.21.15.100:0/8747079 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f807815e7a0 msgr2=0x7f80780aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:12.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.809+0000 7f806e000640 1 --2- 172.21.15.100:0/8747079 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f807815e7a0 0x7f80780aea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.809+0000 7f806e000640 1 -- 172.21.15.100:0/8747079 >> v1:172.21.15.151:6789/0 conn(0x7f80780cf7c0 legacy=0x7f807806e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:12.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.809+0000 7f806e000640 1 -- 172.21.15.100:0/8747079 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80780adf90 con 0x7f80780cee40 2024-04-23T14:32:12.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.809+0000 7f806e000640 1 --2- 172.21.15.100:0/8747079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80780cee40 0x7f80780cf280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8060009870 tx=0x7f806001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a259a8704f1eb418 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:12.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.809+0000 7f8076000640 1 -- 172.21.15.100:0/8747079 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80600a9620 con 0x7f80780cee40 2024-04-23T14:32:12.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.809+0000 7f8076000640 1 -- 172.21.15.100:0/8747079 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80600a97c0 con 0x7f80780cee40 2024-04-23T14:32:12.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.809+0000 7f8076000640 1 -- 172.21.15.100:0/8747079 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80600a9b90 con 0x7f80780cee40 2024-04-23T14:32:12.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f807de00640 1 -- 172.21.15.100:0/8747079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80780cee40 msgr2=0x7f80780cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:12.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f807de00640 1 --2- 172.21.15.100:0/8747079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80780cee40 0x7f80780cf280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8060009870 tx=0x7f806001c560 comp rx=0 tx=0).stop 2024-04-23T14:32:12.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f807de00640 1 -- 172.21.15.100:0/8747079 shutdown_connections 2024-04-23T14:32:12.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f807de00640 1 --2- 172.21.15.100:0/8747079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80780cee40 0x7f80780cf280 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f807de00640 1 --2- 172.21.15.100:0/8747079 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f807815e7a0 0x7f80780aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f807de00640 1 -- 172.21.15.100:0/8747079 >> 172.21.15.100:0/8747079 conn(0x7f80780b5200 msgr2=0x7f80780b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:12.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f807de00640 1 -- 172.21.15.100:0/8747079 shutdown_connections 2024-04-23T14:32:12.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f807de00640 1 -- 172.21.15.100:0/8747079 wait complete. 2024-04-23T14:32:12.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f807de00640 1 Processor -- start 2024-04-23T14:32:12.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f807de00640 1 -- start start 2024-04-23T14:32:12.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f807de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80780cee40 0x7f80781549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:12.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f807de00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f80780cf7c0 0x7f8078096f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:12.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f807de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807815e7a0 0x7f807815cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:12.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f807de00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f807815ed00 con 0x7f807815e7a0 2024-04-23T14:32:12.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f807de00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f807815eb80 con 0x7f80780cee40 2024-04-23T14:32:12.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f807de00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f807816b400 con 0x7f80780cf7c0 2024-04-23T14:32:12.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f8077400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807815e7a0 0x7f807815cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:12.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f8077400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807815e7a0 0x7f807815cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48644/0 (socket says 172.21.15.100:48644) 2024-04-23T14:32:12.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f8077400640 1 -- 172.21.15.100:0/2926961768 learned_addr learned my addr 172.21.15.100:0/2926961768 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:12.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f8076a00640 1 --2- 172.21.15.100:0/2926961768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80780cee40 0x7f80781549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:12.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f806e000640 1 --2- 172.21.15.100:0/2926961768 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f80780cf7c0 0x7f8078096f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:12.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f8077400640 1 -- 172.21.15.100:0/2926961768 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f80780cf7c0 msgr2=0x7f8078096f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:12.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f8077400640 1 --2- 172.21.15.100:0/2926961768 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f80780cf7c0 0x7f8078096f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f8077400640 1 -- 172.21.15.100:0/2926961768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80780cee40 msgr2=0x7f80781549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:12.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f8077400640 1 --2- 172.21.15.100:0/2926961768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80780cee40 0x7f80781549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f8077400640 1 -- 172.21.15.100:0/2926961768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80780943d0 con 0x7f807815e7a0 2024-04-23T14:32:12.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f8077400640 1 --2- 172.21.15.100:0/2926961768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807815e7a0 0x7f807815cdb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f807000c430 tx=0x7f807000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:12.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.813+0000 7f806fe00640 1 -- 172.21.15.100:0/2926961768 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070005220 con 0x7f807815e7a0 2024-04-23T14:32:12.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.817+0000 7f806fe00640 1 -- 172.21.15.100:0/2926961768 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80700053c0 con 0x7f807815e7a0 2024-04-23T14:32:12.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.817+0000 7f806fe00640 1 -- 172.21.15.100:0/2926961768 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070007270 con 0x7f807815e7a0 2024-04-23T14:32:12.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.817+0000 7f807de00640 1 -- 172.21.15.100:0/2926961768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80780d07f0 con 0x7f807815e7a0 2024-04-23T14:32:12.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.817+0000 7f807de00640 1 -- 172.21.15.100:0/2926961768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80780969f0 con 0x7f807815e7a0 2024-04-23T14:32:12.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.817+0000 7f807de00640 1 -- 172.21.15.100:0/2926961768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8078068ad0 con 0x7f807815e7a0 2024-04-23T14:32:12.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.817+0000 7f806fe00640 1 -- 172.21.15.100:0/2926961768 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8070019020 con 0x7f807815e7a0 2024-04-23T14:32:12.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.817+0000 7f806fe00640 1 --2- 172.21.15.100:0/2926961768 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8054068380 0x7f805406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:12.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.817+0000 7f806fe00640 1 -- 172.21.15.100:0/2926961768 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7f8070086160 con 0x7f807815e7a0 2024-04-23T14:32:12.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.817+0000 7f8076a00640 1 --2- 172.21.15.100:0/2926961768 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8054068380 0x7f805406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:12.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.817+0000 7f8076a00640 1 --2- 172.21.15.100:0/2926961768 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8054068380 0x7f805406a840 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8078158730 tx=0x7f806800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:12.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.825+0000 7f806fe00640 1 -- 172.21.15.100:0/2926961768 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f80780ada30 con 0x7f807815e7a0 2024-04-23T14:32:12.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:12.981+0000 7f807de00640 1 -- 172.21.15.100:0/2926961768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f8078099e40 con 0x7f807815e7a0 2024-04-23T14:32:13.214 INFO:tasks.ceph.mds.d.smithi151.stderr:2024-04-23T14:32:13.211+0000 7f70b3000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:32:13.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.629+0000 7f806fe00640 1 -- 172.21.15.100:0/2926961768 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7f8070056360 con 0x7f807815e7a0 2024-04-23T14:32:13.632 INFO:teuthology.orchestra.run.smithi100.stderr:set pool 4 allow_ec_overwrites to true 2024-04-23T14:32:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.633+0000 7f807de00640 1 -- 172.21.15.100:0/2926961768 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8054068380 msgr2=0x7f805406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.633+0000 7f807de00640 1 --2- 172.21.15.100:0/2926961768 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8054068380 0x7f805406a840 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8078158730 tx=0x7f806800b040 comp rx=0 tx=0).stop 2024-04-23T14:32:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.633+0000 7f807de00640 1 -- 172.21.15.100:0/2926961768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807815e7a0 msgr2=0x7f807815cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.633+0000 7f807de00640 1 --2- 172.21.15.100:0/2926961768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807815e7a0 0x7f807815cdb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f807000c430 tx=0x7f807000c8f0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.633+0000 7f807de00640 1 -- 172.21.15.100:0/2926961768 shutdown_connections 2024-04-23T14:32:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.633+0000 7f807de00640 1 --2- 172.21.15.100:0/2926961768 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8054068380 0x7f805406a840 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.633+0000 7f807de00640 1 --2- 172.21.15.100:0/2926961768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807815e7a0 0x7f807815cdb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.633+0000 7f807de00640 1 --2- 172.21.15.100:0/2926961768 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f80780cf7c0 0x7f8078096f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.633+0000 7f807de00640 1 --2- 172.21.15.100:0/2926961768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80780cee40 0x7f80781549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.633+0000 7f807de00640 1 -- 172.21.15.100:0/2926961768 >> 172.21.15.100:0/2926961768 conn(0x7f80780b5200 msgr2=0x7f80780b55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.637+0000 7f807de00640 1 -- 172.21.15.100:0/2926961768 shutdown_connections 2024-04-23T14:32:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.637+0000 7f807de00640 1 -- 172.21.15.100:0/2926961768 wait complete. 2024-04-23T14:32:13.684 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-23T14:32:13.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91e1200640 1 Processor -- start 2024-04-23T14:32:13.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91e1200640 1 -- start start 2024-04-23T14:32:13.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91e1200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f91dc0cc960 0x7f91dc0ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91e1200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f91dc160080 0x7f91dc160510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91e1200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91dc0b1e00 con 0x7f91dc156a00 2024-04-23T14:32:13.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91e1200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f91dc0b2a10 con 0x7f91dc160080 2024-04-23T14:32:13.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91e1200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f91dc0b20e0 con 0x7f91dc0cc960 2024-04-23T14:32:13.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91db400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f91dc156a00 0x7f91dc15d370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:34142/0 (socket says 172.21.15.100:34142) 2024-04-23T14:32:13.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91db400640 1 -- 172.21.15.100:0/3387454559 learned_addr learned my addr 172.21.15.100:0/3387454559 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:13.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91dbe00640 1 --2- 172.21.15.100:0/3387454559 >> v2:172.21.15.151:3300/0 conn(0x7f91dc160080 0x7f91dc160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91d2a00640 1 --2- 172.21.15.100:0/3387454559 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f91dc0cc960 0x7f91dc0ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91daa00640 1 -- 172.21.15.100:0/3387454559 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3600410859 0 0) 0x7f91dc0b1e00 con 0x7f91dc156a00 2024-04-23T14:32:13.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91daa00640 1 -- 172.21.15.100:0/3387454559 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91c0003530 con 0x7f91dc156a00 2024-04-23T14:32:13.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91daa00640 1 -- 172.21.15.100:0/3387454559 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1087935915 0 0) 0x7f91c0003530 con 0x7f91dc156a00 2024-04-23T14:32:13.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91daa00640 1 -- 172.21.15.100:0/3387454559 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f91dc0cc960 msgr2=0x7f91dc0ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:13.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91daa00640 1 --2- 172.21.15.100:0/3387454559 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f91dc0cc960 0x7f91dc0ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91daa00640 1 -- 172.21.15.100:0/3387454559 >> v2:172.21.15.151:3300/0 conn(0x7f91dc160080 msgr2=0x7f91dc160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91daa00640 1 --2- 172.21.15.100:0/3387454559 >> v2:172.21.15.151:3300/0 conn(0x7f91dc160080 0x7f91dc160510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91daa00640 1 -- 172.21.15.100:0/3387454559 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc0b23c0 con 0x7f91dc156a00 2024-04-23T14:32:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91daa00640 1 -- 172.21.15.100:0/3387454559 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f91cc001680 con 0x7f91dc156a00 2024-04-23T14:32:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91d2a00640 1 --2- 172.21.15.100:0/3387454559 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f91dc0cc960 0x7f91dc0ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91daa00640 1 -- 172.21.15.100:0/3387454559 >> v1:172.21.15.100:6789/0 conn(0x7f91dc156a00 legacy=0x7f91dc15d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91daa00640 1 --2- 172.21.15.100:0/3387454559 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f91dc0cc960 0x7f91c00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91daa00640 1 --2- 172.21.15.100:0/3387454559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91c0003f00 0x7f91c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91daa00640 1 --2- 172.21.15.100:0/3387454559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91c0004d50 0x7f91c00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91daa00640 1 -- 172.21.15.100:0/3387454559 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc0b2a10 con 0x7f91c0004d50 2024-04-23T14:32:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91daa00640 1 -- 172.21.15.100:0/3387454559 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc0b20e0 con 0x7f91c0003f00 2024-04-23T14:32:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91daa00640 1 -- 172.21.15.100:0/3387454559 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f91c000b9d0 con 0x7f91dc0cc960 2024-04-23T14:32:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91dbe00640 1 --2- 172.21.15.100:0/3387454559 >> v2:172.21.15.151:3300/0 conn(0x7f91dc160080 0x7f91dc160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91db400640 1 --2- 172.21.15.100:0/3387454559 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f91dc0cc960 0x7f91c00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91dbe00640 1 --2- 172.21.15.100:0/3387454559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91c0004d50 0x7f91c00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.825+0000 7f91d2a00640 1 --2- 172.21.15.100:0/3387454559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91c0003f00 0x7f91c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.829+0000 7f91dbe00640 1 -- 172.21.15.100:0/3387454559 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f91dc0cc960 msgr2=0x7f91c00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.829+0000 7f91dbe00640 1 --2- 172.21.15.100:0/3387454559 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f91dc0cc960 0x7f91c00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.829+0000 7f91dbe00640 1 -- 172.21.15.100:0/3387454559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91c0003f00 msgr2=0x7f91c0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.829+0000 7f91dbe00640 1 --2- 172.21.15.100:0/3387454559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91c0003f00 0x7f91c0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.829+0000 7f91dbe00640 1 -- 172.21.15.100:0/3387454559 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c0003530 con 0x7f91c0004d50 2024-04-23T14:32:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.829+0000 7f91db400640 1 --2- 172.21.15.100:0/3387454559 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f91dc0cc960 0x7f91c00039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:13.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.829+0000 7f91dbe00640 1 --2- 172.21.15.100:0/3387454559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91c0004d50 0x7f91c00051d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f91d400e350 tx=0x7f91d401ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23ca3dc8d64fdd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:13.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.829+0000 7f91daa00640 1 -- 172.21.15.100:0/3387454559 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4009dc0 con 0x7f91c0004d50 2024-04-23T14:32:13.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.829+0000 7f91daa00640 1 -- 172.21.15.100:0/3387454559 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d40039a0 con 0x7f91c0004d50 2024-04-23T14:32:13.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.829+0000 7f91daa00640 1 -- 172.21.15.100:0/3387454559 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4003dd0 con 0x7f91c0004d50 2024-04-23T14:32:13.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.829+0000 7f91e1200640 1 -- 172.21.15.100:0/3387454559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91c0004d50 msgr2=0x7f91c00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.829+0000 7f91e1200640 1 --2- 172.21.15.100:0/3387454559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91c0004d50 0x7f91c00051d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f91d400e350 tx=0x7f91d401ce60 comp rx=0 tx=0).stop 2024-04-23T14:32:13.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.829+0000 7f91dbe00640 1 -- 172.21.15.100:0/3387454559 reap_dead start 2024-04-23T14:32:13.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.829+0000 7f91e1200640 1 -- 172.21.15.100:0/3387454559 shutdown_connections 2024-04-23T14:32:13.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.829+0000 7f91e1200640 1 -- 172.21.15.100:0/3387454559 >> 172.21.15.100:0/3387454559 conn(0x7f91dc072140 msgr2=0x7f91dc0b3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:13.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91e1200640 1 -- 172.21.15.100:0/3387454559 shutdown_connections 2024-04-23T14:32:13.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91e1200640 1 -- 172.21.15.100:0/3387454559 wait complete. 2024-04-23T14:32:13.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91e1200640 1 Processor -- start 2024-04-23T14:32:13.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91e1200640 1 -- start start 2024-04-23T14:32:13.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91e1200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f91c0004d50 0x7f91dc1f7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91e1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91dc0b7a90 0x7f91dc1592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91e1200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91dc0b7f10 0x7f91dc1597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91e1200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc156f60 con 0x7f91dc0b7a90 2024-04-23T14:32:13.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91e1200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc156de0 con 0x7f91dc0b7f10 2024-04-23T14:32:13.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91e1200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f91dc0ccde0 con 0x7f91c0004d50 2024-04-23T14:32:13.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91d2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91dc0b7a90 0x7f91dc1592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91d2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91dc0b7a90 0x7f91dc1592b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48652/0 (socket says 172.21.15.100:48652) 2024-04-23T14:32:13.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91d2a00640 1 -- 172.21.15.100:0/445416524 learned_addr learned my addr 172.21.15.100:0/445416524 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:13.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91db400640 1 --2- 172.21.15.100:0/445416524 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f91c0004d50 0x7f91dc1f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91dbe00640 1 --2- 172.21.15.100:0/445416524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91dc0b7f10 0x7f91dc1597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91d2a00640 1 -- 172.21.15.100:0/445416524 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f91c0004d50 msgr2=0x7f91dc1f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91d2a00640 1 --2- 172.21.15.100:0/445416524 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f91c0004d50 0x7f91dc1f7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91d2a00640 1 -- 172.21.15.100:0/445416524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91dc0b7f10 msgr2=0x7f91dc1597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91d2a00640 1 --2- 172.21.15.100:0/445416524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91dc0b7f10 0x7f91dc1597f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91d2a00640 1 -- 172.21.15.100:0/445416524 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc15d0b0 con 0x7f91dc0b7a90 2024-04-23T14:32:13.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91db400640 1 --2- 172.21.15.100:0/445416524 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f91c0004d50 0x7f91dc1f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:13.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91d2a00640 1 --2- 172.21.15.100:0/445416524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91dc0b7a90 0x7f91dc1592b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f91c4011a00 tx=0x7f91c4011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:13.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91d3e00640 1 -- 172.21.15.100:0/445416524 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c400fb50 con 0x7f91dc0b7a90 2024-04-23T14:32:13.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91d3e00640 1 -- 172.21.15.100:0/445416524 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c400fcf0 con 0x7f91dc0b7a90 2024-04-23T14:32:13.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91d3e00640 1 -- 172.21.15.100:0/445416524 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c400fea0 con 0x7f91dc0b7a90 2024-04-23T14:32:13.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.833+0000 7f91e1200640 1 -- 172.21.15.100:0/445416524 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91dc0c8770 con 0x7f91dc0b7a90 2024-04-23T14:32:13.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.837+0000 7f91e1200640 1 -- 172.21.15.100:0/445416524 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91dc0c8d30 con 0x7f91dc0b7a90 2024-04-23T14:32:13.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.837+0000 7f91e1200640 1 -- 172.21.15.100:0/445416524 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91dc068ad0 con 0x7f91dc0b7a90 2024-04-23T14:32:13.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.837+0000 7f91d3e00640 1 -- 172.21.15.100:0/445416524 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f91c4012040 con 0x7f91dc0b7a90 2024-04-23T14:32:13.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.837+0000 7f91d3e00640 1 --2- 172.21.15.100:0/445416524 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f91b80683b0 0x7f91b806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.837+0000 7f91d3e00640 1 -- 172.21.15.100:0/445416524 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f91c40871a0 con 0x7f91dc0b7a90 2024-04-23T14:32:13.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.837+0000 7f91db400640 1 --2- 172.21.15.100:0/445416524 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f91b80683b0 0x7f91b806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.837+0000 7f91db400640 1 --2- 172.21.15.100:0/445416524 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f91b80683b0 0x7f91b806a870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f91cc000c10 tx=0x7f91cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:13.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:13.841+0000 7f91d3e00640 1 -- 172.21.15.100:0/445416524 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f91c4019070 con 0x7f91dc0b7a90 2024-04-23T14:32:14.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.033+0000 7f91e1200640 1 -- 172.21.15.100:0/445416524 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f91dc0b3480 con 0x7f91dc0b7a90 2024-04-23T14:32:14.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.637+0000 7f91d3e00640 1 -- 172.21.15.100:0/445416524 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f91c40573a0 con 0x7f91dc0b7a90 2024-04-23T14:32:14.640 INFO:teuthology.orchestra.run.smithi100.stderr:added data pool 4 to fsmap 2024-04-23T14:32:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.641+0000 7f91e1200640 1 -- 172.21.15.100:0/445416524 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f91b80683b0 msgr2=0x7f91b806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.641+0000 7f91e1200640 1 --2- 172.21.15.100:0/445416524 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f91b80683b0 0x7f91b806a870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f91cc000c10 tx=0x7f91cc009040 comp rx=0 tx=0).stop 2024-04-23T14:32:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.641+0000 7f91e1200640 1 -- 172.21.15.100:0/445416524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91dc0b7a90 msgr2=0x7f91dc1592b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.641+0000 7f91e1200640 1 --2- 172.21.15.100:0/445416524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91dc0b7a90 0x7f91dc1592b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f91c4011a00 tx=0x7f91c4011ec0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.641+0000 7f91e1200640 1 -- 172.21.15.100:0/445416524 shutdown_connections 2024-04-23T14:32:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.641+0000 7f91e1200640 1 --2- 172.21.15.100:0/445416524 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f91b80683b0 0x7f91b806a870 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.641+0000 7f91e1200640 1 --2- 172.21.15.100:0/445416524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91dc0b7f10 0x7f91dc1597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.641+0000 7f91e1200640 1 --2- 172.21.15.100:0/445416524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91dc0b7a90 0x7f91dc1592b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.641+0000 7f91e1200640 1 --2- 172.21.15.100:0/445416524 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f91c0004d50 0x7f91dc1f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.641+0000 7f91e1200640 1 -- 172.21.15.100:0/445416524 >> 172.21.15.100:0/445416524 conn(0x7f91dc072140 msgr2=0x7f91dc15de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.641+0000 7f91e1200640 1 -- 172.21.15.100:0/445416524 shutdown_connections 2024-04-23T14:32:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.641+0000 7f91e1200640 1 -- 172.21.15.100:0/445416524 wait complete. 2024-04-23T14:32:14.662 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:32:14.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.797+0000 7f9a08200640 1 Processor -- start 2024-04-23T14:32:14.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.797+0000 7f9a08200640 1 -- start start 2024-04-23T14:32:14.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.797+0000 7f9a08200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9a0015b640 0x7f9a000adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:14.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.797+0000 7f9a08200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f9a00073610 0x7f9a0006df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:14.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.797+0000 7f9a08200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a000af080 con 0x7f9a000b2a10 2024-04-23T14:32:14.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.797+0000 7f9a08200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f9a000afbc0 con 0x7f9a00073610 2024-04-23T14:32:14.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.797+0000 7f9a08200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9a0015ba20 con 0x7f9a0015b640 2024-04-23T14:32:14.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.797+0000 7f9a04e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f9a000b2a10 0x7f9a00072f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:34154/0 (socket says 172.21.15.100:34154) 2024-04-23T14:32:14.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.797+0000 7f9a04e00640 1 -- 172.21.15.100:0/3752292396 learned_addr learned my addr 172.21.15.100:0/3752292396 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:14.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.797+0000 7f9a06200640 1 --2- 172.21.15.100:0/3752292396 >> v2:172.21.15.151:3300/0 conn(0x7f9a00073610 0x7f9a0006df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:14.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.797+0000 7f9a05800640 1 --2- 172.21.15.100:0/3752292396 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9a0015b640 0x7f9a000adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:14.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.797+0000 7f99ffe00640 1 -- 172.21.15.100:0/3752292396 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2700325129 0 0) 0x7f9a000af080 con 0x7f9a000b2a10 2024-04-23T14:32:14.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.797+0000 7f99ffe00640 1 -- 172.21.15.100:0/3752292396 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99e8005740 con 0x7f9a000b2a10 2024-04-23T14:32:14.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.797+0000 7f99ffe00640 1 -- 172.21.15.100:0/3752292396 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2439644582 0 0) 0x7f99e8005740 con 0x7f9a000b2a10 2024-04-23T14:32:14.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.797+0000 7f99ffe00640 1 -- 172.21.15.100:0/3752292396 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9a0015b640 msgr2=0x7f9a000adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:14.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.797+0000 7f99ffe00640 1 --2- 172.21.15.100:0/3752292396 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9a0015b640 0x7f9a000adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.797+0000 7f99ffe00640 1 -- 172.21.15.100:0/3752292396 >> v2:172.21.15.151:3300/0 conn(0x7f9a00073610 msgr2=0x7f9a0006df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:14.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.797+0000 7f99ffe00640 1 --2- 172.21.15.100:0/3752292396 >> v2:172.21.15.151:3300/0 conn(0x7f9a00073610 0x7f9a0006df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.797+0000 7f99ffe00640 1 -- 172.21.15.100:0/3752292396 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a000af320 con 0x7f9a000b2a10 2024-04-23T14:32:14.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.797+0000 7f99ffe00640 1 -- 172.21.15.100:0/3752292396 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f99ec001680 con 0x7f9a000b2a10 2024-04-23T14:32:14.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.797+0000 7f9a05800640 1 --2- 172.21.15.100:0/3752292396 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9a0015b640 0x7f9a000adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:14.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.797+0000 7f99ffe00640 1 -- 172.21.15.100:0/3752292396 >> v1:172.21.15.100:6789/0 conn(0x7f9a000b2a10 legacy=0x7f9a00072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:14.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.801+0000 7f99ffe00640 1 --2- 172.21.15.100:0/3752292396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99e8005bd0 0x7f99e8005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:14.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.801+0000 7f99ffe00640 1 --2- 172.21.15.100:0/3752292396 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9a0015b640 0x7f99e8006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:14.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.801+0000 7f99ffe00640 1 --2- 172.21.15.100:0/3752292396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99e8006f60 0x7f99e80073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:14.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.801+0000 7f99ffe00640 1 -- 172.21.15.100:0/3752292396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9a000afbc0 con 0x7f99e8005bd0 2024-04-23T14:32:14.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.801+0000 7f99ffe00640 1 -- 172.21.15.100:0/3752292396 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0015ba20 con 0x7f99e8006f60 2024-04-23T14:32:14.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.801+0000 7f99ffe00640 1 -- 172.21.15.100:0/3752292396 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f99e800dbe0 con 0x7f9a0015b640 2024-04-23T14:32:14.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.801+0000 7f9a06200640 1 --2- 172.21.15.100:0/3752292396 >> v2:172.21.15.151:3300/0 conn(0x7f9a00073610 0x7f9a0006df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:14.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.801+0000 7f9a05800640 1 --2- 172.21.15.100:0/3752292396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99e8005bd0 0x7f99e8005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:14.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.801+0000 7f9a04e00640 1 --2- 172.21.15.100:0/3752292396 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9a0015b640 0x7f99e8006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:14.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.801+0000 7f9a06200640 1 --2- 172.21.15.100:0/3752292396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99e8006f60 0x7f99e80073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:14.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.801+0000 7f9a05800640 1 -- 172.21.15.100:0/3752292396 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9a0015b640 msgr2=0x7f99e8006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:14.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.801+0000 7f9a05800640 1 --2- 172.21.15.100:0/3752292396 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9a0015b640 0x7f99e8006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.801+0000 7f9a05800640 1 -- 172.21.15.100:0/3752292396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99e8006f60 msgr2=0x7f99e80073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:14.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.801+0000 7f9a05800640 1 --2- 172.21.15.100:0/3752292396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99e8006f60 0x7f99e80073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.801+0000 7f9a05800640 1 -- 172.21.15.100:0/3752292396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e8005740 con 0x7f99e8005bd0 2024-04-23T14:32:14.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.801+0000 7f9a05800640 1 --2- 172.21.15.100:0/3752292396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99e8005bd0 0x7f99e8005ff0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009cb0 tx=0x7f99f401cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de29d3919f84c9e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:14.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.801+0000 7f99ffe00640 1 -- 172.21.15.100:0/3752292396 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f40b2070 con 0x7f99e8005bd0 2024-04-23T14:32:14.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.801+0000 7f99ffe00640 1 -- 172.21.15.100:0/3752292396 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f40a5880 con 0x7f99e8005bd0 2024-04-23T14:32:14.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.801+0000 7f99ffe00640 1 -- 172.21.15.100:0/3752292396 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4009880 con 0x7f99e8005bd0 2024-04-23T14:32:14.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a08200640 1 -- 172.21.15.100:0/3752292396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99e8005bd0 msgr2=0x7f99e8005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:14.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a08200640 1 --2- 172.21.15.100:0/3752292396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99e8005bd0 0x7f99e8005ff0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009cb0 tx=0x7f99f401cbe0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a06200640 1 -- 172.21.15.100:0/3752292396 reap_dead start 2024-04-23T14:32:14.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a08200640 1 -- 172.21.15.100:0/3752292396 shutdown_connections 2024-04-23T14:32:14.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a08200640 1 -- 172.21.15.100:0/3752292396 >> 172.21.15.100:0/3752292396 conn(0x7f9a000b1e80 msgr2=0x7f9a000b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:14.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a08200640 1 -- 172.21.15.100:0/3752292396 shutdown_connections 2024-04-23T14:32:14.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a08200640 1 -- 172.21.15.100:0/3752292396 wait complete. 2024-04-23T14:32:14.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a08200640 1 Processor -- start 2024-04-23T14:32:14.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a08200640 1 -- start start 2024-04-23T14:32:14.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a08200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99e8005bd0 0x7f9a000999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:14.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a08200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0015e890 0x7f9a001e78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:14.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a08200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9a001e8e00 0x7f9a001e9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:14.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a08200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0006f0d0 con 0x7f9a0015e890 2024-04-23T14:32:14.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a08200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0006ef50 con 0x7f99e8005bd0 2024-04-23T14:32:14.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a08200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9a0006f250 con 0x7f9a001e8e00 2024-04-23T14:32:14.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a04e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0015e890 0x7f9a001e78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:14.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a04e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0015e890 0x7f9a001e78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48670/0 (socket says 172.21.15.100:48670) 2024-04-23T14:32:14.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a04e00640 1 -- 172.21.15.100:0/3350752999 learned_addr learned my addr 172.21.15.100:0/3350752999 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:14.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a05800640 1 --2- 172.21.15.100:0/3350752999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99e8005bd0 0x7f9a000999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:14.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a06200640 1 --2- 172.21.15.100:0/3350752999 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9a001e8e00 0x7f9a001e9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:14.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a04e00640 1 -- 172.21.15.100:0/3350752999 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9a001e8e00 msgr2=0x7f9a001e9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:14.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a04e00640 1 --2- 172.21.15.100:0/3350752999 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9a001e8e00 0x7f9a001e9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a04e00640 1 -- 172.21.15.100:0/3350752999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99e8005bd0 msgr2=0x7f9a000999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:14.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a04e00640 1 --2- 172.21.15.100:0/3350752999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99e8005bd0 0x7f9a000999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a04e00640 1 -- 172.21.15.100:0/3350752999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a001e8790 con 0x7f9a0015e890 2024-04-23T14:32:14.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a06200640 1 --2- 172.21.15.100:0/3350752999 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9a001e8e00 0x7f9a001e9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:14.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.805+0000 7f9a04e00640 1 --2- 172.21.15.100:0/3350752999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0015e890 0x7f9a001e78a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f99ec007310 tx=0x7f99ec0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:14.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.809+0000 7f99fd600640 1 -- 172.21.15.100:0/3350752999 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec00f9c0 con 0x7f9a0015e890 2024-04-23T14:32:14.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.809+0000 7f99fd600640 1 -- 172.21.15.100:0/3350752999 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99ec018070 con 0x7f9a0015e890 2024-04-23T14:32:14.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.809+0000 7f99fd600640 1 -- 172.21.15.100:0/3350752999 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec00fdc0 con 0x7f9a0015e890 2024-04-23T14:32:14.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.809+0000 7f9a08200640 1 -- 172.21.15.100:0/3350752999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a0015bd90 con 0x7f9a0015e890 2024-04-23T14:32:14.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.809+0000 7f9a08200640 1 -- 172.21.15.100:0/3350752999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a001e84b0 con 0x7f9a0015e890 2024-04-23T14:32:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.809+0000 7f9a08200640 1 -- 172.21.15.100:0/3350752999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a00068ad0 con 0x7f9a0015e890 2024-04-23T14:32:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.809+0000 7f99fd600640 1 -- 172.21.15.100:0/3350752999 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f99ec01d460 con 0x7f9a0015e890 2024-04-23T14:32:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.809+0000 7f99fd600640 1 --2- 172.21.15.100:0/3350752999 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f99e006c790 0x7f99e006ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.809+0000 7f99fd600640 1 -- 172.21.15.100:0/3350752999 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f99ec086d70 con 0x7f9a0015e890 2024-04-23T14:32:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.809+0000 7f9a05800640 1 --2- 172.21.15.100:0/3350752999 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f99e006c790 0x7f99e006ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.809+0000 7f9a05800640 1 --2- 172.21.15.100:0/3350752999 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f99e006c790 0x7f99e006ec50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f99f40af000 tx=0x7f99f40046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:14.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:14.813+0000 7f99fd600640 1 -- 172.21.15.100:0/3350752999 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f99ec057080 con 0x7f9a0015e890 2024-04-23T14:32:15.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.001+0000 7f9a08200640 1 -- 172.21.15.100:0/3350752999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a000b6230 con 0x7f9a0015e890 2024-04-23T14:32:15.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.005+0000 7f99fd600640 1 -- 172.21.15.100:0/3350752999 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5189 (secure 0 0 0) 0x7f99ec056980 con 0x7f9a0015e890 2024-04-23T14:32:15.005 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:15.006 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1456958549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1456958549},{"type":"v1","addr":"172.21.15.100:6836","nonce":1456958549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3655609551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3655609551},{"type":"v1","addr":"172.21.15.100:6837","nonce":3655609551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/4242180972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":4242180972},{"type":"v1","addr":"172.21.15.151:6834","nonce":4242180972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:32:08.976576+0000","modified":"2024-04-23T14:32:14.037173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1533322173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1533322173},{"type":"v1","addr":"172.21.15.151:6835","nonce":1533322173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-23T14:32:15.006 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 7 2024-04-23T14:32:15.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.005+0000 7f9a08200640 1 -- 172.21.15.100:0/3350752999 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f99e006c790 msgr2=0x7f99e006ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.005+0000 7f9a08200640 1 --2- 172.21.15.100:0/3350752999 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f99e006c790 0x7f99e006ec50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f99f40af000 tx=0x7f99f40046d0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.005+0000 7f9a08200640 1 -- 172.21.15.100:0/3350752999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0015e890 msgr2=0x7f9a001e78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.005+0000 7f9a08200640 1 --2- 172.21.15.100:0/3350752999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0015e890 0x7f9a001e78a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f99ec007310 tx=0x7f99ec0077d0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.005+0000 7f9a08200640 1 -- 172.21.15.100:0/3350752999 shutdown_connections 2024-04-23T14:32:15.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.005+0000 7f9a08200640 1 --2- 172.21.15.100:0/3350752999 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f99e006c790 0x7f99e006ec50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.005+0000 7f9a08200640 1 --2- 172.21.15.100:0/3350752999 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9a001e8e00 0x7f9a001e9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.005+0000 7f9a08200640 1 --2- 172.21.15.100:0/3350752999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0015e890 0x7f9a001e78a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.005+0000 7f9a08200640 1 --2- 172.21.15.100:0/3350752999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99e8005bd0 0x7f9a000999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.005+0000 7f9a08200640 1 -- 172.21.15.100:0/3350752999 >> 172.21.15.100:0/3350752999 conn(0x7f9a000b1e80 msgr2=0x7f9a0015eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:15.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.005+0000 7f9a08200640 1 -- 172.21.15.100:0/3350752999 shutdown_connections 2024-04-23T14:32:15.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.005+0000 7f9a08200640 1 -- 172.21.15.100:0/3350752999 wait complete. 2024-04-23T14:32:15.026 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:15.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.157+0000 7f0c01200640 1 Processor -- start 2024-04-23T14:32:15.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.157+0000 7f0c01200640 1 -- start start 2024-04-23T14:32:15.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.161+0000 7f0c01200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bfc15e5a0 0x7f0bfc1621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.161+0000 7f0c01200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f0bfc0ced00 0x7f0bfc0cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.161+0000 7f0c01200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bfc162a20 con 0x7f0bfc0b3930 2024-04-23T14:32:15.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.161+0000 7f0c01200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f0bfc162d00 con 0x7f0bfc0ced00 2024-04-23T14:32:15.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.161+0000 7f0c01200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0bfc162fe0 con 0x7f0bfc15e5a0 2024-04-23T14:32:15.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.161+0000 7f0bfbe00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f0bfc0ced00 0x7f0bfc0cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.161+0000 7f0bfb400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bfc15e5a0 0x7f0bfc1621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.161+0000 7f0bfaa00640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f0bfc0b3930 0x7f0bfc06df80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:35642/0 (socket says 172.21.15.100:35642) 2024-04-23T14:32:15.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.161+0000 7f0bfaa00640 1 -- 172.21.15.100:0/2278343894 learned_addr learned my addr 172.21.15.100:0/2278343894 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:15.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.161+0000 7f0bfbe00640 1 -- 172.21.15.100:0/2278343894 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bfc15e5a0 msgr2=0x7f0bfc1621d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:15.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.161+0000 7f0bfbe00640 1 --2- 172.21.15.100:0/2278343894 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bfc15e5a0 0x7f0bfc1621d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.161+0000 7f0bfbe00640 1 -- 172.21.15.100:0/2278343894 >> v1:172.21.15.151:6789/0 conn(0x7f0bfc0b3930 legacy=0x7f0bfc06df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.161+0000 7f0bfbe00640 1 -- 172.21.15.100:0/2278343894 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bfc06d780 con 0x7f0bfc0ced00 2024-04-23T14:32:15.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.165+0000 7f0bfbe00640 1 --2- 172.21.15.100:0/2278343894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bfc0ced00 0x7f0bfc0cf190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40097c0 tx=0x7f0bf401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a9f2e7c26ce890b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:15.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.165+0000 7f0bfa000640 1 -- 172.21.15.100:0/2278343894 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf40b0070 con 0x7f0bfc0ced00 2024-04-23T14:32:15.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.165+0000 7f0bfa000640 1 -- 172.21.15.100:0/2278343894 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bf4004230 con 0x7f0bfc0ced00 2024-04-23T14:32:15.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.165+0000 7f0bfa000640 1 -- 172.21.15.100:0/2278343894 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4003620 con 0x7f0bfc0ced00 2024-04-23T14:32:15.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.165+0000 7f0c01200640 1 -- 172.21.15.100:0/2278343894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bfc0ced00 msgr2=0x7f0bfc0cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.165+0000 7f0c01200640 1 --2- 172.21.15.100:0/2278343894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bfc0ced00 0x7f0bfc0cf190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40097c0 tx=0x7f0bf401c4a0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.165+0000 7f0c01200640 1 -- 172.21.15.100:0/2278343894 shutdown_connections 2024-04-23T14:32:15.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.165+0000 7f0c01200640 1 --2- 172.21.15.100:0/2278343894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bfc0ced00 0x7f0bfc0cf190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.165+0000 7f0c01200640 1 --2- 172.21.15.100:0/2278343894 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bfc15e5a0 0x7f0bfc1621d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.165+0000 7f0c01200640 1 -- 172.21.15.100:0/2278343894 >> 172.21.15.100:0/2278343894 conn(0x7f0bfc0b3e60 msgr2=0x7f0bfc0b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:15.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.165+0000 7f0c01200640 1 -- 172.21.15.100:0/2278343894 shutdown_connections 2024-04-23T14:32:15.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.165+0000 7f0c01200640 1 -- 172.21.15.100:0/2278343894 wait complete. 2024-04-23T14:32:15.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.165+0000 7f0c01200640 1 Processor -- start 2024-04-23T14:32:15.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.165+0000 7f0c01200640 1 -- start start 2024-04-23T14:32:15.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0c01200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bfc0b3930 0x7f0bfc15f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0c01200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bfc0ced00 0x7f0bfc15b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0c01200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bfc15e5a0 0x7f0bfc161180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0c01200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc16b570 con 0x7f0bfc0ced00 2024-04-23T14:32:15.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0c01200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc15e980 con 0x7f0bfc0b3930 2024-04-23T14:32:15.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0c01200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0bfc16b6f0 con 0x7f0bfc15e5a0 2024-04-23T14:32:15.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0bfaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bfc0ced00 0x7f0bfc15b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0bfaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bfc0ced00 0x7f0bfc15b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48678/0 (socket says 172.21.15.100:48678) 2024-04-23T14:32:15.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0bfaa00640 1 -- 172.21.15.100:0/3405363584 learned_addr learned my addr 172.21.15.100:0/3405363584 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:15.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0bfb400640 1 --2- 172.21.15.100:0/3405363584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bfc0b3930 0x7f0bfc15f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0bfbe00640 1 --2- 172.21.15.100:0/3405363584 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bfc15e5a0 0x7f0bfc161180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0bfaa00640 1 -- 172.21.15.100:0/3405363584 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bfc15e5a0 msgr2=0x7f0bfc161180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0bfaa00640 1 --2- 172.21.15.100:0/3405363584 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bfc15e5a0 0x7f0bfc161180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0bfaa00640 1 -- 172.21.15.100:0/3405363584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bfc0b3930 msgr2=0x7f0bfc15f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0bfaa00640 1 --2- 172.21.15.100:0/3405363584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bfc0b3930 0x7f0bfc15f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0bfaa00640 1 -- 172.21.15.100:0/3405363584 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bfc0c8ee0 con 0x7f0bfc0ced00 2024-04-23T14:32:15.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0bfaa00640 1 --2- 172.21.15.100:0/3405363584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bfc0ced00 0x7f0bfc15b020 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0be800c3a0 tx=0x7f0be800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:15.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0bf3400640 1 -- 172.21.15.100:0/3405363584 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be8017070 con 0x7f0bfc0ced00 2024-04-23T14:32:15.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0bf3400640 1 -- 172.21.15.100:0/3405363584 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0be8005cf0 con 0x7f0bfc0ced00 2024-04-23T14:32:15.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0c01200640 1 -- 172.21.15.100:0/3405363584 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bfc0c8920 con 0x7f0bfc0ced00 2024-04-23T14:32:15.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0bf3400640 1 -- 172.21.15.100:0/3405363584 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be80072f0 con 0x7f0bfc0ced00 2024-04-23T14:32:15.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0c01200640 1 -- 172.21.15.100:0/3405363584 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bfc15a820 con 0x7f0bfc0ced00 2024-04-23T14:32:15.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.169+0000 7f0c01200640 1 -- 172.21.15.100:0/3405363584 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bfc075c20 con 0x7f0bfc0ced00 2024-04-23T14:32:15.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.173+0000 7f0bf3400640 1 -- 172.21.15.100:0/3405363584 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0be801a070 con 0x7f0bfc0ced00 2024-04-23T14:32:15.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.173+0000 7f0bf3400640 1 --2- 172.21.15.100:0/3405363584 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0bdc068380 0x7f0bdc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.173+0000 7f0bf3400640 1 -- 172.21.15.100:0/3405363584 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0be8085fe0 con 0x7f0bfc0ced00 2024-04-23T14:32:15.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.173+0000 7f0bfb400640 1 --2- 172.21.15.100:0/3405363584 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0bdc068380 0x7f0bdc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.173+0000 7f0bfb400640 1 --2- 172.21.15.100:0/3405363584 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0bdc068380 0x7f0bdc06a840 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0113e0 tx=0x7f0bec00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:15.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.177+0000 7f0bf3400640 1 -- 172.21.15.100:0/3405363584 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0be8055bb0 con 0x7f0bfc0ced00 2024-04-23T14:32:15.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.341+0000 7f0c01200640 1 -- 172.21.15.100:0/3405363584 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0bfc0b3710 con 0x7f0bfc0ced00 2024-04-23T14:32:15.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.341+0000 7f0bf3400640 1 -- 172.21.15.100:0/3405363584 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f0be8059eb0 con 0x7f0bfc0ced00 2024-04-23T14:32:15.343 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:15.343 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":24,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:14.037027+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.341+0000 7f0c01200640 1 -- 172.21.15.100:0/3405363584 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0bdc068380 msgr2=0x7f0bdc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.341+0000 7f0c01200640 1 --2- 172.21.15.100:0/3405363584 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0bdc068380 0x7f0bdc06a840 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0113e0 tx=0x7f0bec00b040 comp rx=0 tx=0).stop 2024-04-23T14:32:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.341+0000 7f0c01200640 1 -- 172.21.15.100:0/3405363584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bfc0ced00 msgr2=0x7f0bfc15b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.341+0000 7f0c01200640 1 --2- 172.21.15.100:0/3405363584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bfc0ced00 0x7f0bfc15b020 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0be800c3a0 tx=0x7f0be800c860 comp rx=0 tx=0).stop 2024-04-23T14:32:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.341+0000 7f0c01200640 1 -- 172.21.15.100:0/3405363584 shutdown_connections 2024-04-23T14:32:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.341+0000 7f0c01200640 1 --2- 172.21.15.100:0/3405363584 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0bdc068380 0x7f0bdc06a840 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.341+0000 7f0c01200640 1 --2- 172.21.15.100:0/3405363584 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bfc15e5a0 0x7f0bfc161180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.341+0000 7f0c01200640 1 --2- 172.21.15.100:0/3405363584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bfc0ced00 0x7f0bfc15b020 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.341+0000 7f0c01200640 1 --2- 172.21.15.100:0/3405363584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bfc0b3930 0x7f0bfc15f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.341+0000 7f0c01200640 1 -- 172.21.15.100:0/3405363584 >> 172.21.15.100:0/3405363584 conn(0x7f0bfc0b3e60 msgr2=0x7f0bfc0b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:15.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.345+0000 7f0c01200640 1 -- 172.21.15.100:0/3405363584 shutdown_connections 2024-04-23T14:32:15.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.345+0000 7f0c01200640 1 -- 172.21.15.100:0/3405363584 wait complete. 2024-04-23T14:32:15.365 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:15.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.497+0000 7fd6d7000640 1 Processor -- start 2024-04-23T14:32:15.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.497+0000 7fd6d7000640 1 -- start start 2024-04-23T14:32:15.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.501+0000 7fd6d7000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6d015e730 0x7fd6d00cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.501+0000 7fd6d7000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6d006f450 con 0x7fd6d015eb10 2024-04-23T14:32:15.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.501+0000 7fd6d7000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6d006f170 con 0x7fd6d00d0760 2024-04-23T14:32:15.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.501+0000 7fd6d7000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d0162590 con 0x7fd6d015e730 2024-04-23T14:32:15.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.501+0000 7fd6c7400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fd6d015eb10 0x7fd6d00d0050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:34162/0 (socket says 172.21.15.100:34162) 2024-04-23T14:32:15.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.501+0000 7fd6c7400640 1 -- 172.21.15.100:0/3414026611 learned_addr learned my addr 172.21.15.100:0/3414026611 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:15.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.501+0000 7fd6cfe00640 1 --2- 172.21.15.100:0/3414026611 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6d015e730 0x7fd6d00cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.501+0000 7fd6cf400640 1 -- 172.21.15.100:0/3414026611 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1654187074 0 0) 0x7fd6d006f450 con 0x7fd6d015eb10 2024-04-23T14:32:15.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.501+0000 7fd6cf400640 1 -- 172.21.15.100:0/3414026611 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6b4003530 con 0x7fd6d015eb10 2024-04-23T14:32:15.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.501+0000 7fd6cf400640 1 -- 172.21.15.100:0/3414026611 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3781664958 0 0) 0x7fd6d006f170 con 0x7fd6d00d0760 2024-04-23T14:32:15.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.501+0000 7fd6cf400640 1 -- 172.21.15.100:0/3414026611 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6d006f450 con 0x7fd6d00d0760 2024-04-23T14:32:15.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.501+0000 7fd6cf400640 1 -- 172.21.15.100:0/3414026611 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1278440548 0 0) 0x7fd6b4003530 con 0x7fd6d015eb10 2024-04-23T14:32:15.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.501+0000 7fd6cfe00640 1 -- 172.21.15.100:0/3414026611 >> v1:172.21.15.151:6789/0 conn(0x7fd6d00d0760 legacy=0x7fd6d00ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.501+0000 7fd6cfe00640 1 -- 172.21.15.100:0/3414026611 >> v1:172.21.15.100:6789/0 conn(0x7fd6d015eb10 legacy=0x7fd6d00d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.501+0000 7fd6cfe00640 1 -- 172.21.15.100:0/3414026611 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d0162870 con 0x7fd6d015e730 2024-04-23T14:32:15.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.505+0000 7fd6cfe00640 1 --2- 172.21.15.100:0/3414026611 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6d015e730 0x7fd6d00cf470 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00097f0 tx=0x7fd6d00cd9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=903ec20b4f725954 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:15.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.505+0000 7fd6cf400640 1 -- 172.21.15.100:0/3414026611 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c00b0070 con 0x7fd6d015e730 2024-04-23T14:32:15.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.505+0000 7fd6cf400640 1 -- 172.21.15.100:0/3414026611 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6c00ab400 con 0x7fd6d015e730 2024-04-23T14:32:15.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.505+0000 7fd6cf400640 1 -- 172.21.15.100:0/3414026611 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8001680 con 0x7fd6d015e730 2024-04-23T14:32:15.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.505+0000 7fd6d7000640 1 -- 172.21.15.100:0/3414026611 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6d015e730 msgr2=0x7fd6d00cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.505+0000 7fd6d7000640 1 --2- 172.21.15.100:0/3414026611 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6d015e730 0x7fd6d00cf470 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00097f0 tx=0x7fd6d00cd9f0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.505+0000 7fd6d7000640 1 -- 172.21.15.100:0/3414026611 shutdown_connections 2024-04-23T14:32:15.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.505+0000 7fd6d7000640 1 --2- 172.21.15.100:0/3414026611 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6d015e730 0x7fd6d00cf470 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.505+0000 7fd6d7000640 1 -- 172.21.15.100:0/3414026611 >> 172.21.15.100:0/3414026611 conn(0x7fd6d00b4050 msgr2=0x7fd6d00b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:15.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.505+0000 7fd6d7000640 1 -- 172.21.15.100:0/3414026611 shutdown_connections 2024-04-23T14:32:15.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.505+0000 7fd6d7000640 1 -- 172.21.15.100:0/3414026611 wait complete. 2024-04-23T14:32:15.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.505+0000 7fd6d7000640 1 Processor -- start 2024-04-23T14:32:15.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.505+0000 7fd6d7000640 1 -- start start 2024-04-23T14:32:15.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.505+0000 7fd6d7000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6d00d0760 0x7fd6d00c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.505+0000 7fd6d7000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d015e730 0x7fd6d00cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.505+0000 7fd6d7000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6d015eb10 0x7fd6d00cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.505+0000 7fd6d7000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d016cb80 con 0x7fd6d015e730 2024-04-23T14:32:15.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.505+0000 7fd6d7000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d016ca00 con 0x7fd6d015eb10 2024-04-23T14:32:15.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.505+0000 7fd6d7000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d016cd00 con 0x7fd6d00d0760 2024-04-23T14:32:15.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.505+0000 7fd6c7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d015e730 0x7fd6d00cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.509+0000 7fd6c7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d015e730 0x7fd6d00cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48690/0 (socket says 172.21.15.100:48690) 2024-04-23T14:32:15.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.509+0000 7fd6c7400640 1 -- 172.21.15.100:0/3932083680 learned_addr learned my addr 172.21.15.100:0/3932083680 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:15.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.509+0000 7fd6d5000640 1 --2- 172.21.15.100:0/3932083680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6d015eb10 0x7fd6d00cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.509+0000 7fd6cfe00640 1 --2- 172.21.15.100:0/3932083680 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6d00d0760 0x7fd6d00c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.509+0000 7fd6c7400640 1 -- 172.21.15.100:0/3932083680 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6d00d0760 msgr2=0x7fd6d00c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.509+0000 7fd6c7400640 1 --2- 172.21.15.100:0/3932083680 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6d00d0760 0x7fd6d00c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.509+0000 7fd6c7400640 1 -- 172.21.15.100:0/3932083680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6d015eb10 msgr2=0x7fd6d00cabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.509+0000 7fd6c7400640 1 --2- 172.21.15.100:0/3932083680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6d015eb10 0x7fd6d00cabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.509+0000 7fd6c7400640 1 -- 172.21.15.100:0/3932083680 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d0159e30 con 0x7fd6d015e730 2024-04-23T14:32:15.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.509+0000 7fd6c7400640 1 --2- 172.21.15.100:0/3932083680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d015e730 0x7fd6d00cc0e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8001ec0 tx=0x7fd6b800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:15.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.509+0000 7fd6ccc00640 1 -- 172.21.15.100:0/3932083680 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8005ae0 con 0x7fd6d015e730 2024-04-23T14:32:15.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.509+0000 7fd6ccc00640 1 -- 172.21.15.100:0/3932083680 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b8005c80 con 0x7fd6d015e730 2024-04-23T14:32:15.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.509+0000 7fd6ccc00640 1 -- 172.21.15.100:0/3932083680 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b80072b0 con 0x7fd6d015e730 2024-04-23T14:32:15.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.509+0000 7fd6d7000640 1 -- 172.21.15.100:0/3932083680 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d00c9b30 con 0x7fd6d015e730 2024-04-23T14:32:15.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.509+0000 7fd6d7000640 1 -- 172.21.15.100:0/3932083680 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d00ca0f0 con 0x7fd6d015e730 2024-04-23T14:32:15.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.509+0000 7fd6d7000640 1 -- 172.21.15.100:0/3932083680 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d0068ad0 con 0x7fd6d015e730 2024-04-23T14:32:15.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.509+0000 7fd6ccc00640 1 -- 172.21.15.100:0/3932083680 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd6b8007450 con 0x7fd6d015e730 2024-04-23T14:32:15.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.509+0000 7fd6ccc00640 1 --2- 172.21.15.100:0/3932083680 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd6ac068380 0x7fd6ac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.509+0000 7fd6ccc00640 1 -- 172.21.15.100:0/3932083680 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd6b8086940 con 0x7fd6d015e730 2024-04-23T14:32:15.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.513+0000 7fd6cfe00640 1 --2- 172.21.15.100:0/3932083680 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd6ac068380 0x7fd6ac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.513+0000 7fd6cfe00640 1 --2- 172.21.15.100:0/3932083680 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd6ac068380 0x7fd6ac06a840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00ad000 tx=0x7fd6c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:15.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.513+0000 7fd6ccc00640 1 -- 172.21.15.100:0/3932083680 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd6d0159e30 con 0x7fd6d015e730 2024-04-23T14:32:15.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.673+0000 7fd6d7000640 1 -- 172.21.15.100:0/3932083680 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6d00b31e0 con 0x7fd6d015e730 2024-04-23T14:32:15.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.677+0000 7fd6ccc00640 1 -- 172.21.15.100:0/3932083680 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7fd6b8056560 con 0x7fd6d015e730 2024-04-23T14:32:15.677 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:15.677 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":24,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:14.037027+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:15.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.677+0000 7fd6d7000640 1 -- 172.21.15.100:0/3932083680 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd6ac068380 msgr2=0x7fd6ac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.677+0000 7fd6d7000640 1 --2- 172.21.15.100:0/3932083680 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd6ac068380 0x7fd6ac06a840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00ad000 tx=0x7fd6c0008040 comp rx=0 tx=0).stop 2024-04-23T14:32:15.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.677+0000 7fd6d7000640 1 -- 172.21.15.100:0/3932083680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d015e730 msgr2=0x7fd6d00cc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.677+0000 7fd6d7000640 1 --2- 172.21.15.100:0/3932083680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d015e730 0x7fd6d00cc0e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8001ec0 tx=0x7fd6b800cbe0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.677+0000 7fd6d7000640 1 -- 172.21.15.100:0/3932083680 shutdown_connections 2024-04-23T14:32:15.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.677+0000 7fd6d7000640 1 --2- 172.21.15.100:0/3932083680 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd6ac068380 0x7fd6ac06a840 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.677+0000 7fd6d7000640 1 --2- 172.21.15.100:0/3932083680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6d015eb10 0x7fd6d00cabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.677+0000 7fd6d7000640 1 --2- 172.21.15.100:0/3932083680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d015e730 0x7fd6d00cc0e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.677+0000 7fd6d7000640 1 --2- 172.21.15.100:0/3932083680 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6d00d0760 0x7fd6d00c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.677+0000 7fd6d7000640 1 -- 172.21.15.100:0/3932083680 >> 172.21.15.100:0/3932083680 conn(0x7fd6d00b4050 msgr2=0x7fd6d00b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:15.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.677+0000 7fd6d7000640 1 -- 172.21.15.100:0/3932083680 shutdown_connections 2024-04-23T14:32:15.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.677+0000 7fd6d7000640 1 -- 172.21.15.100:0/3932083680 wait complete. 2024-04-23T14:32:15.712 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:32:15.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.845+0000 7f5171200640 1 Processor -- start 2024-04-23T14:32:15.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.845+0000 7f5171200640 1 -- start start 2024-04-23T14:32:15.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.849+0000 7f5171200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f516c15c470 0x7f516c0b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.849+0000 7f5171200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f516c160de0 0x7f516c0ad160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.849+0000 7f5171200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f516c0ad6a0 0x7f516c06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.849+0000 7f5171200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f516c0adad0 con 0x7f516c15c470 2024-04-23T14:32:15.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.849+0000 7f5171200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f516c160320 con 0x7f516c0ad6a0 2024-04-23T14:32:15.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.849+0000 7f5171200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f516c0cc7e0 con 0x7f516c160de0 2024-04-23T14:32:15.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.849+0000 7f5169e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f516c15c470 0x7f516c0b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.849+0000 7f5169e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f516c15c470 0x7f516c0b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48706/0 (socket says 172.21.15.100:48706) 2024-04-23T14:32:15.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.849+0000 7f5169e00640 1 -- 172.21.15.100:0/1813102513 learned_addr learned my addr 172.21.15.100:0/1813102513 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:15.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.849+0000 7f516a800640 1 --2- 172.21.15.100:0/1813102513 >> v2:172.21.15.151:3300/0 conn(0x7f516c0ad6a0 0x7f516c06bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.849+0000 7f5161400640 1 --2- 172.21.15.100:0/1813102513 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f516c160de0 0x7f516c0ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.849+0000 7f5169e00640 1 -- 172.21.15.100:0/1813102513 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f516c160de0 msgr2=0x7f516c0ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:15.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.849+0000 7f5169e00640 1 --2- 172.21.15.100:0/1813102513 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f516c160de0 0x7f516c0ad160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.849+0000 7f5169e00640 1 -- 172.21.15.100:0/1813102513 >> v2:172.21.15.151:3300/0 conn(0x7f516c0ad6a0 msgr2=0x7f516c06bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:15.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.849+0000 7f5169e00640 1 --2- 172.21.15.100:0/1813102513 >> v2:172.21.15.151:3300/0 conn(0x7f516c0ad6a0 0x7f516c06bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.849+0000 7f5169e00640 1 -- 172.21.15.100:0/1813102513 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516c1605e0 con 0x7f516c15c470 2024-04-23T14:32:15.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.853+0000 7f5169e00640 1 --2- 172.21.15.100:0/1813102513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516c15c470 0x7f516c0b29a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f516c142f30 tx=0x7f515c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d9588633238531 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:15.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.853+0000 7f5169400640 1 -- 172.21.15.100:0/1813102513 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c0b2070 con 0x7f516c15c470 2024-04-23T14:32:15.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.853+0000 7f5169400640 1 -- 172.21.15.100:0/1813102513 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f515c0a9070 con 0x7f516c15c470 2024-04-23T14:32:15.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.853+0000 7f5169400640 1 -- 172.21.15.100:0/1813102513 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c0ad880 con 0x7f516c15c470 2024-04-23T14:32:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.853+0000 7f5171200640 1 -- 172.21.15.100:0/1813102513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516c15c470 msgr2=0x7f516c0b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.853+0000 7f5171200640 1 --2- 172.21.15.100:0/1813102513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516c15c470 0x7f516c0b29a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f516c142f30 tx=0x7f515c01c490 comp rx=0 tx=0).stop 2024-04-23T14:32:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.853+0000 7f5171200640 1 -- 172.21.15.100:0/1813102513 shutdown_connections 2024-04-23T14:32:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.853+0000 7f5171200640 1 --2- 172.21.15.100:0/1813102513 >> v2:172.21.15.151:3300/0 conn(0x7f516c0ad6a0 0x7f516c06bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.853+0000 7f5171200640 1 --2- 172.21.15.100:0/1813102513 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f516c160de0 0x7f516c0ad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.853+0000 7f5171200640 1 --2- 172.21.15.100:0/1813102513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516c15c470 0x7f516c0b29a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.853+0000 7f5171200640 1 -- 172.21.15.100:0/1813102513 >> 172.21.15.100:0/1813102513 conn(0x7f516c0b1e80 msgr2=0x7f516c0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.853+0000 7f5171200640 1 -- 172.21.15.100:0/1813102513 shutdown_connections 2024-04-23T14:32:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.853+0000 7f5171200640 1 -- 172.21.15.100:0/1813102513 wait complete. 2024-04-23T14:32:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.853+0000 7f5171200640 1 Processor -- start 2024-04-23T14:32:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.853+0000 7f5171200640 1 -- start start 2024-04-23T14:32:15.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.853+0000 7f5171200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f516c0ad6a0 0x7f516c142f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.853+0000 7f5171200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f516c15c470 0x7f516c14f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.853+0000 7f5171200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516c160de0 0x7f516c099aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.853+0000 7f5171200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f516c0cc370 con 0x7f516c160de0 2024-04-23T14:32:15.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.853+0000 7f5171200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f516c15ca60 con 0x7f516c0ad6a0 2024-04-23T14:32:15.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.853+0000 7f5171200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f516c0cc4f0 con 0x7f516c15c470 2024-04-23T14:32:15.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.857+0000 7f516a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516c160de0 0x7f516c099aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.857+0000 7f516a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516c160de0 0x7f516c099aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48710/0 (socket says 172.21.15.100:48710) 2024-04-23T14:32:15.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.857+0000 7f516a800640 1 -- 172.21.15.100:0/4143601718 learned_addr learned my addr 172.21.15.100:0/4143601718 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:15.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.857+0000 7f5169e00640 1 --2- 172.21.15.100:0/4143601718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f516c0ad6a0 0x7f516c142f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.857+0000 7f5161400640 1 --2- 172.21.15.100:0/4143601718 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f516c15c470 0x7f516c14f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.857+0000 7f516a800640 1 -- 172.21.15.100:0/4143601718 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f516c15c470 msgr2=0x7f516c14f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.857+0000 7f516a800640 1 --2- 172.21.15.100:0/4143601718 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f516c15c470 0x7f516c14f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.857+0000 7f516a800640 1 -- 172.21.15.100:0/4143601718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f516c0ad6a0 msgr2=0x7f516c142f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.857+0000 7f516a800640 1 --2- 172.21.15.100:0/4143601718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f516c0ad6a0 0x7f516c142f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.857+0000 7f516a800640 1 -- 172.21.15.100:0/4143601718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516c0d0810 con 0x7f516c160de0 2024-04-23T14:32:15.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.857+0000 7f5169e00640 1 --2- 172.21.15.100:0/4143601718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f516c0ad6a0 0x7f516c142f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:15.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.857+0000 7f516a800640 1 --2- 172.21.15.100:0/4143601718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516c160de0 0x7f516c099aa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5164007d40 tx=0x7f516400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:15.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.857+0000 7f5162a00640 1 -- 172.21.15.100:0/4143601718 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5164014420 con 0x7f516c160de0 2024-04-23T14:32:15.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.857+0000 7f5162a00640 1 -- 172.21.15.100:0/4143601718 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5164019070 con 0x7f516c160de0 2024-04-23T14:32:15.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.857+0000 7f5162a00640 1 -- 172.21.15.100:0/4143601718 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5164014820 con 0x7f516c160de0 2024-04-23T14:32:15.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.857+0000 7f5171200640 1 -- 172.21.15.100:0/4143601718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f516c153540 con 0x7f516c160de0 2024-04-23T14:32:15.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.857+0000 7f5171200640 1 -- 172.21.15.100:0/4143601718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f516c0cce30 con 0x7f516c160de0 2024-04-23T14:32:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.857+0000 7f5171200640 1 -- 172.21.15.100:0/4143601718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f516c068ad0 con 0x7f516c160de0 2024-04-23T14:32:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.857+0000 7f5162a00640 1 -- 172.21.15.100:0/4143601718 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f51640149c0 con 0x7f516c160de0 2024-04-23T14:32:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.857+0000 7f5162a00640 1 --2- 172.21.15.100:0/4143601718 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5148068380 0x7f514806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.857+0000 7f5162a00640 1 -- 172.21.15.100:0/4143601718 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f51640874d0 con 0x7f516c160de0 2024-04-23T14:32:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.861+0000 7f5169e00640 1 --2- 172.21.15.100:0/4143601718 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5148068380 0x7f514806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.861+0000 7f5169e00640 1 --2- 172.21.15.100:0/4143601718 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5148068380 0x7f514806a840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f515c0af000 tx=0x7f515c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:15.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:15.861+0000 7f5162a00640 1 -- 172.21.15.100:0/4143601718 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5164057740 con 0x7f516c160de0 2024-04-23T14:32:16.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.053+0000 7f5171200640 1 -- 172.21.15.100:0/4143601718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f516c0b5460 con 0x7f516c160de0 2024-04-23T14:32:16.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.053+0000 7f5162a00640 1 -- 172.21.15.100:0/4143601718 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5189 (secure 0 0 0) 0x7f5164057040 con 0x7f516c160de0 2024-04-23T14:32:16.056 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:16.057 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1456958549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1456958549},{"type":"v1","addr":"172.21.15.100:6836","nonce":1456958549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3655609551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3655609551},{"type":"v1","addr":"172.21.15.100:6837","nonce":3655609551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/4242180972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":4242180972},{"type":"v1","addr":"172.21.15.151:6834","nonce":4242180972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:32:08.976576+0000","modified":"2024-04-23T14:32:14.037173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1533322173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1533322173},{"type":"v1","addr":"172.21.15.151:6835","nonce":1533322173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-23T14:32:16.057 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 7 2024-04-23T14:32:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.057+0000 7f5171200640 1 -- 172.21.15.100:0/4143601718 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5148068380 msgr2=0x7f514806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.057+0000 7f5171200640 1 --2- 172.21.15.100:0/4143601718 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5148068380 0x7f514806a840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f515c0af000 tx=0x7f515c008040 comp rx=0 tx=0).stop 2024-04-23T14:32:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.057+0000 7f5171200640 1 -- 172.21.15.100:0/4143601718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516c160de0 msgr2=0x7f516c099aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.057+0000 7f5171200640 1 --2- 172.21.15.100:0/4143601718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516c160de0 0x7f516c099aa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5164007d40 tx=0x7f516400e4b0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.057+0000 7f5171200640 1 -- 172.21.15.100:0/4143601718 shutdown_connections 2024-04-23T14:32:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.057+0000 7f5171200640 1 --2- 172.21.15.100:0/4143601718 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5148068380 0x7f514806a840 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.057+0000 7f5171200640 1 --2- 172.21.15.100:0/4143601718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516c160de0 0x7f516c099aa0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.057+0000 7f5171200640 1 --2- 172.21.15.100:0/4143601718 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f516c15c470 0x7f516c14f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.057+0000 7f5171200640 1 --2- 172.21.15.100:0/4143601718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f516c0ad6a0 0x7f516c142f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.057+0000 7f5171200640 1 -- 172.21.15.100:0/4143601718 >> 172.21.15.100:0/4143601718 conn(0x7f516c0b1e80 msgr2=0x7f516c06ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.057+0000 7f5171200640 1 -- 172.21.15.100:0/4143601718 shutdown_connections 2024-04-23T14:32:16.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.057+0000 7f5171200640 1 -- 172.21.15.100:0/4143601718 wait complete. 2024-04-23T14:32:16.078 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:16.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.213+0000 7f950f200640 1 Processor -- start 2024-04-23T14:32:16.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.213+0000 7f950f200640 1 -- start start 2024-04-23T14:32:16.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.213+0000 7f950f200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f95080cea60 0x7f95080d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.213+0000 7f950f200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f95080cd460 0x7f95080b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.213+0000 7f950f200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95080b1f90 con 0x7f950806d450 2024-04-23T14:32:16.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.213+0000 7f950f200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f95080b2270 con 0x7f95080cd460 2024-04-23T14:32:16.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.213+0000 7f950f200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f95080b2550 con 0x7f95080cea60 2024-04-23T14:32:16.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.213+0000 7f950d200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f95080cd460 0x7f95080b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.213+0000 7f950d200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f95080cd460 0x7f95080b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48726/0 (socket says 172.21.15.100:48726) 2024-04-23T14:32:16.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.213+0000 7f950d200640 1 -- 172.21.15.100:0/1827605835 learned_addr learned my addr 172.21.15.100:0/1827605835 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:16.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.213+0000 7f9507e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f950806d450 0x7f950806d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:34178/0 (socket says 172.21.15.100:34178) 2024-04-23T14:32:16.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.213+0000 7f94ff400640 1 --2- 172.21.15.100:0/1827605835 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f95080cea60 0x7f95080d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.213+0000 7f950d200640 1 -- 172.21.15.100:0/1827605835 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f95080cea60 msgr2=0x7f95080d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:16.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.213+0000 7f950d200640 1 --2- 172.21.15.100:0/1827605835 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f95080cea60 0x7f95080d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.213+0000 7f950d200640 1 -- 172.21.15.100:0/1827605835 >> v1:172.21.15.100:6789/0 conn(0x7f950806d450 legacy=0x7f950806d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.213+0000 7f950d200640 1 -- 172.21.15.100:0/1827605835 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95080b2830 con 0x7f95080cd460 2024-04-23T14:32:16.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.213+0000 7f9507400640 1 -- 172.21.15.100:0/1827605835 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3381605511 0 0) 0x7f95080b1f90 con 0x7f950806d450 2024-04-23T14:32:16.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.217+0000 7f950d200640 1 --2- 172.21.15.100:0/1827605835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95080cd460 0x7f95080b2d50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9500009870 tx=0x7f950001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd7abc0f33d3117 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:16.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.217+0000 7f9507400640 1 -- 172.21.15.100:0/1827605835 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95000b0070 con 0x7f95080cd460 2024-04-23T14:32:16.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.217+0000 7f9507400640 1 -- 172.21.15.100:0/1827605835 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95000ab600 con 0x7f95080cd460 2024-04-23T14:32:16.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.217+0000 7f9507400640 1 -- 172.21.15.100:0/1827605835 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95000ab9d0 con 0x7f95080cd460 2024-04-23T14:32:16.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.217+0000 7f950f200640 1 -- 172.21.15.100:0/1827605835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95080cd460 msgr2=0x7f95080b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.217+0000 7f950f200640 1 --2- 172.21.15.100:0/1827605835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95080cd460 0x7f95080b2d50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9500009870 tx=0x7f950001c4d0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.217+0000 7f950f200640 1 -- 172.21.15.100:0/1827605835 shutdown_connections 2024-04-23T14:32:16.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.217+0000 7f950f200640 1 --2- 172.21.15.100:0/1827605835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95080cd460 0x7f95080b2d50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.217+0000 7f950f200640 1 --2- 172.21.15.100:0/1827605835 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f95080cea60 0x7f95080d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.217+0000 7f950f200640 1 -- 172.21.15.100:0/1827605835 >> 172.21.15.100:0/1827605835 conn(0x7f95080cee80 msgr2=0x7f95080cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:16.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.217+0000 7f950f200640 1 -- 172.21.15.100:0/1827605835 shutdown_connections 2024-04-23T14:32:16.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.217+0000 7f950f200640 1 -- 172.21.15.100:0/1827605835 wait complete. 2024-04-23T14:32:16.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f950f200640 1 Processor -- start 2024-04-23T14:32:16.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f950f200640 1 -- start start 2024-04-23T14:32:16.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f950f200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f950806d450 0x7f9508154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f950f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95080cd460 0x7f9508155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f950f200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f95080cea60 0x7f95080701e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f950f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f950816b740 con 0x7f95080cd460 2024-04-23T14:32:16.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f950f200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f950816b5c0 con 0x7f950806d450 2024-04-23T14:32:16.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f950f200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f950816b8c0 con 0x7f95080cea60 2024-04-23T14:32:16.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f94ff400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95080cd460 0x7f9508155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f94ff400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95080cd460 0x7f9508155e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48734/0 (socket says 172.21.15.100:48734) 2024-04-23T14:32:16.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f94ff400640 1 -- 172.21.15.100:0/573519450 learned_addr learned my addr 172.21.15.100:0/573519450 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:16.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f950d200640 1 --2- 172.21.15.100:0/573519450 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f95080cea60 0x7f95080701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f9507e00640 1 --2- 172.21.15.100:0/573519450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f950806d450 0x7f9508154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f94ff400640 1 -- 172.21.15.100:0/573519450 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f95080cea60 msgr2=0x7f95080701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f94ff400640 1 --2- 172.21.15.100:0/573519450 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f95080cea60 0x7f95080701e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f94ff400640 1 -- 172.21.15.100:0/573519450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f950806d450 msgr2=0x7f9508154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f94ff400640 1 --2- 172.21.15.100:0/573519450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f950806d450 0x7f9508154f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f94ff400640 1 -- 172.21.15.100:0/573519450 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95081566d0 con 0x7f95080cd460 2024-04-23T14:32:16.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f9507e00640 1 --2- 172.21.15.100:0/573519450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f950806d450 0x7f9508154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:16.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f94ff400640 1 --2- 172.21.15.100:0/573519450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95080cd460 0x7f9508155e80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f94f000d800 tx=0x7f94f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:16.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f9504c00640 1 -- 172.21.15.100:0/573519450 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f000b6c0 con 0x7f95080cd460 2024-04-23T14:32:16.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f9504c00640 1 -- 172.21.15.100:0/573519450 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94f0019070 con 0x7f95080cd460 2024-04-23T14:32:16.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f9504c00640 1 -- 172.21.15.100:0/573519450 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f000ba60 con 0x7f95080cd460 2024-04-23T14:32:16.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f950f200640 1 -- 172.21.15.100:0/573519450 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95080c5e40 con 0x7f95080cd460 2024-04-23T14:32:16.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f950f200640 1 -- 172.21.15.100:0/573519450 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95080c6400 con 0x7f95080cd460 2024-04-23T14:32:16.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.221+0000 7f950f200640 1 -- 172.21.15.100:0/573519450 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9508068ad0 con 0x7f95080cd460 2024-04-23T14:32:16.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.225+0000 7f9504c00640 1 -- 172.21.15.100:0/573519450 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f94f000bc00 con 0x7f95080cd460 2024-04-23T14:32:16.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.225+0000 7f9504c00640 1 --2- 172.21.15.100:0/573519450 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f94e4068380 0x7f94e406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.225+0000 7f9504c00640 1 -- 172.21.15.100:0/573519450 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f94f0087140 con 0x7f95080cd460 2024-04-23T14:32:16.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.225+0000 7f9507e00640 1 --2- 172.21.15.100:0/573519450 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f94e4068380 0x7f94e406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.225+0000 7f9507e00640 1 --2- 172.21.15.100:0/573519450 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f94e4068380 0x7f94e406a840 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f94f8007fd0 tx=0x7f94f8005b80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:16.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.229+0000 7f9504c00640 1 -- 172.21.15.100:0/573519450 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f94f00573b0 con 0x7f95080cd460 2024-04-23T14:32:16.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.389+0000 7f950f200640 1 -- 172.21.15.100:0/573519450 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95080b1b60 con 0x7f95080cd460 2024-04-23T14:32:16.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.389+0000 7f9504c00640 1 -- 172.21.15.100:0/573519450 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f94f0056cb0 con 0x7f95080cd460 2024-04-23T14:32:16.391 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:16.391 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":24,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:14.037027+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:16.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.389+0000 7f950f200640 1 -- 172.21.15.100:0/573519450 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f94e4068380 msgr2=0x7f94e406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.389+0000 7f950f200640 1 --2- 172.21.15.100:0/573519450 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f94e4068380 0x7f94e406a840 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f94f8007fd0 tx=0x7f94f8005b80 comp rx=0 tx=0).stop 2024-04-23T14:32:16.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.389+0000 7f950f200640 1 -- 172.21.15.100:0/573519450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95080cd460 msgr2=0x7f9508155e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.389+0000 7f950f200640 1 --2- 172.21.15.100:0/573519450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95080cd460 0x7f9508155e80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f94f000d800 tx=0x7f94f000dcc0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.389+0000 7f950f200640 1 -- 172.21.15.100:0/573519450 shutdown_connections 2024-04-23T14:32:16.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.389+0000 7f950f200640 1 --2- 172.21.15.100:0/573519450 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f94e4068380 0x7f94e406a840 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.389+0000 7f950f200640 1 --2- 172.21.15.100:0/573519450 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f95080cea60 0x7f95080701e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.389+0000 7f950f200640 1 --2- 172.21.15.100:0/573519450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95080cd460 0x7f9508155e80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.389+0000 7f950f200640 1 --2- 172.21.15.100:0/573519450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f950806d450 0x7f9508154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.389+0000 7f950f200640 1 -- 172.21.15.100:0/573519450 >> 172.21.15.100:0/573519450 conn(0x7f95080cee80 msgr2=0x7f950815d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:16.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.389+0000 7f950f200640 1 -- 172.21.15.100:0/573519450 shutdown_connections 2024-04-23T14:32:16.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.389+0000 7f950f200640 1 -- 172.21.15.100:0/573519450 wait complete. 2024-04-23T14:32:16.814 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:32:08.976576+0000', 'modified': '2024-04-23T14:32:14.037173+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4279}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4279': {'gid': 4279, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.151:6835/1533322173', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6833', 'nonce': 1533322173}, {'type': 'v1', 'addr': '172.21.15.151:6835', 'nonce': 1533322173}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4279, 'qdb_cluster': [4279]} 2024-04-23T14:32:16.815 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-23T14:32:16.815 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-23T14:32:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.948+0000 7f9ae7a00640 1 Processor -- start 2024-04-23T14:32:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.948+0000 7f9ae7a00640 1 -- start start 2024-04-23T14:32:16.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9ae7a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ae00d0190 0x7f9ae00d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9ae7a00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ae00cee00 con 0x7f9ae015b660 2024-04-23T14:32:16.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9ae7a00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ae006bc50 con 0x7f9ae00c5160 2024-04-23T14:32:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9ae7a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9ae015ba40 con 0x7f9ae00d0190 2024-04-23T14:32:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9ae5000640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f9ae015b660 0x7f9ae00c6820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:34188/0 (socket says 172.21.15.100:34188) 2024-04-23T14:32:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9ae5000640 1 -- 172.21.15.100:0/1021261360 learned_addr learned my addr 172.21.15.100:0/1021261360 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9adfe00640 1 --2- 172.21.15.100:0/1021261360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ae00d0190 0x7f9ae00d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9adf400640 1 -- 172.21.15.100:0/1021261360 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3982950786 0 0) 0x7f9ad4001120 con 0x7f9ae015b660 2024-04-23T14:32:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9adf400640 1 -- 172.21.15.100:0/1021261360 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ac8003530 con 0x7f9ae015b660 2024-04-23T14:32:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9adf400640 1 -- 172.21.15.100:0/1021261360 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2027077406 0 0) 0x7f9ae006bc50 con 0x7f9ae00c5160 2024-04-23T14:32:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9adf400640 1 -- 172.21.15.100:0/1021261360 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ad4001120 con 0x7f9ae00c5160 2024-04-23T14:32:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9adf400640 1 -- 172.21.15.100:0/1021261360 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2880172429 0 0) 0x7f9ac8003530 con 0x7f9ae015b660 2024-04-23T14:32:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9adf400640 1 -- 172.21.15.100:0/1021261360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ae00d0190 msgr2=0x7f9ae00d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9adf400640 1 --2- 172.21.15.100:0/1021261360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ae00d0190 0x7f9ae00d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9adf400640 1 -- 172.21.15.100:0/1021261360 >> v1:172.21.15.151:6789/0 conn(0x7f9ae00c5160 legacy=0x7f9ae00c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9adf400640 1 -- 172.21.15.100:0/1021261360 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae006bf30 con 0x7f9ae015b660 2024-04-23T14:32:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9adf400640 1 -- 172.21.15.100:0/1021261360 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f9ad4001850 con 0x7f9ae015b660 2024-04-23T14:32:16.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9adfe00640 1 --2- 172.21.15.100:0/1021261360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ae00d0190 0x7f9ae00d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9adf400640 1 -- 172.21.15.100:0/1021261360 >> v1:172.21.15.100:6789/0 conn(0x7f9ae015b660 legacy=0x7f9ae00c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9adf400640 1 --2- 172.21.15.100:0/1021261360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ae00d0190 0x7f9ac8003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9adf400640 1 --2- 172.21.15.100:0/1021261360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ac80040c0 0x7f9ac8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9adf400640 1 --2- 172.21.15.100:0/1021261360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac8004d50 0x7f9ac80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9adf400640 1 -- 172.21.15.100:0/1021261360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac800ba80 con 0x7f9ac8004d50 2024-04-23T14:32:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9adf400640 1 -- 172.21.15.100:0/1021261360 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac800bc00 con 0x7f9ac80040c0 2024-04-23T14:32:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9adf400640 1 -- 172.21.15.100:0/1021261360 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9ac800be00 con 0x7f9ae00d0190 2024-04-23T14:32:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9ae5a00640 1 --2- 172.21.15.100:0/1021261360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac8004d50 0x7f9ac80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9ae5000640 1 --2- 172.21.15.100:0/1021261360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ae00d0190 0x7f9ac8003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9adfe00640 1 --2- 172.21.15.100:0/1021261360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ac80040c0 0x7f9ac8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9ae5a00640 1 -- 172.21.15.100:0/1021261360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ae00d0190 msgr2=0x7f9ac8003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9ae5a00640 1 --2- 172.21.15.100:0/1021261360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ae00d0190 0x7f9ac8003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9ae5a00640 1 -- 172.21.15.100:0/1021261360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ac80040c0 msgr2=0x7f9ac8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:16.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9ae5a00640 1 --2- 172.21.15.100:0/1021261360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ac80040c0 0x7f9ac8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.952+0000 7f9ae5a00640 1 -- 172.21.15.100:0/1021261360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac8003530 con 0x7f9ac8004d50 2024-04-23T14:32:16.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.956+0000 7f9ae5a00640 1 --2- 172.21.15.100:0/1021261360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac8004d50 0x7f9ac80051d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8001fd0 tx=0x7f9ad801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e547b327fabea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:16.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.956+0000 7f9adf400640 1 -- 172.21.15.100:0/1021261360 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8001c20 con 0x7f9ac8004d50 2024-04-23T14:32:16.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.956+0000 7f9adf400640 1 -- 172.21.15.100:0/1021261360 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad8003ba0 con 0x7f9ac8004d50 2024-04-23T14:32:16.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.956+0000 7f9adf400640 1 -- 172.21.15.100:0/1021261360 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad80050d0 con 0x7f9ac8004d50 2024-04-23T14:32:16.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.956+0000 7f9ae7a00640 1 -- 172.21.15.100:0/1021261360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac8004d50 msgr2=0x7f9ac80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.956+0000 7f9ae7a00640 1 --2- 172.21.15.100:0/1021261360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac8004d50 0x7f9ac80051d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8001fd0 tx=0x7f9ad801c900 comp rx=0 tx=0).stop 2024-04-23T14:32:16.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.956+0000 7f9ae5a00640 1 -- 172.21.15.100:0/1021261360 reap_dead start 2024-04-23T14:32:16.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.956+0000 7f9ae7a00640 1 -- 172.21.15.100:0/1021261360 shutdown_connections 2024-04-23T14:32:16.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.956+0000 7f9ae7a00640 1 -- 172.21.15.100:0/1021261360 >> 172.21.15.100:0/1021261360 conn(0x7f9ae00b5310 msgr2=0x7f9ae00b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:16.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.956+0000 7f9ae7a00640 1 -- 172.21.15.100:0/1021261360 shutdown_connections 2024-04-23T14:32:16.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.956+0000 7f9ae7a00640 1 -- 172.21.15.100:0/1021261360 wait complete. 2024-04-23T14:32:16.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.956+0000 7f9ae7a00640 1 Processor -- start 2024-04-23T14:32:16.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.956+0000 7f9ae7a00640 1 -- start start 2024-04-23T14:32:16.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9ae7a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac8004d50 0x7f9ae015e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9ae7a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ae00adb70 0x7f9ae00c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9ae7a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ae00c80c0 0x7f9ae00c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9ae7a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae006c7d0 con 0x7f9ac8004d50 2024-04-23T14:32:16.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9ae7a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae006c4f0 con 0x7f9ae00adb70 2024-04-23T14:32:16.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9ae7a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9ae00c6f30 con 0x7f9ae00c80c0 2024-04-23T14:32:16.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9ae5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac8004d50 0x7f9ae015e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9ae5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac8004d50 0x7f9ae015e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48758/0 (socket says 172.21.15.100:48758) 2024-04-23T14:32:16.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9ae5000640 1 -- 172.21.15.100:0/1110293571 learned_addr learned my addr 172.21.15.100:0/1110293571 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:16.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9adfe00640 1 --2- 172.21.15.100:0/1110293571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ae00adb70 0x7f9ae00c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9ae5a00640 1 --2- 172.21.15.100:0/1110293571 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ae00c80c0 0x7f9ae00c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9ae5000640 1 -- 172.21.15.100:0/1110293571 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ae00c80c0 msgr2=0x7f9ae00c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9ae5000640 1 --2- 172.21.15.100:0/1110293571 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ae00c80c0 0x7f9ae00c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9ae5000640 1 -- 172.21.15.100:0/1110293571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ae00adb70 msgr2=0x7f9ae00c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9ae5000640 1 --2- 172.21.15.100:0/1110293571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ae00adb70 0x7f9ae00c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9ae5000640 1 -- 172.21.15.100:0/1110293571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae00ae070 con 0x7f9ac8004d50 2024-04-23T14:32:16.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9adfe00640 1 --2- 172.21.15.100:0/1110293571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ae00adb70 0x7f9ae00c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:16.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9ae5000640 1 --2- 172.21.15.100:0/1110293571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac8004d50 0x7f9ae015e180 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4012540 tx=0x7f9ad4012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:16.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9adcc00640 1 -- 172.21.15.100:0/1110293571 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4014810 con 0x7f9ac8004d50 2024-04-23T14:32:16.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9adcc00640 1 -- 172.21.15.100:0/1110293571 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad401b070 con 0x7f9ac8004d50 2024-04-23T14:32:16.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9adcc00640 1 -- 172.21.15.100:0/1110293571 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4014c30 con 0x7f9ac8004d50 2024-04-23T14:32:16.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9ae7a00640 1 -- 172.21.15.100:0/1110293571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae00ae350 con 0x7f9ac8004d50 2024-04-23T14:32:16.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9ae7a00640 1 -- 172.21.15.100:0/1110293571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae00ae910 con 0x7f9ac8004d50 2024-04-23T14:32:16.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.960+0000 7f9adcc00640 1 -- 172.21.15.100:0/1110293571 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9ad4016480 con 0x7f9ac8004d50 2024-04-23T14:32:16.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9adcc00640 1 --2- 172.21.15.100:0/1110293571 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9ac0068380 0x7f9ac006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9adcc00640 1 -- 172.21.15.100:0/1110293571 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9ad408a1e0 con 0x7f9ac8004d50 2024-04-23T14:32:16.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9adfe00640 1 --2- 172.21.15.100:0/1110293571 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9ac0068380 0x7f9ac006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9aeae48000 1 Processor -- start 2024-04-23T14:32:16.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9aeae48000 1 -- start start 2024-04-23T14:32:16.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9adfe00640 1 --2- 172.21.15.100:0/1110293571 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9ac0068380 0x7f9ac006a840 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00c6d40 tx=0x7f9acc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:16.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9aeae48000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ebd049b5b0 0x55ebd049b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9aeae48000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd049bed0 0x55ebd044ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9aeae48000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55ebd044f140 0x55ebd044f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9aeae48000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ebd03386f0 con 0x55ebd049bed0 2024-04-23T14:32:16.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9aeae48000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ebd029f4d0 con 0x55ebd049b5b0 2024-04-23T14:32:16.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9aeae48000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x55ebd0239bf0 con 0x55ebd044f140 2024-04-23T14:32:16.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9adfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd049bed0 0x55ebd044ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9adfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd049bed0 0x55ebd044ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48768/0 (socket says 172.21.15.100:48768) 2024-04-23T14:32:16.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9adfe00640 1 -- 172.21.15.100:0/2058961082 learned_addr learned my addr 172.21.15.100:0/2058961082 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:16.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9ae5000640 1 --2- 172.21.15.100:0/2058961082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ebd049b5b0 0x55ebd049b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9ae5a00640 1 --2- 172.21.15.100:0/2058961082 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55ebd044f140 0x55ebd044f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9adfe00640 1 -- 172.21.15.100:0/2058961082 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55ebd044f140 msgr2=0x55ebd044f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:16.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9adfe00640 1 --2- 172.21.15.100:0/2058961082 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55ebd044f140 0x55ebd044f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9adfe00640 1 -- 172.21.15.100:0/2058961082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ebd049b5b0 msgr2=0x55ebd049b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:16.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9adfe00640 1 --2- 172.21.15.100:0/2058961082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ebd049b5b0 0x55ebd049b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9adfe00640 1 -- 172.21.15.100:0/2058961082 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebd0239e40 con 0x55ebd049bed0 2024-04-23T14:32:16.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9adfe00640 1 --2- 172.21.15.100:0/2058961082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd049bed0 0x55ebd044ec00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9acc014ed0 tx=0x7f9acc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4745ef65658d74 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:16.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9ad1600640 1 -- 172.21.15.100:0/2058961082 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc0044c0 con 0x55ebd049bed0 2024-04-23T14:32:16.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9ad1600640 1 -- 172.21.15.100:0/2058961082 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9acc029070 con 0x55ebd049bed0 2024-04-23T14:32:16.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.964+0000 7f9ad1600640 1 -- 172.21.15.100:0/2058961082 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc024460 con 0x55ebd049bed0 2024-04-23T14:32:16.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9aeae48000 1 -- 172.21.15.100:0/2058961082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd049bed0 msgr2=0x55ebd044ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9aeae48000 1 --2- 172.21.15.100:0/2058961082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd049bed0 0x55ebd044ec00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9acc014ed0 tx=0x7f9acc017000 comp rx=0 tx=0).stop 2024-04-23T14:32:16.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9aeae48000 1 -- 172.21.15.100:0/2058961082 shutdown_connections 2024-04-23T14:32:16.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9aeae48000 1 --2- 172.21.15.100:0/2058961082 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55ebd044f140 0x55ebd044f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9aeae48000 1 --2- 172.21.15.100:0/2058961082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd049bed0 0x55ebd044ec00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9aeae48000 1 --2- 172.21.15.100:0/2058961082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ebd049b5b0 0x55ebd049b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9aeae48000 1 -- 172.21.15.100:0/2058961082 >> 172.21.15.100:0/2058961082 conn(0x55ebd047c350 msgr2=0x55ebd047e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:16.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9aeae48000 1 -- 172.21.15.100:0/2058961082 shutdown_connections 2024-04-23T14:32:16.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9aeae48000 1 -- 172.21.15.100:0/2058961082 wait complete. 2024-04-23T14:32:16.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9aeae48000 1 -- start start 2024-04-23T14:32:16.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9aeae48000 1 Processor -- start 2024-04-23T14:32:16.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9aeae48000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ebd044f140 0x55ebd0455250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9aeae48000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd049b5b0 0x55ebd0455790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9aeae48000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55ebd049bed0 0x55ebd0455cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9aeae48000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ebd035be50 con 0x55ebd049b5b0 2024-04-23T14:32:16.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9aeae48000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ebd025ca60 con 0x55ebd044f140 2024-04-23T14:32:16.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9aeae48000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x55ebd025acf0 con 0x55ebd049bed0 2024-04-23T14:32:16.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9aeae48000 10 client.0 _ll_register_callbacks cb 0x55ebd043ca70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:32:16.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9adfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd049b5b0 0x55ebd0455790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9adfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd049b5b0 0x55ebd0455790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48772/0 (socket says 172.21.15.100:48772) 2024-04-23T14:32:16.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9adfe00640 1 -- 172.21.15.100:0/944862068 learned_addr learned my addr 172.21.15.100:0/944862068 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:16.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9ae5000640 1 --2- 172.21.15.100:0/944862068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ebd044f140 0x55ebd0455250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9ae5a00640 1 --2- 172.21.15.100:0/944862068 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55ebd049bed0 0x55ebd0455cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9adfe00640 1 -- 172.21.15.100:0/944862068 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55ebd049bed0 msgr2=0x55ebd0455cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:16.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9adfe00640 1 --2- 172.21.15.100:0/944862068 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55ebd049bed0 0x55ebd0455cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9adfe00640 1 -- 172.21.15.100:0/944862068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ebd044f140 msgr2=0x55ebd0455250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:16.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9adfe00640 1 --2- 172.21.15.100:0/944862068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ebd044f140 0x55ebd0455250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9adfe00640 1 -- 172.21.15.100:0/944862068 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebd026ae00 con 0x55ebd049b5b0 2024-04-23T14:32:16.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9adfe00640 1 --2- 172.21.15.100:0/944862068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd049b5b0 0x55ebd0455790 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9acc026040 tx=0x7f9acc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb9544ac1b79088 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:16.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9ac6000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-04-23T14:32:16.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9ac6000640 1 -- 172.21.15.100:0/944862068 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc006240 con 0x55ebd049b5b0 2024-04-23T14:32:16.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9aeae48000 1 -- 172.21.15.100:0/944862068 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9acc029070 con 0x55ebd049b5b0 2024-04-23T14:32:16.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9ac6000640 1 -- 172.21.15.100:0/944862068 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9acc0063e0 con 0x55ebd049b5b0 2024-04-23T14:32:16.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9ac6000640 1 -- 172.21.15.100:0/944862068 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f9acc03e050 con 0x55ebd049b5b0 2024-04-23T14:32:16.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9aeae48000 10 client.4401 fetch_fsmap learned FSMap version 7 2024-04-23T14:32:16.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9aeae48000 1 -- 172.21.15.100:0/944862068 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55ebd0321670 con 0x55ebd049b5b0 2024-04-23T14:32:16.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.968+0000 7f9ac6000640 1 -- 172.21.15.100:0/944862068 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9acc029070 con 0x55ebd049b5b0 2024-04-23T14:32:16.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9ac6000640 1 -- 172.21.15.100:0/944862068 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f9acc041020 con 0x55ebd049b5b0 2024-04-23T14:32:16.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9aeae48000 10 client.4401 fetch_fsmap finished waiting for FSMap version 7 2024-04-23T14:32:16.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9aeae48000 10 client.4401 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-23T14:32:16.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9aeae48000 20 client.4401 populate_metadata read hostname 'smithi100' 2024-04-23T14:32:16.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9aeae48000 1 --2- 172.21.15.100:0/944862068 >> [v2:172.21.15.151:6833/1533322173,v1:172.21.15.151:6835/1533322173] conn(0x55ebd04565a0 0x55ebd04acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9aeae48000 4 client.4401 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:32:16.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9aeae48000 1 -- 172.21.15.100:0/944862068 --> [v2:172.21.15.151:6833/1533322173,v1:172.21.15.151:6835/1533322173] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebd0238bc0 con 0x55ebd04565a0 2024-04-23T14:32:16.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9ae5000640 1 --2- 172.21.15.100:0/944862068 >> [v2:172.21.15.151:6833/1533322173,v1:172.21.15.151:6835/1533322173] conn(0x55ebd04565a0 0x55ebd04acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9ae5000640 1 --2- 172.21.15.100:0/944862068 >> [v2:172.21.15.151:6833/1533322173,v1:172.21.15.151:6835/1533322173] conn(0x55ebd04565a0 0x55ebd04acd30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fefc2efb40f41ee3 server_cookie=379a489405bd731d in_seq=0 out_seq=0 2024-04-23T14:32:16.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9ac6000640 10 client.4401 ms_handle_connect on v2:172.21.15.151:6833/1533322173 2024-04-23T14:32:16.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9ac6000640 1 -- 172.21.15.100:0/944862068 <== mds.0 v2:172.21.15.151:6833/1533322173 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ebd0238bc0 con 0x55ebd04565a0 2024-04-23T14:32:16.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9ac6000640 10 client.4401 handle_command_reply: tid=0 2024-04-23T14:32:16.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9aeae48000 1 client.4401 shutdown 2024-04-23T14:32:16.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9aeae48000 1 -- 172.21.15.100:0/944862068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd049b5b0 msgr2=0x55ebd0455790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9aeae48000 1 --2- 172.21.15.100:0/944862068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd049b5b0 0x55ebd0455790 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9acc026040 tx=0x7f9acc004740 comp rx=0 tx=0).stop 2024-04-23T14:32:16.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9aeae48000 1 -- 172.21.15.100:0/944862068 shutdown_connections 2024-04-23T14:32:16.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9aeae48000 1 --2- 172.21.15.100:0/944862068 >> [v2:172.21.15.151:6833/1533322173,v1:172.21.15.151:6835/1533322173] conn(0x55ebd04565a0 0x55ebd04acd30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9aeae48000 1 --2- 172.21.15.100:0/944862068 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55ebd049bed0 0x55ebd0455cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9aeae48000 1 --2- 172.21.15.100:0/944862068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd049b5b0 0x55ebd0455790 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9ac6000640 0 client.4401 ms_handle_reset on v2:172.21.15.151:6833/1533322173 2024-04-23T14:32:16.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9aeae48000 1 --2- 172.21.15.100:0/944862068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ebd044f140 0x55ebd0455250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9aeae48000 1 -- 172.21.15.100:0/944862068 >> 172.21.15.100:0/944862068 conn(0x55ebd047c350 msgr2=0x55ebd047e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:16.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9aeae48000 1 -- 172.21.15.100:0/944862068 shutdown_connections 2024-04-23T14:32:16.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9aeae48000 1 -- 172.21.15.100:0/944862068 wait complete. 2024-04-23T14:32:16.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.972+0000 7f9aeae48000 20 client.4401 trim_cache size 0 max 16384 2024-04-23T14:32:16.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9aeae48000 1 Processor -- start 2024-04-23T14:32:16.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9aeae48000 1 -- start start 2024-04-23T14:32:16.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9aeae48000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd044f140 0x55ebd049aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9aeae48000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ebd049b5b0 0x55ebd049afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9aeae48000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55ebd049bed0 0x55ebd04540e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9aeae48000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae0201590 con 0x55ebd044f140 2024-04-23T14:32:16.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9aeae48000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae02008f0 con 0x55ebd049b5b0 2024-04-23T14:32:16.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9aeae48000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9ae0200f40 con 0x55ebd049bed0 2024-04-23T14:32:16.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9ae5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd044f140 0x55ebd049aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9ae5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd044f140 0x55ebd049aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48788/0 (socket says 172.21.15.100:48788) 2024-04-23T14:32:16.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9ae5000640 1 -- 172.21.15.100:0/1690871638 learned_addr learned my addr 172.21.15.100:0/1690871638 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:16.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9ae5000640 1 -- 172.21.15.100:0/1690871638 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55ebd049bed0 msgr2=0x55ebd04540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:16.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9ae5000640 1 --2- 172.21.15.100:0/1690871638 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55ebd049bed0 0x55ebd04540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9ae5000640 1 -- 172.21.15.100:0/1690871638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ebd049b5b0 msgr2=0x55ebd049afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:16.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9ae5000640 1 --2- 172.21.15.100:0/1690871638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ebd049b5b0 0x55ebd049afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9ae5000640 1 -- 172.21.15.100:0/1690871638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebd047ee90 con 0x55ebd044f140 2024-04-23T14:32:16.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9ae5000640 1 --2- 172.21.15.100:0/1690871638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd044f140 0x55ebd049aa90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40599c0 tx=0x7f9ad4088200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e664b57c22a410 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:16.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9ad0c00640 1 -- 172.21.15.100:0/1690871638 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad40628a0 con 0x55ebd044f140 2024-04-23T14:32:16.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9ad0c00640 1 -- 172.21.15.100:0/1690871638 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad4085070 con 0x55ebd044f140 2024-04-23T14:32:16.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9ad0c00640 1 -- 172.21.15.100:0/1690871638 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4057fc0 con 0x55ebd044f140 2024-04-23T14:32:17.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9aeae48000 1 -- 172.21.15.100:0/1690871638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd044f140 msgr2=0x55ebd049aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:17.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9aeae48000 1 --2- 172.21.15.100:0/1690871638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd044f140 0x55ebd049aa90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40599c0 tx=0x7f9ad4088200 comp rx=0 tx=0).stop 2024-04-23T14:32:17.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9aeae48000 1 -- 172.21.15.100:0/1690871638 shutdown_connections 2024-04-23T14:32:17.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9aeae48000 1 --2- 172.21.15.100:0/1690871638 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55ebd049bed0 0x55ebd04540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9aeae48000 1 --2- 172.21.15.100:0/1690871638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ebd049b5b0 0x55ebd049afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9aeae48000 1 --2- 172.21.15.100:0/1690871638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd044f140 0x55ebd049aa90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9aeae48000 1 -- 172.21.15.100:0/1690871638 >> 172.21.15.100:0/1690871638 conn(0x55ebd047c350 msgr2=0x55ebd04539d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:17.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9aeae48000 1 -- 172.21.15.100:0/1690871638 shutdown_connections 2024-04-23T14:32:17.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9aeae48000 1 -- 172.21.15.100:0/1690871638 wait complete. 2024-04-23T14:32:17.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9aeae48000 1 -- start start 2024-04-23T14:32:17.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.992+0000 7f9aeae48000 1 Processor -- start 2024-04-23T14:32:17.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd044f140 0x55ebd04add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:17.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55ebd049b5b0 0x55ebd04ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:17.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ebd049bed0 0x55ebd04ae800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:17.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ebd02a3b00 con 0x55ebd044f140 2024-04-23T14:32:17.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ebd029e7e0 con 0x55ebd049bed0 2024-04-23T14:32:17.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x55ebd02ac6e0 con 0x55ebd049b5b0 2024-04-23T14:32:17.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 10 client.0 _ll_register_callbacks cb 0x7f9ad408a0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:32:17.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9ae5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd044f140 0x55ebd04add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:17.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9ae5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd044f140 0x55ebd04add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48790/0 (socket says 172.21.15.100:48790) 2024-04-23T14:32:17.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9ae5000640 1 -- 172.21.15.100:0/2765533010 learned_addr learned my addr 172.21.15.100:0/2765533010 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:17.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9adfe00640 1 --2- 172.21.15.100:0/2765533010 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55ebd049b5b0 0x55ebd04ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:17.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9ae5a00640 1 --2- 172.21.15.100:0/2765533010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ebd049bed0 0x55ebd04ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:17.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9ae5000640 1 -- 172.21.15.100:0/2765533010 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55ebd049b5b0 msgr2=0x55ebd04ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:17.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9ae5000640 1 --2- 172.21.15.100:0/2765533010 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55ebd049b5b0 0x55ebd04ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9ae5000640 1 -- 172.21.15.100:0/2765533010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ebd049bed0 msgr2=0x55ebd04ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:17.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9ae5000640 1 --2- 172.21.15.100:0/2765533010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ebd049bed0 0x55ebd04ae800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9ae5000640 1 -- 172.21.15.100:0/2765533010 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebd02f8340 con 0x55ebd044f140 2024-04-23T14:32:17.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9ae5000640 1 --2- 172.21.15.100:0/2765533010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd044f140 0x55ebd04add80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4059c50 tx=0x7f9ad4022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e80edd00b2ccbc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:17.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9ac6000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-04-23T14:32:17.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9ac6000640 1 -- 172.21.15.100:0/2765533010 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad407d060 con 0x55ebd044f140 2024-04-23T14:32:17.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9ac6000640 1 -- 172.21.15.100:0/2765533010 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9ad40883b0 con 0x55ebd044f140 2024-04-23T14:32:17.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 1 -- 172.21.15.100:0/2765533010 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ad4085070 con 0x55ebd044f140 2024-04-23T14:32:17.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9ac6000640 1 -- 172.21.15.100:0/2765533010 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f9ad4056330 con 0x55ebd044f140 2024-04-23T14:32:17.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 10 client.4407 fetch_fsmap learned FSMap version 7 2024-04-23T14:32:17.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 1 -- 172.21.15.100:0/2765533010 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55ebd02317c0 con 0x55ebd044f140 2024-04-23T14:32:17.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9ac6000640 1 -- 172.21.15.100:0/2765533010 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad4085070 con 0x55ebd044f140 2024-04-23T14:32:17.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9ac6000640 1 -- 172.21.15.100:0/2765533010 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f9ad4062410 con 0x55ebd044f140 2024-04-23T14:32:17.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 10 client.4407 fetch_fsmap finished waiting for FSMap version 7 2024-04-23T14:32:17.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 10 client.4407 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-23T14:32:17.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 20 client.4407 populate_metadata read hostname 'smithi100' 2024-04-23T14:32:17.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 1 --2- 172.21.15.100:0/2765533010 >> [v2:172.21.15.151:6833/1533322173,v1:172.21.15.151:6835/1533322173] conn(0x55ebd04af160 0x55ebd051b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:17.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 4 client.4407 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:32:17.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 1 -- 172.21.15.100:0/2765533010 --> [v2:172.21.15.151:6833/1533322173,v1:172.21.15.151:6835/1533322173] -- command(tid 0: {"prefix": "status"}) -- 0x55ebd02744e0 con 0x55ebd04af160 2024-04-23T14:32:17.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9adfe00640 1 --2- 172.21.15.100:0/2765533010 >> [v2:172.21.15.151:6833/1533322173,v1:172.21.15.151:6835/1533322173] conn(0x55ebd04af160 0x55ebd051b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:17.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9ac6000640 10 client.4407 ms_handle_connect on v2:172.21.15.151:6833/1533322173 2024-04-23T14:32:17.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9adfe00640 1 --2- 172.21.15.100:0/2765533010 >> [v2:172.21.15.151:6833/1533322173,v1:172.21.15.151:6835/1533322173] conn(0x55ebd04af160 0x55ebd051b540 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99b2f7c9828a25f2 server_cookie=77122262b2d85c00 in_seq=0 out_seq=0 2024-04-23T14:32:17.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9ac6000640 1 -- 172.21.15.100:0/2765533010 <== mds.0 v2:172.21.15.151:6833/1533322173 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55ebd02744e0 con 0x55ebd04af160 2024-04-23T14:32:17.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9ac6000640 10 client.4407 handle_command_reply: tid=0 2024-04-23T14:32:17.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 1 client.4407 shutdown 2024-04-23T14:32:17.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 1 -- 172.21.15.100:0/2765533010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd044f140 msgr2=0x55ebd04add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:17.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 1 --2- 172.21.15.100:0/2765533010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd044f140 0x55ebd04add80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4059c50 tx=0x7f9ad4022040 comp rx=0 tx=0).stop 2024-04-23T14:32:17.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 1 -- 172.21.15.100:0/2765533010 shutdown_connections 2024-04-23T14:32:17.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 1 --2- 172.21.15.100:0/2765533010 >> [v2:172.21.15.151:6833/1533322173,v1:172.21.15.151:6835/1533322173] conn(0x55ebd04af160 0x55ebd051b540 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 1 --2- 172.21.15.100:0/2765533010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ebd049bed0 0x55ebd04ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 1 --2- 172.21.15.100:0/2765533010 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55ebd049b5b0 0x55ebd04ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 1 --2- 172.21.15.100:0/2765533010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ebd044f140 0x55ebd04add80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9ac6000640 0 client.4407 ms_handle_reset on v2:172.21.15.151:6833/1533322173 2024-04-23T14:32:17.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 1 -- 172.21.15.100:0/2765533010 >> 172.21.15.100:0/2765533010 conn(0x55ebd047c350 msgr2=0x55ebd0454260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:17.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 1 -- 172.21.15.100:0/2765533010 shutdown_connections 2024-04-23T14:32:17.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 1 -- 172.21.15.100:0/2765533010 wait complete. 2024-04-23T14:32:17.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:16.996+0000 7f9aeae48000 20 client.4407 trim_cache size 0 max 16384 2024-04-23T14:32:17.007 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:32:17.007 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "167d2af1-01c4-4f87-8bce-0de22791f823", 2024-04-23T14:32:17.007 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-04-23T14:32:17.007 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 4279, 2024-04-23T14:32:17.007 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-04-23T14:32:17.007 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-04-23T14:32:17.008 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-04-23T14:32:17.008 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 7.3983532189999996, 2024-04-23T14:32:17.008 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 7, 2024-04-23T14:32:17.008 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 24, 2024-04-23T14:32:17.008 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 24, 2024-04-23T14:32:17.008 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 14.783562232 2024-04-23T14:32:17.008 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:32:17.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.000+0000 7f9ae7a00640 1 -- 172.21.15.100:0/1110293571 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9ac0068380 msgr2=0x7f9ac006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:17.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.000+0000 7f9ae7a00640 1 --2- 172.21.15.100:0/1110293571 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9ac0068380 0x7f9ac006a840 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00c6d40 tx=0x7f9acc008040 comp rx=0 tx=0).stop 2024-04-23T14:32:17.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.000+0000 7f9ae7a00640 1 -- 172.21.15.100:0/1110293571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac8004d50 msgr2=0x7f9ae015e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:17.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.000+0000 7f9ae7a00640 1 --2- 172.21.15.100:0/1110293571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac8004d50 0x7f9ae015e180 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4012540 tx=0x7f9ad4012a00 comp rx=0 tx=0).stop 2024-04-23T14:32:17.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.000+0000 7f9ae7a00640 1 -- 172.21.15.100:0/1110293571 shutdown_connections 2024-04-23T14:32:17.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.000+0000 7f9ae7a00640 1 --2- 172.21.15.100:0/1110293571 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9ac0068380 0x7f9ac006a840 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.000+0000 7f9ae7a00640 1 --2- 172.21.15.100:0/1110293571 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ae00c80c0 0x7f9ae00c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.000+0000 7f9ae7a00640 1 --2- 172.21.15.100:0/1110293571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ae00adb70 0x7f9ae00c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.000+0000 7f9ae7a00640 1 --2- 172.21.15.100:0/1110293571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac8004d50 0x7f9ae015e180 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.000+0000 7f9ae7a00640 1 -- 172.21.15.100:0/1110293571 >> 172.21.15.100:0/1110293571 conn(0x7f9ae00b5310 msgr2=0x7f9ae0072680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:17.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.000+0000 7f9ae7a00640 1 -- 172.21.15.100:0/1110293571 shutdown_connections 2024-04-23T14:32:17.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.000+0000 7f9ae7a00640 1 -- 172.21.15.100:0/1110293571 wait complete. 2024-04-23T14:32:17.020 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-23T14:32:17.020 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-23T14:32:17.020 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:32:17.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.152+0000 7fcf13600640 1 Processor -- start 2024-04-23T14:32:17.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.152+0000 7fcf13600640 1 -- start start 2024-04-23T14:32:17.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.156+0000 7fcf13600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcf0c0b2a10 0x7fcf0c0adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:17.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.156+0000 7fcf13600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c073610 0x7fcf0c06df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:17.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.156+0000 7fcf13600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf0c0af080 con 0x7fcf0c15b640 2024-04-23T14:32:17.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.156+0000 7fcf13600640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fcf0c0afbc0 con 0x7fcf0c0b2a10 2024-04-23T14:32:17.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.156+0000 7fcf13600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fcf0c15ba20 con 0x7fcf0c073610 2024-04-23T14:32:17.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.156+0000 7fcf0be00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcf0c0b2a10 0x7fcf0c0adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:17.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.156+0000 7fcf10c00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fcf0c15b640 0x7fcf0c072f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:34200/0 (socket says 172.21.15.100:34200) 2024-04-23T14:32:17.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.156+0000 7fcf0be00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcf0c0b2a10 0x7fcf0c0adfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48796/0 (socket says 172.21.15.100:48796) 2024-04-23T14:32:17.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.156+0000 7fcf10c00640 1 -- 172.21.15.100:0/3207969740 learned_addr learned my addr 172.21.15.100:0/3207969740 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:17.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.156+0000 7fcf11600640 1 --2- 172.21.15.100:0/3207969740 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c073610 0x7fcf0c06df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:17.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.156+0000 7fcf0be00640 1 -- 172.21.15.100:0/3207969740 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c073610 msgr2=0x7fcf0c06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:17.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.156+0000 7fcf0be00640 1 --2- 172.21.15.100:0/3207969740 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c073610 0x7fcf0c06df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.156+0000 7fcf0be00640 1 -- 172.21.15.100:0/3207969740 >> v1:172.21.15.100:6789/0 conn(0x7fcf0c15b640 legacy=0x7fcf0c072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:17.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.156+0000 7fcf0be00640 1 -- 172.21.15.100:0/3207969740 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0c0af320 con 0x7fcf0c0b2a10 2024-04-23T14:32:17.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.156+0000 7fcf0b400640 1 -- 172.21.15.100:0/3207969740 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2767955739 0 0) 0x7fcf0c0af080 con 0x7fcf0c15b640 2024-04-23T14:32:17.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.156+0000 7fcf0be00640 1 --2- 172.21.15.100:0/3207969740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf0c0b2a10 0x7fcf0c0adfc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcef80097c0 tx=0x7fcef801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b6d176c80b8d19 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:17.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.156+0000 7fcf0b400640 1 -- 172.21.15.100:0/3207969740 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef80a7070 con 0x7fcf0c0b2a10 2024-04-23T14:32:17.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.156+0000 7fcf0b400640 1 -- 172.21.15.100:0/3207969740 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcef80b0070 con 0x7fcf0c0b2a10 2024-04-23T14:32:17.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.156+0000 7fcf0b400640 1 -- 172.21.15.100:0/3207969740 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef80ab890 con 0x7fcf0c0b2a10 2024-04-23T14:32:17.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.160+0000 7fcf13600640 1 -- 172.21.15.100:0/3207969740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf0c0b2a10 msgr2=0x7fcf0c0adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:17.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.160+0000 7fcf13600640 1 --2- 172.21.15.100:0/3207969740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf0c0b2a10 0x7fcf0c0adfc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcef80097c0 tx=0x7fcef801c470 comp rx=0 tx=0).stop 2024-04-23T14:32:17.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.160+0000 7fcf13600640 1 -- 172.21.15.100:0/3207969740 shutdown_connections 2024-04-23T14:32:17.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.160+0000 7fcf13600640 1 --2- 172.21.15.100:0/3207969740 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c073610 0x7fcf0c06df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.160+0000 7fcf13600640 1 --2- 172.21.15.100:0/3207969740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf0c0b2a10 0x7fcf0c0adfc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.160+0000 7fcf13600640 1 -- 172.21.15.100:0/3207969740 >> 172.21.15.100:0/3207969740 conn(0x7fcf0c0b1e80 msgr2=0x7fcf0c0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:17.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.160+0000 7fcf13600640 1 -- 172.21.15.100:0/3207969740 shutdown_connections 2024-04-23T14:32:17.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.160+0000 7fcf13600640 1 -- 172.21.15.100:0/3207969740 wait complete. 2024-04-23T14:32:17.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.160+0000 7fcf13600640 1 Processor -- start 2024-04-23T14:32:17.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.160+0000 7fcf13600640 1 -- start start 2024-04-23T14:32:17.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.160+0000 7fcf13600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf0c073610 0x7fcf0c1fa990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.160+0000 7fcf13600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf0c0b2a10 0x7fcf0c1f7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.160+0000 7fcf13600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c15b640 0x7fcf0c14a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.160+0000 7fcf13600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c06f0d0 con 0x7fcf0c073610 2024-04-23T14:32:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.160+0000 7fcf13600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c06ef50 con 0x7fcf0c0b2a10 2024-04-23T14:32:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.160+0000 7fcf13600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fcf0c06f250 con 0x7fcf0c15b640 2024-04-23T14:32:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.160+0000 7fcf10c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf0c073610 0x7fcf0c1fa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.160+0000 7fcf10c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf0c073610 0x7fcf0c1fa990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48806/0 (socket says 172.21.15.100:48806) 2024-04-23T14:32:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.160+0000 7fcf10c00640 1 -- 172.21.15.100:0/2018150153 learned_addr learned my addr 172.21.15.100:0/2018150153 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.160+0000 7fcf0be00640 1 --2- 172.21.15.100:0/2018150153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf0c0b2a10 0x7fcf0c1f7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.160+0000 7fcf11600640 1 --2- 172.21.15.100:0/2018150153 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c15b640 0x7fcf0c14a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.164+0000 7fcf10c00640 1 -- 172.21.15.100:0/2018150153 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c15b640 msgr2=0x7fcf0c14a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.164+0000 7fcf10c00640 1 --2- 172.21.15.100:0/2018150153 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c15b640 0x7fcf0c14a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.164+0000 7fcf10c00640 1 -- 172.21.15.100:0/2018150153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf0c0b2a10 msgr2=0x7fcf0c1f7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.164+0000 7fcf10c00640 1 --2- 172.21.15.100:0/2018150153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf0c0b2a10 0x7fcf0c1f7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.164+0000 7fcf10c00640 1 -- 172.21.15.100:0/2018150153 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0c1faf70 con 0x7fcf0c073610 2024-04-23T14:32:17.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.164+0000 7fcf11600640 1 --2- 172.21.15.100:0/2018150153 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c15b640 0x7fcf0c14a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:17.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.164+0000 7fcf10c00640 1 --2- 172.21.15.100:0/2018150153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf0c073610 0x7fcf0c1fa990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00c7c0 tx=0x7fcefc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:17.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.164+0000 7fcf08c00640 1 -- 172.21.15.100:0/2018150153 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc018070 con 0x7fcf0c073610 2024-04-23T14:32:17.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.164+0000 7fcf08c00640 1 -- 172.21.15.100:0/2018150153 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcefc00a640 con 0x7fcf0c073610 2024-04-23T14:32:17.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.164+0000 7fcf08c00640 1 -- 172.21.15.100:0/2018150153 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc00aa30 con 0x7fcf0c073610 2024-04-23T14:32:17.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.164+0000 7fcf13600640 1 -- 172.21.15.100:0/2018150153 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf0c1fb250 con 0x7fcf0c073610 2024-04-23T14:32:17.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.164+0000 7fcf13600640 1 -- 172.21.15.100:0/2018150153 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf0c1fb810 con 0x7fcf0c073610 2024-04-23T14:32:17.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.164+0000 7fcf13600640 1 -- 172.21.15.100:0/2018150153 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf0c068ad0 con 0x7fcf0c073610 2024-04-23T14:32:17.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.164+0000 7fcf08c00640 1 -- 172.21.15.100:0/2018150153 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcefc00abd0 con 0x7fcf0c073610 2024-04-23T14:32:17.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.164+0000 7fcf08c00640 1 --2- 172.21.15.100:0/2018150153 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fceec068380 0x7fceec06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:17.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.164+0000 7fcf08c00640 1 -- 172.21.15.100:0/2018150153 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fcefc0862c0 con 0x7fcf0c073610 2024-04-23T14:32:17.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.164+0000 7fcf0be00640 1 --2- 172.21.15.100:0/2018150153 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fceec068380 0x7fceec06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:17.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.168+0000 7fcf0be00640 1 --2- 172.21.15.100:0/2018150153 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fceec068380 0x7fceec06a840 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcef80af000 tx=0x7fcef80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:17.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.168+0000 7fcf08c00640 1 -- 172.21.15.100:0/2018150153 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcefc055f80 con 0x7fcf0c073610 2024-04-23T14:32:17.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.364+0000 7fcf13600640 1 -- 172.21.15.100:0/2018150153 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcf0c0b5310 con 0x7fcf0c073610 2024-04-23T14:32:17.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.364+0000 7fcf08c00640 1 -- 172.21.15.100:0/2018150153 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7fcefc05a280 con 0x7fcf0c073610 2024-04-23T14:32:17.367 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-23T14:32:17.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.364+0000 7fcf13600640 1 -- 172.21.15.100:0/2018150153 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fceec068380 msgr2=0x7fceec06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:17.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.364+0000 7fcf13600640 1 --2- 172.21.15.100:0/2018150153 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fceec068380 0x7fceec06a840 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcef80af000 tx=0x7fcef80a7000 comp rx=0 tx=0).stop 2024-04-23T14:32:17.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.364+0000 7fcf13600640 1 -- 172.21.15.100:0/2018150153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf0c073610 msgr2=0x7fcf0c1fa990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:17.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.364+0000 7fcf13600640 1 --2- 172.21.15.100:0/2018150153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf0c073610 0x7fcf0c1fa990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00c7c0 tx=0x7fcefc00cc80 comp rx=0 tx=0).stop 2024-04-23T14:32:17.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.364+0000 7fcf13600640 1 -- 172.21.15.100:0/2018150153 shutdown_connections 2024-04-23T14:32:17.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.364+0000 7fcf13600640 1 --2- 172.21.15.100:0/2018150153 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fceec068380 0x7fceec06a840 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.364+0000 7fcf13600640 1 --2- 172.21.15.100:0/2018150153 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c15b640 0x7fcf0c14a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.364+0000 7fcf13600640 1 --2- 172.21.15.100:0/2018150153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf0c0b2a10 0x7fcf0c1f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.364+0000 7fcf13600640 1 --2- 172.21.15.100:0/2018150153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf0c073610 0x7fcf0c1fa990 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.368+0000 7fcf13600640 1 -- 172.21.15.100:0/2018150153 >> 172.21.15.100:0/2018150153 conn(0x7fcf0c0b1e80 msgr2=0x7fcf0c072670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:17.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.368+0000 7fcf13600640 1 -- 172.21.15.100:0/2018150153 shutdown_connections 2024-04-23T14:32:17.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17.368+0000 7fcf13600640 1 -- 172.21.15.100:0/2018150153 wait complete. 2024-04-23T14:32:17.388 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-23T14:32:17.388 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:32:17.388 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-04-23T14:32:17.388 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-23T14:32:17.388 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-23T14:32:17.388 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-23T14:32:17.388 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:32:17.388 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:32:17.388 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-23T14:32:17.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:17.389 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-04-23T14:32:17.398 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:32:17.398 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:32:17.398 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:32:17.398 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:17.398 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-23T14:32:17.398 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:17.398 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:32:17.398 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:17.398 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-04-23T14:32:17.398 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:32:17.398 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:17.399 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-04-23T14:32:17.399 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:32:17.399 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:17.399 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:32:17.399 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:17.399 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-23T14:32:17.399 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42446sec preferred_lft 42446sec 2024-04-23T14:32:17.399 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-04-23T14:32:17.399 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:17.399 INFO:teuthology.orchestra.run.smithi100.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-23T14:32:17.399 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 02:42:ff:fc:d1:cc brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:17.399 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-23T14:32:17.399 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:17.400 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-23T14:32:17.400 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:17.400 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-23T14:32:17.400 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add name ceph-brx type bridge 2024-04-23T14:32:17.400 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr flush dev ceph-brx 2024-04-23T14:32:17.400 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set ceph-brx up 2024-04-23T14:32:17.400 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-23T14:32:17.400 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-23T14:32:17.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:17.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:17.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:17.553 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:32:17.571 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-04-23T14:32:17.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:17.574 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-04-23T14:32:17.652 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-04-23T14:32:17.652 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:32:17.652 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-23T14:32:17.652 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-23T14:32:17.652 INFO:teuthology.orchestra.run.smithi100.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-23T14:32:17.652 INFO:teuthology.orchestra.run.smithi100.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-23T14:32:17.652 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-23T14:32:17.652 INFO:teuthology.orchestra.run.smithi100.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:32:17.652 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:32:17.652 INFO:teuthology.orchestra.run.smithi100.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:32:17.652 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:32:17.653 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:17.653 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-23T14:32:17.653 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-23T14:32:17.653 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-23T14:32:17.653 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-23T14:32:17.653 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-23T14:32:17.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:17.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:17.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:17.791 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-23T14:32:17.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:17.800 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2024-04-23T14:32:17.849 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:17.849 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-23T14:32:17.849 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-23T14:32:17.849 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-23T14:32:17.849 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-23T14:32:17.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:17.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:17.974 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-23T14:32:17.974 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:17.974 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-23T14:32:17.974 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-23T14:32:17.974 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-23T14:32:17.975 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-23T14:32:17.975 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-23T14:32:17.975 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-23T14:32:17.975 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-23T14:32:18.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:18.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:18.147 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:18.147 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-23T14:32:18.147 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-04-23T14:32:18.147 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.0 master ceph-brx 2024-04-23T14:32:18.147 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-23T14:32:18.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:18.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:18.271 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-23T14:32:18.271 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:32:18.272 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-23T14:32:18.321 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-23T14:32:18.321 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:32:18.321 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-23T14:32:18.376 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-23T14:32:18.440 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:32:18.493 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-23T14:32:18.494 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-23T14:32:18.494 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-23T14:32:18.494 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-23T14:32:18.494 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-23T14:32:18.494 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-23T14:32:18.494 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-23T14:32:18.494 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-23T14:32:18.494 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-23T14:32:18.494 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-23T14:32:18.494 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-23T14:32:18.495 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-23T14:32:18.495 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:32:18.495 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-23T14:32:18.495 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-23T14:32:18.495 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-23T14:32:18.495 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-23T14:32:18.495 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-23T14:32:18.495 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-23T14:32:18.495 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-23T14:32:18.495 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:32:18.495 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-23T14:32:18.495 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-23T14:32:18.495 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-23T14:32:18.495 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:32:18.496 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:32:18.496 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:32:18.496 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:32:18.496 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-23T14:32:18.496 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-23T14:32:18.496 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-23T14:32:18.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:18.497 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-23T14:32:18.548 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:32:18.548 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-23T14:32:18.591 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-23T14:32:18.612 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:32:18.662 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-23T14:32:18.662 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-23T14:32:18.662 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-23T14:32:18.662 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-23T14:32:18.662 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-23T14:32:18.662 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-23T14:32:18.662 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-23T14:32:18.663 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-23T14:32:18.663 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-23T14:32:18.663 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-23T14:32:18.663 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-23T14:32:18.663 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-23T14:32:18.663 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:32:18.663 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-23T14:32:18.663 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-23T14:32:18.663 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-23T14:32:18.663 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-23T14:32:18.663 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-23T14:32:18.663 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-23T14:32:18.664 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-23T14:32:18.664 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:32:18.664 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-23T14:32:18.664 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-23T14:32:18.664 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-23T14:32:18.664 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:32:18.664 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:32:18.664 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:32:18.664 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:32:18.664 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-23T14:32:18.664 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-23T14:32:18.664 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-23T14:32:18.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:18.666 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-23T14:32:18.688 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi100.stderr:2024-04-23T14:32:18.684+0000 7fa9f5d0f580 -1 init, newargv = 0x55a034ddb6c0 newargc=15 2024-04-23T14:32:18.688 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi100.stderr:ceph-fuse[33968]: starting ceph client 2024-04-23T14:32:18.697 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi100.stderr:ceph-fuse[33968]: starting fuse 2024-04-23T14:32:19.796 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-23T14:32:19.819 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:32:19.875 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-23T14:32:19.875 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-23T14:32:19.875 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-23T14:32:19.875 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-23T14:32:19.875 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-23T14:32:19.875 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-23T14:32:19.875 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-23T14:32:19.875 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-23T14:32:19.875 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-23T14:32:19.875 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-23T14:32:19.876 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-23T14:32:19.876 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-23T14:32:19.876 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:32:19.876 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-23T14:32:19.876 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-23T14:32:19.876 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-23T14:32:19.876 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-23T14:32:19.876 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-23T14:32:19.876 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-23T14:32:19.876 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-23T14:32:19.876 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:32:19.876 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-23T14:32:19.876 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-23T14:32:19.876 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-23T14:32:19.876 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:32:19.877 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:32:19.877 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:32:19.877 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:32:19.877 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-23T14:32:19.877 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-23T14:32:19.877 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-23T14:32:19.877 INFO:teuthology.orchestra.run.smithi100.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-23T14:32:19.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:19.878 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-23T14:32:19.933 INFO:teuthology.orchestra.run.smithi100.stdout:44 2024-04-23T14:32:19.933 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-23T14:32:19.934 DEBUG:teuthology.orchestra.run.smithi100:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:32:19.934 DEBUG:teuthology.orchestra.run.smithi100:> import glob 2024-04-23T14:32:19.934 DEBUG:teuthology.orchestra.run.smithi100:> import re 2024-04-23T14:32:19.934 DEBUG:teuthology.orchestra.run.smithi100:> import os 2024-04-23T14:32:19.934 DEBUG:teuthology.orchestra.run.smithi100:> import subprocess 2024-04-23T14:32:19.934 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:32:19.934 DEBUG:teuthology.orchestra.run.smithi100:> def _find_admin_socket(client_name): 2024-04-23T14:32:19.934 DEBUG:teuthology.orchestra.run.smithi100:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-23T14:32:19.934 DEBUG:teuthology.orchestra.run.smithi100:> files = glob.glob(asok_path) 2024-04-23T14:32:19.934 DEBUG:teuthology.orchestra.run.smithi100:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-23T14:32:19.934 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:32:19.934 DEBUG:teuthology.orchestra.run.smithi100:> # Given a non-glob path, it better be there 2024-04-23T14:32:19.934 DEBUG:teuthology.orchestra.run.smithi100:> if "*" not in asok_path: 2024-04-23T14:32:19.935 DEBUG:teuthology.orchestra.run.smithi100:> assert(len(files) == 1) 2024-04-23T14:32:19.935 DEBUG:teuthology.orchestra.run.smithi100:> return files[0] 2024-04-23T14:32:19.935 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:32:19.935 DEBUG:teuthology.orchestra.run.smithi100:> for f in files: 2024-04-23T14:32:19.935 DEBUG:teuthology.orchestra.run.smithi100:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:32:19.935 DEBUG:teuthology.orchestra.run.smithi100:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:32:19.935 DEBUG:teuthology.orchestra.run.smithi100:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:32:19.935 DEBUG:teuthology.orchestra.run.smithi100:> contents = proc_f.read() 2024-04-23T14:32:19.935 DEBUG:teuthology.orchestra.run.smithi100:> if mountpoint in contents: 2024-04-23T14:32:19.935 DEBUG:teuthology.orchestra.run.smithi100:> return f 2024-04-23T14:32:19.935 DEBUG:teuthology.orchestra.run.smithi100:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:32:19.935 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:32:19.935 DEBUG:teuthology.orchestra.run.smithi100:> print(_find_admin_socket("client.admin")) 2024-04-23T14:32:19.935 DEBUG:teuthology.orchestra.run.smithi100:> ' 2024-04-23T14:32:20.063 INFO:teuthology.orchestra.run.smithi100.stdout:/var/run/ceph/ceph-client.admin.33968.asok 2024-04-23T14:32:20.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:20.075 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.33968.asok 2024-04-23T14:32:20.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:20.075 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.33968.asok status 2024-04-23T14:32:20.248 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:32:20.248 INFO:teuthology.orchestra.run.smithi100.stdout: "metadata": { 2024-04-23T14:32:20.248 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:32:20.248 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:32:20.248 INFO:teuthology.orchestra.run.smithi100.stdout: "entity_id": "admin", 2024-04-23T14:32:20.248 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": "smithi100", 2024-04-23T14:32:20.249 INFO:teuthology.orchestra.run.smithi100.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-23T14:32:20.249 INFO:teuthology.orchestra.run.smithi100.stdout: "pid": "33968", 2024-04-23T14:32:20.249 INFO:teuthology.orchestra.run.smithi100.stdout: "root": "/" 2024-04-23T14:32:20.249 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:32:20.249 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_count": 0, 2024-04-23T14:32:20.249 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_pinned_count": 0, 2024-04-23T14:32:20.249 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 4425, 2024-04-23T14:32:20.249 INFO:teuthology.orchestra.run.smithi100.stdout: "inst": { 2024-04-23T14:32:20.249 INFO:teuthology.orchestra.run.smithi100.stdout: "name": { 2024-04-23T14:32:20.249 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "client", 2024-04-23T14:32:20.249 INFO:teuthology.orchestra.run.smithi100.stdout: "num": 4425 2024-04-23T14:32:20.249 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:32:20.249 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-23T14:32:20.249 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-23T14:32:20.249 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:32:20.250 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2022928680 2024-04-23T14:32:20.250 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:32:20.250 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:32:20.250 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-23T14:32:20.250 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-23T14:32:20.250 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:32:20.250 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2022928680 2024-04-23T14:32:20.250 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:32:20.250 INFO:teuthology.orchestra.run.smithi100.stdout: "inst_str": "client.4425 192.168.0.1:0/2022928680", 2024-04-23T14:32:20.250 INFO:teuthology.orchestra.run.smithi100.stdout: "addr_str": "192.168.0.1:0/2022928680", 2024-04-23T14:32:20.250 INFO:teuthology.orchestra.run.smithi100.stdout: "inode_count": 1, 2024-04-23T14:32:20.251 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_epoch": 7, 2024-04-23T14:32:20.251 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch": 24, 2024-04-23T14:32:20.251 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:32:20.251 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklisted": false, 2024-04-23T14:32:20.251 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs" 2024-04-23T14:32:20.251 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:32:20.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:20.263 DEBUG:teuthology.orchestra.run.smithi100:> stat --file-system '--printf=%T 2024-04-23T14:32:20.263 DEBUG:teuthology.orchestra.run.smithi100:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-23T14:32:20.272 INFO:teuthology.orchestra.run.smithi100.stdout:fuseblk 2024-04-23T14:32:20.273 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-23T14:32:20.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:20.273 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-23T14:32:20.341 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-23T14:32:20.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-23T14:32:21.533 INFO:teuthology.orchestra.run.smithi100.stdout:# file: . 2024-04-23T14:32:21.533 INFO:teuthology.orchestra.run.smithi100.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-23T14:32:21.533 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:21.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:21.542 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:32:21.542 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:32:21.552 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:21.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.736+0000 7f848c400640 1 Processor -- start 2024-04-23T14:32:21.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.736+0000 7f848c400640 1 -- start start 2024-04-23T14:32:21.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.740+0000 7f848c400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f848415ad40 0x7f84840af390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:21.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.740+0000 7f848c400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84840af970 0x7f8484072760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:21.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.740+0000 7f848c400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f8484072ca0 0x7f848415e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:21.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.740+0000 7f848c400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f84840ad720 con 0x7f848415ad40 2024-04-23T14:32:21.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.740+0000 7f848c400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f848406d410 con 0x7f8484072ca0 2024-04-23T14:32:21.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.740+0000 7f848c400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f848406d130 con 0x7f84840af970 2024-04-23T14:32:21.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.740+0000 7f8489a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f848415ad40 0x7f84840af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:21.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.740+0000 7f8489a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f848415ad40 0x7f84840af390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39610/0 (socket says 172.21.15.100:39610) 2024-04-23T14:32:21.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.740+0000 7f8489a00640 1 -- 172.21.15.100:0/622006957 learned_addr learned my addr 172.21.15.100:0/622006957 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:21.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.740+0000 7f8489000640 1 --2- 172.21.15.100:0/622006957 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84840af970 0x7f8484072760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:21.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.740+0000 7f848a400640 1 --2- 172.21.15.100:0/622006957 >> v2:172.21.15.151:3300/0 conn(0x7f8484072ca0 0x7f848415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:21.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.740+0000 7f8489a00640 1 -- 172.21.15.100:0/622006957 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84840af970 msgr2=0x7f8484072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:21.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.740+0000 7f8489a00640 1 --2- 172.21.15.100:0/622006957 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84840af970 0x7f8484072760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.740+0000 7f8489a00640 1 -- 172.21.15.100:0/622006957 >> v2:172.21.15.151:3300/0 conn(0x7f8484072ca0 msgr2=0x7f848415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:21.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.740+0000 7f8489a00640 1 --2- 172.21.15.100:0/622006957 >> v2:172.21.15.151:3300/0 conn(0x7f8484072ca0 0x7f848415e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.740+0000 7f8489a00640 1 -- 172.21.15.100:0/622006957 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8484071e10 con 0x7f848415ad40 2024-04-23T14:32:21.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.740+0000 7f848a400640 1 --2- 172.21.15.100:0/622006957 >> v2:172.21.15.151:3300/0 conn(0x7f8484072ca0 0x7f848415e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:21.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.744+0000 7f8489a00640 1 --2- 172.21.15.100:0/622006957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f848415ad40 0x7f84840af390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8484075360 tx=0x7f847801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8c6487220a3be2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:21.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.744+0000 7f8483e00640 1 -- 172.21.15.100:0/622006957 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84780b0070 con 0x7f848415ad40 2024-04-23T14:32:21.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.744+0000 7f8483e00640 1 -- 172.21.15.100:0/622006957 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84780ab600 con 0x7f848415ad40 2024-04-23T14:32:21.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.744+0000 7f8483e00640 1 -- 172.21.15.100:0/622006957 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84780ab9d0 con 0x7f848415ad40 2024-04-23T14:32:21.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.744+0000 7f848c400640 1 -- 172.21.15.100:0/622006957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f848415ad40 msgr2=0x7f84840af390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:21.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.744+0000 7f848c400640 1 --2- 172.21.15.100:0/622006957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f848415ad40 0x7f84840af390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8484075360 tx=0x7f847801c490 comp rx=0 tx=0).stop 2024-04-23T14:32:21.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.744+0000 7f848c400640 1 -- 172.21.15.100:0/622006957 shutdown_connections 2024-04-23T14:32:21.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.744+0000 7f848c400640 1 --2- 172.21.15.100:0/622006957 >> v2:172.21.15.151:3300/0 conn(0x7f8484072ca0 0x7f848415e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.744+0000 7f848c400640 1 --2- 172.21.15.100:0/622006957 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84840af970 0x7f8484072760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.744+0000 7f848c400640 1 --2- 172.21.15.100:0/622006957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f848415ad40 0x7f84840af390 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.744+0000 7f848c400640 1 -- 172.21.15.100:0/622006957 >> 172.21.15.100:0/622006957 conn(0x7f84840b1e80 msgr2=0x7f84840b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:21.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.744+0000 7f848c400640 1 -- 172.21.15.100:0/622006957 shutdown_connections 2024-04-23T14:32:21.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.744+0000 7f848c400640 1 -- 172.21.15.100:0/622006957 wait complete. 2024-04-23T14:32:21.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.744+0000 7f848c400640 1 Processor -- start 2024-04-23T14:32:21.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.744+0000 7f848c400640 1 -- start start 2024-04-23T14:32:21.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f848c400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8484072ca0 0x7f8484150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:21.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f848c400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84840af970 0x7f84841645f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:21.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f848c400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f848415ad40 0x7f8484164b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:21.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f848c400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f84840730b0 con 0x7f8484072ca0 2024-04-23T14:32:21.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f848c400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f84840aeeb0 con 0x7f848415ad40 2024-04-23T14:32:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f848c400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f848415ec90 con 0x7f84840af970 2024-04-23T14:32:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f8489a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8484072ca0 0x7f8484150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f8489a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8484072ca0 0x7f8484150900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39618/0 (socket says 172.21.15.100:39618) 2024-04-23T14:32:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f8489a00640 1 -- 172.21.15.100:0/3152943543 learned_addr learned my addr 172.21.15.100:0/3152943543 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f8489000640 1 --2- 172.21.15.100:0/3152943543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84840af970 0x7f84841645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f848a400640 1 --2- 172.21.15.100:0/3152943543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f848415ad40 0x7f8484164b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f8489a00640 1 -- 172.21.15.100:0/3152943543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84840af970 msgr2=0x7f84841645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f8489a00640 1 --2- 172.21.15.100:0/3152943543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84840af970 0x7f84841645f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f8489a00640 1 -- 172.21.15.100:0/3152943543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f848415ad40 msgr2=0x7f8484164b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f8489a00640 1 --2- 172.21.15.100:0/3152943543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f848415ad40 0x7f8484164b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f8489a00640 1 -- 172.21.15.100:0/3152943543 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84841512f0 con 0x7f8484072ca0 2024-04-23T14:32:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f8489000640 1 --2- 172.21.15.100:0/3152943543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84840af970 0x7f84841645f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f8489a00640 1 --2- 172.21.15.100:0/3152943543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8484072ca0 0x7f8484150900 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f847801cd30 tx=0x7f8478019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f8481600640 1 -- 172.21.15.100:0/3152943543 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84780abe60 con 0x7f8484072ca0 2024-04-23T14:32:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f8481600640 1 -- 172.21.15.100:0/3152943543 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84780b4640 con 0x7f8484072ca0 2024-04-23T14:32:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f8481600640 1 -- 172.21.15.100:0/3152943543 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84780b0040 con 0x7f8484072ca0 2024-04-23T14:32:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f848c400640 1 -- 172.21.15.100:0/3152943543 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84841515d0 con 0x7f8484072ca0 2024-04-23T14:32:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f848c400640 1 -- 172.21.15.100:0/3152943543 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f848413b3f0 con 0x7f8484072ca0 2024-04-23T14:32:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f848c400640 1 -- 172.21.15.100:0/3152943543 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8484068ad0 con 0x7f8484072ca0 2024-04-23T14:32:21.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.748+0000 7f8481600640 1 -- 172.21.15.100:0/3152943543 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f84780ab4c0 con 0x7f8484072ca0 2024-04-23T14:32:21.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.752+0000 7f8481600640 1 --2- 172.21.15.100:0/3152943543 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8464068380 0x7f846406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:21.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.752+0000 7f8481600640 1 -- 172.21.15.100:0/3152943543 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f84781207e0 con 0x7f8484072ca0 2024-04-23T14:32:21.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.752+0000 7f8489000640 1 --2- 172.21.15.100:0/3152943543 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8464068380 0x7f846406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:21.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.752+0000 7f8489000640 1 --2- 172.21.15.100:0/3152943543 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8464068380 0x7f846406a840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8484153ac0 tx=0x7f8470008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:21.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.756+0000 7f8481600640 1 -- 172.21.15.100:0/3152943543 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8478016110 con 0x7f8484072ca0 2024-04-23T14:32:21.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.916+0000 7f848c400640 1 -- 172.21.15.100:0/3152943543 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8484155c50 con 0x7f8484072ca0 2024-04-23T14:32:21.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.916+0000 7f8481600640 1 -- 172.21.15.100:0/3152943543 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f84780f0ae0 con 0x7f8484072ca0 2024-04-23T14:32:21.919 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:21.919 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":24,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:14.037027+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:21.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.916+0000 7f848c400640 1 -- 172.21.15.100:0/3152943543 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8464068380 msgr2=0x7f846406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:21.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.916+0000 7f848c400640 1 --2- 172.21.15.100:0/3152943543 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8464068380 0x7f846406a840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8484153ac0 tx=0x7f8470008040 comp rx=0 tx=0).stop 2024-04-23T14:32:21.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.916+0000 7f848c400640 1 -- 172.21.15.100:0/3152943543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8484072ca0 msgr2=0x7f8484150900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:21.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.916+0000 7f848c400640 1 --2- 172.21.15.100:0/3152943543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8484072ca0 0x7f8484150900 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f847801cd30 tx=0x7f8478019230 comp rx=0 tx=0).stop 2024-04-23T14:32:21.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.916+0000 7f848c400640 1 -- 172.21.15.100:0/3152943543 shutdown_connections 2024-04-23T14:32:21.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.916+0000 7f848c400640 1 --2- 172.21.15.100:0/3152943543 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8464068380 0x7f846406a840 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.916+0000 7f848c400640 1 --2- 172.21.15.100:0/3152943543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f848415ad40 0x7f8484164b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.916+0000 7f848c400640 1 --2- 172.21.15.100:0/3152943543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84840af970 0x7f84841645f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.916+0000 7f848c400640 1 --2- 172.21.15.100:0/3152943543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8484072ca0 0x7f8484150900 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.916+0000 7f848c400640 1 -- 172.21.15.100:0/3152943543 >> 172.21.15.100:0/3152943543 conn(0x7f84840b1e80 msgr2=0x7f848406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:21.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.916+0000 7f848c400640 1 -- 172.21.15.100:0/3152943543 shutdown_connections 2024-04-23T14:32:21.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:21.916+0000 7f848c400640 1 -- 172.21.15.100:0/3152943543 wait complete. 2024-04-23T14:32:21.940 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:32:06.835560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:32:07.930489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-23T14:32:11.636814+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-23T14:32:21.941 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:22.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.076+0000 7f8556000640 1 Processor -- start 2024-04-23T14:32:22.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.076+0000 7f8556000640 1 -- start start 2024-04-23T14:32:22.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.076+0000 7f8556000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85500cf370 0x7f855015cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.076+0000 7f8556000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85501602b0 con 0x7f855015db10 2024-04-23T14:32:22.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.076+0000 7f8556000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8550160590 con 0x7f855015c360 2024-04-23T14:32:22.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.076+0000 7f8556000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f855015dfa0 con 0x7f85500cf370 2024-04-23T14:32:22.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.080+0000 7f854fe00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f855015db10 0x7f85500cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:41028/0 (socket says 172.21.15.100:41028) 2024-04-23T14:32:22.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.080+0000 7f854fe00640 1 -- 172.21.15.100:0/1956695616 learned_addr learned my addr 172.21.15.100:0/1956695616 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:22.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.080+0000 7f8555600640 1 --2- 172.21.15.100:0/1956695616 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85500cf370 0x7f855015cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.080+0000 7f854f400640 1 -- 172.21.15.100:0/1956695616 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3145452958 0 0) 0x7f85501602b0 con 0x7f855015db10 2024-04-23T14:32:22.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.080+0000 7f854f400640 1 -- 172.21.15.100:0/1956695616 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8534003530 con 0x7f855015db10 2024-04-23T14:32:22.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.080+0000 7f8555600640 1 -- 172.21.15.100:0/1956695616 >> v1:172.21.15.151:6789/0 conn(0x7f855015c360 legacy=0x7f85500cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.080+0000 7f8555600640 1 -- 172.21.15.100:0/1956695616 >> v1:172.21.15.100:6789/0 conn(0x7f855015db10 legacy=0x7f85500cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.080+0000 7f8555600640 1 -- 172.21.15.100:0/1956695616 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8550160870 con 0x7f85500cf370 2024-04-23T14:32:22.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.080+0000 7f8555600640 1 --2- 172.21.15.100:0/1956695616 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85500cf370 0x7f855015cca0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f85480097f0 tx=0x7f854801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d327aa20d589799f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:22.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f854f400640 1 -- 172.21.15.100:0/1956695616 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85480b1070 con 0x7f85500cf370 2024-04-23T14:32:22.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f854f400640 1 -- 172.21.15.100:0/1956695616 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85480ac410 con 0x7f85500cf370 2024-04-23T14:32:22.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f854f400640 1 -- 172.21.15.100:0/1956695616 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85480ac7e0 con 0x7f85500cf370 2024-04-23T14:32:22.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f8556000640 1 -- 172.21.15.100:0/1956695616 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85500cf370 msgr2=0x7f855015cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f8556000640 1 --2- 172.21.15.100:0/1956695616 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85500cf370 0x7f855015cca0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f85480097f0 tx=0x7f854801c4b0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f8556000640 1 -- 172.21.15.100:0/1956695616 shutdown_connections 2024-04-23T14:32:22.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f8556000640 1 --2- 172.21.15.100:0/1956695616 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85500cf370 0x7f855015cca0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f8556000640 1 -- 172.21.15.100:0/1956695616 >> 172.21.15.100:0/1956695616 conn(0x7f85500b3df0 msgr2=0x7f85500b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:22.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f8556000640 1 -- 172.21.15.100:0/1956695616 shutdown_connections 2024-04-23T14:32:22.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f8556000640 1 -- 172.21.15.100:0/1956695616 wait complete. 2024-04-23T14:32:22.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f8556000640 1 Processor -- start 2024-04-23T14:32:22.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f8556000640 1 -- start start 2024-04-23T14:32:22.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f8556000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85500cf370 0x7f8550154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f8556000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f855015c360 0x7f8550167b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f8556000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f855015db10 0x7f855015b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f8556000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f85500ccfe0 con 0x7f85500cf370 2024-04-23T14:32:22.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f8556000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f855015ff40 con 0x7f855015c360 2024-04-23T14:32:22.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f8556000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f85500cd160 con 0x7f855015db10 2024-04-23T14:32:22.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f8554c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85500cf370 0x7f8550154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f8554c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85500cf370 0x7f8550154ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39622/0 (socket says 172.21.15.100:39622) 2024-04-23T14:32:22.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f8554c00640 1 -- 172.21.15.100:0/3709323218 learned_addr learned my addr 172.21.15.100:0/3709323218 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:22.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f8554c00640 1 -- 172.21.15.100:0/3709323218 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f855015db10 msgr2=0x7f855015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f8555600640 1 --2- 172.21.15.100:0/3709323218 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f855015db10 0x7f855015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.084+0000 7f854fe00640 1 --2- 172.21.15.100:0/3709323218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f855015c360 0x7f8550167b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.088+0000 7f8554c00640 1 --2- 172.21.15.100:0/3709323218 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f855015db10 0x7f855015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.088+0000 7f8554c00640 1 -- 172.21.15.100:0/3709323218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f855015c360 msgr2=0x7f8550167b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.088+0000 7f8554c00640 1 --2- 172.21.15.100:0/3709323218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f855015c360 0x7f8550167b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.088+0000 7f8554c00640 1 -- 172.21.15.100:0/3709323218 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8550165480 con 0x7f85500cf370 2024-04-23T14:32:22.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.088+0000 7f8555600640 1 --2- 172.21.15.100:0/3709323218 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f855015db10 0x7f855015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:22.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.088+0000 7f854fe00640 1 --2- 172.21.15.100:0/3709323218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f855015c360 0x7f8550167b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:22.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.088+0000 7f8554c00640 1 --2- 172.21.15.100:0/3709323218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85500cf370 0x7f8550154ea0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f854000c790 tx=0x7f854000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:22.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.088+0000 7f854cc00640 1 -- 172.21.15.100:0/3709323218 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540016020 con 0x7f85500cf370 2024-04-23T14:32:22.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.088+0000 7f854cc00640 1 -- 172.21.15.100:0/3709323218 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8540006d60 con 0x7f85500cf370 2024-04-23T14:32:22.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.088+0000 7f854cc00640 1 -- 172.21.15.100:0/3709323218 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540003bc0 con 0x7f85500cf370 2024-04-23T14:32:22.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.088+0000 7f8556000640 1 -- 172.21.15.100:0/3709323218 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8550165760 con 0x7f85500cf370 2024-04-23T14:32:22.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.088+0000 7f8556000640 1 -- 172.21.15.100:0/3709323218 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8550165d20 con 0x7f85500cf370 2024-04-23T14:32:22.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.088+0000 7f8556000640 1 -- 172.21.15.100:0/3709323218 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8550068ad0 con 0x7f85500cf370 2024-04-23T14:32:22.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.088+0000 7f854cc00640 1 -- 172.21.15.100:0/3709323218 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f854000a850 con 0x7f85500cf370 2024-04-23T14:32:22.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.088+0000 7f854cc00640 1 --2- 172.21.15.100:0/3709323218 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8530068380 0x7f853006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.088+0000 7f854cc00640 1 -- 172.21.15.100:0/3709323218 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8540086b10 con 0x7f85500cf370 2024-04-23T14:32:22.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.088+0000 7f854fe00640 1 --2- 172.21.15.100:0/3709323218 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8530068380 0x7f853006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.092+0000 7f854fe00640 1 --2- 172.21.15.100:0/3709323218 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8530068380 0x7f853006a840 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f855015d160 tx=0x7f853c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:22.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.092+0000 7f854cc00640 1 -- 172.21.15.100:0/3709323218 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8540020020 con 0x7f85500cf370 2024-04-23T14:32:22.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.252+0000 7f8556000640 1 -- 172.21.15.100:0/3709323218 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f85500cc640 con 0x7f85500cf370 2024-04-23T14:32:22.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.252+0000 7f854cc00640 1 -- 172.21.15.100:0/3709323218 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f8540019310 con 0x7f85500cf370 2024-04-23T14:32:22.256 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:32:22.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.256+0000 7f8556000640 1 -- 172.21.15.100:0/3709323218 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8530068380 msgr2=0x7f853006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.256+0000 7f8556000640 1 --2- 172.21.15.100:0/3709323218 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8530068380 0x7f853006a840 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f855015d160 tx=0x7f853c009040 comp rx=0 tx=0).stop 2024-04-23T14:32:22.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.256+0000 7f8556000640 1 -- 172.21.15.100:0/3709323218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85500cf370 msgr2=0x7f8550154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.256+0000 7f8556000640 1 --2- 172.21.15.100:0/3709323218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85500cf370 0x7f8550154ea0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f854000c790 tx=0x7f854000cc50 comp rx=0 tx=0).stop 2024-04-23T14:32:22.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.256+0000 7f8556000640 1 -- 172.21.15.100:0/3709323218 shutdown_connections 2024-04-23T14:32:22.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.256+0000 7f8556000640 1 --2- 172.21.15.100:0/3709323218 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8530068380 0x7f853006a840 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.256+0000 7f8556000640 1 --2- 172.21.15.100:0/3709323218 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f855015db10 0x7f855015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.256+0000 7f8556000640 1 --2- 172.21.15.100:0/3709323218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f855015c360 0x7f8550167b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.256+0000 7f8556000640 1 --2- 172.21.15.100:0/3709323218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85500cf370 0x7f8550154ea0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.256+0000 7f8556000640 1 -- 172.21.15.100:0/3709323218 >> 172.21.15.100:0/3709323218 conn(0x7f85500b3df0 msgr2=0x7f8550070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:22.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.256+0000 7f8556000640 1 -- 172.21.15.100:0/3709323218 shutdown_connections 2024-04-23T14:32:22.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.256+0000 7f8556000640 1 -- 172.21.15.100:0/3709323218 wait complete. 2024-04-23T14:32:22.277 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:32:22.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.412+0000 7f76bfa00640 1 Processor -- start 2024-04-23T14:32:22.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.412+0000 7f76bfa00640 1 -- start start 2024-04-23T14:32:22.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.416+0000 7f76bfa00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f76b815c7a0 0x7f76b80af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.416+0000 7f76bfa00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f76b806bb20 0x7f76b80ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.416+0000 7f76bfa00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f76b80cd130 0x7f76b806e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.416+0000 7f76bfa00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f76b80cd5a0 con 0x7f76b80cd130 2024-04-23T14:32:22.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.416+0000 7f76bfa00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f76b806d450 con 0x7f76b815c7a0 2024-04-23T14:32:22.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.416+0000 7f76bfa00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f76b806cbb0 con 0x7f76b806bb20 2024-04-23T14:32:22.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.416+0000 7f76bda00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f76b80cd130 0x7f76b806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.416+0000 7f76bda00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f76b80cd130 0x7f76b806e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39638/0 (socket says 172.21.15.100:39638) 2024-04-23T14:32:22.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.416+0000 7f76bda00640 1 -- 172.21.15.100:0/244122924 learned_addr learned my addr 172.21.15.100:0/244122924 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:22.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.416+0000 7f76b7e00640 1 --2- 172.21.15.100:0/244122924 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f76b806bb20 0x7f76b80ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.416+0000 7f76bd000640 1 --2- 172.21.15.100:0/244122924 >> v2:172.21.15.151:3300/0 conn(0x7f76b815c7a0 0x7f76b80af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.416+0000 7f76bda00640 1 -- 172.21.15.100:0/244122924 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f76b806bb20 msgr2=0x7f76b80ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:22.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.416+0000 7f76bda00640 1 --2- 172.21.15.100:0/244122924 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f76b806bb20 0x7f76b80ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.416+0000 7f76bda00640 1 -- 172.21.15.100:0/244122924 >> v2:172.21.15.151:3300/0 conn(0x7f76b815c7a0 msgr2=0x7f76b80af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:22.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.416+0000 7f76bda00640 1 --2- 172.21.15.100:0/244122924 >> v2:172.21.15.151:3300/0 conn(0x7f76b815c7a0 0x7f76b80af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.416+0000 7f76bda00640 1 -- 172.21.15.100:0/244122924 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b806ce90 con 0x7f76b80cd130 2024-04-23T14:32:22.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76bda00640 1 --2- 172.21.15.100:0/244122924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76b80cd130 0x7f76b806e290 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f76b80716b0 tx=0x7f76b001c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dffea28a0ccb51ca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:22.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76b7400640 1 -- 172.21.15.100:0/244122924 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b00a9430 con 0x7f76b80cd130 2024-04-23T14:32:22.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76b7400640 1 -- 172.21.15.100:0/244122924 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76b00ae070 con 0x7f76b80cd130 2024-04-23T14:32:22.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76b7400640 1 -- 172.21.15.100:0/244122924 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b00a97f0 con 0x7f76b80cd130 2024-04-23T14:32:22.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76bfa00640 1 -- 172.21.15.100:0/244122924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76b80cd130 msgr2=0x7f76b806e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76bfa00640 1 --2- 172.21.15.100:0/244122924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76b80cd130 0x7f76b806e290 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f76b80716b0 tx=0x7f76b001c6a0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76bfa00640 1 -- 172.21.15.100:0/244122924 shutdown_connections 2024-04-23T14:32:22.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76bfa00640 1 --2- 172.21.15.100:0/244122924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76b80cd130 0x7f76b806e290 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76bfa00640 1 --2- 172.21.15.100:0/244122924 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f76b806bb20 0x7f76b80ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76bfa00640 1 --2- 172.21.15.100:0/244122924 >> v2:172.21.15.151:3300/0 conn(0x7f76b815c7a0 0x7f76b80af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76bfa00640 1 -- 172.21.15.100:0/244122924 >> 172.21.15.100:0/244122924 conn(0x7f76b80b5200 msgr2=0x7f76b80b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:22.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76bfa00640 1 -- 172.21.15.100:0/244122924 shutdown_connections 2024-04-23T14:32:22.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76bfa00640 1 -- 172.21.15.100:0/244122924 wait complete. 2024-04-23T14:32:22.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76bfa00640 1 Processor -- start 2024-04-23T14:32:22.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76bfa00640 1 -- start start 2024-04-23T14:32:22.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76bfa00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f76b806bb20 0x7f76b815d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76bfa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76b80cd130 0x7f76b815d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76bfa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76b815c7a0 0x7f76b8097810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76bfa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f76b80ad7c0 con 0x7f76b80cd130 2024-04-23T14:32:22.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76bfa00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f76b80ad640 con 0x7f76b815c7a0 2024-04-23T14:32:22.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76bfa00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f76b80ad940 con 0x7f76b806bb20 2024-04-23T14:32:22.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76b7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76b80cd130 0x7f76b815d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76b7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76b80cd130 0x7f76b815d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39654/0 (socket says 172.21.15.100:39654) 2024-04-23T14:32:22.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76b7e00640 1 -- 172.21.15.100:0/2957938929 learned_addr learned my addr 172.21.15.100:0/2957938929 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:22.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76bd000640 1 --2- 172.21.15.100:0/2957938929 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f76b806bb20 0x7f76b815d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.420+0000 7f76bda00640 1 --2- 172.21.15.100:0/2957938929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76b815c7a0 0x7f76b8097810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.424+0000 7f76b7e00640 1 -- 172.21.15.100:0/2957938929 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f76b806bb20 msgr2=0x7f76b815d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.424+0000 7f76b7e00640 1 --2- 172.21.15.100:0/2957938929 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f76b806bb20 0x7f76b815d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.424+0000 7f76b7e00640 1 -- 172.21.15.100:0/2957938929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76b815c7a0 msgr2=0x7f76b8097810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.424+0000 7f76b7e00640 1 --2- 172.21.15.100:0/2957938929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76b815c7a0 0x7f76b8097810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.424+0000 7f76b7e00640 1 -- 172.21.15.100:0/2957938929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b815e190 con 0x7f76b80cd130 2024-04-23T14:32:22.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.424+0000 7f76b7e00640 1 --2- 172.21.15.100:0/2957938929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76b80cd130 0x7f76b815d8e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f76a400d7d0 tx=0x7f76a400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:22.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.424+0000 7f76b4c00640 1 -- 172.21.15.100:0/2957938929 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a400b6c0 con 0x7f76b80cd130 2024-04-23T14:32:22.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.424+0000 7f76b4c00640 1 -- 172.21.15.100:0/2957938929 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a4019070 con 0x7f76b80cd130 2024-04-23T14:32:22.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.424+0000 7f76b4c00640 1 -- 172.21.15.100:0/2957938929 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a400baa0 con 0x7f76b80cd130 2024-04-23T14:32:22.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.424+0000 7f76bfa00640 1 -- 172.21.15.100:0/2957938929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b815e470 con 0x7f76b80cd130 2024-04-23T14:32:22.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.424+0000 7f76bfa00640 1 -- 172.21.15.100:0/2957938929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b815ea30 con 0x7f76b80cd130 2024-04-23T14:32:22.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.424+0000 7f76bfa00640 1 -- 172.21.15.100:0/2957938929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b8068ad0 con 0x7f76b80cd130 2024-04-23T14:32:22.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.424+0000 7f76b4c00640 1 -- 172.21.15.100:0/2957938929 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f76a400bc40 con 0x7f76b80cd130 2024-04-23T14:32:22.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.424+0000 7f76b4c00640 1 --2- 172.21.15.100:0/2957938929 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7698068380 0x7f769806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.424+0000 7f76b4c00640 1 -- 172.21.15.100:0/2957938929 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f76a40872d0 con 0x7f76b80cd130 2024-04-23T14:32:22.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.428+0000 7f76bd000640 1 --2- 172.21.15.100:0/2957938929 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7698068380 0x7f769806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.428+0000 7f76bd000640 1 --2- 172.21.15.100:0/2957938929 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7698068380 0x7f769806a840 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00d8d0 tx=0x7f76ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:22.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.432+0000 7f76b4c00640 1 -- 172.21.15.100:0/2957938929 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f76a4057420 con 0x7f76b80cd130 2024-04-23T14:32:22.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.584+0000 7f76bfa00640 1 -- 172.21.15.100:0/2957938929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f76b806c840 con 0x7f76b80cd130 2024-04-23T14:32:22.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.584+0000 7f76b4c00640 1 -- 172.21.15.100:0/2957938929 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f76a4056d20 con 0x7f76b80cd130 2024-04-23T14:32:22.589 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:32:22.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.588+0000 7f76bfa00640 1 -- 172.21.15.100:0/2957938929 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7698068380 msgr2=0x7f769806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.588+0000 7f76bfa00640 1 --2- 172.21.15.100:0/2957938929 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7698068380 0x7f769806a840 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00d8d0 tx=0x7f76ac008040 comp rx=0 tx=0).stop 2024-04-23T14:32:22.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.588+0000 7f76bfa00640 1 -- 172.21.15.100:0/2957938929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76b80cd130 msgr2=0x7f76b815d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.588+0000 7f76bfa00640 1 --2- 172.21.15.100:0/2957938929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76b80cd130 0x7f76b815d8e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f76a400d7d0 tx=0x7f76a400dc90 comp rx=0 tx=0).stop 2024-04-23T14:32:22.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.588+0000 7f76bfa00640 1 -- 172.21.15.100:0/2957938929 shutdown_connections 2024-04-23T14:32:22.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.588+0000 7f76bfa00640 1 --2- 172.21.15.100:0/2957938929 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7698068380 0x7f769806a840 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.588+0000 7f76bfa00640 1 --2- 172.21.15.100:0/2957938929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76b815c7a0 0x7f76b8097810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.588+0000 7f76bfa00640 1 --2- 172.21.15.100:0/2957938929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76b80cd130 0x7f76b815d8e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.588+0000 7f76bfa00640 1 --2- 172.21.15.100:0/2957938929 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f76b806bb20 0x7f76b815d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.588+0000 7f76bfa00640 1 -- 172.21.15.100:0/2957938929 >> 172.21.15.100:0/2957938929 conn(0x7f76b80b5200 msgr2=0x7f76b8152df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:22.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.588+0000 7f76bfa00640 1 -- 172.21.15.100:0/2957938929 shutdown_connections 2024-04-23T14:32:22.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.588+0000 7f76bfa00640 1 -- 172.21.15.100:0/2957938929 wait complete. 2024-04-23T14:32:22.610 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:32:22.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.744+0000 7f2eb5400640 1 Processor -- start 2024-04-23T14:32:22.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.744+0000 7f2eb5400640 1 -- start start 2024-04-23T14:32:22.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.744+0000 7f2eb5400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2eb006bb20 0x7f2eb00ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.744+0000 7f2eb5400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2eb00cd130 0x7f2eb00ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.744+0000 7f2eb5400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2eb006cbb0 con 0x7f2eb015c7a0 2024-04-23T14:32:22.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.748+0000 7f2eb5400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f2eb00cd5a0 con 0x7f2eb00cd130 2024-04-23T14:32:22.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.748+0000 7f2eb5400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f2eb006d450 con 0x7f2eb006bb20 2024-04-23T14:32:22.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.748+0000 7f2eaf200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2eb00cd130 0x7f2eb00ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.748+0000 7f2eaf200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2eb00cd130 0x7f2eb00ad430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39666/0 (socket says 172.21.15.100:39666) 2024-04-23T14:32:22.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.748+0000 7f2eaf200640 1 -- 172.21.15.100:0/1952582355 learned_addr learned my addr 172.21.15.100:0/1952582355 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:22.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.748+0000 7f2eade00640 1 --2- 172.21.15.100:0/1952582355 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2eb006bb20 0x7f2eb00ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.748+0000 7f2eaf200640 1 -- 172.21.15.100:0/1952582355 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2eb006bb20 msgr2=0x7f2eb00ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:22.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.748+0000 7f2eaf200640 1 --2- 172.21.15.100:0/1952582355 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2eb006bb20 0x7f2eb00ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.748+0000 7f2eaf200640 1 -- 172.21.15.100:0/1952582355 >> v1:172.21.15.151:6789/0 conn(0x7f2eb015c7a0 legacy=0x7f2eb00af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.748+0000 7f2eaf200640 1 -- 172.21.15.100:0/1952582355 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb006ce90 con 0x7f2eb00cd130 2024-04-23T14:32:22.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.748+0000 7f2eaf200640 1 --2- 172.21.15.100:0/1952582355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2eb00cd130 0x7f2eb00ad430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009e50 tx=0x7f2ea801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd505672d578a59f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:22.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.748+0000 7f2ead400640 1 -- 172.21.15.100:0/1952582355 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea80a5660 con 0x7f2eb00cd130 2024-04-23T14:32:22.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.748+0000 7f2ead400640 1 -- 172.21.15.100:0/1952582355 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea80a5800 con 0x7f2eb00cd130 2024-04-23T14:32:22.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.748+0000 7f2ead400640 1 -- 172.21.15.100:0/1952582355 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea80a5bd0 con 0x7f2eb00cd130 2024-04-23T14:32:22.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eb5400640 1 -- 172.21.15.100:0/1952582355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2eb00cd130 msgr2=0x7f2eb00ad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eb5400640 1 --2- 172.21.15.100:0/1952582355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2eb00cd130 0x7f2eb00ad430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009e50 tx=0x7f2ea801c9a0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eb5400640 1 -- 172.21.15.100:0/1952582355 shutdown_connections 2024-04-23T14:32:22.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eb5400640 1 --2- 172.21.15.100:0/1952582355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2eb00cd130 0x7f2eb00ad430 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eb5400640 1 --2- 172.21.15.100:0/1952582355 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2eb006bb20 0x7f2eb00ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eb5400640 1 -- 172.21.15.100:0/1952582355 >> 172.21.15.100:0/1952582355 conn(0x7f2eb00b5200 msgr2=0x7f2eb00b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:22.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eb5400640 1 -- 172.21.15.100:0/1952582355 shutdown_connections 2024-04-23T14:32:22.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eb5400640 1 -- 172.21.15.100:0/1952582355 wait complete. 2024-04-23T14:32:22.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eb5400640 1 Processor -- start 2024-04-23T14:32:22.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eb5400640 1 -- start start 2024-04-23T14:32:22.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eb5400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2eb006bb20 0x7f2eb0099930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eb5400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2eb00cd130 0x7f2eb01655a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eb5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2eb015c7a0 0x7f2eb0165ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eb5400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb00ae510 con 0x7f2eb015c7a0 2024-04-23T14:32:22.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eb5400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb00ae390 con 0x7f2eb00cd130 2024-04-23T14:32:22.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eb5400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f2eb00ae690 con 0x7f2eb006bb20 2024-04-23T14:32:22.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eaf200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2eb015c7a0 0x7f2eb0165ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eaf200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2eb015c7a0 0x7f2eb0165ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39668/0 (socket says 172.21.15.100:39668) 2024-04-23T14:32:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eaf200640 1 -- 172.21.15.100:0/2518728800 learned_addr learned my addr 172.21.15.100:0/2518728800 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eade00640 1 --2- 172.21.15.100:0/2518728800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2eb00cd130 0x7f2eb01655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eae800640 1 --2- 172.21.15.100:0/2518728800 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2eb006bb20 0x7f2eb0099930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eaf200640 1 -- 172.21.15.100:0/2518728800 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2eb006bb20 msgr2=0x7f2eb0099930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eaf200640 1 --2- 172.21.15.100:0/2518728800 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2eb006bb20 0x7f2eb0099930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eaf200640 1 -- 172.21.15.100:0/2518728800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2eb00cd130 msgr2=0x7f2eb01655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eaf200640 1 --2- 172.21.15.100:0/2518728800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2eb00cd130 0x7f2eb01655a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.752+0000 7f2eaf200640 1 -- 172.21.15.100:0/2518728800 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb0099e70 con 0x7f2eb015c7a0 2024-04-23T14:32:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.756+0000 7f2eaf200640 1 --2- 172.21.15.100:0/2518728800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2eb015c7a0 0x7f2eb0165ae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2ea801a5d0 tx=0x7f2ea8019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.756+0000 7f2ea6a00640 1 -- 172.21.15.100:0/2518728800 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea80bc020 con 0x7f2eb015c7a0 2024-04-23T14:32:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.756+0000 7f2ea6a00640 1 -- 172.21.15.100:0/2518728800 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea80b8880 con 0x7f2eb015c7a0 2024-04-23T14:32:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.756+0000 7f2ea6a00640 1 -- 172.21.15.100:0/2518728800 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea80a5660 con 0x7f2eb015c7a0 2024-04-23T14:32:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.756+0000 7f2eb5400640 1 -- 172.21.15.100:0/2518728800 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb00c3d10 con 0x7f2eb015c7a0 2024-04-23T14:32:22.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.756+0000 7f2eb5400640 1 -- 172.21.15.100:0/2518728800 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb0166530 con 0x7f2eb015c7a0 2024-04-23T14:32:22.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.756+0000 7f2eb5400640 1 -- 172.21.15.100:0/2518728800 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb0068ad0 con 0x7f2eb015c7a0 2024-04-23T14:32:22.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.756+0000 7f2ea6a00640 1 -- 172.21.15.100:0/2518728800 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2ea80bf020 con 0x7f2eb015c7a0 2024-04-23T14:32:22.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.756+0000 7f2ea6a00640 1 --2- 172.21.15.100:0/2518728800 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2e90068380 0x7f2e9006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.756+0000 7f2ea6a00640 1 -- 172.21.15.100:0/2518728800 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2ea8121d40 con 0x7f2eb015c7a0 2024-04-23T14:32:22.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.756+0000 7f2eae800640 1 --2- 172.21.15.100:0/2518728800 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2e90068380 0x7f2e9006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.760+0000 7f2eae800640 1 --2- 172.21.15.100:0/2518728800 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2e90068380 0x7f2e9006a840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2eb01faab0 tx=0x7f2ea0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:22.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.760+0000 7f2ea6a00640 1 -- 172.21.15.100:0/2518728800 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2ea80c6030 con 0x7f2eb015c7a0 2024-04-23T14:32:22.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.916+0000 7f2eb5400640 1 -- 172.21.15.100:0/2518728800 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2eb0166700 con 0x7f2eb015c7a0 2024-04-23T14:32:22.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.916+0000 7f2ea6a00640 1 -- 172.21.15.100:0/2518728800 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f2ea8016620 con 0x7f2eb015c7a0 2024-04-23T14:32:22.920 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:32:22.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.920+0000 7f2eb5400640 1 -- 172.21.15.100:0/2518728800 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2e90068380 msgr2=0x7f2e9006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.920+0000 7f2eb5400640 1 --2- 172.21.15.100:0/2518728800 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2e90068380 0x7f2e9006a840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2eb01faab0 tx=0x7f2ea0009040 comp rx=0 tx=0).stop 2024-04-23T14:32:22.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.920+0000 7f2eb5400640 1 -- 172.21.15.100:0/2518728800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2eb015c7a0 msgr2=0x7f2eb0165ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.920+0000 7f2eb5400640 1 --2- 172.21.15.100:0/2518728800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2eb015c7a0 0x7f2eb0165ae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2ea801a5d0 tx=0x7f2ea8019740 comp rx=0 tx=0).stop 2024-04-23T14:32:22.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.920+0000 7f2eb5400640 1 -- 172.21.15.100:0/2518728800 shutdown_connections 2024-04-23T14:32:22.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.920+0000 7f2eb5400640 1 --2- 172.21.15.100:0/2518728800 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2e90068380 0x7f2e9006a840 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.920+0000 7f2eb5400640 1 --2- 172.21.15.100:0/2518728800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2eb015c7a0 0x7f2eb0165ae0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.920+0000 7f2eb5400640 1 --2- 172.21.15.100:0/2518728800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2eb00cd130 0x7f2eb01655a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.920+0000 7f2eb5400640 1 --2- 172.21.15.100:0/2518728800 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2eb006bb20 0x7f2eb0099930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.920+0000 7f2eb5400640 1 -- 172.21.15.100:0/2518728800 >> 172.21.15.100:0/2518728800 conn(0x7f2eb00b5200 msgr2=0x7f2eb00b55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:22.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.920+0000 7f2eb5400640 1 -- 172.21.15.100:0/2518728800 shutdown_connections 2024-04-23T14:32:22.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:22.920+0000 7f2eb5400640 1 -- 172.21.15.100:0/2518728800 wait complete. 2024-04-23T14:32:22.940 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-23T14:32:23.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.072+0000 7ff05d200640 1 Processor -- start 2024-04-23T14:32:23.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.076+0000 7ff05d200640 1 -- start start 2024-04-23T14:32:23.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.076+0000 7ff05d200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff05815c7a0 0x7ff0580af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.076+0000 7ff05d200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff05806bb20 0x7ff0580ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.076+0000 7ff05d200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff05806cbb0 con 0x7ff0580cd130 2024-04-23T14:32:23.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.076+0000 7ff05d200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff0580cd5a0 con 0x7ff05806bb20 2024-04-23T14:32:23.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.076+0000 7ff05d200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff05806d450 con 0x7ff05815c7a0 2024-04-23T14:32:23.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.076+0000 7ff055400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff05806bb20 0x7ff0580ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.076+0000 7ff056800640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff0580cd130 0x7ff05806e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:41036/0 (socket says 172.21.15.100:41036) 2024-04-23T14:32:23.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.076+0000 7ff056800640 1 -- 172.21.15.100:0/2477017856 learned_addr learned my addr 172.21.15.100:0/2477017856 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:23.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.076+0000 7ff055e00640 1 --2- 172.21.15.100:0/2477017856 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff05815c7a0 0x7ff0580af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.076+0000 7ff055400640 1 -- 172.21.15.100:0/2477017856 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff05815c7a0 msgr2=0x7ff0580af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:23.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.076+0000 7ff055400640 1 --2- 172.21.15.100:0/2477017856 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff05815c7a0 0x7ff0580af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.076+0000 7ff055400640 1 -- 172.21.15.100:0/2477017856 >> v1:172.21.15.100:6789/0 conn(0x7ff0580cd130 legacy=0x7ff05806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.076+0000 7ff055400640 1 -- 172.21.15.100:0/2477017856 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05806ce90 con 0x7ff05806bb20 2024-04-23T14:32:23.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.076+0000 7ff054a00640 1 -- 172.21.15.100:0/2477017856 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2734359375 0 0) 0x7ff05806cbb0 con 0x7ff0580cd130 2024-04-23T14:32:23.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.076+0000 7ff055e00640 1 --2- 172.21.15.100:0/2477017856 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff05815c7a0 0x7ff0580af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:23.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.080+0000 7ff055400640 1 --2- 172.21.15.100:0/2477017856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff05806bb20 0x7ff0580ccbf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff0440097c0 tx=0x7ff04401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb5315a18a50898 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:23.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.080+0000 7ff054a00640 1 -- 172.21.15.100:0/2477017856 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0440b0070 con 0x7ff05806bb20 2024-04-23T14:32:23.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.080+0000 7ff054a00640 1 -- 172.21.15.100:0/2477017856 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0440ab600 con 0x7ff05806bb20 2024-04-23T14:32:23.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.080+0000 7ff054a00640 1 -- 172.21.15.100:0/2477017856 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0440ab9d0 con 0x7ff05806bb20 2024-04-23T14:32:23.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.080+0000 7ff05d200640 1 -- 172.21.15.100:0/2477017856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff05806bb20 msgr2=0x7ff0580ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.080+0000 7ff05d200640 1 --2- 172.21.15.100:0/2477017856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff05806bb20 0x7ff0580ccbf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff0440097c0 tx=0x7ff04401c410 comp rx=0 tx=0).stop 2024-04-23T14:32:23.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.080+0000 7ff05d200640 1 -- 172.21.15.100:0/2477017856 shutdown_connections 2024-04-23T14:32:23.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.080+0000 7ff05d200640 1 --2- 172.21.15.100:0/2477017856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff05806bb20 0x7ff0580ccbf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.080+0000 7ff05d200640 1 --2- 172.21.15.100:0/2477017856 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff05815c7a0 0x7ff0580af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.080+0000 7ff05d200640 1 -- 172.21.15.100:0/2477017856 >> 172.21.15.100:0/2477017856 conn(0x7ff0580b5200 msgr2=0x7ff0580b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:23.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.080+0000 7ff05d200640 1 -- 172.21.15.100:0/2477017856 shutdown_connections 2024-04-23T14:32:23.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.080+0000 7ff05d200640 1 -- 172.21.15.100:0/2477017856 wait complete. 2024-04-23T14:32:23.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.080+0000 7ff05d200640 1 Processor -- start 2024-04-23T14:32:23.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.080+0000 7ff05d200640 1 -- start start 2024-04-23T14:32:23.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.080+0000 7ff05d200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff05806bb20 0x7ff0581566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.080+0000 7ff05d200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0580cd130 0x7ff058099800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.080+0000 7ff05d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff05815c7a0 0x7ff058165e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.080+0000 7ff05d200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff058169590 con 0x7ff05815c7a0 2024-04-23T14:32:23.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.080+0000 7ff05d200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff058169410 con 0x7ff05806bb20 2024-04-23T14:32:23.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.080+0000 7ff05d200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff058169710 con 0x7ff0580cd130 2024-04-23T14:32:23.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.084+0000 7ff056800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff05815c7a0 0x7ff058165e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.084+0000 7ff056800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff05815c7a0 0x7ff058165e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39676/0 (socket says 172.21.15.100:39676) 2024-04-23T14:32:23.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.084+0000 7ff056800640 1 -- 172.21.15.100:0/1834105386 learned_addr learned my addr 172.21.15.100:0/1834105386 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:23.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.084+0000 7ff056800640 1 -- 172.21.15.100:0/1834105386 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0580cd130 msgr2=0x7ff058099800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.084+0000 7ff056800640 1 --2- 172.21.15.100:0/1834105386 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0580cd130 0x7ff058099800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.084+0000 7ff056800640 1 -- 172.21.15.100:0/1834105386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff05806bb20 msgr2=0x7ff0581566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.084+0000 7ff056800640 1 --2- 172.21.15.100:0/1834105386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff05806bb20 0x7ff0581566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.084+0000 7ff056800640 1 -- 172.21.15.100:0/1834105386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff058156be0 con 0x7ff05815c7a0 2024-04-23T14:32:23.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.084+0000 7ff056800640 1 --2- 172.21.15.100:0/1834105386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff05815c7a0 0x7ff058165e00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff05000c4b0 tx=0x7ff05000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:23.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.084+0000 7ff04e000640 1 -- 172.21.15.100:0/1834105386 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff050018070 con 0x7ff05815c7a0 2024-04-23T14:32:23.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.084+0000 7ff04e000640 1 -- 172.21.15.100:0/1834105386 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0500074e0 con 0x7ff05815c7a0 2024-04-23T14:32:23.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.084+0000 7ff04e000640 1 -- 172.21.15.100:0/1834105386 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff050004a30 con 0x7ff05815c7a0 2024-04-23T14:32:23.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.084+0000 7ff05d200640 1 -- 172.21.15.100:0/1834105386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff05814eed0 con 0x7ff05815c7a0 2024-04-23T14:32:23.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.084+0000 7ff05d200640 1 -- 172.21.15.100:0/1834105386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff05814f490 con 0x7ff05815c7a0 2024-04-23T14:32:23.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.084+0000 7ff05d200640 1 -- 172.21.15.100:0/1834105386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff058068ad0 con 0x7ff05815c7a0 2024-04-23T14:32:23.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.084+0000 7ff04e000640 1 -- 172.21.15.100:0/1834105386 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff050004bd0 con 0x7ff05815c7a0 2024-04-23T14:32:23.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.084+0000 7ff04e000640 1 --2- 172.21.15.100:0/1834105386 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff038068380 0x7ff03806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.088+0000 7ff04e000640 1 -- 172.21.15.100:0/1834105386 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff0500860a0 con 0x7ff05815c7a0 2024-04-23T14:32:23.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.088+0000 7ff055e00640 1 --2- 172.21.15.100:0/1834105386 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff038068380 0x7ff03806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.088+0000 7ff055e00640 1 --2- 172.21.15.100:0/1834105386 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff038068380 0x7ff03806a840 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff0480094c0 tx=0x7ff048009570 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:23.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.092+0000 7ff04e000640 1 -- 172.21.15.100:0/1834105386 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff050056280 con 0x7ff05815c7a0 2024-04-23T14:32:23.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.244+0000 7ff05d200640 1 -- 172.21.15.100:0/1834105386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff05815bcd0 con 0x7ff05815c7a0 2024-04-23T14:32:23.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.248+0000 7ff04e000640 1 -- 172.21.15.100:0/1834105386 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7ff050055b80 con 0x7ff05815c7a0 2024-04-23T14:32:23.249 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:32:23.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.248+0000 7ff05d200640 1 -- 172.21.15.100:0/1834105386 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff038068380 msgr2=0x7ff03806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.248+0000 7ff05d200640 1 --2- 172.21.15.100:0/1834105386 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff038068380 0x7ff03806a840 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff0480094c0 tx=0x7ff048009570 comp rx=0 tx=0).stop 2024-04-23T14:32:23.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.248+0000 7ff05d200640 1 -- 172.21.15.100:0/1834105386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff05815c7a0 msgr2=0x7ff058165e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.248+0000 7ff05d200640 1 --2- 172.21.15.100:0/1834105386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff05815c7a0 0x7ff058165e00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff05000c4b0 tx=0x7ff05000c970 comp rx=0 tx=0).stop 2024-04-23T14:32:23.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.248+0000 7ff05d200640 1 -- 172.21.15.100:0/1834105386 shutdown_connections 2024-04-23T14:32:23.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.248+0000 7ff05d200640 1 --2- 172.21.15.100:0/1834105386 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff038068380 0x7ff03806a840 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.248+0000 7ff05d200640 1 --2- 172.21.15.100:0/1834105386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff05815c7a0 0x7ff058165e00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.248+0000 7ff05d200640 1 --2- 172.21.15.100:0/1834105386 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0580cd130 0x7ff058099800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.248+0000 7ff05d200640 1 --2- 172.21.15.100:0/1834105386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff05806bb20 0x7ff0581566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.248+0000 7ff05d200640 1 -- 172.21.15.100:0/1834105386 >> 172.21.15.100:0/1834105386 conn(0x7ff0580b5200 msgr2=0x7ff0580cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:23.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.248+0000 7ff05d200640 1 -- 172.21.15.100:0/1834105386 shutdown_connections 2024-04-23T14:32:23.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.248+0000 7ff05d200640 1 -- 172.21.15.100:0/1834105386 wait complete. 2024-04-23T14:32:23.270 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:32:23.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.404+0000 7f689ea00640 1 Processor -- start 2024-04-23T14:32:23.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.404+0000 7f689ea00640 1 -- start start 2024-04-23T14:32:23.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.404+0000 7f689ea00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f68980cde20 0x7f68980cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.404+0000 7f689ea00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f68980cbee0 0x7f68980cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.404+0000 7f689ea00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68980ce280 con 0x7f689815d0c0 2024-04-23T14:32:23.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.404+0000 7f689ea00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f68980cff50 con 0x7f68980cbee0 2024-04-23T14:32:23.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.404+0000 7f689ea00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f68980ceda0 con 0x7f68980cde20 2024-04-23T14:32:23.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.404+0000 7f689ca00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f68980cbee0 0x7f68980cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.404+0000 7f689ca00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f68980cbee0 0x7f68980cf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39690/0 (socket says 172.21.15.100:39690) 2024-04-23T14:32:23.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.404+0000 7f689ca00640 1 -- 172.21.15.100:0/4029335238 learned_addr learned my addr 172.21.15.100:0/4029335238 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:23.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.404+0000 7f6897400640 1 --2- 172.21.15.100:0/4029335238 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f68980cde20 0x7f68980cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.404+0000 7f689ca00640 1 -- 172.21.15.100:0/4029335238 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f68980cde20 msgr2=0x7f68980cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:23.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.404+0000 7f689ca00640 1 --2- 172.21.15.100:0/4029335238 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f68980cde20 0x7f68980cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.404+0000 7f689ca00640 1 -- 172.21.15.100:0/4029335238 >> v1:172.21.15.151:6789/0 conn(0x7f689815d0c0 legacy=0x7f689815d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.404+0000 7f689ca00640 1 -- 172.21.15.100:0/4029335238 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68980ce500 con 0x7f68980cbee0 2024-04-23T14:32:23.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.408+0000 7f689ca00640 1 --2- 172.21.15.100:0/4029335238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68980cbee0 0x7f68980cf160 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6890009870 tx=0x7f689001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175465d7ee835205 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:23.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.408+0000 7f6896a00640 1 -- 172.21.15.100:0/4029335238 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68900b0070 con 0x7f68980cbee0 2024-04-23T14:32:23.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.408+0000 7f6896a00640 1 -- 172.21.15.100:0/4029335238 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68900ab600 con 0x7f68980cbee0 2024-04-23T14:32:23.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.408+0000 7f6896a00640 1 -- 172.21.15.100:0/4029335238 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68900ab9d0 con 0x7f68980cbee0 2024-04-23T14:32:23.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.408+0000 7f689ea00640 1 -- 172.21.15.100:0/4029335238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68980cbee0 msgr2=0x7f68980cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.408+0000 7f689ea00640 1 --2- 172.21.15.100:0/4029335238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68980cbee0 0x7f68980cf160 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6890009870 tx=0x7f689001c4d0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.408+0000 7f689ea00640 1 -- 172.21.15.100:0/4029335238 shutdown_connections 2024-04-23T14:32:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.408+0000 7f689ea00640 1 --2- 172.21.15.100:0/4029335238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68980cbee0 0x7f68980cf160 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.408+0000 7f689ea00640 1 --2- 172.21.15.100:0/4029335238 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f68980cde20 0x7f68980cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.408+0000 7f689ea00640 1 -- 172.21.15.100:0/4029335238 >> 172.21.15.100:0/4029335238 conn(0x7f68980b5310 msgr2=0x7f68980b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.408+0000 7f689ea00640 1 -- 172.21.15.100:0/4029335238 shutdown_connections 2024-04-23T14:32:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.408+0000 7f689ea00640 1 -- 172.21.15.100:0/4029335238 wait complete. 2024-04-23T14:32:23.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f689ea00640 1 Processor -- start 2024-04-23T14:32:23.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f689ea00640 1 -- start start 2024-04-23T14:32:23.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f689ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68980cbee0 0x7f689815f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f689ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68980cde20 0x7f6898159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f689ea00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f689815d0c0 0x7f6898151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f689ea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6898152fb0 con 0x7f68980cbee0 2024-04-23T14:32:23.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f689ea00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6898152e30 con 0x7f68980cde20 2024-04-23T14:32:23.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f689ea00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f6898153130 con 0x7f689815d0c0 2024-04-23T14:32:23.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f6897e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68980cbee0 0x7f689815f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f6897e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68980cbee0 0x7f689815f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39698/0 (socket says 172.21.15.100:39698) 2024-04-23T14:32:23.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f6897e00640 1 -- 172.21.15.100:0/3020889008 learned_addr learned my addr 172.21.15.100:0/3020889008 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:23.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f689ca00640 1 --2- 172.21.15.100:0/3020889008 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f689815d0c0 0x7f6898151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f6897400640 1 --2- 172.21.15.100:0/3020889008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68980cde20 0x7f6898159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f6897e00640 1 -- 172.21.15.100:0/3020889008 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f689815d0c0 msgr2=0x7f6898151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f6897e00640 1 --2- 172.21.15.100:0/3020889008 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f689815d0c0 0x7f6898151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f6897e00640 1 -- 172.21.15.100:0/3020889008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68980cde20 msgr2=0x7f6898159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f6897e00640 1 --2- 172.21.15.100:0/3020889008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68980cde20 0x7f6898159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f6897e00640 1 -- 172.21.15.100:0/3020889008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6898166980 con 0x7f68980cbee0 2024-04-23T14:32:23.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f6897e00640 1 --2- 172.21.15.100:0/3020889008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68980cbee0 0x7f689815f880 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f688800c730 tx=0x7f688800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:23.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f688fe00640 1 -- 172.21.15.100:0/3020889008 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888016020 con 0x7f68980cbee0 2024-04-23T14:32:23.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f688fe00640 1 -- 172.21.15.100:0/3020889008 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6888006d60 con 0x7f68980cbee0 2024-04-23T14:32:23.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f688fe00640 1 -- 172.21.15.100:0/3020889008 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888003bc0 con 0x7f68980cbee0 2024-04-23T14:32:23.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f689ea00640 1 -- 172.21.15.100:0/3020889008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6898166c60 con 0x7f68980cbee0 2024-04-23T14:32:23.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.412+0000 7f689ea00640 1 -- 172.21.15.100:0/3020889008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6898159d50 con 0x7f68980cbee0 2024-04-23T14:32:23.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.416+0000 7f689ea00640 1 -- 172.21.15.100:0/3020889008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f689806dd40 con 0x7f68980cbee0 2024-04-23T14:32:23.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.416+0000 7f688fe00640 1 -- 172.21.15.100:0/3020889008 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f688800a850 con 0x7f68980cbee0 2024-04-23T14:32:23.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.416+0000 7f688fe00640 1 --2- 172.21.15.100:0/3020889008 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6878068380 0x7f687806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.416+0000 7f688fe00640 1 -- 172.21.15.100:0/3020889008 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6888086880 con 0x7f68980cbee0 2024-04-23T14:32:23.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.416+0000 7f6897400640 1 --2- 172.21.15.100:0/3020889008 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6878068380 0x7f687806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.416+0000 7f6897400640 1 --2- 172.21.15.100:0/3020889008 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6878068380 0x7f687806a840 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f68981fafa0 tx=0x7f6884009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:23.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.420+0000 7f688fe00640 1 -- 172.21.15.100:0/3020889008 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6888020020 con 0x7f68980cbee0 2024-04-23T14:32:23.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.588+0000 7f689ea00640 1 -- 172.21.15.100:0/3020889008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6898068ad0 con 0x7f68980cbee0 2024-04-23T14:32:23.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.588+0000 7f688fe00640 1 -- 172.21.15.100:0/3020889008 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f68980ce280 con 0x7f68980cbee0 2024-04-23T14:32:23.592 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-04-23T14:32:23.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.592+0000 7f689ea00640 1 -- 172.21.15.100:0/3020889008 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6878068380 msgr2=0x7f687806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.592+0000 7f689ea00640 1 --2- 172.21.15.100:0/3020889008 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6878068380 0x7f687806a840 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f68981fafa0 tx=0x7f6884009190 comp rx=0 tx=0).stop 2024-04-23T14:32:23.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.592+0000 7f689ea00640 1 -- 172.21.15.100:0/3020889008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68980cbee0 msgr2=0x7f689815f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.592+0000 7f689ea00640 1 --2- 172.21.15.100:0/3020889008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68980cbee0 0x7f689815f880 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f688800c730 tx=0x7f688800cbf0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.592+0000 7f689ea00640 1 -- 172.21.15.100:0/3020889008 shutdown_connections 2024-04-23T14:32:23.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.592+0000 7f689ea00640 1 --2- 172.21.15.100:0/3020889008 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6878068380 0x7f687806a840 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.592+0000 7f689ea00640 1 --2- 172.21.15.100:0/3020889008 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f689815d0c0 0x7f6898151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.592+0000 7f689ea00640 1 --2- 172.21.15.100:0/3020889008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68980cde20 0x7f6898159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.592+0000 7f689ea00640 1 --2- 172.21.15.100:0/3020889008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68980cbee0 0x7f689815f880 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.592+0000 7f689ea00640 1 -- 172.21.15.100:0/3020889008 >> 172.21.15.100:0/3020889008 conn(0x7f68980b5310 msgr2=0x7f689815d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:23.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.592+0000 7f689ea00640 1 -- 172.21.15.100:0/3020889008 shutdown_connections 2024-04-23T14:32:23.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.592+0000 7f689ea00640 1 -- 172.21.15.100:0/3020889008 wait complete. 2024-04-23T14:32:23.613 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:32:23.613 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:32:23.622 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:23.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.796+0000 7fe155000640 1 Processor -- start 2024-04-23T14:32:23.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.796+0000 7fe155000640 1 -- start start 2024-04-23T14:32:23.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.800+0000 7fe155000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fe15006d350 0x7fe150073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.800+0000 7fe155000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe150073880 con 0x7fe15006ef80 2024-04-23T14:32:23.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.800+0000 7fe155000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1500b47f0 con 0x7fe15015fa90 2024-04-23T14:32:23.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.800+0000 7fe155000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fe15006eaf0 con 0x7fe15006d350 2024-04-23T14:32:23.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.800+0000 7fe14e800640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe15006ef80 0x7fe15006f430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:41046/0 (socket says 172.21.15.100:41046) 2024-04-23T14:32:23.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.800+0000 7fe14e800640 1 -- 172.21.15.100:0/2937888682 learned_addr learned my addr 172.21.15.100:0/2937888682 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:23.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.800+0000 7fe14de00640 1 --2- 172.21.15.100:0/2937888682 >> v2:172.21.15.151:3300/0 conn(0x7fe15006d350 0x7fe150073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.800+0000 7fe14d400640 1 -- 172.21.15.100:0/2937888682 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3579662425 0 0) 0x7fe150073880 con 0x7fe15006ef80 2024-04-23T14:32:23.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.800+0000 7fe14d400640 1 -- 172.21.15.100:0/2937888682 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe134003530 con 0x7fe15006ef80 2024-04-23T14:32:23.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.800+0000 7fe14d400640 1 -- 172.21.15.100:0/2937888682 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2203822535 0 0) 0x7fe1500b47f0 con 0x7fe15015fa90 2024-04-23T14:32:23.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.800+0000 7fe14d400640 1 -- 172.21.15.100:0/2937888682 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe150073880 con 0x7fe15015fa90 2024-04-23T14:32:23.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.800+0000 7fe14de00640 1 -- 172.21.15.100:0/2937888682 >> v1:172.21.15.151:6789/0 conn(0x7fe15015fa90 legacy=0x7fe15006d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.800+0000 7fe14de00640 1 -- 172.21.15.100:0/2937888682 >> v1:172.21.15.100:6789/0 conn(0x7fe15006ef80 legacy=0x7fe15006f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.800+0000 7fe14d400640 1 -- 172.21.15.100:0/2937888682 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1310818991 0 0) 0x7fe134003530 con 0x7fe15006ef80 2024-04-23T14:32:23.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.800+0000 7fe14de00640 1 -- 172.21.15.100:0/2937888682 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1500b4ad0 con 0x7fe15006d350 2024-04-23T14:32:23.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.804+0000 7fe14de00640 1 --2- 172.21.15.100:0/2937888682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe15006d350 0x7fe150073340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe1400097c0 tx=0x7fe14001c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=348211453c7047c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:23.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.804+0000 7fe14d400640 1 -- 172.21.15.100:0/2937888682 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1400b0070 con 0x7fe15006d350 2024-04-23T14:32:23.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.804+0000 7fe14d400640 1 -- 172.21.15.100:0/2937888682 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1400ab400 con 0x7fe15006d350 2024-04-23T14:32:23.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.804+0000 7fe14d400640 1 -- 172.21.15.100:0/2937888682 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148001a00 con 0x7fe15006d350 2024-04-23T14:32:23.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.804+0000 7fe155000640 1 -- 172.21.15.100:0/2937888682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe15006d350 msgr2=0x7fe150073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.804+0000 7fe155000640 1 --2- 172.21.15.100:0/2937888682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe15006d350 0x7fe150073340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe1400097c0 tx=0x7fe14001c330 comp rx=0 tx=0).stop 2024-04-23T14:32:23.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.804+0000 7fe155000640 1 -- 172.21.15.100:0/2937888682 shutdown_connections 2024-04-23T14:32:23.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.804+0000 7fe155000640 1 --2- 172.21.15.100:0/2937888682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe15006d350 0x7fe150073340 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.804+0000 7fe155000640 1 -- 172.21.15.100:0/2937888682 >> 172.21.15.100:0/2937888682 conn(0x7fe150072600 msgr2=0x7fe150072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:23.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.804+0000 7fe155000640 1 -- 172.21.15.100:0/2937888682 shutdown_connections 2024-04-23T14:32:23.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.804+0000 7fe155000640 1 -- 172.21.15.100:0/2937888682 wait complete. 2024-04-23T14:32:23.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.804+0000 7fe155000640 1 Processor -- start 2024-04-23T14:32:23.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.804+0000 7fe155000640 1 -- start start 2024-04-23T14:32:23.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.804+0000 7fe155000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe15006d350 0x7fe150156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.804+0000 7fe155000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe15006ef80 0x7fe1501578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.804+0000 7fe155000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe15015fa90 0x7fe150153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.804+0000 7fe155000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe15016d330 con 0x7fe15006d350 2024-04-23T14:32:23.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.804+0000 7fe155000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe15016d1b0 con 0x7fe15006ef80 2024-04-23T14:32:23.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.804+0000 7fe155000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe15016d4b0 con 0x7fe15015fa90 2024-04-23T14:32:23.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.808+0000 7fe14de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe15006d350 0x7fe150156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.808+0000 7fe14de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe15006d350 0x7fe150156c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39708/0 (socket says 172.21.15.100:39708) 2024-04-23T14:32:23.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.808+0000 7fe14de00640 1 -- 172.21.15.100:0/2390858462 learned_addr learned my addr 172.21.15.100:0/2390858462 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:23.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.808+0000 7fe14de00640 1 -- 172.21.15.100:0/2390858462 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe15015fa90 msgr2=0x7fe150153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.808+0000 7fe14de00640 1 --2- 172.21.15.100:0/2390858462 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe15015fa90 0x7fe150153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.808+0000 7fe14de00640 1 -- 172.21.15.100:0/2390858462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe15006ef80 msgr2=0x7fe1501578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.808+0000 7fe14de00640 1 --2- 172.21.15.100:0/2390858462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe15006ef80 0x7fe1501578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.808+0000 7fe14de00640 1 -- 172.21.15.100:0/2390858462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1500729e0 con 0x7fe15006d350 2024-04-23T14:32:23.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.808+0000 7fe14de00640 1 --2- 172.21.15.100:0/2390858462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe15006d350 0x7fe150156c90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe140000c10 tx=0x7fe1400190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:23.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.808+0000 7fe146a00640 1 -- 172.21.15.100:0/2390858462 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1400b9020 con 0x7fe15006d350 2024-04-23T14:32:23.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.808+0000 7fe146a00640 1 -- 172.21.15.100:0/2390858462 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe140003a30 con 0x7fe15006d350 2024-04-23T14:32:23.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.808+0000 7fe155000640 1 -- 172.21.15.100:0/2390858462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe150157320 con 0x7fe15006d350 2024-04-23T14:32:23.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.808+0000 7fe146a00640 1 -- 172.21.15.100:0/2390858462 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1400b0040 con 0x7fe15006d350 2024-04-23T14:32:23.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.808+0000 7fe155000640 1 -- 172.21.15.100:0/2390858462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe150150990 con 0x7fe15006d350 2024-04-23T14:32:23.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.808+0000 7fe155000640 1 -- 172.21.15.100:0/2390858462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe150068ad0 con 0x7fe15006d350 2024-04-23T14:32:23.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.808+0000 7fe146a00640 1 -- 172.21.15.100:0/2390858462 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe1400bc020 con 0x7fe15006d350 2024-04-23T14:32:23.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.812+0000 7fe146a00640 1 --2- 172.21.15.100:0/2390858462 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe12c068380 0x7fe12c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.812+0000 7fe146a00640 1 -- 172.21.15.100:0/2390858462 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe140120d80 con 0x7fe15006d350 2024-04-23T14:32:23.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.812+0000 7fe145400640 1 --2- 172.21.15.100:0/2390858462 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe12c068380 0x7fe12c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.812+0000 7fe145400640 1 --2- 172.21.15.100:0/2390858462 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe12c068380 0x7fe12c06a840 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe138007fd0 tx=0x7fe138007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:23.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.816+0000 7fe146a00640 1 -- 172.21.15.100:0/2390858462 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe140124030 con 0x7fe15006d350 2024-04-23T14:32:23.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.972+0000 7fe155000640 1 -- 172.21.15.100:0/2390858462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe150073120 con 0x7fe15006d350 2024-04-23T14:32:23.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.976+0000 7fe146a00640 1 -- 172.21.15.100:0/2390858462 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7fe1400c3030 con 0x7fe15006d350 2024-04-23T14:32:23.978 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:23.978 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":24,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:14.037027+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:23.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.976+0000 7fe155000640 1 -- 172.21.15.100:0/2390858462 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe12c068380 msgr2=0x7fe12c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.976+0000 7fe155000640 1 --2- 172.21.15.100:0/2390858462 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe12c068380 0x7fe12c06a840 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe138007fd0 tx=0x7fe138007960 comp rx=0 tx=0).stop 2024-04-23T14:32:23.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.976+0000 7fe155000640 1 -- 172.21.15.100:0/2390858462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe15006d350 msgr2=0x7fe150156c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.976+0000 7fe155000640 1 --2- 172.21.15.100:0/2390858462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe15006d350 0x7fe150156c90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe140000c10 tx=0x7fe1400190d0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.976+0000 7fe155000640 1 -- 172.21.15.100:0/2390858462 shutdown_connections 2024-04-23T14:32:23.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.976+0000 7fe155000640 1 --2- 172.21.15.100:0/2390858462 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe12c068380 0x7fe12c06a840 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.976+0000 7fe155000640 1 --2- 172.21.15.100:0/2390858462 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe15015fa90 0x7fe150153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.976+0000 7fe155000640 1 --2- 172.21.15.100:0/2390858462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe15006ef80 0x7fe1501578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.976+0000 7fe155000640 1 --2- 172.21.15.100:0/2390858462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe15006d350 0x7fe150156c90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.976+0000 7fe155000640 1 -- 172.21.15.100:0/2390858462 >> 172.21.15.100:0/2390858462 conn(0x7fe150072600 msgr2=0x7fe1500cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:23.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.976+0000 7fe155000640 1 -- 172.21.15.100:0/2390858462 shutdown_connections 2024-04-23T14:32:23.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:23.976+0000 7fe155000640 1 -- 172.21.15.100:0/2390858462 wait complete. 2024-04-23T14:32:23.999 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:32:06.835560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:32:07.930489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-23T14:32:11.636814+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-23T14:32:23.999 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:24.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.136+0000 7ff9e2a00640 1 Processor -- start 2024-04-23T14:32:24.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.136+0000 7ff9e2a00640 1 -- start start 2024-04-23T14:32:24.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.136+0000 7ff9e2a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff9dc06c6c0 0x7ff9dc0ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.136+0000 7ff9e2a00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff9dc0ce620 0x7ff9dc0add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.136+0000 7ff9e2a00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9dc06dd60 con 0x7ff9dc06f8c0 2024-04-23T14:32:24.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.136+0000 7ff9e2a00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7ff9dc0ad050 con 0x7ff9dc0ce620 2024-04-23T14:32:24.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.136+0000 7ff9e2a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff9dc0ad8f0 con 0x7ff9dc06c6c0 2024-04-23T14:32:24.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.136+0000 7ff9d3400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff9dc06c6c0 0x7ff9dc0ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.136+0000 7ff9e0a00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff9dc0ce620 0x7ff9dc0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.136+0000 7ff9dbe00640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7ff9dc06f8c0 0x7ff9dc0d0170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:40332/0 (socket says 172.21.15.100:40332) 2024-04-23T14:32:24.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.136+0000 7ff9d3400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff9dc06c6c0 0x7ff9dc0ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:55712/0 (socket says 172.21.15.100:55712) 2024-04-23T14:32:24.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.136+0000 7ff9dbe00640 1 -- 172.21.15.100:0/2642000732 learned_addr learned my addr 172.21.15.100:0/2642000732 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:24.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.136+0000 7ff9d3400640 1 -- 172.21.15.100:0/2642000732 >> v2:172.21.15.151:3300/0 conn(0x7ff9dc0ce620 msgr2=0x7ff9dc0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:24.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.136+0000 7ff9d3400640 1 --2- 172.21.15.100:0/2642000732 >> v2:172.21.15.151:3300/0 conn(0x7ff9dc0ce620 0x7ff9dc0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.136+0000 7ff9d3400640 1 -- 172.21.15.100:0/2642000732 >> v1:172.21.15.151:6789/0 conn(0x7ff9dc06f8c0 legacy=0x7ff9dc0d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.136+0000 7ff9d3400640 1 -- 172.21.15.100:0/2642000732 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc0d09c0 con 0x7ff9dc06c6c0 2024-04-23T14:32:24.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.140+0000 7ff9d3400640 1 --2- 172.21.15.100:0/2642000732 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff9dc06c6c0 0x7ff9dc0ce0e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40097c0 tx=0x7ff9c401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b29e20e08c8b26cd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:24.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.140+0000 7ff9db400640 1 -- 172.21.15.100:0/2642000732 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40b1070 con 0x7ff9dc06c6c0 2024-04-23T14:32:24.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.140+0000 7ff9db400640 1 -- 172.21.15.100:0/2642000732 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9c40ac410 con 0x7ff9dc06c6c0 2024-04-23T14:32:24.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.140+0000 7ff9db400640 1 -- 172.21.15.100:0/2642000732 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40ac7e0 con 0x7ff9dc06c6c0 2024-04-23T14:32:24.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.140+0000 7ff9e2a00640 1 -- 172.21.15.100:0/2642000732 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff9dc06c6c0 msgr2=0x7ff9dc0ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.140+0000 7ff9e2a00640 1 --2- 172.21.15.100:0/2642000732 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff9dc06c6c0 0x7ff9dc0ce0e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40097c0 tx=0x7ff9c401c4a0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.140+0000 7ff9e2a00640 1 -- 172.21.15.100:0/2642000732 shutdown_connections 2024-04-23T14:32:24.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.140+0000 7ff9e2a00640 1 --2- 172.21.15.100:0/2642000732 >> v2:172.21.15.151:3300/0 conn(0x7ff9dc0ce620 0x7ff9dc0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.140+0000 7ff9e2a00640 1 --2- 172.21.15.100:0/2642000732 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff9dc06c6c0 0x7ff9dc0ce0e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.140+0000 7ff9e2a00640 1 -- 172.21.15.100:0/2642000732 >> 172.21.15.100:0/2642000732 conn(0x7ff9dc0b5200 msgr2=0x7ff9dc0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:24.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.140+0000 7ff9e2a00640 1 -- 172.21.15.100:0/2642000732 shutdown_connections 2024-04-23T14:32:24.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.140+0000 7ff9e2a00640 1 -- 172.21.15.100:0/2642000732 wait complete. 2024-04-23T14:32:24.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9e2a00640 1 Processor -- start 2024-04-23T14:32:24.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9e2a00640 1 -- start start 2024-04-23T14:32:24.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9e2a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff9dc06c6c0 0x7ff9dc098dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9e2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9dc06f8c0 0x7ff9dc0c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9e2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9dc0ce620 0x7ff9dc0cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9e2a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc0aeea0 con 0x7ff9dc06f8c0 2024-04-23T14:32:24.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9e2a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc0aed20 con 0x7ff9dc0ce620 2024-04-23T14:32:24.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9e2a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff9dc168400 con 0x7ff9dc06c6c0 2024-04-23T14:32:24.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9d3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9dc06f8c0 0x7ff9dc0c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9d3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9dc06f8c0 0x7ff9dc0c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39714/0 (socket says 172.21.15.100:39714) 2024-04-23T14:32:24.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9d3400640 1 -- 172.21.15.100:0/2786426739 learned_addr learned my addr 172.21.15.100:0/2786426739 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:24.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9dbe00640 1 --2- 172.21.15.100:0/2786426739 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff9dc06c6c0 0x7ff9dc098dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9e0a00640 1 --2- 172.21.15.100:0/2786426739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9dc0ce620 0x7ff9dc0cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9d3400640 1 -- 172.21.15.100:0/2786426739 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff9dc06c6c0 msgr2=0x7ff9dc098dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9d3400640 1 --2- 172.21.15.100:0/2786426739 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff9dc06c6c0 0x7ff9dc098dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9d3400640 1 -- 172.21.15.100:0/2786426739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9dc0ce620 msgr2=0x7ff9dc0cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9d3400640 1 --2- 172.21.15.100:0/2786426739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9dc0ce620 0x7ff9dc0cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9d3400640 1 -- 172.21.15.100:0/2786426739 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc159e70 con 0x7ff9dc06f8c0 2024-04-23T14:32:24.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9e0a00640 1 --2- 172.21.15.100:0/2786426739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9dc0ce620 0x7ff9dc0cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:24.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9d3400640 1 --2- 172.21.15.100:0/2786426739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9dc06f8c0 0x7ff9dc0c5d50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff9c401a0d0 tx=0x7ff9c4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:24.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9d8c00640 1 -- 172.21.15.100:0/2786426739 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40b9020 con 0x7ff9dc06f8c0 2024-04-23T14:32:24.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9d8c00640 1 -- 172.21.15.100:0/2786426739 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9c40acd90 con 0x7ff9dc06f8c0 2024-04-23T14:32:24.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9d8c00640 1 -- 172.21.15.100:0/2786426739 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40b1040 con 0x7ff9dc06f8c0 2024-04-23T14:32:24.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9e2a00640 1 -- 172.21.15.100:0/2786426739 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9dc06e980 con 0x7ff9dc06f8c0 2024-04-23T14:32:24.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9e2a00640 1 -- 172.21.15.100:0/2786426739 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9dc098010 con 0x7ff9dc06f8c0 2024-04-23T14:32:24.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.144+0000 7ff9e2a00640 1 -- 172.21.15.100:0/2786426739 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9dc068ad0 con 0x7ff9dc06f8c0 2024-04-23T14:32:24.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.148+0000 7ff9d8c00640 1 -- 172.21.15.100:0/2786426739 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff9c40bc020 con 0x7ff9dc06f8c0 2024-04-23T14:32:24.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.148+0000 7ff9d8c00640 1 --2- 172.21.15.100:0/2786426739 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff9b8068380 0x7ff9b806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.148+0000 7ff9d8c00640 1 -- 172.21.15.100:0/2786426739 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff9c4120560 con 0x7ff9dc06f8c0 2024-04-23T14:32:24.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.148+0000 7ff9dbe00640 1 --2- 172.21.15.100:0/2786426739 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff9b8068380 0x7ff9b806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.148+0000 7ff9dbe00640 1 --2- 172.21.15.100:0/2786426739 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff9b8068380 0x7ff9b806a840 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc000f70 tx=0x7ff9cc007e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:24.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.152+0000 7ff9d8c00640 1 -- 172.21.15.100:0/2786426739 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff9c40c3030 con 0x7ff9dc06f8c0 2024-04-23T14:32:24.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.308+0000 7ff9e2a00640 1 -- 172.21.15.100:0/2786426739 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff9dc06cc00 con 0x7ff9dc06f8c0 2024-04-23T14:32:24.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.308+0000 7ff9d8c00640 1 -- 172.21.15.100:0/2786426739 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7ff9c4016090 con 0x7ff9dc06f8c0 2024-04-23T14:32:24.311 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:32:24.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.308+0000 7ff9e2a00640 1 -- 172.21.15.100:0/2786426739 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff9b8068380 msgr2=0x7ff9b806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.308+0000 7ff9e2a00640 1 --2- 172.21.15.100:0/2786426739 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff9b8068380 0x7ff9b806a840 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc000f70 tx=0x7ff9cc007e90 comp rx=0 tx=0).stop 2024-04-23T14:32:24.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.308+0000 7ff9e2a00640 1 -- 172.21.15.100:0/2786426739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9dc06f8c0 msgr2=0x7ff9dc0c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.308+0000 7ff9e2a00640 1 --2- 172.21.15.100:0/2786426739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9dc06f8c0 0x7ff9dc0c5d50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff9c401a0d0 tx=0x7ff9c4019240 comp rx=0 tx=0).stop 2024-04-23T14:32:24.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.308+0000 7ff9e2a00640 1 -- 172.21.15.100:0/2786426739 shutdown_connections 2024-04-23T14:32:24.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.308+0000 7ff9e2a00640 1 --2- 172.21.15.100:0/2786426739 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff9b8068380 0x7ff9b806a840 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.308+0000 7ff9e2a00640 1 --2- 172.21.15.100:0/2786426739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9dc0ce620 0x7ff9dc0cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.308+0000 7ff9e2a00640 1 --2- 172.21.15.100:0/2786426739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9dc06f8c0 0x7ff9dc0c5d50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.308+0000 7ff9e2a00640 1 --2- 172.21.15.100:0/2786426739 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff9dc06c6c0 0x7ff9dc098dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.308+0000 7ff9e2a00640 1 -- 172.21.15.100:0/2786426739 >> 172.21.15.100:0/2786426739 conn(0x7ff9dc0b5200 msgr2=0x7ff9dc0af020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:24.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.308+0000 7ff9e2a00640 1 -- 172.21.15.100:0/2786426739 shutdown_connections 2024-04-23T14:32:24.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.308+0000 7ff9e2a00640 1 -- 172.21.15.100:0/2786426739 wait complete. 2024-04-23T14:32:24.332 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:32:24.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.468+0000 7f075c000640 1 Processor -- start 2024-04-23T14:32:24.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.468+0000 7f075c000640 1 -- start start 2024-04-23T14:32:24.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.468+0000 7f075c000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f075415c7a0 0x7f07540af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.468+0000 7f075c000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f075406bb20 0x7f07540ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.468+0000 7f075c000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f07540cd130 0x7f075406e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.468+0000 7f075c000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f07540cd5a0 con 0x7f075415c7a0 2024-04-23T14:32:24.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.468+0000 7f075c000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f075406d450 con 0x7f07540cd130 2024-04-23T14:32:24.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.468+0000 7f075c000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f075406cbb0 con 0x7f075406bb20 2024-04-23T14:32:24.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.468+0000 7f0759600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f075415c7a0 0x7f07540af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.468+0000 7f0759600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f075415c7a0 0x7f07540af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39722/0 (socket says 172.21.15.100:39722) 2024-04-23T14:32:24.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.468+0000 7f0759600640 1 -- 172.21.15.100:0/4118850669 learned_addr learned my addr 172.21.15.100:0/4118850669 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:24.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.468+0000 7f0758c00640 1 --2- 172.21.15.100:0/4118850669 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f075406bb20 0x7f07540ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.468+0000 7f075a000640 1 --2- 172.21.15.100:0/4118850669 >> v2:172.21.15.151:3300/0 conn(0x7f07540cd130 0x7f075406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.468+0000 7f0759600640 1 -- 172.21.15.100:0/4118850669 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f075406bb20 msgr2=0x7f07540ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:24.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.468+0000 7f0759600640 1 --2- 172.21.15.100:0/4118850669 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f075406bb20 0x7f07540ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.468+0000 7f0759600640 1 -- 172.21.15.100:0/4118850669 >> v2:172.21.15.151:3300/0 conn(0x7f07540cd130 msgr2=0x7f075406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:24.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.468+0000 7f0759600640 1 --2- 172.21.15.100:0/4118850669 >> v2:172.21.15.151:3300/0 conn(0x7f07540cd130 0x7f075406e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.468+0000 7f0759600640 1 -- 172.21.15.100:0/4118850669 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075406ce90 con 0x7f075415c7a0 2024-04-23T14:32:24.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.472+0000 7f0759600640 1 --2- 172.21.15.100:0/4118850669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f075415c7a0 0x7f07540af190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0754075550 tx=0x7f074801c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9602a55ab93e6e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:24.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.472+0000 7f0753e00640 1 -- 172.21.15.100:0/4118850669 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07480a9430 con 0x7f075415c7a0 2024-04-23T14:32:24.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.472+0000 7f0753e00640 1 -- 172.21.15.100:0/4118850669 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07480ae070 con 0x7f075415c7a0 2024-04-23T14:32:24.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.472+0000 7f0753e00640 1 -- 172.21.15.100:0/4118850669 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07480a97f0 con 0x7f075415c7a0 2024-04-23T14:32:24.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.472+0000 7f075c000640 1 -- 172.21.15.100:0/4118850669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f075415c7a0 msgr2=0x7f07540af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.472+0000 7f075c000640 1 --2- 172.21.15.100:0/4118850669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f075415c7a0 0x7f07540af190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0754075550 tx=0x7f074801c6a0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.472+0000 7f075c000640 1 -- 172.21.15.100:0/4118850669 shutdown_connections 2024-04-23T14:32:24.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.472+0000 7f075c000640 1 --2- 172.21.15.100:0/4118850669 >> v2:172.21.15.151:3300/0 conn(0x7f07540cd130 0x7f075406e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.472+0000 7f075c000640 1 --2- 172.21.15.100:0/4118850669 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f075406bb20 0x7f07540ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.472+0000 7f075c000640 1 --2- 172.21.15.100:0/4118850669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f075415c7a0 0x7f07540af190 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.472+0000 7f075c000640 1 -- 172.21.15.100:0/4118850669 >> 172.21.15.100:0/4118850669 conn(0x7f07540b5200 msgr2=0x7f07540b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:24.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.472+0000 7f075c000640 1 -- 172.21.15.100:0/4118850669 shutdown_connections 2024-04-23T14:32:24.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.472+0000 7f075c000640 1 -- 172.21.15.100:0/4118850669 wait complete. 2024-04-23T14:32:24.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.472+0000 7f075c000640 1 Processor -- start 2024-04-23T14:32:24.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.472+0000 7f075c000640 1 -- start start 2024-04-23T14:32:24.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f075c000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f075406bb20 0x7f07540c2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f075c000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f07540cd130 0x7f07540923f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f075c000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f075415c7a0 0x7f0754092930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f075c000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f07540ad7c0 con 0x7f075415c7a0 2024-04-23T14:32:24.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f075c000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07540ad640 con 0x7f075406bb20 2024-04-23T14:32:24.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f075c000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f07540ad940 con 0x7f07540cd130 2024-04-23T14:32:24.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f075a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f075415c7a0 0x7f0754092930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f075a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f075415c7a0 0x7f0754092930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39736/0 (socket says 172.21.15.100:39736) 2024-04-23T14:32:24.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f075a000640 1 -- 172.21.15.100:0/525889685 learned_addr learned my addr 172.21.15.100:0/525889685 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:24.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f0759600640 1 --2- 172.21.15.100:0/525889685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f075406bb20 0x7f07540c2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f0758c00640 1 --2- 172.21.15.100:0/525889685 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f07540cd130 0x7f07540923f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f075a000640 1 -- 172.21.15.100:0/525889685 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f07540cd130 msgr2=0x7f07540923f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f075a000640 1 --2- 172.21.15.100:0/525889685 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f07540cd130 0x7f07540923f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f075a000640 1 -- 172.21.15.100:0/525889685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f075406bb20 msgr2=0x7f07540c2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f075a000640 1 --2- 172.21.15.100:0/525889685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f075406bb20 0x7f07540c2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f075a000640 1 -- 172.21.15.100:0/525889685 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0754090d70 con 0x7f075415c7a0 2024-04-23T14:32:24.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f0758c00640 1 --2- 172.21.15.100:0/525889685 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f07540cd130 0x7f07540923f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:24.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f075a000640 1 --2- 172.21.15.100:0/525889685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f075415c7a0 0x7f0754092930 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f074c00d800 tx=0x7f074c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:24.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f0751600640 1 -- 172.21.15.100:0/525889685 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c00b6c0 con 0x7f075415c7a0 2024-04-23T14:32:24.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f0751600640 1 -- 172.21.15.100:0/525889685 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f074c019070 con 0x7f075415c7a0 2024-04-23T14:32:24.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f075c000640 1 -- 172.21.15.100:0/525889685 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0754090fe0 con 0x7f075415c7a0 2024-04-23T14:32:24.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f0751600640 1 -- 172.21.15.100:0/525889685 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c00baa0 con 0x7f075415c7a0 2024-04-23T14:32:24.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f075c000640 1 -- 172.21.15.100:0/525889685 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07541538c0 con 0x7f075415c7a0 2024-04-23T14:32:24.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f075c000640 1 -- 172.21.15.100:0/525889685 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0754068ad0 con 0x7f075415c7a0 2024-04-23T14:32:24.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.476+0000 7f0751600640 1 -- 172.21.15.100:0/525889685 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f074c00bc40 con 0x7f075415c7a0 2024-04-23T14:32:24.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.480+0000 7f0751600640 1 --2- 172.21.15.100:0/525889685 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0734068380 0x7f073406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.480+0000 7f0751600640 1 -- 172.21.15.100:0/525889685 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f074c087100 con 0x7f075415c7a0 2024-04-23T14:32:24.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.480+0000 7f0759600640 1 --2- 172.21.15.100:0/525889685 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0734068380 0x7f073406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.480+0000 7f0759600640 1 --2- 172.21.15.100:0/525889685 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0734068380 0x7f073406a840 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f07480ad000 tx=0x7f07480b4040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:24.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.484+0000 7f0751600640 1 -- 172.21.15.100:0/525889685 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f074c0572e0 con 0x7f075415c7a0 2024-04-23T14:32:24.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.640+0000 7f075c000640 1 -- 172.21.15.100:0/525889685 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f07540ce570 con 0x7f075415c7a0 2024-04-23T14:32:24.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.640+0000 7f0751600640 1 -- 172.21.15.100:0/525889685 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f074c056be0 con 0x7f075415c7a0 2024-04-23T14:32:24.645 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:32:24.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.644+0000 7f075c000640 1 -- 172.21.15.100:0/525889685 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0734068380 msgr2=0x7f073406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.644+0000 7f075c000640 1 --2- 172.21.15.100:0/525889685 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0734068380 0x7f073406a840 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f07480ad000 tx=0x7f07480b4040 comp rx=0 tx=0).stop 2024-04-23T14:32:24.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.644+0000 7f075c000640 1 -- 172.21.15.100:0/525889685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f075415c7a0 msgr2=0x7f0754092930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.644+0000 7f075c000640 1 --2- 172.21.15.100:0/525889685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f075415c7a0 0x7f0754092930 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f074c00d800 tx=0x7f074c00dcc0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.644+0000 7f075c000640 1 -- 172.21.15.100:0/525889685 shutdown_connections 2024-04-23T14:32:24.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.644+0000 7f075c000640 1 --2- 172.21.15.100:0/525889685 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0734068380 0x7f073406a840 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.644+0000 7f075c000640 1 --2- 172.21.15.100:0/525889685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f075415c7a0 0x7f0754092930 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.644+0000 7f075c000640 1 --2- 172.21.15.100:0/525889685 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f07540cd130 0x7f07540923f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.644+0000 7f075c000640 1 --2- 172.21.15.100:0/525889685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f075406bb20 0x7f07540c2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.644+0000 7f075c000640 1 -- 172.21.15.100:0/525889685 >> 172.21.15.100:0/525889685 conn(0x7f07540b5200 msgr2=0x7f075415ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:24.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.644+0000 7f075c000640 1 -- 172.21.15.100:0/525889685 shutdown_connections 2024-04-23T14:32:24.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.644+0000 7f075c000640 1 -- 172.21.15.100:0/525889685 wait complete. 2024-04-23T14:32:24.668 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:32:24.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.804+0000 7fea42e00640 1 Processor -- start 2024-04-23T14:32:24.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.804+0000 7fea42e00640 1 -- start start 2024-04-23T14:32:24.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.804+0000 7fea42e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fea3c15eb10 0x7fea3c0d4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.804+0000 7fea42e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea3c0b39c0 0x7fea3c0d4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.804+0000 7fea42e00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea3c0cfc40 con 0x7fea3c0d3e40 2024-04-23T14:32:24.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.804+0000 7fea42e00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fea3c0d4f90 con 0x7fea3c15eb10 2024-04-23T14:32:24.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.804+0000 7fea42e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fea3c0cfee0 con 0x7fea3c0b39c0 2024-04-23T14:32:24.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.804+0000 7fea3be00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fea3c15eb10 0x7fea3c0d4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.804+0000 7fea33400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fea3c0d3e40 0x7fea3c0d2240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:41050/0 (socket says 172.21.15.100:41050) 2024-04-23T14:32:24.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.804+0000 7fea33400640 1 -- 172.21.15.100:0/1390682436 learned_addr learned my addr 172.21.15.100:0/1390682436 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:24.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.804+0000 7fea40e00640 1 --2- 172.21.15.100:0/1390682436 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea3c0b39c0 0x7fea3c0d4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.804+0000 7fea3be00640 1 -- 172.21.15.100:0/1390682436 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea3c0b39c0 msgr2=0x7fea3c0d4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:24.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.804+0000 7fea3be00640 1 --2- 172.21.15.100:0/1390682436 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea3c0b39c0 0x7fea3c0d4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.804+0000 7fea3be00640 1 -- 172.21.15.100:0/1390682436 >> v1:172.21.15.100:6789/0 conn(0x7fea3c0d3e40 legacy=0x7fea3c0d2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.804+0000 7fea3be00640 1 -- 172.21.15.100:0/1390682436 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3c0d01c0 con 0x7fea3c15eb10 2024-04-23T14:32:24.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.808+0000 7fea3be00640 1 --2- 172.21.15.100:0/1390682436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea3c15eb10 0x7fea3c0d4510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009d90 tx=0x7fea2c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5eec610c663c251 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:24.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.808+0000 7fea3b400640 1 -- 172.21.15.100:0/1390682436 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0b2070 con 0x7fea3c15eb10 2024-04-23T14:32:24.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.808+0000 7fea3b400640 1 -- 172.21.15.100:0/1390682436 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea2c0a5860 con 0x7fea3c15eb10 2024-04-23T14:32:24.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.808+0000 7fea3b400640 1 -- 172.21.15.100:0/1390682436 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0a5c30 con 0x7fea3c15eb10 2024-04-23T14:32:24.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.808+0000 7fea42e00640 1 -- 172.21.15.100:0/1390682436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea3c15eb10 msgr2=0x7fea3c0d4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.808+0000 7fea42e00640 1 --2- 172.21.15.100:0/1390682436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea3c15eb10 0x7fea3c0d4510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009d90 tx=0x7fea2c01c970 comp rx=0 tx=0).stop 2024-04-23T14:32:24.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.808+0000 7fea42e00640 1 -- 172.21.15.100:0/1390682436 shutdown_connections 2024-04-23T14:32:24.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.808+0000 7fea42e00640 1 --2- 172.21.15.100:0/1390682436 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea3c0b39c0 0x7fea3c0d4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.808+0000 7fea42e00640 1 --2- 172.21.15.100:0/1390682436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea3c15eb10 0x7fea3c0d4510 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.808+0000 7fea42e00640 1 -- 172.21.15.100:0/1390682436 >> 172.21.15.100:0/1390682436 conn(0x7fea3c0b9a30 msgr2=0x7fea3c0b75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:24.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.808+0000 7fea42e00640 1 -- 172.21.15.100:0/1390682436 shutdown_connections 2024-04-23T14:32:24.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.808+0000 7fea42e00640 1 -- 172.21.15.100:0/1390682436 wait complete. 2024-04-23T14:32:24.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea42e00640 1 Processor -- start 2024-04-23T14:32:24.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea42e00640 1 -- start start 2024-04-23T14:32:24.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea42e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea3c0b39c0 0x7fea3c14fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea42e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea3c15eb10 0x7fea3c1589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea42e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea3c158f00 0x7fea3c0b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea42e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c0d55a0 con 0x7fea3c0b39c0 2024-04-23T14:32:24.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea42e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c0d5420 con 0x7fea3c15eb10 2024-04-23T14:32:24.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea42e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fea3c0b12c0 con 0x7fea3c158f00 2024-04-23T14:32:24.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea3be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea3c0b39c0 0x7fea3c14fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea3be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea3c0b39c0 0x7fea3c14fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39744/0 (socket says 172.21.15.100:39744) 2024-04-23T14:32:24.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea3be00640 1 -- 172.21.15.100:0/2795775660 learned_addr learned my addr 172.21.15.100:0/2795775660 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:24.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea3be00640 1 -- 172.21.15.100:0/2795775660 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea3c158f00 msgr2=0x7fea3c0b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea3be00640 1 --2- 172.21.15.100:0/2795775660 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea3c158f00 0x7fea3c0b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea3be00640 1 -- 172.21.15.100:0/2795775660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea3c15eb10 msgr2=0x7fea3c1589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea3be00640 1 --2- 172.21.15.100:0/2795775660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea3c15eb10 0x7fea3c1589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea3be00640 1 -- 172.21.15.100:0/2795775660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3c14ff60 con 0x7fea3c0b39c0 2024-04-23T14:32:24.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea3be00640 1 --2- 172.21.15.100:0/2795775660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea3c0b39c0 0x7fea3c14fa20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0b0040 tx=0x7fea2c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:24.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea38c00640 1 -- 172.21.15.100:0/2795775660 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0b2040 con 0x7fea3c0b39c0 2024-04-23T14:32:24.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea38c00640 1 -- 172.21.15.100:0/2795775660 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea2c0bf070 con 0x7fea3c0b39c0 2024-04-23T14:32:24.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea38c00640 1 -- 172.21.15.100:0/2795775660 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0b88b0 con 0x7fea3c0b39c0 2024-04-23T14:32:24.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea42e00640 1 -- 172.21.15.100:0/2795775660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3c06e370 con 0x7fea3c0b39c0 2024-04-23T14:32:24.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea42e00640 1 -- 172.21.15.100:0/2795775660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea3c06d5b0 con 0x7fea3c0b39c0 2024-04-23T14:32:24.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.812+0000 7fea42e00640 1 -- 172.21.15.100:0/2795775660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea3c068ad0 con 0x7fea3c0b39c0 2024-04-23T14:32:24.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.816+0000 7fea38c00640 1 -- 172.21.15.100:0/2795775660 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fea2c0b8a50 con 0x7fea3c0b39c0 2024-04-23T14:32:24.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.816+0000 7fea38c00640 1 --2- 172.21.15.100:0/2795775660 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fea18068380 0x7fea1806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.816+0000 7fea38c00640 1 -- 172.21.15.100:0/2795775660 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fea2c121de0 con 0x7fea3c0b39c0 2024-04-23T14:32:24.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.816+0000 7fea33400640 1 --2- 172.21.15.100:0/2795775660 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fea18068380 0x7fea1806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.816+0000 7fea33400640 1 --2- 172.21.15.100:0/2795775660 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fea18068380 0x7fea1806a840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fea24000f70 tx=0x7fea24007a90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:24.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.820+0000 7fea38c00640 1 -- 172.21.15.100:0/2795775660 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fea3c06e370 con 0x7fea3c0b39c0 2024-04-23T14:32:24.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.972+0000 7fea42e00640 1 -- 172.21.15.100:0/2795775660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fea3c0cecc0 con 0x7fea3c0b39c0 2024-04-23T14:32:24.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.972+0000 7fea38c00640 1 -- 172.21.15.100:0/2795775660 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fea2c016560 con 0x7fea3c0b39c0 2024-04-23T14:32:24.977 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:32:24.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.976+0000 7fea42e00640 1 -- 172.21.15.100:0/2795775660 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fea18068380 msgr2=0x7fea1806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.976+0000 7fea42e00640 1 --2- 172.21.15.100:0/2795775660 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fea18068380 0x7fea1806a840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fea24000f70 tx=0x7fea24007a90 comp rx=0 tx=0).stop 2024-04-23T14:32:24.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.976+0000 7fea42e00640 1 -- 172.21.15.100:0/2795775660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea3c0b39c0 msgr2=0x7fea3c14fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.976+0000 7fea42e00640 1 --2- 172.21.15.100:0/2795775660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea3c0b39c0 0x7fea3c14fa20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0b0040 tx=0x7fea2c019710 comp rx=0 tx=0).stop 2024-04-23T14:32:24.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.976+0000 7fea42e00640 1 -- 172.21.15.100:0/2795775660 shutdown_connections 2024-04-23T14:32:24.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.976+0000 7fea42e00640 1 --2- 172.21.15.100:0/2795775660 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fea18068380 0x7fea1806a840 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.976+0000 7fea42e00640 1 --2- 172.21.15.100:0/2795775660 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea3c158f00 0x7fea3c0b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.976+0000 7fea42e00640 1 --2- 172.21.15.100:0/2795775660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea3c15eb10 0x7fea3c1589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.976+0000 7fea42e00640 1 --2- 172.21.15.100:0/2795775660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea3c0b39c0 0x7fea3c14fa20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.976+0000 7fea42e00640 1 -- 172.21.15.100:0/2795775660 >> 172.21.15.100:0/2795775660 conn(0x7fea3c0b9a30 msgr2=0x7fea3c0d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:24.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.976+0000 7fea42e00640 1 -- 172.21.15.100:0/2795775660 shutdown_connections 2024-04-23T14:32:24.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:24.976+0000 7fea42e00640 1 -- 172.21.15.100:0/2795775660 wait complete. 2024-04-23T14:32:24.998 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-23T14:32:25.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.132+0000 7faca4000640 1 Processor -- start 2024-04-23T14:32:25.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.132+0000 7faca4000640 1 -- start start 2024-04-23T14:32:25.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.132+0000 7faca4000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fac9c15c7a0 0x7fac9c0af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.132+0000 7faca4000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fac9c06bb20 0x7fac9c0ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.132+0000 7faca4000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac9c0cd540 con 0x7fac9c0cd130 2024-04-23T14:32:25.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.132+0000 7faca4000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fac9c06d450 con 0x7fac9c06bb20 2024-04-23T14:32:25.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.132+0000 7faca4000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fac9c06cbb0 con 0x7fac9c15c7a0 2024-04-23T14:32:25.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.132+0000 7faca0c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fac9c06bb20 0x7fac9c0ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.132+0000 7faca0c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fac9c06bb20 0x7fac9c0ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39754/0 (socket says 172.21.15.100:39754) 2024-04-23T14:32:25.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.132+0000 7faca0c00640 1 -- 172.21.15.100:0/2672434906 learned_addr learned my addr 172.21.15.100:0/2672434906 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:25.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.136+0000 7faca1600640 1 --2- 172.21.15.100:0/2672434906 >> v2:172.21.15.151:3300/0 conn(0x7fac9c15c7a0 0x7fac9c0af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.136+0000 7faca0c00640 1 -- 172.21.15.100:0/2672434906 >> v2:172.21.15.151:3300/0 conn(0x7fac9c15c7a0 msgr2=0x7fac9c0af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:25.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.136+0000 7faca0c00640 1 --2- 172.21.15.100:0/2672434906 >> v2:172.21.15.151:3300/0 conn(0x7fac9c15c7a0 0x7fac9c0af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.136+0000 7faca0c00640 1 -- 172.21.15.100:0/2672434906 >> v1:172.21.15.100:6789/0 conn(0x7fac9c0cd130 legacy=0x7fac9c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.136+0000 7faca0c00640 1 -- 172.21.15.100:0/2672434906 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c06ce90 con 0x7fac9c06bb20 2024-04-23T14:32:25.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.136+0000 7faca0c00640 1 --2- 172.21.15.100:0/2672434906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac9c06bb20 0x7fac9c0ccbf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0716b0 tx=0x7fac8801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417f31ef85534783 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:25.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.136+0000 7fac9be00640 1 -- 172.21.15.100:0/2672434906 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac880b0070 con 0x7fac9c06bb20 2024-04-23T14:32:25.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.136+0000 7fac9be00640 1 -- 172.21.15.100:0/2672434906 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac880ab600 con 0x7fac9c06bb20 2024-04-23T14:32:25.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.136+0000 7fac9be00640 1 -- 172.21.15.100:0/2672434906 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac880ab9d0 con 0x7fac9c06bb20 2024-04-23T14:32:25.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.136+0000 7faca4000640 1 -- 172.21.15.100:0/2672434906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac9c06bb20 msgr2=0x7fac9c0ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.136+0000 7faca4000640 1 --2- 172.21.15.100:0/2672434906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac9c06bb20 0x7fac9c0ccbf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0716b0 tx=0x7fac8801c490 comp rx=0 tx=0).stop 2024-04-23T14:32:25.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.136+0000 7faca4000640 1 -- 172.21.15.100:0/2672434906 shutdown_connections 2024-04-23T14:32:25.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.136+0000 7faca4000640 1 --2- 172.21.15.100:0/2672434906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac9c06bb20 0x7fac9c0ccbf0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.136+0000 7faca4000640 1 --2- 172.21.15.100:0/2672434906 >> v2:172.21.15.151:3300/0 conn(0x7fac9c15c7a0 0x7fac9c0af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.136+0000 7faca4000640 1 -- 172.21.15.100:0/2672434906 >> 172.21.15.100:0/2672434906 conn(0x7fac9c0b5200 msgr2=0x7fac9c0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:25.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca4000640 1 -- 172.21.15.100:0/2672434906 shutdown_connections 2024-04-23T14:32:25.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca4000640 1 -- 172.21.15.100:0/2672434906 wait complete. 2024-04-23T14:32:25.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca4000640 1 Processor -- start 2024-04-23T14:32:25.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca4000640 1 -- start start 2024-04-23T14:32:25.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca4000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac9c06bb20 0x7fac9c0cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca4000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fac9c0cd130 0x7fac9c0cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca4000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac9c15c7a0 0x7fac9c071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca4000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c169590 con 0x7fac9c15c7a0 2024-04-23T14:32:25.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca4000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c169410 con 0x7fac9c06bb20 2024-04-23T14:32:25.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca4000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fac9c169710 con 0x7fac9c0cd130 2024-04-23T14:32:25.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac9c15c7a0 0x7fac9c071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac9c15c7a0 0x7fac9c071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39756/0 (socket says 172.21.15.100:39756) 2024-04-23T14:32:25.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca2000640 1 -- 172.21.15.100:0/3564289002 learned_addr learned my addr 172.21.15.100:0/3564289002 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:25.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca1600640 1 --2- 172.21.15.100:0/3564289002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac9c06bb20 0x7fac9c0cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca0c00640 1 --2- 172.21.15.100:0/3564289002 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fac9c0cd130 0x7fac9c0cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca2000640 1 -- 172.21.15.100:0/3564289002 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fac9c0cd130 msgr2=0x7fac9c0cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca2000640 1 --2- 172.21.15.100:0/3564289002 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fac9c0cd130 0x7fac9c0cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca2000640 1 -- 172.21.15.100:0/3564289002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac9c06bb20 msgr2=0x7fac9c0cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca2000640 1 --2- 172.21.15.100:0/3564289002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac9c06bb20 0x7fac9c0cbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca2000640 1 -- 172.21.15.100:0/3564289002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c08c120 con 0x7fac9c15c7a0 2024-04-23T14:32:25.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca1600640 1 --2- 172.21.15.100:0/3564289002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac9c06bb20 0x7fac9c0cbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:25.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca0c00640 1 --2- 172.21.15.100:0/3564289002 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fac9c0cd130 0x7fac9c0cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:25.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca2000640 1 --2- 172.21.15.100:0/3564289002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac9c15c7a0 0x7fac9c071c00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fac9400c770 tx=0x7fac9400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:25.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7fac99600640 1 -- 172.21.15.100:0/3564289002 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94003b80 con 0x7fac9c15c7a0 2024-04-23T14:32:25.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7fac99600640 1 -- 172.21.15.100:0/3564289002 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac94003d20 con 0x7fac9c15c7a0 2024-04-23T14:32:25.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7fac99600640 1 -- 172.21.15.100:0/3564289002 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9400aa90 con 0x7fac9c15c7a0 2024-04-23T14:32:25.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.140+0000 7faca4000640 1 -- 172.21.15.100:0/3564289002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9c08c400 con 0x7fac9c15c7a0 2024-04-23T14:32:25.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.144+0000 7faca4000640 1 -- 172.21.15.100:0/3564289002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac9c0c72b0 con 0x7fac9c15c7a0 2024-04-23T14:32:25.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.144+0000 7faca4000640 1 -- 172.21.15.100:0/3564289002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac9c068ad0 con 0x7fac9c15c7a0 2024-04-23T14:32:25.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.144+0000 7fac99600640 1 -- 172.21.15.100:0/3564289002 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fac9400ac30 con 0x7fac9c15c7a0 2024-04-23T14:32:25.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.144+0000 7fac99600640 1 --2- 172.21.15.100:0/3564289002 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fac7c068380 0x7fac7c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.144+0000 7fac99600640 1 -- 172.21.15.100:0/3564289002 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fac94086c80 con 0x7fac9c15c7a0 2024-04-23T14:32:25.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.144+0000 7faca1600640 1 --2- 172.21.15.100:0/3564289002 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fac7c068380 0x7fac7c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.144+0000 7faca1600640 1 --2- 172.21.15.100:0/3564289002 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fac7c068380 0x7fac7c06a840 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fac900097a0 tx=0x7fac90008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:25.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.148+0000 7fac99600640 1 -- 172.21.15.100:0/3564289002 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fac9c0cd540 con 0x7fac9c15c7a0 2024-04-23T14:32:25.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.304+0000 7faca4000640 1 -- 172.21.15.100:0/3564289002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fac9c15a840 con 0x7fac9c15c7a0 2024-04-23T14:32:25.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.304+0000 7fac99600640 1 -- 172.21.15.100:0/3564289002 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7fac94056e60 con 0x7fac9c15c7a0 2024-04-23T14:32:25.308 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:32:25.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.308+0000 7faca4000640 1 -- 172.21.15.100:0/3564289002 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fac7c068380 msgr2=0x7fac7c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.308+0000 7faca4000640 1 --2- 172.21.15.100:0/3564289002 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fac7c068380 0x7fac7c06a840 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fac900097a0 tx=0x7fac90008040 comp rx=0 tx=0).stop 2024-04-23T14:32:25.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.308+0000 7faca4000640 1 -- 172.21.15.100:0/3564289002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac9c15c7a0 msgr2=0x7fac9c071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.308+0000 7faca4000640 1 --2- 172.21.15.100:0/3564289002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac9c15c7a0 0x7fac9c071c00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fac9400c770 tx=0x7fac9400cc30 comp rx=0 tx=0).stop 2024-04-23T14:32:25.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.308+0000 7faca4000640 1 -- 172.21.15.100:0/3564289002 shutdown_connections 2024-04-23T14:32:25.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.308+0000 7faca4000640 1 --2- 172.21.15.100:0/3564289002 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fac7c068380 0x7fac7c06a840 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.308+0000 7faca4000640 1 --2- 172.21.15.100:0/3564289002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac9c15c7a0 0x7fac9c071c00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.308+0000 7faca4000640 1 --2- 172.21.15.100:0/3564289002 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fac9c0cd130 0x7fac9c0cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.308+0000 7faca4000640 1 --2- 172.21.15.100:0/3564289002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac9c06bb20 0x7fac9c0cbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.308+0000 7faca4000640 1 -- 172.21.15.100:0/3564289002 >> 172.21.15.100:0/3564289002 conn(0x7fac9c0b5200 msgr2=0x7fac9c15ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:25.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.308+0000 7faca4000640 1 -- 172.21.15.100:0/3564289002 shutdown_connections 2024-04-23T14:32:25.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.308+0000 7faca4000640 1 -- 172.21.15.100:0/3564289002 wait complete. 2024-04-23T14:32:25.329 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:32:25.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.460+0000 7f971d200640 1 Processor -- start 2024-04-23T14:32:25.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.460+0000 7f971d200640 1 -- start start 2024-04-23T14:32:25.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.464+0000 7f971d200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f971815b660 0x7f97180c6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.464+0000 7f971d200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f97180d08a0 0x7f97180c5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.464+0000 7f971d200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97180cee00 con 0x7f97180c6d60 2024-04-23T14:32:25.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.464+0000 7f971d200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f971815ba40 con 0x7f97180d08a0 2024-04-23T14:32:25.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.464+0000 7f971d200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f971806bc50 con 0x7f971815b660 2024-04-23T14:32:25.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.464+0000 7f9716800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f97180d08a0 0x7f97180c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.464+0000 7f9715e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f971815b660 0x7f97180c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.464+0000 7f9716800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f97180d08a0 0x7f97180c5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:59922/0 (socket says 172.21.15.100:59922) 2024-04-23T14:32:25.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.464+0000 7f9716800640 1 -- 172.21.15.100:0/507377383 learned_addr learned my addr 172.21.15.100:0/507377383 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:25.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.464+0000 7f9715e00640 1 -- 172.21.15.100:0/507377383 >> v2:172.21.15.151:3300/0 conn(0x7f97180d08a0 msgr2=0x7f97180c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:25.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.464+0000 7f9715e00640 1 --2- 172.21.15.100:0/507377383 >> v2:172.21.15.151:3300/0 conn(0x7f97180d08a0 0x7f97180c5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.464+0000 7f9715e00640 1 -- 172.21.15.100:0/507377383 >> v1:172.21.15.151:6789/0 conn(0x7f97180c6d60 legacy=0x7f97180d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.464+0000 7f9715e00640 1 -- 172.21.15.100:0/507377383 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971806bf30 con 0x7f971815b660 2024-04-23T14:32:25.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.464+0000 7f9716800640 1 --2- 172.21.15.100:0/507377383 >> v2:172.21.15.151:3300/0 conn(0x7f97180d08a0 0x7f97180c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:25.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.464+0000 7f9715e00640 1 --2- 172.21.15.100:0/507377383 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f971815b660 0x7f97180c6820 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f97080097c0 tx=0x7f970801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1009508b66e8fced server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:25.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.468+0000 7f9715400640 1 -- 172.21.15.100:0/507377383 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97080b2070 con 0x7f971815b660 2024-04-23T14:32:25.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.468+0000 7f9715400640 1 -- 172.21.15.100:0/507377383 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97080ad680 con 0x7f971815b660 2024-04-23T14:32:25.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.468+0000 7f9715400640 1 -- 172.21.15.100:0/507377383 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97080ada50 con 0x7f971815b660 2024-04-23T14:32:25.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.468+0000 7f971d200640 1 -- 172.21.15.100:0/507377383 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f971815b660 msgr2=0x7f97180c6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.468+0000 7f971d200640 1 --2- 172.21.15.100:0/507377383 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f971815b660 0x7f97180c6820 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f97080097c0 tx=0x7f970801c470 comp rx=0 tx=0).stop 2024-04-23T14:32:25.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.468+0000 7f971d200640 1 -- 172.21.15.100:0/507377383 shutdown_connections 2024-04-23T14:32:25.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.468+0000 7f971d200640 1 --2- 172.21.15.100:0/507377383 >> v2:172.21.15.151:3300/0 conn(0x7f97180d08a0 0x7f97180c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.468+0000 7f971d200640 1 --2- 172.21.15.100:0/507377383 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f971815b660 0x7f97180c6820 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.468+0000 7f971d200640 1 -- 172.21.15.100:0/507377383 >> 172.21.15.100:0/507377383 conn(0x7f97180b5310 msgr2=0x7f97180b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:25.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.468+0000 7f971d200640 1 -- 172.21.15.100:0/507377383 shutdown_connections 2024-04-23T14:32:25.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.468+0000 7f971d200640 1 -- 172.21.15.100:0/507377383 wait complete. 2024-04-23T14:32:25.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.468+0000 7f971d200640 1 Processor -- start 2024-04-23T14:32:25.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.468+0000 7f971d200640 1 -- start start 2024-04-23T14:32:25.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.468+0000 7f971d200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f97180c6d60 0x7f971806f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.468+0000 7f971d200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97180d08a0 0x7f9718149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.468+0000 7f971d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971815b660 0x7f97181646f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.468+0000 7f971d200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f97180c6030 con 0x7f971815b660 2024-04-23T14:32:25.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.468+0000 7f971d200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f97180c5eb0 con 0x7f97180d08a0 2024-04-23T14:32:25.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.468+0000 7f971d200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f97180c61b0 con 0x7f97180c6d60 2024-04-23T14:32:25.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.472+0000 7f9716800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971815b660 0x7f97181646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.472+0000 7f9716800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971815b660 0x7f97181646f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39764/0 (socket says 172.21.15.100:39764) 2024-04-23T14:32:25.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.472+0000 7f9716800640 1 -- 172.21.15.100:0/1480414689 learned_addr learned my addr 172.21.15.100:0/1480414689 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:25.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.472+0000 7f9715e00640 1 --2- 172.21.15.100:0/1480414689 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f97180c6d60 0x7f971806f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.472+0000 7f970d400640 1 --2- 172.21.15.100:0/1480414689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97180d08a0 0x7f9718149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.472+0000 7f9716800640 1 -- 172.21.15.100:0/1480414689 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f97180c6d60 msgr2=0x7f971806f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.472+0000 7f9716800640 1 --2- 172.21.15.100:0/1480414689 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f97180c6d60 0x7f971806f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.472+0000 7f9716800640 1 -- 172.21.15.100:0/1480414689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97180d08a0 msgr2=0x7f9718149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.472+0000 7f9716800640 1 --2- 172.21.15.100:0/1480414689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97180d08a0 0x7f9718149de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.472+0000 7f9716800640 1 -- 172.21.15.100:0/1480414689 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9718164f40 con 0x7f971815b660 2024-04-23T14:32:25.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.472+0000 7f970d400640 1 --2- 172.21.15.100:0/1480414689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97180d08a0 0x7f9718149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:25.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.472+0000 7f9715e00640 1 --2- 172.21.15.100:0/1480414689 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f97180c6d60 0x7f971806f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:25.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.472+0000 7f9716800640 1 --2- 172.21.15.100:0/1480414689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971815b660 0x7f97181646f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f971000efc0 tx=0x7f971000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:25.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.472+0000 7f970ea00640 1 -- 172.21.15.100:0/1480414689 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971000cbe0 con 0x7f971815b660 2024-04-23T14:32:25.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.472+0000 7f970ea00640 1 -- 172.21.15.100:0/1480414689 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f971000cd80 con 0x7f971815b660 2024-04-23T14:32:25.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.472+0000 7f970ea00640 1 -- 172.21.15.100:0/1480414689 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710003be0 con 0x7f971815b660 2024-04-23T14:32:25.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.472+0000 7f971d200640 1 -- 172.21.15.100:0/1480414689 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9718165220 con 0x7f971815b660 2024-04-23T14:32:25.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.472+0000 7f971d200640 1 -- 172.21.15.100:0/1480414689 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97181657e0 con 0x7f971815b660 2024-04-23T14:32:25.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.472+0000 7f971d200640 1 -- 172.21.15.100:0/1480414689 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9718068ad0 con 0x7f971815b660 2024-04-23T14:32:25.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.472+0000 7f970ea00640 1 -- 172.21.15.100:0/1480414689 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f971001b030 con 0x7f971815b660 2024-04-23T14:32:25.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.472+0000 7f970ea00640 1 --2- 172.21.15.100:0/1480414689 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f96f40683b0 0x7f96f406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.476+0000 7f970ea00640 1 -- 172.21.15.100:0/1480414689 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f97100871d0 con 0x7f971815b660 2024-04-23T14:32:25.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.476+0000 7f9715e00640 1 --2- 172.21.15.100:0/1480414689 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f96f40683b0 0x7f96f406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.476+0000 7f9715e00640 1 --2- 172.21.15.100:0/1480414689 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f96f40683b0 0x7f96f406a870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f97080af000 tx=0x7f97080a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:25.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.480+0000 7f970ea00640 1 -- 172.21.15.100:0/1480414689 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9710022020 con 0x7f971815b660 2024-04-23T14:32:25.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.652+0000 7f971d200640 1 -- 172.21.15.100:0/1480414689 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f971806d3a0 con 0x7f971815b660 2024-04-23T14:32:25.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.652+0000 7f970ea00640 1 -- 172.21.15.100:0/1480414689 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f9710010040 con 0x7f971815b660 2024-04-23T14:32:25.683 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-04-23T14:32:25.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.652+0000 7f971d200640 1 -- 172.21.15.100:0/1480414689 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f96f40683b0 msgr2=0x7f96f406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.652+0000 7f971d200640 1 --2- 172.21.15.100:0/1480414689 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f96f40683b0 0x7f96f406a870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f97080af000 tx=0x7f97080a7000 comp rx=0 tx=0).stop 2024-04-23T14:32:25.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.652+0000 7f971d200640 1 -- 172.21.15.100:0/1480414689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971815b660 msgr2=0x7f97181646f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.652+0000 7f971d200640 1 --2- 172.21.15.100:0/1480414689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971815b660 0x7f97181646f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f971000efc0 tx=0x7f971000c490 comp rx=0 tx=0).stop 2024-04-23T14:32:25.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.652+0000 7f971d200640 1 -- 172.21.15.100:0/1480414689 shutdown_connections 2024-04-23T14:32:25.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.652+0000 7f971d200640 1 --2- 172.21.15.100:0/1480414689 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f96f40683b0 0x7f96f406a870 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.652+0000 7f971d200640 1 --2- 172.21.15.100:0/1480414689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971815b660 0x7f97181646f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.652+0000 7f971d200640 1 --2- 172.21.15.100:0/1480414689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97180d08a0 0x7f9718149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.652+0000 7f971d200640 1 --2- 172.21.15.100:0/1480414689 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f97180c6d60 0x7f971806f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.652+0000 7f971d200640 1 -- 172.21.15.100:0/1480414689 >> 172.21.15.100:0/1480414689 conn(0x7f97180b5310 msgr2=0x7f97180b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:25.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.652+0000 7f971d200640 1 -- 172.21.15.100:0/1480414689 shutdown_connections 2024-04-23T14:32:25.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:25.652+0000 7f971d200640 1 -- 172.21.15.100:0/1480414689 wait complete. 2024-04-23T14:32:25.685 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi100.front.sepia.ceph.com... 2024-04-23T14:32:25.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:25.685 DEBUG:teuthology.orchestra.run.smithi100:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-23T14:32:25.911 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi100.stderr:ceph-fuse[33968]: fuse finished with error 0 and tester_r 0 2024-04-23T14:32:25.915 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:32:32.016 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:32:32.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:32.016 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-23T14:32:32.041 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:32.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1639400640 1 Processor -- start 2024-04-23T14:32:32.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1639400640 1 -- start start 2024-04-23T14:32:32.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1639400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16340b6930 0x7f16340d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1639400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16340b5700 con 0x7f163415c400 2024-04-23T14:32:32.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1639400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16340cd580 con 0x7f16340b6480 2024-04-23T14:32:32.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1639400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f16340d03e0 con 0x7f16340b6930 2024-04-23T14:32:32.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1633e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f163415c400 0x7f16340b5d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57170/0 (socket says 172.21.15.100:57170) 2024-04-23T14:32:32.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1633e00640 1 -- 172.21.15.100:0/3376888822 learned_addr learned my addr 172.21.15.100:0/3376888822 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:32.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1638a00640 1 --2- 172.21.15.100:0/3376888822 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16340b6930 0x7f16340d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1633400640 1 -- 172.21.15.100:0/3376888822 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2025743371 0 0) 0x7f16340b5700 con 0x7f163415c400 2024-04-23T14:32:32.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1633400640 1 -- 172.21.15.100:0/3376888822 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1618003530 con 0x7f163415c400 2024-04-23T14:32:32.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1633400640 1 -- 172.21.15.100:0/3376888822 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 970293554 0 0) 0x7f1618003530 con 0x7f163415c400 2024-04-23T14:32:32.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1633400640 1 -- 172.21.15.100:0/3376888822 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16340b6930 msgr2=0x7f16340d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:32.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1633400640 1 --2- 172.21.15.100:0/3376888822 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16340b6930 0x7f16340d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1633400640 1 -- 172.21.15.100:0/3376888822 >> v1:172.21.15.151:6789/0 conn(0x7f16340b6480 legacy=0x7f16340d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1633400640 1 -- 172.21.15.100:0/3376888822 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16340d0070 con 0x7f163415c400 2024-04-23T14:32:32.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1633400640 1 -- 172.21.15.100:0/3376888822 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f16240016a0 con 0x7f163415c400 2024-04-23T14:32:32.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1633400640 1 -- 172.21.15.100:0/3376888822 >> v1:172.21.15.100:6789/0 conn(0x7f163415c400 legacy=0x7f16340b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1633400640 1 --2- 172.21.15.100:0/3376888822 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16340b6930 0x7f16180039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1633400640 1 --2- 172.21.15.100:0/3376888822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1618003f30 0x7f1618004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1633400640 1 --2- 172.21.15.100:0/3376888822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1618004d50 0x7f16180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1633400640 1 -- 172.21.15.100:0/3376888822 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f16340d03e0 con 0x7f1618003f30 2024-04-23T14:32:32.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1633400640 1 -- 172.21.15.100:0/3376888822 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f161800bae0 con 0x7f1618004d50 2024-04-23T14:32:32.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1633400640 1 -- 172.21.15.100:0/3376888822 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f161800bce0 con 0x7f16340b6930 2024-04-23T14:32:32.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f162b400640 1 --2- 172.21.15.100:0/3376888822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1618003f30 0x7f1618004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1638a00640 1 --2- 172.21.15.100:0/3376888822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1618004d50 0x7f16180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1633e00640 1 --2- 172.21.15.100:0/3376888822 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16340b6930 0x7f16180039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f162b400640 1 -- 172.21.15.100:0/3376888822 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16340b6930 msgr2=0x7f16180039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:32.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f162b400640 1 --2- 172.21.15.100:0/3376888822 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16340b6930 0x7f16180039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f162b400640 1 -- 172.21.15.100:0/3376888822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1618004d50 msgr2=0x7f16180051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:32.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f162b400640 1 --2- 172.21.15.100:0/3376888822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1618004d50 0x7f16180051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f162b400640 1 -- 172.21.15.100:0/3376888822 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1618003530 con 0x7f1618003f30 2024-04-23T14:32:32.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.227+0000 7f1633e00640 1 --2- 172.21.15.100:0/3376888822 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16340b6930 0x7f16180039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:32.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.231+0000 7f162b400640 1 --2- 172.21.15.100:0/3376888822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1618003f30 0x7f1618004810 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f161c001860 tx=0x7f161c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6d50f67208f12f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:32.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.231+0000 7f1633400640 1 -- 172.21.15.100:0/3376888822 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c0b0070 con 0x7f1618003f30 2024-04-23T14:32:32.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.231+0000 7f1633400640 1 -- 172.21.15.100:0/3376888822 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f161c0ab600 con 0x7f1618003f30 2024-04-23T14:32:32.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.231+0000 7f1633400640 1 -- 172.21.15.100:0/3376888822 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c001460 con 0x7f1618003f30 2024-04-23T14:32:32.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.231+0000 7f1639400640 1 -- 172.21.15.100:0/3376888822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1618003f30 msgr2=0x7f1618004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.231+0000 7f1639400640 1 --2- 172.21.15.100:0/3376888822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1618003f30 0x7f1618004810 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f161c001860 tx=0x7f161c01c500 comp rx=0 tx=0).stop 2024-04-23T14:32:32.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.231+0000 7f1638a00640 1 -- 172.21.15.100:0/3376888822 reap_dead start 2024-04-23T14:32:32.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.231+0000 7f1639400640 1 -- 172.21.15.100:0/3376888822 shutdown_connections 2024-04-23T14:32:32.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.231+0000 7f1639400640 1 -- 172.21.15.100:0/3376888822 >> 172.21.15.100:0/3376888822 conn(0x7f16340d2320 msgr2=0x7f16340d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:32.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.231+0000 7f1639400640 1 -- 172.21.15.100:0/3376888822 shutdown_connections 2024-04-23T14:32:32.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.231+0000 7f1639400640 1 -- 172.21.15.100:0/3376888822 wait complete. 2024-04-23T14:32:32.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1639400640 1 Processor -- start 2024-04-23T14:32:32.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1639400640 1 -- start start 2024-04-23T14:32:32.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1639400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1618003f30 0x7f163415c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1639400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f163415a280 0x7f163415f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1639400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f163415e2e0 0x7f16341f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1639400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f16340ce4f0 con 0x7f1618003f30 2024-04-23T14:32:32.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1639400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f16340ce370 con 0x7f163415e2e0 2024-04-23T14:32:32.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1639400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1634168380 con 0x7f163415a280 2024-04-23T14:32:32.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1633e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1618003f30 0x7f163415c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1633e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1618003f30 0x7f163415c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57180/0 (socket says 172.21.15.100:57180) 2024-04-23T14:32:32.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1633e00640 1 -- 172.21.15.100:0/733784263 learned_addr learned my addr 172.21.15.100:0/733784263 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:32.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f162b400640 1 --2- 172.21.15.100:0/733784263 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f163415a280 0x7f163415f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1638a00640 1 --2- 172.21.15.100:0/733784263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f163415e2e0 0x7f16341f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1633e00640 1 -- 172.21.15.100:0/733784263 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f163415a280 msgr2=0x7f163415f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1633e00640 1 --2- 172.21.15.100:0/733784263 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f163415a280 0x7f163415f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1633e00640 1 -- 172.21.15.100:0/733784263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f163415e2e0 msgr2=0x7f16341f5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1633e00640 1 --2- 172.21.15.100:0/733784263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f163415e2e0 0x7f16341f5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1633e00640 1 -- 172.21.15.100:0/733784263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163415d240 con 0x7f1618003f30 2024-04-23T14:32:32.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1638a00640 1 --2- 172.21.15.100:0/733784263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f163415e2e0 0x7f16341f5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:32.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1633e00640 1 --2- 172.21.15.100:0/733784263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1618003f30 0x7f163415c9f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f162400e7b0 tx=0x7f162400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:32.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1630c00640 1 -- 172.21.15.100:0/733784263 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162400ee60 con 0x7f1618003f30 2024-04-23T14:32:32.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1630c00640 1 -- 172.21.15.100:0/733784263 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1624018070 con 0x7f1618003f30 2024-04-23T14:32:32.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1630c00640 1 -- 172.21.15.100:0/733784263 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162400caf0 con 0x7f1618003f30 2024-04-23T14:32:32.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1639400640 1 -- 172.21.15.100:0/733784263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f163415d520 con 0x7f1618003f30 2024-04-23T14:32:32.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1639400640 1 -- 172.21.15.100:0/733784263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f163415dae0 con 0x7f1618003f30 2024-04-23T14:32:32.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.235+0000 7f1639400640 1 -- 172.21.15.100:0/733784263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1634075c20 con 0x7f1618003f30 2024-04-23T14:32:32.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.239+0000 7f1630c00640 1 -- 172.21.15.100:0/733784263 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f162400cc90 con 0x7f1618003f30 2024-04-23T14:32:32.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.239+0000 7f1630c00640 1 --2- 172.21.15.100:0/733784263 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f16100683b0 0x7f161006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.239+0000 7f1630c00640 1 -- 172.21.15.100:0/733784263 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1624086b90 con 0x7f1618003f30 2024-04-23T14:32:32.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.239+0000 7f162b400640 1 --2- 172.21.15.100:0/733784263 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f16100683b0 0x7f161006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.239+0000 7f162b400640 1 --2- 172.21.15.100:0/733784263 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f16100683b0 0x7f161006a870 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f161c0ad000 tx=0x7f161c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:32.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.243+0000 7f1630c00640 1 -- 172.21.15.100:0/733784263 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1624056d80 con 0x7f1618003f30 2024-04-23T14:32:32.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.399+0000 7f1639400640 1 -- 172.21.15.100:0/733784263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16340b6210 con 0x7f1618003f30 2024-04-23T14:32:32.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.399+0000 7f1630c00640 1 -- 172.21.15.100:0/733784263 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f1624056680 con 0x7f1618003f30 2024-04-23T14:32:32.405 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:32.405 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":24,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:14.037027+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:32.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.403+0000 7f1639400640 1 -- 172.21.15.100:0/733784263 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f16100683b0 msgr2=0x7f161006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.403+0000 7f1639400640 1 --2- 172.21.15.100:0/733784263 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f16100683b0 0x7f161006a870 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f161c0ad000 tx=0x7f161c0a6000 comp rx=0 tx=0).stop 2024-04-23T14:32:32.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.403+0000 7f1639400640 1 -- 172.21.15.100:0/733784263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1618003f30 msgr2=0x7f163415c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.403+0000 7f1639400640 1 --2- 172.21.15.100:0/733784263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1618003f30 0x7f163415c9f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f162400e7b0 tx=0x7f162400ec70 comp rx=0 tx=0).stop 2024-04-23T14:32:32.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.403+0000 7f1639400640 1 -- 172.21.15.100:0/733784263 shutdown_connections 2024-04-23T14:32:32.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.403+0000 7f1639400640 1 --2- 172.21.15.100:0/733784263 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f16100683b0 0x7f161006a870 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.403+0000 7f1639400640 1 --2- 172.21.15.100:0/733784263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f163415e2e0 0x7f16341f5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.403+0000 7f1639400640 1 --2- 172.21.15.100:0/733784263 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f163415a280 0x7f163415f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.403+0000 7f1639400640 1 --2- 172.21.15.100:0/733784263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1618003f30 0x7f163415c9f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.403+0000 7f1639400640 1 -- 172.21.15.100:0/733784263 >> 172.21.15.100:0/733784263 conn(0x7f16340d2320 msgr2=0x7f16340cd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:32.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.403+0000 7f1639400640 1 -- 172.21.15.100:0/733784263 shutdown_connections 2024-04-23T14:32:32.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.403+0000 7f1639400640 1 -- 172.21.15.100:0/733784263 wait complete. 2024-04-23T14:32:32.427 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:32.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.555+0000 7fb728c00640 1 Processor -- start 2024-04-23T14:32:32.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.555+0000 7fb728c00640 1 -- start start 2024-04-23T14:32:32.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.559+0000 7fb728c00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fb72415e730 0x7fb7240cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.559+0000 7fb728c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb72415eb10 0x7fb7240d0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.559+0000 7fb728c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb724163070 0x7fb7240d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.559+0000 7fb728c00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fb72406f450 con 0x7fb72415eb10 2024-04-23T14:32:32.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.559+0000 7fb728c00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fb72406f170 con 0x7fb72415e730 2024-04-23T14:32:32.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.559+0000 7fb728c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb724162590 con 0x7fb724163070 2024-04-23T14:32:32.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.559+0000 7fb721400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb72415eb10 0x7fb7240d0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.559+0000 7fb721400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb72415eb10 0x7fb7240d0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57186/0 (socket says 172.21.15.100:57186) 2024-04-23T14:32:32.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.559+0000 7fb721400640 1 -- 172.21.15.100:0/181888499 learned_addr learned my addr 172.21.15.100:0/181888499 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:32.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.559+0000 7fb722800640 1 --2- 172.21.15.100:0/181888499 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb724163070 0x7fb7240d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.559+0000 7fb721e00640 1 --2- 172.21.15.100:0/181888499 >> v2:172.21.15.151:3300/0 conn(0x7fb72415e730 0x7fb7240cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.559+0000 7fb721400640 1 -- 172.21.15.100:0/181888499 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb724163070 msgr2=0x7fb7240d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:32.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.559+0000 7fb721400640 1 --2- 172.21.15.100:0/181888499 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb724163070 0x7fb7240d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.559+0000 7fb721400640 1 -- 172.21.15.100:0/181888499 >> v2:172.21.15.151:3300/0 conn(0x7fb72415e730 msgr2=0x7fb7240cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:32.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.559+0000 7fb721400640 1 --2- 172.21.15.100:0/181888499 >> v2:172.21.15.151:3300/0 conn(0x7fb72415e730 0x7fb7240cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.559+0000 7fb721400640 1 -- 172.21.15.100:0/181888499 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb724162870 con 0x7fb72415eb10 2024-04-23T14:32:32.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.559+0000 7fb721e00640 1 --2- 172.21.15.100:0/181888499 >> v2:172.21.15.151:3300/0 conn(0x7fb72415e730 0x7fb7240cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:32.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.563+0000 7fb721400640 1 --2- 172.21.15.100:0/181888499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb72415eb10 0x7fb7240d0050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb7240b8610 tx=0x7fb71001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca94e2983f00a839 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:32.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.563+0000 7fb720a00640 1 -- 172.21.15.100:0/181888499 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7100b2070 con 0x7fb72415eb10 2024-04-23T14:32:32.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.563+0000 7fb720a00640 1 -- 172.21.15.100:0/181888499 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7100a9070 con 0x7fb72415eb10 2024-04-23T14:32:32.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.563+0000 7fb720a00640 1 -- 172.21.15.100:0/181888499 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7100ad880 con 0x7fb72415eb10 2024-04-23T14:32:32.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.563+0000 7fb728c00640 1 -- 172.21.15.100:0/181888499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb72415eb10 msgr2=0x7fb7240d0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.563+0000 7fb728c00640 1 --2- 172.21.15.100:0/181888499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb72415eb10 0x7fb7240d0050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb7240b8610 tx=0x7fb71001c490 comp rx=0 tx=0).stop 2024-04-23T14:32:32.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.563+0000 7fb728c00640 1 -- 172.21.15.100:0/181888499 shutdown_connections 2024-04-23T14:32:32.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.563+0000 7fb728c00640 1 --2- 172.21.15.100:0/181888499 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb724163070 0x7fb7240d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.563+0000 7fb728c00640 1 --2- 172.21.15.100:0/181888499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb72415eb10 0x7fb7240d0050 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.563+0000 7fb728c00640 1 --2- 172.21.15.100:0/181888499 >> v2:172.21.15.151:3300/0 conn(0x7fb72415e730 0x7fb7240cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.563+0000 7fb728c00640 1 -- 172.21.15.100:0/181888499 >> 172.21.15.100:0/181888499 conn(0x7fb7240b4050 msgr2=0x7fb7240b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:32.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.563+0000 7fb728c00640 1 -- 172.21.15.100:0/181888499 shutdown_connections 2024-04-23T14:32:32.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.563+0000 7fb728c00640 1 -- 172.21.15.100:0/181888499 wait complete. 2024-04-23T14:32:32.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.563+0000 7fb728c00640 1 Processor -- start 2024-04-23T14:32:32.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.563+0000 7fb728c00640 1 -- start start 2024-04-23T14:32:32.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb728c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb72415e730 0x7fb7240c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb728c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb72415eb10 0x7fb7240cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb728c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb724163070 0x7fb7240cc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb728c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb7240cd670 con 0x7fb72415eb10 2024-04-23T14:32:32.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb728c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7240cd4f0 con 0x7fb72415e730 2024-04-23T14:32:32.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb728c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb7240cd7f0 con 0x7fb724163070 2024-04-23T14:32:32.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb721400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb72415eb10 0x7fb7240cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb721400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb72415eb10 0x7fb7240cbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57190/0 (socket says 172.21.15.100:57190) 2024-04-23T14:32:32.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb721400640 1 -- 172.21.15.100:0/814444632 learned_addr learned my addr 172.21.15.100:0/814444632 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:32.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb722800640 1 --2- 172.21.15.100:0/814444632 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb724163070 0x7fb7240cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb721e00640 1 --2- 172.21.15.100:0/814444632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb72415e730 0x7fb7240c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb721400640 1 -- 172.21.15.100:0/814444632 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb724163070 msgr2=0x7fb7240cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb721400640 1 --2- 172.21.15.100:0/814444632 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb724163070 0x7fb7240cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb721400640 1 -- 172.21.15.100:0/814444632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb72415e730 msgr2=0x7fb7240c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb721400640 1 --2- 172.21.15.100:0/814444632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb72415e730 0x7fb7240c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb721400640 1 -- 172.21.15.100:0/814444632 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb724155810 con 0x7fb72415eb10 2024-04-23T14:32:32.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb721e00640 1 --2- 172.21.15.100:0/814444632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb72415e730 0x7fb7240c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:32.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb721400640 1 --2- 172.21.15.100:0/814444632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb72415eb10 0x7fb7240cbb00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb71001ab40 tx=0x7fb710019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:32.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb71a000640 1 -- 172.21.15.100:0/814444632 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7100b2040 con 0x7fb72415eb10 2024-04-23T14:32:32.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb71a000640 1 -- 172.21.15.100:0/814444632 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7100b6620 con 0x7fb72415eb10 2024-04-23T14:32:32.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb71a000640 1 -- 172.21.15.100:0/814444632 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7100b6a10 con 0x7fb72415eb10 2024-04-23T14:32:32.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb728c00640 1 -- 172.21.15.100:0/814444632 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb724155af0 con 0x7fb72415eb10 2024-04-23T14:32:32.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb728c00640 1 -- 172.21.15.100:0/814444632 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72415f140 con 0x7fb72415eb10 2024-04-23T14:32:32.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb728c00640 1 -- 172.21.15.100:0/814444632 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb724068ad0 con 0x7fb72415eb10 2024-04-23T14:32:32.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.567+0000 7fb71a000640 1 -- 172.21.15.100:0/814444632 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb7100ab070 con 0x7fb72415eb10 2024-04-23T14:32:32.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.571+0000 7fb71a000640 1 --2- 172.21.15.100:0/814444632 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb7040683b0 0x7fb70406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.571+0000 7fb71a000640 1 -- 172.21.15.100:0/814444632 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb7101208e0 con 0x7fb72415eb10 2024-04-23T14:32:32.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.571+0000 7fb721e00640 1 --2- 172.21.15.100:0/814444632 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb7040683b0 0x7fb70406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.571+0000 7fb721e00640 1 --2- 172.21.15.100:0/814444632 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb7040683b0 0x7fb70406a870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb7140044f0 tx=0x7fb714003a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:32.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.575+0000 7fb71a000640 1 -- 172.21.15.100:0/814444632 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb710016110 con 0x7fb72415eb10 2024-04-23T14:32:32.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.735+0000 7fb728c00640 1 -- 172.21.15.100:0/814444632 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7240c5530 con 0x7fb72415eb10 2024-04-23T14:32:32.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.735+0000 7fb71a000640 1 -- 172.21.15.100:0/814444632 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7fb7100f0c70 con 0x7fb72415eb10 2024-04-23T14:32:32.740 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:32.740 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":24,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:14.037027+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:32.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.739+0000 7fb728c00640 1 -- 172.21.15.100:0/814444632 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb7040683b0 msgr2=0x7fb70406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.739+0000 7fb728c00640 1 --2- 172.21.15.100:0/814444632 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb7040683b0 0x7fb70406a870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb7140044f0 tx=0x7fb714003a40 comp rx=0 tx=0).stop 2024-04-23T14:32:32.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.739+0000 7fb728c00640 1 -- 172.21.15.100:0/814444632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb72415eb10 msgr2=0x7fb7240cbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.739+0000 7fb728c00640 1 --2- 172.21.15.100:0/814444632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb72415eb10 0x7fb7240cbb00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb71001ab40 tx=0x7fb710019230 comp rx=0 tx=0).stop 2024-04-23T14:32:32.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.739+0000 7fb728c00640 1 -- 172.21.15.100:0/814444632 shutdown_connections 2024-04-23T14:32:32.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.739+0000 7fb728c00640 1 --2- 172.21.15.100:0/814444632 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb7040683b0 0x7fb70406a870 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.739+0000 7fb728c00640 1 --2- 172.21.15.100:0/814444632 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb724163070 0x7fb7240cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.739+0000 7fb728c00640 1 --2- 172.21.15.100:0/814444632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb72415eb10 0x7fb7240cbb00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.739+0000 7fb728c00640 1 --2- 172.21.15.100:0/814444632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb72415e730 0x7fb7240c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.739+0000 7fb728c00640 1 -- 172.21.15.100:0/814444632 >> 172.21.15.100:0/814444632 conn(0x7fb7240b4050 msgr2=0x7fb7240cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:32.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.739+0000 7fb728c00640 1 -- 172.21.15.100:0/814444632 shutdown_connections 2024-04-23T14:32:32.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.739+0000 7fb728c00640 1 -- 172.21.15.100:0/814444632 wait complete. 2024-04-23T14:32:32.763 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-23T14:32:32.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.899+0000 7f38f7000640 1 Processor -- start 2024-04-23T14:32:32.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.899+0000 7f38f7000640 1 -- start start 2024-04-23T14:32:32.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.899+0000 7f38f7000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f38f006f8c0 0x7f38f006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.899+0000 7f38f7000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f38f00d0170 0x7f38f00ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.899+0000 7f38f7000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f38f00ce620 0x7f38f015f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.899+0000 7f38f7000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f38f00ad050 con 0x7f38f00d0170 2024-04-23T14:32:32.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.899+0000 7f38f7000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f38f00ad8f0 con 0x7f38f006f8c0 2024-04-23T14:32:32.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.899+0000 7f38f7000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f38f006dd60 con 0x7f38f00ce620 2024-04-23T14:32:32.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.899+0000 7f38ef400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f38f00d0170 0x7f38f00ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.899+0000 7f38ef400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f38f00d0170 0x7f38f00ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57204/0 (socket says 172.21.15.100:57204) 2024-04-23T14:32:32.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.899+0000 7f38ef400640 1 -- 172.21.15.100:0/2793783015 learned_addr learned my addr 172.21.15.100:0/2793783015 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:32.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.899+0000 7f38efe00640 1 --2- 172.21.15.100:0/2793783015 >> v2:172.21.15.151:3300/0 conn(0x7f38f006f8c0 0x7f38f006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.899+0000 7f38f5000640 1 --2- 172.21.15.100:0/2793783015 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f38f00ce620 0x7f38f015f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.899+0000 7f38ef400640 1 -- 172.21.15.100:0/2793783015 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f38f00ce620 msgr2=0x7f38f015f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:32.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.899+0000 7f38ef400640 1 --2- 172.21.15.100:0/2793783015 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f38f00ce620 0x7f38f015f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.899+0000 7f38ef400640 1 -- 172.21.15.100:0/2793783015 >> v2:172.21.15.151:3300/0 conn(0x7f38f006f8c0 msgr2=0x7f38f006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:32.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.899+0000 7f38ef400640 1 --2- 172.21.15.100:0/2793783015 >> v2:172.21.15.151:3300/0 conn(0x7f38f006f8c0 0x7f38f006c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.899+0000 7f38ef400640 1 -- 172.21.15.100:0/2793783015 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f00d09c0 con 0x7f38f00d0170 2024-04-23T14:32:32.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.899+0000 7f38efe00640 1 --2- 172.21.15.100:0/2793783015 >> v2:172.21.15.151:3300/0 conn(0x7f38f006f8c0 0x7f38f006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:32.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.899+0000 7f38f5000640 1 --2- 172.21.15.100:0/2793783015 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f38f00ce620 0x7f38f015f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:32.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.903+0000 7f38ef400640 1 --2- 172.21.15.100:0/2793783015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38f00d0170 0x7f38f00ce0e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f38f0075570 tx=0x7f38dc01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354604e33abe8e35 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:32.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.903+0000 7f38eea00640 1 -- 172.21.15.100:0/2793783015 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc0b0070 con 0x7f38f00d0170 2024-04-23T14:32:32.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.903+0000 7f38eea00640 1 -- 172.21.15.100:0/2793783015 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38dc0ab600 con 0x7f38f00d0170 2024-04-23T14:32:32.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.903+0000 7f38eea00640 1 -- 172.21.15.100:0/2793783015 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc0ab9d0 con 0x7f38f00d0170 2024-04-23T14:32:32.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.903+0000 7f38f7000640 1 -- 172.21.15.100:0/2793783015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38f00d0170 msgr2=0x7f38f00ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.903+0000 7f38f7000640 1 --2- 172.21.15.100:0/2793783015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38f00d0170 0x7f38f00ce0e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f38f0075570 tx=0x7f38dc01c260 comp rx=0 tx=0).stop 2024-04-23T14:32:32.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.903+0000 7f38f7000640 1 -- 172.21.15.100:0/2793783015 shutdown_connections 2024-04-23T14:32:32.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.903+0000 7f38f7000640 1 --2- 172.21.15.100:0/2793783015 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f38f00ce620 0x7f38f015f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.903+0000 7f38f7000640 1 --2- 172.21.15.100:0/2793783015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38f00d0170 0x7f38f00ce0e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.903+0000 7f38f7000640 1 --2- 172.21.15.100:0/2793783015 >> v2:172.21.15.151:3300/0 conn(0x7f38f006f8c0 0x7f38f006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.903+0000 7f38f7000640 1 -- 172.21.15.100:0/2793783015 >> 172.21.15.100:0/2793783015 conn(0x7f38f00b5200 msgr2=0x7f38f00b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:32.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.903+0000 7f38f7000640 1 -- 172.21.15.100:0/2793783015 shutdown_connections 2024-04-23T14:32:32.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.903+0000 7f38f7000640 1 -- 172.21.15.100:0/2793783015 wait complete. 2024-04-23T14:32:32.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38f7000640 1 Processor -- start 2024-04-23T14:32:32.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38f7000640 1 -- start start 2024-04-23T14:32:32.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38f7000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38f006f8c0 0x7f38f015a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38f7000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f38f00ce620 0x7f38f00725c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38f7000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38f00d0170 0x7f38f00afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38f7000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f38f00adee0 con 0x7f38f00d0170 2024-04-23T14:32:32.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38f7000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f38f00add60 con 0x7f38f006f8c0 2024-04-23T14:32:32.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38f7000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f38f00ae060 con 0x7f38f00ce620 2024-04-23T14:32:32.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38f5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38f00d0170 0x7f38f00afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38efe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38f006f8c0 0x7f38f015a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38f5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38f00d0170 0x7f38f00afce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57216/0 (socket says 172.21.15.100:57216) 2024-04-23T14:32:32.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38f5000640 1 -- 172.21.15.100:0/1842554974 learned_addr learned my addr 172.21.15.100:0/1842554974 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:32.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38ef400640 1 --2- 172.21.15.100:0/1842554974 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f38f00ce620 0x7f38f00725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38f5000640 1 -- 172.21.15.100:0/1842554974 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f38f00ce620 msgr2=0x7f38f00725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38f5000640 1 --2- 172.21.15.100:0/1842554974 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f38f00ce620 0x7f38f00725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38f5000640 1 -- 172.21.15.100:0/1842554974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38f006f8c0 msgr2=0x7f38f015a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38f5000640 1 --2- 172.21.15.100:0/1842554974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38f006f8c0 0x7f38f015a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38f5000640 1 -- 172.21.15.100:0/1842554974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f015af10 con 0x7f38f00d0170 2024-04-23T14:32:32.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38f5000640 1 --2- 172.21.15.100:0/1842554974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38f00d0170 0x7f38f00afce0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f38e800c7f0 tx=0x7f38e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:32.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38e7e00640 1 -- 172.21.15.100:0/1842554974 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e8015070 con 0x7f38f00d0170 2024-04-23T14:32:32.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38e7e00640 1 -- 172.21.15.100:0/1842554974 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38e8007860 con 0x7f38f00d0170 2024-04-23T14:32:32.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38e7e00640 1 -- 172.21.15.100:0/1842554974 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e8007c50 con 0x7f38f00d0170 2024-04-23T14:32:32.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38f7000640 1 -- 172.21.15.100:0/1842554974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f015b370 con 0x7f38f00d0170 2024-04-23T14:32:32.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.907+0000 7f38f7000640 1 -- 172.21.15.100:0/1842554974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f015b930 con 0x7f38f00d0170 2024-04-23T14:32:32.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.911+0000 7f38f7000640 1 -- 172.21.15.100:0/1842554974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38f0068ad0 con 0x7f38f00d0170 2024-04-23T14:32:32.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.911+0000 7f38e7e00640 1 -- 172.21.15.100:0/1842554974 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f38e8018070 con 0x7f38f00d0170 2024-04-23T14:32:32.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.911+0000 7f38e7e00640 1 --2- 172.21.15.100:0/1842554974 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f38d0068380 0x7f38d006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.911+0000 7f38e7e00640 1 -- 172.21.15.100:0/1842554974 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f38e8086d50 con 0x7f38f00d0170 2024-04-23T14:32:32.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.911+0000 7f38efe00640 1 --2- 172.21.15.100:0/1842554974 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f38d0068380 0x7f38d006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.911+0000 7f38efe00640 1 --2- 172.21.15.100:0/1842554974 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f38d0068380 0x7f38d006a840 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f38e000d620 tx=0x7f38e0007810 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:32.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:32.915+0000 7f38e7e00640 1 -- 172.21.15.100:0/1842554974 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f38e8056970 con 0x7f38f00d0170 2024-04-23T14:32:33.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.103+0000 7f38f7000640 1 -- 172.21.15.100:0/1842554974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f38f006cc00 con 0x7f38f00d0170 2024-04-23T14:32:33.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.683+0000 7f38e7e00640 1 -- 172.21.15.100:0/1842554974 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f38e805ac70 con 0x7f38f00d0170 2024-04-23T14:32:33.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.687+0000 7f38f7000640 1 -- 172.21.15.100:0/1842554974 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f38d0068380 msgr2=0x7f38d006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:33.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.687+0000 7f38f7000640 1 --2- 172.21.15.100:0/1842554974 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f38d0068380 0x7f38d006a840 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f38e000d620 tx=0x7f38e0007810 comp rx=0 tx=0).stop 2024-04-23T14:32:33.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.687+0000 7f38f7000640 1 -- 172.21.15.100:0/1842554974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38f00d0170 msgr2=0x7f38f00afce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:33.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.687+0000 7f38f7000640 1 --2- 172.21.15.100:0/1842554974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38f00d0170 0x7f38f00afce0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f38e800c7f0 tx=0x7f38e800ccb0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.687+0000 7f38f7000640 1 -- 172.21.15.100:0/1842554974 shutdown_connections 2024-04-23T14:32:33.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.687+0000 7f38f7000640 1 --2- 172.21.15.100:0/1842554974 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f38d0068380 0x7f38d006a840 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.687+0000 7f38f7000640 1 --2- 172.21.15.100:0/1842554974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38f00d0170 0x7f38f00afce0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.687+0000 7f38f7000640 1 --2- 172.21.15.100:0/1842554974 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f38f00ce620 0x7f38f00725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.687+0000 7f38f7000640 1 --2- 172.21.15.100:0/1842554974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38f006f8c0 0x7f38f015a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.687+0000 7f38f7000640 1 -- 172.21.15.100:0/1842554974 >> 172.21.15.100:0/1842554974 conn(0x7f38f00b5200 msgr2=0x7f38f015e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:33.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.687+0000 7f38f7000640 1 -- 172.21.15.100:0/1842554974 shutdown_connections 2024-04-23T14:32:33.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.687+0000 7f38f7000640 1 -- 172.21.15.100:0/1842554974 wait complete. 2024-04-23T14:32:33.710 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-23T14:32:33.710 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-23T14:32:33.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.847+0000 7fe61dc00640 1 Processor -- start 2024-04-23T14:32:33.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.847+0000 7fe61dc00640 1 -- start start 2024-04-23T14:32:33.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.847+0000 7fe61dc00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe61815c7a0 0x7fe6180af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:33.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.847+0000 7fe61dc00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe61806bb20 0x7fe6180ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:33.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.847+0000 7fe61dc00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe61806cbb0 con 0x7fe6180cd130 2024-04-23T14:32:33.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.847+0000 7fe61dc00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fe6180cd5a0 con 0x7fe61806bb20 2024-04-23T14:32:33.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.847+0000 7fe61dc00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe61806d450 con 0x7fe61815c7a0 2024-04-23T14:32:33.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.847+0000 7fe60e000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe61806bb20 0x7fe6180ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:33.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.847+0000 7fe60e000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe61806bb20 0x7fe6180ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57232/0 (socket says 172.21.15.100:57232) 2024-04-23T14:32:33.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.847+0000 7fe60e000640 1 -- 172.21.15.100:0/781255020 learned_addr learned my addr 172.21.15.100:0/781255020 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:33.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.847+0000 7fe616a00640 1 --2- 172.21.15.100:0/781255020 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe61815c7a0 0x7fe6180af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:33.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.847+0000 7fe60e000640 1 -- 172.21.15.100:0/781255020 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe61815c7a0 msgr2=0x7fe6180af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:33.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.847+0000 7fe60e000640 1 --2- 172.21.15.100:0/781255020 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe61815c7a0 0x7fe6180af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.847+0000 7fe60e000640 1 -- 172.21.15.100:0/781255020 >> v1:172.21.15.151:6789/0 conn(0x7fe6180cd130 legacy=0x7fe61806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:33.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.847+0000 7fe60e000640 1 -- 172.21.15.100:0/781255020 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61806ce90 con 0x7fe61806bb20 2024-04-23T14:32:33.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.847+0000 7fe60e000640 1 --2- 172.21.15.100:0/781255020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe61806bb20 0x7fe6180ccbf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe600009870 tx=0x7fe60001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd093f7b00423e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:33.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.847+0000 7fe616000640 1 -- 172.21.15.100:0/781255020 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6000b0070 con 0x7fe61806bb20 2024-04-23T14:32:33.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.847+0000 7fe616000640 1 -- 172.21.15.100:0/781255020 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6000ab600 con 0x7fe61806bb20 2024-04-23T14:32:33.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.847+0000 7fe616000640 1 -- 172.21.15.100:0/781255020 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6000ab9d0 con 0x7fe61806bb20 2024-04-23T14:32:33.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe61dc00640 1 -- 172.21.15.100:0/781255020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe61806bb20 msgr2=0x7fe6180ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:33.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe61dc00640 1 --2- 172.21.15.100:0/781255020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe61806bb20 0x7fe6180ccbf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe600009870 tx=0x7fe60001c4d0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe61dc00640 1 -- 172.21.15.100:0/781255020 shutdown_connections 2024-04-23T14:32:33.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe61dc00640 1 --2- 172.21.15.100:0/781255020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe61806bb20 0x7fe6180ccbf0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe61dc00640 1 --2- 172.21.15.100:0/781255020 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe61815c7a0 0x7fe6180af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe61dc00640 1 -- 172.21.15.100:0/781255020 >> 172.21.15.100:0/781255020 conn(0x7fe6180b5200 msgr2=0x7fe6180b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:33.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe61dc00640 1 -- 172.21.15.100:0/781255020 shutdown_connections 2024-04-23T14:32:33.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe61dc00640 1 -- 172.21.15.100:0/781255020 wait complete. 2024-04-23T14:32:33.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe61dc00640 1 Processor -- start 2024-04-23T14:32:33.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe61dc00640 1 -- start start 2024-04-23T14:32:33.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe61dc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe61806bb20 0x7fe618159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:33.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe61dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6180cd130 0x7fe61815a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:33.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe61dc00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe61815c7a0 0x7fe6181564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:33.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe61dc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe618169590 con 0x7fe6180cd130 2024-04-23T14:32:33.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe61dc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe618169410 con 0x7fe61806bb20 2024-04-23T14:32:33.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe61dc00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe618169710 con 0x7fe61815c7a0 2024-04-23T14:32:33.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe60e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6180cd130 0x7fe61815a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:33.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe60e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6180cd130 0x7fe61815a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57236/0 (socket says 172.21.15.100:57236) 2024-04-23T14:32:33.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe60e000640 1 -- 172.21.15.100:0/1223969462 learned_addr learned my addr 172.21.15.100:0/1223969462 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:33.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe616a00640 1 --2- 172.21.15.100:0/1223969462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe61806bb20 0x7fe618159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:33.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe617400640 1 --2- 172.21.15.100:0/1223969462 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe61815c7a0 0x7fe6181564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:33.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe60e000640 1 -- 172.21.15.100:0/1223969462 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe61815c7a0 msgr2=0x7fe6181564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:33.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe60e000640 1 --2- 172.21.15.100:0/1223969462 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe61815c7a0 0x7fe6181564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe60e000640 1 -- 172.21.15.100:0/1223969462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe61806bb20 msgr2=0x7fe618159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:33.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe60e000640 1 --2- 172.21.15.100:0/1223969462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe61806bb20 0x7fe618159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.851+0000 7fe60e000640 1 -- 172.21.15.100:0/1223969462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6180af500 con 0x7fe6180cd130 2024-04-23T14:32:33.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.855+0000 7fe60e000640 1 --2- 172.21.15.100:0/1223969462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6180cd130 0x7fe61815a100 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe6000ae040 tx=0x7fe600019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:33.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.855+0000 7fe60f400640 1 -- 172.21.15.100:0/1223969462 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6000b0040 con 0x7fe6180cd130 2024-04-23T14:32:33.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.855+0000 7fe60f400640 1 -- 172.21.15.100:0/1223969462 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6000b4420 con 0x7fe6180cd130 2024-04-23T14:32:33.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.855+0000 7fe61dc00640 1 -- 172.21.15.100:0/1223969462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6180cf0b0 con 0x7fe6180cd130 2024-04-23T14:32:33.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.855+0000 7fe60f400640 1 -- 172.21.15.100:0/1223969462 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6000b4810 con 0x7fe6180cd130 2024-04-23T14:32:33.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.855+0000 7fe61dc00640 1 -- 172.21.15.100:0/1223969462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6180cf670 con 0x7fe6180cd130 2024-04-23T14:32:33.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.855+0000 7fe61dc00640 1 -- 172.21.15.100:0/1223969462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe618068ad0 con 0x7fe6180cd130 2024-04-23T14:32:33.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.855+0000 7fe60f400640 1 -- 172.21.15.100:0/1223969462 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe6000b49b0 con 0x7fe6180cd130 2024-04-23T14:32:33.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.855+0000 7fe60f400640 1 --2- 172.21.15.100:0/1223969462 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe5f4068380 0x7fe5f406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:33.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.855+0000 7fe60f400640 1 -- 172.21.15.100:0/1223969462 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe600120970 con 0x7fe6180cd130 2024-04-23T14:32:33.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.855+0000 7fe616a00640 1 --2- 172.21.15.100:0/1223969462 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe5f4068380 0x7fe5f406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:33.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.859+0000 7fe616a00640 1 --2- 172.21.15.100:0/1223969462 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe5f4068380 0x7fe5f406a840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe608003fd0 tx=0x7fe6080039a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:33.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:33.859+0000 7fe60f400640 1 -- 172.21.15.100:0/1223969462 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe600016150 con 0x7fe6180cd130 2024-04-23T14:32:34.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.055+0000 7fe61dc00640 1 -- 172.21.15.100:0/1223969462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe618074f20 con 0x7fe6180cd130 2024-04-23T14:32:34.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.691+0000 7fe60f400640 1 -- 172.21.15.100:0/1223969462 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7fe6000f0be0 con 0x7fe6180cd130 2024-04-23T14:32:34.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.695+0000 7fe61dc00640 1 -- 172.21.15.100:0/1223969462 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe5f4068380 msgr2=0x7fe5f406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:34.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.695+0000 7fe61dc00640 1 --2- 172.21.15.100:0/1223969462 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe5f4068380 0x7fe5f406a840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe608003fd0 tx=0x7fe6080039a0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.695+0000 7fe61dc00640 1 -- 172.21.15.100:0/1223969462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6180cd130 msgr2=0x7fe61815a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:34.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.695+0000 7fe61dc00640 1 --2- 172.21.15.100:0/1223969462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6180cd130 0x7fe61815a100 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe6000ae040 tx=0x7fe600019270 comp rx=0 tx=0).stop 2024-04-23T14:32:34.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.695+0000 7fe61dc00640 1 -- 172.21.15.100:0/1223969462 shutdown_connections 2024-04-23T14:32:34.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.695+0000 7fe61dc00640 1 --2- 172.21.15.100:0/1223969462 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe5f4068380 0x7fe5f406a840 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.695+0000 7fe61dc00640 1 --2- 172.21.15.100:0/1223969462 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe61815c7a0 0x7fe6181564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.695+0000 7fe61dc00640 1 --2- 172.21.15.100:0/1223969462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6180cd130 0x7fe61815a100 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.695+0000 7fe61dc00640 1 --2- 172.21.15.100:0/1223969462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe61806bb20 0x7fe618159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.695+0000 7fe61dc00640 1 -- 172.21.15.100:0/1223969462 >> 172.21.15.100:0/1223969462 conn(0x7fe6180b5200 msgr2=0x7fe6180cade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:34.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.695+0000 7fe61dc00640 1 -- 172.21.15.100:0/1223969462 shutdown_connections 2024-04-23T14:32:34.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.695+0000 7fe61dc00640 1 -- 172.21.15.100:0/1223969462 wait complete. 2024-04-23T14:32:34.717 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-23T14:32:34.717 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:32:34.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.851+0000 7fa150600640 1 Processor -- start 2024-04-23T14:32:34.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.855+0000 7fa150600640 1 -- start start 2024-04-23T14:32:34.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.855+0000 7fa150600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa14815c7e0 0x7fa1480d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:34.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.855+0000 7fa150600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1480d04a0 con 0x7fa14815d600 2024-04-23T14:32:34.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.855+0000 7fa150600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1480d0780 con 0x7fa1480cca50 2024-04-23T14:32:34.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.855+0000 7fa150600640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fa14815db00 con 0x7fa14815c7e0 2024-04-23T14:32:34.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.855+0000 7fa14d200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa14815c7e0 0x7fa1480d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:34.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.855+0000 7fa14e600640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fa14815d600 0x7fa1480d17a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57180/0 (socket says 172.21.15.100:57180) 2024-04-23T14:32:34.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.855+0000 7fa14e600640 1 -- 172.21.15.100:0/1809040449 learned_addr learned my addr 172.21.15.100:0/1809040449 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:34.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.855+0000 7fa14d200640 1 -- 172.21.15.100:0/1809040449 >> v1:172.21.15.151:6789/0 conn(0x7fa1480cca50 legacy=0x7fa1480cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:34.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.855+0000 7fa14d200640 1 -- 172.21.15.100:0/1809040449 >> v1:172.21.15.100:6789/0 conn(0x7fa14815d600 legacy=0x7fa1480d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:34.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.855+0000 7fa14d200640 1 -- 172.21.15.100:0/1809040449 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1480d0a60 con 0x7fa14815c7e0 2024-04-23T14:32:34.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.855+0000 7fa147e00640 1 -- 172.21.15.100:0/1809040449 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 896083059 0 0) 0x7fa1480d04a0 con 0x7fa14815d600 2024-04-23T14:32:34.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.859+0000 7fa14d200640 1 --2- 172.21.15.100:0/1809040449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa14815c7e0 0x7fa1480d1260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa1340097c0 tx=0x7fa13401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9846f9c8565a427 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:34.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.859+0000 7fa147e00640 1 -- 172.21.15.100:0/1809040449 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1340ab600 con 0x7fa14815c7e0 2024-04-23T14:32:34.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.859+0000 7fa147e00640 1 -- 172.21.15.100:0/1809040449 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1340b0070 con 0x7fa14815c7e0 2024-04-23T14:32:34.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.859+0000 7fa147e00640 1 -- 172.21.15.100:0/1809040449 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1340ab9c0 con 0x7fa14815c7e0 2024-04-23T14:32:34.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.859+0000 7fa150600640 1 -- 172.21.15.100:0/1809040449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa14815c7e0 msgr2=0x7fa1480d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:34.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.859+0000 7fa150600640 1 --2- 172.21.15.100:0/1809040449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa14815c7e0 0x7fa1480d1260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa1340097c0 tx=0x7fa13401c3e0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.859+0000 7fa150600640 1 -- 172.21.15.100:0/1809040449 shutdown_connections 2024-04-23T14:32:34.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.859+0000 7fa150600640 1 --2- 172.21.15.100:0/1809040449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa14815c7e0 0x7fa1480d1260 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.859+0000 7fa150600640 1 -- 172.21.15.100:0/1809040449 >> 172.21.15.100:0/1809040449 conn(0x7fa1480cee80 msgr2=0x7fa1480cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:34.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.859+0000 7fa150600640 1 -- 172.21.15.100:0/1809040449 shutdown_connections 2024-04-23T14:32:34.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.859+0000 7fa150600640 1 -- 172.21.15.100:0/1809040449 wait complete. 2024-04-23T14:32:34.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.859+0000 7fa150600640 1 Processor -- start 2024-04-23T14:32:34.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.859+0000 7fa150600640 1 -- start start 2024-04-23T14:32:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa150600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1480cca50 0x7fa14814b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa150600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa14815c7e0 0x7fa148149230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa150600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa14815d600 0x7fa148149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa150600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa14816ac60 con 0x7fa1480cca50 2024-04-23T14:32:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa150600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa14816aae0 con 0x7fa14815c7e0 2024-04-23T14:32:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa150600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa14816ade0 con 0x7fa14815d600 2024-04-23T14:32:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa14dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1480cca50 0x7fa14814b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa14dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1480cca50 0x7fa14814b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57242/0 (socket says 172.21.15.100:57242) 2024-04-23T14:32:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa14dc00640 1 -- 172.21.15.100:0/2139780075 learned_addr learned my addr 172.21.15.100:0/2139780075 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa14d200640 1 --2- 172.21.15.100:0/2139780075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa14815c7e0 0x7fa148149230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa14e600640 1 --2- 172.21.15.100:0/2139780075 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa14815d600 0x7fa148149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa14dc00640 1 -- 172.21.15.100:0/2139780075 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa14815d600 msgr2=0x7fa148149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa14dc00640 1 --2- 172.21.15.100:0/2139780075 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa14815d600 0x7fa148149770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa14dc00640 1 -- 172.21.15.100:0/2139780075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa14815c7e0 msgr2=0x7fa148149230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa14dc00640 1 --2- 172.21.15.100:0/2139780075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa14815c7e0 0x7fa148149230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa14dc00640 1 -- 172.21.15.100:0/2139780075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14814b560 con 0x7fa1480cca50 2024-04-23T14:32:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa14dc00640 1 --2- 172.21.15.100:0/2139780075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1480cca50 0x7fa14814b020 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00c530 tx=0x7fa13c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa145600640 1 -- 172.21.15.100:0/2139780075 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c016020 con 0x7fa1480cca50 2024-04-23T14:32:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa145600640 1 -- 172.21.15.100:0/2139780075 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa13c005220 con 0x7fa1480cca50 2024-04-23T14:32:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa150600640 1 -- 172.21.15.100:0/2139780075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa148159f10 con 0x7fa1480cca50 2024-04-23T14:32:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa145600640 1 -- 172.21.15.100:0/2139780075 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c007030 con 0x7fa1480cca50 2024-04-23T14:32:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa150600640 1 -- 172.21.15.100:0/2139780075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa14814ab00 con 0x7fa1480cca50 2024-04-23T14:32:34.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa150600640 1 -- 172.21.15.100:0/2139780075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa148068ad0 con 0x7fa1480cca50 2024-04-23T14:32:34.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa145600640 1 -- 172.21.15.100:0/2139780075 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa13c0071d0 con 0x7fa1480cca50 2024-04-23T14:32:34.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.863+0000 7fa145600640 1 --2- 172.21.15.100:0/2139780075 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa128068380 0x7fa12806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:34.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.867+0000 7fa145600640 1 -- 172.21.15.100:0/2139780075 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa13c086680 con 0x7fa1480cca50 2024-04-23T14:32:34.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.867+0000 7fa14d200640 1 --2- 172.21.15.100:0/2139780075 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa128068380 0x7fa12806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:34.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.867+0000 7fa14d200640 1 --2- 172.21.15.100:0/2139780075 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa128068380 0x7fa12806a840 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa1340af000 tx=0x7fa1340a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:34.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:34.867+0000 7fa145600640 1 -- 172.21.15.100:0/2139780075 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa13c020020 con 0x7fa1480cca50 2024-04-23T14:32:35.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.059+0000 7fa150600640 1 -- 172.21.15.100:0/2139780075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa148167d80 con 0x7fa1480cca50 2024-04-23T14:32:35.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.059+0000 7fa145600640 1 -- 172.21.15.100:0/2139780075 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7fa13c0074c0 con 0x7fa1480cca50 2024-04-23T14:32:35.060 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:35.060 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1456958549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1456958549},{"type":"v1","addr":"172.21.15.100:6836","nonce":1456958549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3655609551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3655609551},{"type":"v1","addr":"172.21.15.100:6837","nonce":3655609551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/4242180972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":4242180972},{"type":"v1","addr":"172.21.15.151:6834","nonce":4242180972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:32:08.976576+0000","modified":"2024-04-23T14:32:34.057740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1533322173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1533322173},{"type":"v1","addr":"172.21.15.151:6835","nonce":1533322173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-23T14:32:35.060 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 9 2024-04-23T14:32:35.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.059+0000 7fa150600640 1 -- 172.21.15.100:0/2139780075 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa128068380 msgr2=0x7fa12806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.059+0000 7fa150600640 1 --2- 172.21.15.100:0/2139780075 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa128068380 0x7fa12806a840 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa1340af000 tx=0x7fa1340a7000 comp rx=0 tx=0).stop 2024-04-23T14:32:35.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.059+0000 7fa150600640 1 -- 172.21.15.100:0/2139780075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1480cca50 msgr2=0x7fa14814b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.059+0000 7fa150600640 1 --2- 172.21.15.100:0/2139780075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1480cca50 0x7fa14814b020 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00c530 tx=0x7fa13c00c9f0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.059+0000 7fa150600640 1 -- 172.21.15.100:0/2139780075 shutdown_connections 2024-04-23T14:32:35.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.059+0000 7fa150600640 1 --2- 172.21.15.100:0/2139780075 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa128068380 0x7fa12806a840 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.059+0000 7fa150600640 1 --2- 172.21.15.100:0/2139780075 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa14815d600 0x7fa148149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.059+0000 7fa150600640 1 --2- 172.21.15.100:0/2139780075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa14815c7e0 0x7fa148149230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.059+0000 7fa150600640 1 --2- 172.21.15.100:0/2139780075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1480cca50 0x7fa14814b020 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.059+0000 7fa150600640 1 -- 172.21.15.100:0/2139780075 >> 172.21.15.100:0/2139780075 conn(0x7fa1480cee80 msgr2=0x7fa1480cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:35.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.059+0000 7fa150600640 1 -- 172.21.15.100:0/2139780075 shutdown_connections 2024-04-23T14:32:35.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.059+0000 7fa150600640 1 -- 172.21.15.100:0/2139780075 wait complete. 2024-04-23T14:32:35.081 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:35.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.215+0000 7f5f2ee00640 1 Processor -- start 2024-04-23T14:32:35.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.215+0000 7f5f2ee00640 1 -- start start 2024-04-23T14:32:35.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.215+0000 7f5f2ee00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5f280cca50 0x7f5f280cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.215+0000 7f5f2ee00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f2815c7e0 0x7f5f28160170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.215+0000 7f5f2ee00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f280cd370 con 0x7f5f2815d600 2024-04-23T14:32:35.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.215+0000 7f5f2ee00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f5f280d2270 con 0x7f5f280cca50 2024-04-23T14:32:35.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.215+0000 7f5f2ee00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5f2815db00 con 0x7f5f2815c7e0 2024-04-23T14:32:35.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.215+0000 7f5f27e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5f280cca50 0x7f5f280cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.215+0000 7f5f27e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5f280cca50 0x7f5f280cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57256/0 (socket says 172.21.15.100:57256) 2024-04-23T14:32:35.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.215+0000 7f5f27e00640 1 -- 172.21.15.100:0/222619988 learned_addr learned my addr 172.21.15.100:0/222619988 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:35.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.215+0000 7f5f1f400640 1 --2- 172.21.15.100:0/222619988 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f2815c7e0 0x7f5f28160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.215+0000 7f5f27e00640 1 -- 172.21.15.100:0/222619988 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f2815c7e0 msgr2=0x7f5f28160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:35.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.215+0000 7f5f27e00640 1 --2- 172.21.15.100:0/222619988 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f2815c7e0 0x7f5f28160170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.215+0000 7f5f27e00640 1 -- 172.21.15.100:0/222619988 >> v1:172.21.15.151:6789/0 conn(0x7f5f2815d600 legacy=0x7f5f281606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:35.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.215+0000 7f5f27e00640 1 -- 172.21.15.100:0/222619988 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f280d04a0 con 0x7f5f280cca50 2024-04-23T14:32:35.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.219+0000 7f5f27e00640 1 --2- 172.21.15.100:0/222619988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f280cca50 0x7f5f280cce30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009870 tx=0x7f5f1801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad289db5d6985e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:35.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.219+0000 7f5f27400640 1 -- 172.21.15.100:0/222619988 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f180ab620 con 0x7f5f280cca50 2024-04-23T14:32:35.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.219+0000 7f5f27400640 1 -- 172.21.15.100:0/222619988 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f180b0070 con 0x7f5f280cca50 2024-04-23T14:32:35.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.219+0000 7f5f27400640 1 -- 172.21.15.100:0/222619988 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f180ab9e0 con 0x7f5f280cca50 2024-04-23T14:32:35.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.219+0000 7f5f2ee00640 1 -- 172.21.15.100:0/222619988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f280cca50 msgr2=0x7f5f280cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.219+0000 7f5f2ee00640 1 --2- 172.21.15.100:0/222619988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f280cca50 0x7f5f280cce30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009870 tx=0x7f5f1801c460 comp rx=0 tx=0).stop 2024-04-23T14:32:35.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.219+0000 7f5f2ee00640 1 -- 172.21.15.100:0/222619988 shutdown_connections 2024-04-23T14:32:35.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.219+0000 7f5f2ee00640 1 --2- 172.21.15.100:0/222619988 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f2815c7e0 0x7f5f28160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.219+0000 7f5f2ee00640 1 --2- 172.21.15.100:0/222619988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f280cca50 0x7f5f280cce30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.219+0000 7f5f2ee00640 1 -- 172.21.15.100:0/222619988 >> 172.21.15.100:0/222619988 conn(0x7f5f280cee80 msgr2=0x7f5f280cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:35.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.219+0000 7f5f2ee00640 1 -- 172.21.15.100:0/222619988 shutdown_connections 2024-04-23T14:32:35.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.219+0000 7f5f2ee00640 1 -- 172.21.15.100:0/222619988 wait complete. 2024-04-23T14:32:35.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f2ee00640 1 Processor -- start 2024-04-23T14:32:35.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f2ee00640 1 -- start start 2024-04-23T14:32:35.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f2ee00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f280cca50 0x7f5f280c5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f2ee00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f2815c7e0 0x7f5f2814c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f2ee00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2815d600 0x7f5f281f9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f2ee00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5f280b5430 con 0x7f5f2815d600 2024-04-23T14:32:35.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f2ee00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5f280b52b0 con 0x7f5f2815c7e0 2024-04-23T14:32:35.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f2ee00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5f280b3bf0 con 0x7f5f280cca50 2024-04-23T14:32:35.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f2ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2815d600 0x7f5f281f9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f2ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2815d600 0x7f5f281f9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57258/0 (socket says 172.21.15.100:57258) 2024-04-23T14:32:35.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f2ce00640 1 -- 172.21.15.100:0/2012031638 learned_addr learned my addr 172.21.15.100:0/2012031638 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:35.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f27e00640 1 --2- 172.21.15.100:0/2012031638 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f280cca50 0x7f5f280c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f1f400640 1 --2- 172.21.15.100:0/2012031638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f2815c7e0 0x7f5f2814c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f2ce00640 1 -- 172.21.15.100:0/2012031638 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f280cca50 msgr2=0x7f5f280c5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f2ce00640 1 --2- 172.21.15.100:0/2012031638 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f280cca50 0x7f5f280c5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f2ce00640 1 -- 172.21.15.100:0/2012031638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f2815c7e0 msgr2=0x7f5f2814c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f2ce00640 1 --2- 172.21.15.100:0/2012031638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f2815c7e0 0x7f5f2814c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f2ce00640 1 -- 172.21.15.100:0/2012031638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f281f9c70 con 0x7f5f2815d600 2024-04-23T14:32:35.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f2ce00640 1 --2- 172.21.15.100:0/2012031638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2815d600 0x7f5f281f9220 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000c780 tx=0x7f5f2000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:35.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f24c00640 1 -- 172.21.15.100:0/2012031638 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20016020 con 0x7f5f2815d600 2024-04-23T14:32:35.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f24c00640 1 -- 172.21.15.100:0/2012031638 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2000a9f0 con 0x7f5f2815d600 2024-04-23T14:32:35.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f24c00640 1 -- 172.21.15.100:0/2012031638 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2000ae20 con 0x7f5f2815d600 2024-04-23T14:32:35.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f2ee00640 1 -- 172.21.15.100:0/2012031638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f281f9f50 con 0x7f5f2815d600 2024-04-23T14:32:35.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.223+0000 7f5f2ee00640 1 -- 172.21.15.100:0/2012031638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f280c4c70 con 0x7f5f2815d600 2024-04-23T14:32:35.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.227+0000 7f5f2ee00640 1 -- 172.21.15.100:0/2012031638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f280c4e70 con 0x7f5f2815d600 2024-04-23T14:32:35.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.227+0000 7f5f24c00640 1 -- 172.21.15.100:0/2012031638 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5f20019020 con 0x7f5f2815d600 2024-04-23T14:32:35.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.227+0000 7f5f24c00640 1 --2- 172.21.15.100:0/2012031638 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5f040683b0 0x7f5f0406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.227+0000 7f5f24c00640 1 -- 172.21.15.100:0/2012031638 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5f20086b70 con 0x7f5f2815d600 2024-04-23T14:32:35.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.227+0000 7f5f27e00640 1 --2- 172.21.15.100:0/2012031638 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5f040683b0 0x7f5f0406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.227+0000 7f5f27e00640 1 --2- 172.21.15.100:0/2012031638 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5f040683b0 0x7f5f0406a870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5f180af000 tx=0x7f5f180a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:35.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.231+0000 7f5f24c00640 1 -- 172.21.15.100:0/2012031638 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5f280c4e70 con 0x7f5f2815d600 2024-04-23T14:32:35.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.387+0000 7f5f2ee00640 1 -- 172.21.15.100:0/2012031638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f28068ad0 con 0x7f5f2815d600 2024-04-23T14:32:35.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.391+0000 7f5f24c00640 1 -- 172.21.15.100:0/2012031638 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f5f20020030 con 0x7f5f2815d600 2024-04-23T14:32:35.390 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:35.390 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":24,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:14.037027+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:35.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.391+0000 7f5f2ee00640 1 -- 172.21.15.100:0/2012031638 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5f040683b0 msgr2=0x7f5f0406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.391+0000 7f5f2ee00640 1 --2- 172.21.15.100:0/2012031638 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5f040683b0 0x7f5f0406a870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5f180af000 tx=0x7f5f180a7000 comp rx=0 tx=0).stop 2024-04-23T14:32:35.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.391+0000 7f5f2ee00640 1 -- 172.21.15.100:0/2012031638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2815d600 msgr2=0x7f5f281f9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.391+0000 7f5f2ee00640 1 --2- 172.21.15.100:0/2012031638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2815d600 0x7f5f281f9220 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000c780 tx=0x7f5f2000cc40 comp rx=0 tx=0).stop 2024-04-23T14:32:35.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.391+0000 7f5f2ee00640 1 -- 172.21.15.100:0/2012031638 shutdown_connections 2024-04-23T14:32:35.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.391+0000 7f5f2ee00640 1 --2- 172.21.15.100:0/2012031638 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5f040683b0 0x7f5f0406a870 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.391+0000 7f5f2ee00640 1 --2- 172.21.15.100:0/2012031638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2815d600 0x7f5f281f9220 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.391+0000 7f5f2ee00640 1 --2- 172.21.15.100:0/2012031638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f2815c7e0 0x7f5f2814c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.391+0000 7f5f2ee00640 1 --2- 172.21.15.100:0/2012031638 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f280cca50 0x7f5f280c5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.391+0000 7f5f2ee00640 1 -- 172.21.15.100:0/2012031638 >> 172.21.15.100:0/2012031638 conn(0x7f5f280cee80 msgr2=0x7f5f2806f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:35.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.391+0000 7f5f2ee00640 1 -- 172.21.15.100:0/2012031638 shutdown_connections 2024-04-23T14:32:35.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.391+0000 7f5f2ee00640 1 -- 172.21.15.100:0/2012031638 wait complete. 2024-04-23T14:32:35.412 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-23T14:32:35.412 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:32:35.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.547+0000 7fb2e5a00640 1 Processor -- start 2024-04-23T14:32:35.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.547+0000 7fb2e5a00640 1 -- start start 2024-04-23T14:32:35.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.547+0000 7fb2e5a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb2e006d450 0x7fb2e006d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.547+0000 7fb2e5a00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2e006dd70 con 0x7fb2e00cea60 2024-04-23T14:32:35.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.547+0000 7fb2e5a00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2e00b1f90 con 0x7fb2e00cd460 2024-04-23T14:32:35.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.547+0000 7fb2e5a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb2e00b2270 con 0x7fb2e006d450 2024-04-23T14:32:35.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.547+0000 7fb2de000640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fb2e00cea60 0x7fb2e00d0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57188/0 (socket says 172.21.15.100:57188) 2024-04-23T14:32:35.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.547+0000 7fb2de000640 1 -- 172.21.15.100:0/1566724698 learned_addr learned my addr 172.21.15.100:0/1566724698 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:35.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.547+0000 7fb2dea00640 1 --2- 172.21.15.100:0/1566724698 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb2e006d450 0x7fb2e006d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.547+0000 7fb2dd600640 1 -- 172.21.15.100:0/1566724698 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1185215005 0 0) 0x7fb2e006dd70 con 0x7fb2e00cea60 2024-04-23T14:32:35.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.547+0000 7fb2dd600640 1 -- 172.21.15.100:0/1566724698 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2c4003530 con 0x7fb2e00cea60 2024-04-23T14:32:35.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.547+0000 7fb2dea00640 1 -- 172.21.15.100:0/1566724698 >> v1:172.21.15.151:6789/0 conn(0x7fb2e00cd460 legacy=0x7fb2e00b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.547+0000 7fb2dd600640 1 -- 172.21.15.100:0/1566724698 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 434739705 0 0) 0x7fb2e00b1f90 con 0x7fb2e00cd460 2024-04-23T14:32:35.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.547+0000 7fb2dea00640 1 -- 172.21.15.100:0/1566724698 >> v1:172.21.15.100:6789/0 conn(0x7fb2e00cea60 legacy=0x7fb2e00d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.547+0000 7fb2dea00640 1 -- 172.21.15.100:0/1566724698 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e00b2550 con 0x7fb2e006d450 2024-04-23T14:32:35.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.551+0000 7fb2dea00640 1 --2- 172.21.15.100:0/1566724698 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb2e006d450 0x7fb2e006d830 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00097f0 tx=0x7fb2d001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35864e67b2f29c44 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:35.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.551+0000 7fb2dd600640 1 -- 172.21.15.100:0/1566724698 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d00b0070 con 0x7fb2e006d450 2024-04-23T14:32:35.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.551+0000 7fb2dd600640 1 -- 172.21.15.100:0/1566724698 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2d00ab600 con 0x7fb2e006d450 2024-04-23T14:32:35.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.551+0000 7fb2dd600640 1 -- 172.21.15.100:0/1566724698 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d00ab9b0 con 0x7fb2e006d450 2024-04-23T14:32:35.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.551+0000 7fb2e5a00640 1 -- 172.21.15.100:0/1566724698 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb2e006d450 msgr2=0x7fb2e006d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.551+0000 7fb2e5a00640 1 --2- 172.21.15.100:0/1566724698 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb2e006d450 0x7fb2e006d830 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00097f0 tx=0x7fb2d001c450 comp rx=0 tx=0).stop 2024-04-23T14:32:35.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.551+0000 7fb2e5a00640 1 -- 172.21.15.100:0/1566724698 shutdown_connections 2024-04-23T14:32:35.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.551+0000 7fb2e5a00640 1 --2- 172.21.15.100:0/1566724698 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb2e006d450 0x7fb2e006d830 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.551+0000 7fb2e5a00640 1 -- 172.21.15.100:0/1566724698 >> 172.21.15.100:0/1566724698 conn(0x7fb2e00cee80 msgr2=0x7fb2e00cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:35.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.551+0000 7fb2e5a00640 1 -- 172.21.15.100:0/1566724698 shutdown_connections 2024-04-23T14:32:35.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.551+0000 7fb2e5a00640 1 -- 172.21.15.100:0/1566724698 wait complete. 2024-04-23T14:32:35.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2e5a00640 1 Processor -- start 2024-04-23T14:32:35.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2e5a00640 1 -- start start 2024-04-23T14:32:35.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2e5a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2e006d450 0x7fb2e00c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2e5a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb2e00cd460 0x7fb2e0151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2e5a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2e00cea60 0x7fb2e00d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2e5a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e016cd50 con 0x7fb2e00cea60 2024-04-23T14:32:35.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2e5a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e016cbd0 con 0x7fb2e006d450 2024-04-23T14:32:35.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2e5a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb2e016ced0 con 0x7fb2e00cd460 2024-04-23T14:32:35.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2df400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2e00cea60 0x7fb2e00d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2df400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2e00cea60 0x7fb2e00d0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57262/0 (socket says 172.21.15.100:57262) 2024-04-23T14:32:35.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2df400640 1 -- 172.21.15.100:0/4153166535 learned_addr learned my addr 172.21.15.100:0/4153166535 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:35.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2de000640 1 --2- 172.21.15.100:0/4153166535 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb2e00cd460 0x7fb2e0151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2dea00640 1 --2- 172.21.15.100:0/4153166535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2e006d450 0x7fb2e00c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2df400640 1 -- 172.21.15.100:0/4153166535 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb2e00cd460 msgr2=0x7fb2e0151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2df400640 1 --2- 172.21.15.100:0/4153166535 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb2e00cd460 0x7fb2e0151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2df400640 1 -- 172.21.15.100:0/4153166535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2e006d450 msgr2=0x7fb2e00c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2df400640 1 --2- 172.21.15.100:0/4153166535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2e006d450 0x7fb2e00c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2df400640 1 -- 172.21.15.100:0/4153166535 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e00cc200 con 0x7fb2e00cea60 2024-04-23T14:32:35.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2df400640 1 --2- 172.21.15.100:0/4153166535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2e00cea60 0x7fb2e00d0a80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800c780 tx=0x7fb2d800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:35.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2d6a00640 1 -- 172.21.15.100:0/4153166535 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8003db0 con 0x7fb2e00cea60 2024-04-23T14:32:35.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2d6a00640 1 -- 172.21.15.100:0/4153166535 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2d8005070 con 0x7fb2e00cea60 2024-04-23T14:32:35.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2d6a00640 1 -- 172.21.15.100:0/4153166535 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8005460 con 0x7fb2e00cea60 2024-04-23T14:32:35.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2e5a00640 1 -- 172.21.15.100:0/4153166535 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e00c6a90 con 0x7fb2e00cea60 2024-04-23T14:32:35.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.555+0000 7fb2e5a00640 1 -- 172.21.15.100:0/4153166535 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e00c7050 con 0x7fb2e00cea60 2024-04-23T14:32:35.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.559+0000 7fb2e5a00640 1 -- 172.21.15.100:0/4153166535 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e0068ad0 con 0x7fb2e00cea60 2024-04-23T14:32:35.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.559+0000 7fb2d6a00640 1 -- 172.21.15.100:0/4153166535 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb2d8005600 con 0x7fb2e00cea60 2024-04-23T14:32:35.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.559+0000 7fb2d6a00640 1 --2- 172.21.15.100:0/4153166535 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb2c0068380 0x7fb2c006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.559+0000 7fb2d6a00640 1 -- 172.21.15.100:0/4153166535 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb2d8086b50 con 0x7fb2e00cea60 2024-04-23T14:32:35.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.559+0000 7fb2dea00640 1 --2- 172.21.15.100:0/4153166535 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb2c0068380 0x7fb2c006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.559+0000 7fb2dea00640 1 --2- 172.21.15.100:0/4153166535 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb2c0068380 0x7fb2c006a840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00ad000 tx=0x7fb2d00a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:35.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.563+0000 7fb2d6a00640 1 -- 172.21.15.100:0/4153166535 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb2d8056770 con 0x7fb2e00cea60 2024-04-23T14:32:35.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.719+0000 7fb2e5a00640 1 -- 172.21.15.100:0/4153166535 --> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb2e00b1b60 con 0x7fb2c0068380 2024-04-23T14:32:35.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.727+0000 7fb2d6a00640 1 -- 172.21.15.100:0/4153166535 <== mgr.4101 v2:172.21.15.100:6832/2859853957 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332537 (secure 0 0 0) 0x7fb2e00b1b60 con 0x7fb2c0068380 2024-04-23T14:32:35.731 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:35.733 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-04-23T14:32:35.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.731+0000 7fb2e5a00640 1 -- 172.21.15.100:0/4153166535 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb2c0068380 msgr2=0x7fb2c006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.731+0000 7fb2e5a00640 1 --2- 172.21.15.100:0/4153166535 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb2c0068380 0x7fb2c006a840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00ad000 tx=0x7fb2d00a6000 comp rx=0 tx=0).stop 2024-04-23T14:32:35.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.731+0000 7fb2e5a00640 1 -- 172.21.15.100:0/4153166535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2e00cea60 msgr2=0x7fb2e00d0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.731+0000 7fb2e5a00640 1 --2- 172.21.15.100:0/4153166535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2e00cea60 0x7fb2e00d0a80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800c780 tx=0x7fb2d800cc40 comp rx=0 tx=0).stop 2024-04-23T14:32:35.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.731+0000 7fb2e5a00640 1 -- 172.21.15.100:0/4153166535 shutdown_connections 2024-04-23T14:32:35.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.731+0000 7fb2e5a00640 1 --2- 172.21.15.100:0/4153166535 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb2c0068380 0x7fb2c006a840 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.731+0000 7fb2e5a00640 1 --2- 172.21.15.100:0/4153166535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2e00cea60 0x7fb2e00d0a80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.731+0000 7fb2e5a00640 1 --2- 172.21.15.100:0/4153166535 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb2e00cd460 0x7fb2e0151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.731+0000 7fb2e5a00640 1 --2- 172.21.15.100:0/4153166535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2e006d450 0x7fb2e00c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.731+0000 7fb2e5a00640 1 -- 172.21.15.100:0/4153166535 >> 172.21.15.100:0/4153166535 conn(0x7fb2e00cee80 msgr2=0x7fb2e006f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:35.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.731+0000 7fb2e5a00640 1 -- 172.21.15.100:0/4153166535 shutdown_connections 2024-04-23T14:32:35.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.731+0000 7fb2e5a00640 1 -- 172.21.15.100:0/4153166535 wait complete. 2024-04-23T14:32:35.751 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-23T14:32:34.740754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219376,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755344,"statfs":{"total":773094113280,"available":772869472256,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002942"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640233+0000","last_change":"2024-04-23T14:32:08.611525+0000","last_active":"2024-04-23T14:32:14.640233+0000","last_peered":"2024-04-23T14:32:14.640233+0000","last_clean":"2024-04-23T14:32:14.640233+0000","last_became_active":"2024-04-23T14:32:08.611163+0000","last_became_peered":"2024-04-23T14:32:08.611163+0000","last_unstale":"2024-04-23T14:32:14.640233+0000","last_undegraded":"2024-04-23T14:32:14.640233+0000","last_fullsized":"2024-04-23T14:32:14.640233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:08:50.925677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181653+0000","last_change":"2024-04-23T14:32:09.625243+0000","last_active":"2024-04-23T14:32:15.181653+0000","last_peered":"2024-04-23T14:32:15.181653+0000","last_clean":"2024-04-23T14:32:15.181653+0000","last_became_active":"2024-04-23T14:32:09.624765+0000","last_became_peered":"2024-04-23T14:32:09.624765+0000","last_unstale":"2024-04-23T14:32:15.181653+0000","last_undegraded":"2024-04-23T14:32:15.181653+0000","last_fullsized":"2024-04-23T14:32:15.181653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:50.876901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181363+0000","last_change":"2024-04-23T14:32:13.779583+0000","last_active":"2024-04-23T14:32:15.181363+0000","last_peered":"2024-04-23T14:32:15.181363+0000","last_clean":"2024-04-23T14:32:15.181363+0000","last_became_active":"2024-04-23T14:32:13.779224+0000","last_became_peered":"2024-04-23T14:32:13.779224+0000","last_unstale":"2024-04-23T14:32:15.181363+0000","last_undegraded":"2024-04-23T14:32:15.181363+0000","last_fullsized":"2024-04-23T14:32:15.181363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:55.041747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181862+0000","last_change":"2024-04-23T14:32:08.587781+0000","last_active":"2024-04-23T14:32:15.181862+0000","last_peered":"2024-04-23T14:32:15.181862+0000","last_clean":"2024-04-23T14:32:15.181862+0000","last_became_active":"2024-04-23T14:32:08.587435+0000","last_became_peered":"2024-04-23T14:32:08.587435+0000","last_unstale":"2024-04-23T14:32:15.181862+0000","last_undegraded":"2024-04-23T14:32:15.181862+0000","last_fullsized":"2024-04-23T14:32:15.181862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:51.650822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640102+0000","last_change":"2024-04-23T14:32:09.621038+0000","last_active":"2024-04-23T14:32:14.640102+0000","last_peered":"2024-04-23T14:32:14.640102+0000","last_clean":"2024-04-23T14:32:14.640102+0000","last_became_active":"2024-04-23T14:32:09.620776+0000","last_became_peered":"2024-04-23T14:32:09.620776+0000","last_unstale":"2024-04-23T14:32:14.640102+0000","last_undegraded":"2024-04-23T14:32:14.640102+0000","last_fullsized":"2024-04-23T14:32:14.640102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:12.933947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151642+0000","last_change":"2024-04-23T14:32:13.671138+0000","last_active":"2024-04-23T14:32:15.151642+0000","last_peered":"2024-04-23T14:32:15.151642+0000","last_clean":"2024-04-23T14:32:15.151642+0000","last_became_active":"2024-04-23T14:32:13.670514+0000","last_became_peered":"2024-04-23T14:32:13.670514+0000","last_unstale":"2024-04-23T14:32:15.151642+0000","last_undegraded":"2024-04-23T14:32:15.151642+0000","last_fullsized":"2024-04-23T14:32:15.151642+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:26.271219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179964+0000","last_change":"2024-04-23T14:32:08.616316+0000","last_active":"2024-04-23T14:32:15.179964+0000","last_peered":"2024-04-23T14:32:15.179964+0000","last_clean":"2024-04-23T14:32:15.179964+0000","last_became_active":"2024-04-23T14:32:08.616020+0000","last_became_peered":"2024-04-23T14:32:08.616020+0000","last_unstale":"2024-04-23T14:32:15.179964+0000","last_undegraded":"2024-04-23T14:32:15.179964+0000","last_fullsized":"2024-04-23T14:32:15.179964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:53:40.004436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640291+0000","last_change":"2024-04-23T14:32:09.635504+0000","last_active":"2024-04-23T14:32:14.640291+0000","last_peered":"2024-04-23T14:32:14.640291+0000","last_clean":"2024-04-23T14:32:14.640291+0000","last_became_active":"2024-04-23T14:32:09.635179+0000","last_became_peered":"2024-04-23T14:32:09.635179+0000","last_unstale":"2024-04-23T14:32:14.640291+0000","last_undegraded":"2024-04-23T14:32:14.640291+0000","last_fullsized":"2024-04-23T14:32:14.640291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:56.096479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151795+0000","last_change":"2024-04-23T14:32:13.667280+0000","last_active":"2024-04-23T14:32:15.151795+0000","last_peered":"2024-04-23T14:32:15.151795+0000","last_clean":"2024-04-23T14:32:15.151795+0000","last_became_active":"2024-04-23T14:32:13.666861+0000","last_became_peered":"2024-04-23T14:32:13.666861+0000","last_unstale":"2024-04-23T14:32:15.151795+0000","last_undegraded":"2024-04-23T14:32:15.151795+0000","last_fullsized":"2024-04-23T14:32:15.151795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:56.509994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179707+0000","last_change":"2024-04-23T14:32:08.615668+0000","last_active":"2024-04-23T14:32:15.179707+0000","last_peered":"2024-04-23T14:32:15.179707+0000","last_clean":"2024-04-23T14:32:15.179707+0000","last_became_active":"2024-04-23T14:32:08.615384+0000","last_became_peered":"2024-04-23T14:32:08.615384+0000","last_unstale":"2024-04-23T14:32:15.179707+0000","last_undegraded":"2024-04-23T14:32:15.179707+0000","last_fullsized":"2024-04-23T14:32:15.179707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:11.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640191+0000","last_change":"2024-04-23T14:32:09.634014+0000","last_active":"2024-04-23T14:32:14.640191+0000","last_peered":"2024-04-23T14:32:14.640191+0000","last_clean":"2024-04-23T14:32:14.640191+0000","last_became_active":"2024-04-23T14:32:09.633633+0000","last_became_peered":"2024-04-23T14:32:09.633633+0000","last_unstale":"2024-04-23T14:32:14.640191+0000","last_undegraded":"2024-04-23T14:32:14.640191+0000","last_fullsized":"2024-04-23T14:32:14.640191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:47:31.805399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151732+0000","last_change":"2024-04-23T14:32:13.661944+0000","last_active":"2024-04-23T14:32:15.151732+0000","last_peered":"2024-04-23T14:32:15.151732+0000","last_clean":"2024-04-23T14:32:15.151732+0000","last_became_active":"2024-04-23T14:32:13.661584+0000","last_became_peered":"2024-04-23T14:32:13.661584+0000","last_unstale":"2024-04-23T14:32:15.151732+0000","last_undegraded":"2024-04-23T14:32:15.151732+0000","last_fullsized":"2024-04-23T14:32:15.151732+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:54.927834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641604+0000","last_change":"2024-04-23T14:32:09.645728+0000","last_active":"2024-04-23T14:32:14.641604+0000","last_peered":"2024-04-23T14:32:14.641604+0000","last_clean":"2024-04-23T14:32:14.641604+0000","last_became_active":"2024-04-23T14:32:09.641089+0000","last_became_peered":"2024-04-23T14:32:09.641089+0000","last_unstale":"2024-04-23T14:32:14.641604+0000","last_undegraded":"2024-04-23T14:32:14.641604+0000","last_fullsized":"2024-04-23T14:32:14.641604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:53.674486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179794+0000","last_change":"2024-04-23T14:32:08.616537+0000","last_active":"2024-04-23T14:32:15.179794+0000","last_peered":"2024-04-23T14:32:15.179794+0000","last_clean":"2024-04-23T14:32:15.179794+0000","last_became_active":"2024-04-23T14:32:08.616145+0000","last_became_peered":"2024-04-23T14:32:08.616145+0000","last_unstale":"2024-04-23T14:32:15.179794+0000","last_undegraded":"2024-04-23T14:32:15.179794+0000","last_fullsized":"2024-04-23T14:32:15.179794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:00.056567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152246+0000","last_change":"2024-04-23T14:32:13.667224+0000","last_active":"2024-04-23T14:32:15.152246+0000","last_peered":"2024-04-23T14:32:15.152246+0000","last_clean":"2024-04-23T14:32:15.152246+0000","last_became_active":"2024-04-23T14:32:13.666816+0000","last_became_peered":"2024-04-23T14:32:13.666816+0000","last_unstale":"2024-04-23T14:32:15.152246+0000","last_undegraded":"2024-04-23T14:32:15.152246+0000","last_fullsized":"2024-04-23T14:32:15.152246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:22.418010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181037+0000","last_change":"2024-04-23T14:32:09.622489+0000","last_active":"2024-04-23T14:32:15.181037+0000","last_peered":"2024-04-23T14:32:15.181037+0000","last_clean":"2024-04-23T14:32:15.181037+0000","last_became_active":"2024-04-23T14:32:09.617206+0000","last_became_peered":"2024-04-23T14:32:09.617206+0000","last_unstale":"2024-04-23T14:32:15.181037+0000","last_undegraded":"2024-04-23T14:32:15.181037+0000","last_fullsized":"2024-04-23T14:32:15.181037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:03.128923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182693+0000","last_change":"2024-04-23T14:32:08.600628+0000","last_active":"2024-04-23T14:32:15.182693+0000","last_peered":"2024-04-23T14:32:15.182693+0000","last_clean":"2024-04-23T14:32:15.182693+0000","last_became_active":"2024-04-23T14:32:08.600318+0000","last_became_peered":"2024-04-23T14:32:08.600318+0000","last_unstale":"2024-04-23T14:32:15.182693+0000","last_undegraded":"2024-04-23T14:32:15.182693+0000","last_fullsized":"2024-04-23T14:32:15.182693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:54.495261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152192+0000","last_change":"2024-04-23T14:32:13.661586+0000","last_active":"2024-04-23T14:32:15.152192+0000","last_peered":"2024-04-23T14:32:15.152192+0000","last_clean":"2024-04-23T14:32:15.152192+0000","last_became_active":"2024-04-23T14:32:13.660406+0000","last_became_peered":"2024-04-23T14:32:13.660406+0000","last_unstale":"2024-04-23T14:32:15.152192+0000","last_undegraded":"2024-04-23T14:32:15.152192+0000","last_fullsized":"2024-04-23T14:32:15.152192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:47:35.856332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181164+0000","last_change":"2024-04-23T14:32:09.627921+0000","last_active":"2024-04-23T14:32:15.181164+0000","last_peered":"2024-04-23T14:32:15.181164+0000","last_clean":"2024-04-23T14:32:15.181164+0000","last_became_active":"2024-04-23T14:32:09.622420+0000","last_became_peered":"2024-04-23T14:32:09.622420+0000","last_unstale":"2024-04-23T14:32:15.181164+0000","last_undegraded":"2024-04-23T14:32:15.181164+0000","last_fullsized":"2024-04-23T14:32:15.181164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:53.674486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180797+0000","last_change":"2024-04-23T14:32:08.612306+0000","last_active":"2024-04-23T14:32:15.180797+0000","last_peered":"2024-04-23T14:32:15.180797+0000","last_clean":"2024-04-23T14:32:15.180797+0000","last_became_active":"2024-04-23T14:32:08.612018+0000","last_became_peered":"2024-04-23T14:32:08.612018+0000","last_unstale":"2024-04-23T14:32:15.180797+0000","last_undegraded":"2024-04-23T14:32:15.180797+0000","last_fullsized":"2024-04-23T14:32:15.180797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:59:03.639740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.183304+0000","last_change":"2024-04-23T14:32:13.780668+0000","last_active":"2024-04-23T14:32:15.183304+0000","last_peered":"2024-04-23T14:32:15.183304+0000","last_clean":"2024-04-23T14:32:15.183304+0000","last_became_active":"2024-04-23T14:32:13.780323+0000","last_became_peered":"2024-04-23T14:32:13.780323+0000","last_unstale":"2024-04-23T14:32:15.183304+0000","last_undegraded":"2024-04-23T14:32:15.183304+0000","last_fullsized":"2024-04-23T14:32:15.183304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:12:23.488042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151992+0000","last_change":"2024-04-23T14:32:09.627663+0000","last_active":"2024-04-23T14:32:15.151992+0000","last_peered":"2024-04-23T14:32:15.151992+0000","last_clean":"2024-04-23T14:32:15.151992+0000","last_became_active":"2024-04-23T14:32:09.620464+0000","last_became_peered":"2024-04-23T14:32:09.620464+0000","last_unstale":"2024-04-23T14:32:15.151992+0000","last_undegraded":"2024-04-23T14:32:15.151992+0000","last_fullsized":"2024-04-23T14:32:15.151992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:52.656649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179300+0000","last_change":"2024-04-23T14:32:08.612870+0000","last_active":"2024-04-23T14:32:15.179300+0000","last_peered":"2024-04-23T14:32:15.179300+0000","last_clean":"2024-04-23T14:32:15.179300+0000","last_became_active":"2024-04-23T14:32:08.610830+0000","last_became_peered":"2024-04-23T14:32:08.610830+0000","last_unstale":"2024-04-23T14:32:15.179300+0000","last_undegraded":"2024-04-23T14:32:15.179300+0000","last_fullsized":"2024-04-23T14:32:15.179300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:51.650822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152268+0000","last_change":"2024-04-23T14:32:13.660771+0000","last_active":"2024-04-23T14:32:15.152268+0000","last_peered":"2024-04-23T14:32:15.152268+0000","last_clean":"2024-04-23T14:32:15.152268+0000","last_became_active":"2024-04-23T14:32:13.660291+0000","last_became_peered":"2024-04-23T14:32:13.660291+0000","last_unstale":"2024-04-23T14:32:15.152268+0000","last_undegraded":"2024-04-23T14:32:15.152268+0000","last_fullsized":"2024-04-23T14:32:15.152268+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:28.688885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640262+0000","last_change":"2024-04-23T14:32:09.612595+0000","last_active":"2024-04-23T14:32:14.640262+0000","last_peered":"2024-04-23T14:32:14.640262+0000","last_clean":"2024-04-23T14:32:14.640262+0000","last_became_active":"2024-04-23T14:32:09.612168+0000","last_became_peered":"2024-04-23T14:32:09.612168+0000","last_unstale":"2024-04-23T14:32:14.640262+0000","last_undegraded":"2024-04-23T14:32:14.640262+0000","last_fullsized":"2024-04-23T14:32:14.640262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:03.128923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.642236+0000","last_change":"2024-04-23T14:32:08.615099+0000","last_active":"2024-04-23T14:32:14.642236+0000","last_peered":"2024-04-23T14:32:14.642236+0000","last_clean":"2024-04-23T14:32:14.642236+0000","last_became_active":"2024-04-23T14:32:08.614350+0000","last_became_peered":"2024-04-23T14:32:08.614350+0000","last_unstale":"2024-04-23T14:32:14.642236+0000","last_undegraded":"2024-04-23T14:32:14.642236+0000","last_fullsized":"2024-04-23T14:32:14.642236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:59:03.639740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151716+0000","last_change":"2024-04-23T14:32:13.775674+0000","last_active":"2024-04-23T14:32:15.151716+0000","last_peered":"2024-04-23T14:32:15.151716+0000","last_clean":"2024-04-23T14:32:15.151716+0000","last_became_active":"2024-04-23T14:32:13.774695+0000","last_became_peered":"2024-04-23T14:32:13.774695+0000","last_unstale":"2024-04-23T14:32:15.151716+0000","last_undegraded":"2024-04-23T14:32:15.151716+0000","last_fullsized":"2024-04-23T14:32:15.151716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:13.117688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182184+0000","last_change":"2024-04-23T14:32:09.630421+0000","last_active":"2024-04-23T14:32:15.182184+0000","last_peered":"2024-04-23T14:32:15.182184+0000","last_clean":"2024-04-23T14:32:15.182184+0000","last_became_active":"2024-04-23T14:32:09.630104+0000","last_became_peered":"2024-04-23T14:32:09.630104+0000","last_unstale":"2024-04-23T14:32:15.182184+0000","last_undegraded":"2024-04-23T14:32:15.182184+0000","last_fullsized":"2024-04-23T14:32:15.182184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:56.096479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.183107+0000","last_change":"2024-04-23T14:32:08.607576+0000","last_active":"2024-04-23T14:32:15.183107+0000","last_peered":"2024-04-23T14:32:15.183107+0000","last_clean":"2024-04-23T14:32:15.183107+0000","last_became_active":"2024-04-23T14:32:08.606611+0000","last_became_peered":"2024-04-23T14:32:08.606611+0000","last_unstale":"2024-04-23T14:32:15.183107+0000","last_undegraded":"2024-04-23T14:32:15.183107+0000","last_fullsized":"2024-04-23T14:32:15.183107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:00.056567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180827+0000","last_change":"2024-04-23T14:32:13.778406+0000","last_active":"2024-04-23T14:32:15.180827+0000","last_peered":"2024-04-23T14:32:15.180827+0000","last_clean":"2024-04-23T14:32:15.180827+0000","last_became_active":"2024-04-23T14:32:13.778034+0000","last_became_peered":"2024-04-23T14:32:13.778034+0000","last_unstale":"2024-04-23T14:32:15.180827+0000","last_undegraded":"2024-04-23T14:32:15.180827+0000","last_fullsized":"2024-04-23T14:32:15.180827+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:21:36.288246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181961+0000","last_change":"2024-04-23T14:32:09.613694+0000","last_active":"2024-04-23T14:32:15.181961+0000","last_peered":"2024-04-23T14:32:15.181961+0000","last_clean":"2024-04-23T14:32:15.181961+0000","last_became_active":"2024-04-23T14:32:09.613413+0000","last_became_peered":"2024-04-23T14:32:09.613413+0000","last_unstale":"2024-04-23T14:32:15.181961+0000","last_undegraded":"2024-04-23T14:32:15.181961+0000","last_fullsized":"2024-04-23T14:32:15.181961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:12.933947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180126+0000","last_change":"2024-04-23T14:32:08.612923+0000","last_active":"2024-04-23T14:32:15.180126+0000","last_peered":"2024-04-23T14:32:15.180126+0000","last_clean":"2024-04-23T14:32:15.180126+0000","last_became_active":"2024-04-23T14:32:08.610847+0000","last_became_peered":"2024-04-23T14:32:08.610847+0000","last_unstale":"2024-04-23T14:32:15.180126+0000","last_undegraded":"2024-04-23T14:32:15.180126+0000","last_fullsized":"2024-04-23T14:32:15.180126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:40:25.063520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640453+0000","last_change":"2024-04-23T14:32:13.775013+0000","last_active":"2024-04-23T14:32:14.640453+0000","last_peered":"2024-04-23T14:32:14.640453+0000","last_clean":"2024-04-23T14:32:14.640453+0000","last_became_active":"2024-04-23T14:32:13.774565+0000","last_became_peered":"2024-04-23T14:32:13.774565+0000","last_unstale":"2024-04-23T14:32:14.640453+0000","last_undegraded":"2024-04-23T14:32:14.640453+0000","last_fullsized":"2024-04-23T14:32:14.640453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:26:08.003427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640083+0000","last_change":"2024-04-23T14:32:09.633886+0000","last_active":"2024-04-23T14:32:14.640083+0000","last_peered":"2024-04-23T14:32:14.640083+0000","last_clean":"2024-04-23T14:32:14.640083+0000","last_became_active":"2024-04-23T14:32:09.633258+0000","last_became_peered":"2024-04-23T14:32:09.633258+0000","last_unstale":"2024-04-23T14:32:14.640083+0000","last_undegraded":"2024-04-23T14:32:14.640083+0000","last_fullsized":"2024-04-23T14:32:14.640083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:24.637952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182012+0000","last_change":"2024-04-23T14:32:08.602248+0000","last_active":"2024-04-23T14:32:15.182012+0000","last_peered":"2024-04-23T14:32:15.182012+0000","last_clean":"2024-04-23T14:32:15.182012+0000","last_became_active":"2024-04-23T14:32:08.601913+0000","last_became_peered":"2024-04-23T14:32:08.601913+0000","last_unstale":"2024-04-23T14:32:15.182012+0000","last_undegraded":"2024-04-23T14:32:15.182012+0000","last_fullsized":"2024-04-23T14:32:15.182012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:11.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180987+0000","last_change":"2024-04-23T14:32:13.666952+0000","last_active":"2024-04-23T14:32:15.180987+0000","last_peered":"2024-04-23T14:32:15.180987+0000","last_clean":"2024-04-23T14:32:15.180987+0000","last_became_active":"2024-04-23T14:32:13.663335+0000","last_became_peered":"2024-04-23T14:32:13.663335+0000","last_unstale":"2024-04-23T14:32:15.180987+0000","last_undegraded":"2024-04-23T14:32:15.180987+0000","last_fullsized":"2024-04-23T14:32:15.180987+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:22.418010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152000+0000","last_change":"2024-04-23T14:32:09.627852+0000","last_active":"2024-04-23T14:32:15.152000+0000","last_peered":"2024-04-23T14:32:15.152000+0000","last_clean":"2024-04-23T14:32:15.152000+0000","last_became_active":"2024-04-23T14:32:09.627575+0000","last_became_peered":"2024-04-23T14:32:09.627575+0000","last_unstale":"2024-04-23T14:32:15.152000+0000","last_undegraded":"2024-04-23T14:32:15.152000+0000","last_fullsized":"2024-04-23T14:32:15.152000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:55.501089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640731+0000","last_change":"2024-04-23T14:32:08.603753+0000","last_active":"2024-04-23T14:32:14.640731+0000","last_peered":"2024-04-23T14:32:14.640731+0000","last_clean":"2024-04-23T14:32:14.640731+0000","last_became_active":"2024-04-23T14:32:08.602944+0000","last_became_peered":"2024-04-23T14:32:08.602944+0000","last_unstale":"2024-04-23T14:32:14.640731+0000","last_undegraded":"2024-04-23T14:32:14.640731+0000","last_fullsized":"2024-04-23T14:32:14.640731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:54.495261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181918+0000","last_change":"2024-04-23T14:32:13.667268+0000","last_active":"2024-04-23T14:32:15.181918+0000","last_peered":"2024-04-23T14:32:15.181918+0000","last_clean":"2024-04-23T14:32:15.181918+0000","last_became_active":"2024-04-23T14:32:13.666836+0000","last_became_peered":"2024-04-23T14:32:13.666836+0000","last_unstale":"2024-04-23T14:32:15.181918+0000","last_undegraded":"2024-04-23T14:32:15.181918+0000","last_fullsized":"2024-04-23T14:32:15.181918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:14:16.782670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180774+0000","last_change":"2024-04-23T14:32:09.613896+0000","last_active":"2024-04-23T14:32:15.180774+0000","last_peered":"2024-04-23T14:32:15.180774+0000","last_clean":"2024-04-23T14:32:15.180774+0000","last_became_active":"2024-04-23T14:32:09.613535+0000","last_became_peered":"2024-04-23T14:32:09.613535+0000","last_unstale":"2024-04-23T14:32:15.180774+0000","last_undegraded":"2024-04-23T14:32:15.180774+0000","last_fullsized":"2024-04-23T14:32:15.180774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:50:24.043980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181650+0000","last_change":"2024-04-23T14:32:08.606268+0000","last_active":"2024-04-23T14:32:15.181650+0000","last_peered":"2024-04-23T14:32:15.181650+0000","last_clean":"2024-04-23T14:32:15.181650+0000","last_became_active":"2024-04-23T14:32:08.605958+0000","last_became_peered":"2024-04-23T14:32:08.605958+0000","last_unstale":"2024-04-23T14:32:15.181650+0000","last_undegraded":"2024-04-23T14:32:15.181650+0000","last_fullsized":"2024-04-23T14:32:15.181650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:45:45.243948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.183441+0000","last_change":"2024-04-23T14:32:13.664183+0000","last_active":"2024-04-23T14:32:15.183441+0000","last_peered":"2024-04-23T14:32:15.183441+0000","last_clean":"2024-04-23T14:32:15.183441+0000","last_became_active":"2024-04-23T14:32:13.663836+0000","last_became_peered":"2024-04-23T14:32:13.663836+0000","last_unstale":"2024-04-23T14:32:15.183441+0000","last_undegraded":"2024-04-23T14:32:15.183441+0000","last_fullsized":"2024-04-23T14:32:15.183441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:09:12.045283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639679+0000","last_change":"2024-04-23T14:32:09.633686+0000","last_active":"2024-04-23T14:32:14.639679+0000","last_peered":"2024-04-23T14:32:14.639679+0000","last_clean":"2024-04-23T14:32:14.639679+0000","last_became_active":"2024-04-23T14:32:09.631825+0000","last_became_peered":"2024-04-23T14:32:09.631825+0000","last_unstale":"2024-04-23T14:32:14.639679+0000","last_undegraded":"2024-04-23T14:32:14.639679+0000","last_fullsized":"2024-04-23T14:32:14.639679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:23:57.869625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179303+0000","last_change":"2024-04-23T14:32:08.613375+0000","last_active":"2024-04-23T14:32:15.179303+0000","last_peered":"2024-04-23T14:32:15.179303+0000","last_clean":"2024-04-23T14:32:15.179303+0000","last_became_active":"2024-04-23T14:32:08.613129+0000","last_became_peered":"2024-04-23T14:32:08.613129+0000","last_unstale":"2024-04-23T14:32:15.179303+0000","last_undegraded":"2024-04-23T14:32:15.179303+0000","last_fullsized":"2024-04-23T14:32:15.179303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:54.495261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640202+0000","last_change":"2024-04-23T14:32:13.779707+0000","last_active":"2024-04-23T14:32:14.640202+0000","last_peered":"2024-04-23T14:32:14.640202+0000","last_clean":"2024-04-23T14:32:14.640202+0000","last_became_active":"2024-04-23T14:32:13.779357+0000","last_became_peered":"2024-04-23T14:32:13.779357+0000","last_unstale":"2024-04-23T14:32:14.640202+0000","last_undegraded":"2024-04-23T14:32:14.640202+0000","last_fullsized":"2024-04-23T14:32:14.640202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:12:23.488042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181182+0000","last_change":"2024-04-23T14:32:09.621797+0000","last_active":"2024-04-23T14:32:15.181182+0000","last_peered":"2024-04-23T14:32:15.181182+0000","last_clean":"2024-04-23T14:32:15.181182+0000","last_became_active":"2024-04-23T14:32:09.618825+0000","last_became_peered":"2024-04-23T14:32:09.618825+0000","last_unstale":"2024-04-23T14:32:15.181182+0000","last_undegraded":"2024-04-23T14:32:15.181182+0000","last_fullsized":"2024-04-23T14:32:15.181182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:22.220286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639965+0000","last_change":"2024-04-23T14:32:08.603338+0000","last_active":"2024-04-23T14:32:14.639965+0000","last_peered":"2024-04-23T14:32:14.639965+0000","last_clean":"2024-04-23T14:32:14.639965+0000","last_became_active":"2024-04-23T14:32:08.602563+0000","last_became_peered":"2024-04-23T14:32:08.602563+0000","last_unstale":"2024-04-23T14:32:14.639965+0000","last_undegraded":"2024-04-23T14:32:14.639965+0000","last_fullsized":"2024-04-23T14:32:14.639965+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:45:45.243948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179439+0000","last_change":"2024-04-23T14:31:58.314235+0000","last_active":"2024-04-23T14:32:15.179439+0000","last_peered":"2024-04-23T14:32:15.179439+0000","last_clean":"2024-04-23T14:32:15.179439+0000","last_became_active":"2024-04-23T14:31:58.313165+0000","last_became_peered":"2024-04-23T14:31:58.313165+0000","last_unstale":"2024-04-23T14:32:15.179439+0000","last_undegraded":"2024-04-23T14:32:15.179439+0000","last_fullsized":"2024-04-23T14:32:15.179439+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_clean_scrub_stamp":"2024-04-23T14:31:57.140712+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:44.170436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641636+0000","last_change":"2024-04-23T14:32:13.777292+0000","last_active":"2024-04-23T14:32:14.641636+0000","last_peered":"2024-04-23T14:32:14.641636+0000","last_clean":"2024-04-23T14:32:14.641636+0000","last_became_active":"2024-04-23T14:32:13.776933+0000","last_became_peered":"2024-04-23T14:32:13.776933+0000","last_unstale":"2024-04-23T14:32:14.641636+0000","last_undegraded":"2024-04-23T14:32:14.641636+0000","last_fullsized":"2024-04-23T14:32:14.641636+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:47:35.856332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179793+0000","last_change":"2024-04-23T14:32:09.639015+0000","last_active":"2024-04-23T14:32:15.179793+0000","last_peered":"2024-04-23T14:32:15.179793+0000","last_clean":"2024-04-23T14:32:15.179793+0000","last_became_active":"2024-04-23T14:32:09.638622+0000","last_became_peered":"2024-04-23T14:32:09.638622+0000","last_unstale":"2024-04-23T14:32:15.179793+0000","last_undegraded":"2024-04-23T14:32:15.179793+0000","last_fullsized":"2024-04-23T14:32:15.179793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:18.367077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181794+0000","last_change":"2024-04-23T14:32:08.607926+0000","last_active":"2024-04-23T14:32:15.181794+0000","last_peered":"2024-04-23T14:32:15.181794+0000","last_clean":"2024-04-23T14:32:15.181794+0000","last_became_active":"2024-04-23T14:32:08.607649+0000","last_became_peered":"2024-04-23T14:32:08.607649+0000","last_unstale":"2024-04-23T14:32:15.181794+0000","last_undegraded":"2024-04-23T14:32:15.181794+0000","last_fullsized":"2024-04-23T14:32:15.181794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:59:03.639740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641732+0000","last_change":"2024-04-23T14:32:13.774449+0000","last_active":"2024-04-23T14:32:14.641732+0000","last_peered":"2024-04-23T14:32:14.641732+0000","last_clean":"2024-04-23T14:32:14.641732+0000","last_became_active":"2024-04-23T14:32:13.773937+0000","last_became_peered":"2024-04-23T14:32:13.773937+0000","last_unstale":"2024-04-23T14:32:14.641732+0000","last_undegraded":"2024-04-23T14:32:14.641732+0000","last_fullsized":"2024-04-23T14:32:14.641732+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:22.418010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640399+0000","last_change":"2024-04-23T14:32:09.626405+0000","last_active":"2024-04-23T14:32:14.640399+0000","last_peered":"2024-04-23T14:32:14.640399+0000","last_clean":"2024-04-23T14:32:14.640399+0000","last_became_active":"2024-04-23T14:32:09.626056+0000","last_became_peered":"2024-04-23T14:32:09.626056+0000","last_unstale":"2024-04-23T14:32:14.640399+0000","last_undegraded":"2024-04-23T14:32:14.640399+0000","last_fullsized":"2024-04-23T14:32:14.640399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:18.367077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182898+0000","last_change":"2024-04-23T14:32:08.608148+0000","last_active":"2024-04-23T14:32:15.182898+0000","last_peered":"2024-04-23T14:32:15.182898+0000","last_clean":"2024-04-23T14:32:15.182898+0000","last_became_active":"2024-04-23T14:32:08.607623+0000","last_became_peered":"2024-04-23T14:32:08.607623+0000","last_unstale":"2024-04-23T14:32:15.182898+0000","last_undegraded":"2024-04-23T14:32:15.182898+0000","last_fullsized":"2024-04-23T14:32:15.182898+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:51.650822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640246+0000","last_change":"2024-04-23T14:32:13.663295+0000","last_active":"2024-04-23T14:32:14.640246+0000","last_peered":"2024-04-23T14:32:14.640246+0000","last_clean":"2024-04-23T14:32:14.640246+0000","last_became_active":"2024-04-23T14:32:13.662900+0000","last_became_peered":"2024-04-23T14:32:13.662900+0000","last_unstale":"2024-04-23T14:32:14.640246+0000","last_undegraded":"2024-04-23T14:32:14.640246+0000","last_fullsized":"2024-04-23T14:32:14.640246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:09:12.045283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180115+0000","last_change":"2024-04-23T14:32:09.641393+0000","last_active":"2024-04-23T14:32:15.180115+0000","last_peered":"2024-04-23T14:32:15.180115+0000","last_clean":"2024-04-23T14:32:15.180115+0000","last_became_active":"2024-04-23T14:32:09.641075+0000","last_became_peered":"2024-04-23T14:32:09.641075+0000","last_unstale":"2024-04-23T14:32:15.180115+0000","last_undegraded":"2024-04-23T14:32:15.180115+0000","last_fullsized":"2024-04-23T14:32:15.180115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:26:03.952494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182813+0000","last_change":"2024-04-23T14:32:08.607178+0000","last_active":"2024-04-23T14:32:15.182813+0000","last_peered":"2024-04-23T14:32:15.182813+0000","last_clean":"2024-04-23T14:32:15.182813+0000","last_became_active":"2024-04-23T14:32:08.606863+0000","last_became_peered":"2024-04-23T14:32:08.606863+0000","last_unstale":"2024-04-23T14:32:15.182813+0000","last_undegraded":"2024-04-23T14:32:15.182813+0000","last_fullsized":"2024-04-23T14:32:15.182813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:35.854078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639398+0000","last_change":"2024-04-23T14:32:13.773533+0000","last_active":"2024-04-23T14:32:14.639398+0000","last_peered":"2024-04-23T14:32:14.639398+0000","last_clean":"2024-04-23T14:32:14.639398+0000","last_became_active":"2024-04-23T14:32:13.773057+0000","last_became_peered":"2024-04-23T14:32:13.773057+0000","last_unstale":"2024-04-23T14:32:14.639398+0000","last_undegraded":"2024-04-23T14:32:14.639398+0000","last_fullsized":"2024-04-23T14:32:14.639398+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:08.115745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180012+0000","last_change":"2024-04-23T14:32:09.641317+0000","last_active":"2024-04-23T14:32:15.180012+0000","last_peered":"2024-04-23T14:32:15.180012+0000","last_clean":"2024-04-23T14:32:15.180012+0000","last_became_active":"2024-04-23T14:32:09.640930+0000","last_became_peered":"2024-04-23T14:32:09.640930+0000","last_unstale":"2024-04-23T14:32:15.180012+0000","last_undegraded":"2024-04-23T14:32:15.180012+0000","last_fullsized":"2024-04-23T14:32:15.180012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:24.637952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152438+0000","last_change":"2024-04-23T14:32:08.601936+0000","last_active":"2024-04-23T14:32:15.152438+0000","last_peered":"2024-04-23T14:32:15.152438+0000","last_clean":"2024-04-23T14:32:15.152438+0000","last_became_active":"2024-04-23T14:32:08.601653+0000","last_became_peered":"2024-04-23T14:32:08.601653+0000","last_unstale":"2024-04-23T14:32:15.152438+0000","last_undegraded":"2024-04-23T14:32:15.152438+0000","last_fullsized":"2024-04-23T14:32:15.152438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:07:08.639311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151588+0000","last_change":"2024-04-23T14:32:13.664132+0000","last_active":"2024-04-23T14:32:15.151588+0000","last_peered":"2024-04-23T14:32:15.151588+0000","last_clean":"2024-04-23T14:32:15.151588+0000","last_became_active":"2024-04-23T14:32:13.663741+0000","last_became_peered":"2024-04-23T14:32:13.663741+0000","last_unstale":"2024-04-23T14:32:15.151588+0000","last_undegraded":"2024-04-23T14:32:15.151588+0000","last_fullsized":"2024-04-23T14:32:15.151588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:07.179856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179517+0000","last_change":"2024-04-23T14:32:09.641942+0000","last_active":"2024-04-23T14:32:15.179517+0000","last_peered":"2024-04-23T14:32:15.179517+0000","last_clean":"2024-04-23T14:32:15.179517+0000","last_became_active":"2024-04-23T14:32:09.641680+0000","last_became_peered":"2024-04-23T14:32:09.641680+0000","last_unstale":"2024-04-23T14:32:15.179517+0000","last_undegraded":"2024-04-23T14:32:15.179517+0000","last_fullsized":"2024-04-23T14:32:15.179517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:50.876901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181283+0000","last_change":"2024-04-23T14:32:08.603405+0000","last_active":"2024-04-23T14:32:15.181283+0000","last_peered":"2024-04-23T14:32:15.181283+0000","last_clean":"2024-04-23T14:32:15.181283+0000","last_became_active":"2024-04-23T14:32:08.603116+0000","last_became_peered":"2024-04-23T14:32:08.603116+0000","last_unstale":"2024-04-23T14:32:15.181283+0000","last_undegraded":"2024-04-23T14:32:15.181283+0000","last_fullsized":"2024-04-23T14:32:15.181283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:43:58.790056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640138+0000","last_change":"2024-04-23T14:32:13.660242+0000","last_active":"2024-04-23T14:32:14.640138+0000","last_peered":"2024-04-23T14:32:14.640138+0000","last_clean":"2024-04-23T14:32:14.640138+0000","last_became_active":"2024-04-23T14:32:13.659712+0000","last_became_peered":"2024-04-23T14:32:13.659712+0000","last_unstale":"2024-04-23T14:32:14.640138+0000","last_undegraded":"2024-04-23T14:32:14.640138+0000","last_fullsized":"2024-04-23T14:32:14.640138+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:14:16.782670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641152+0000","last_change":"2024-04-23T14:32:09.619773+0000","last_active":"2024-04-23T14:32:14.641152+0000","last_peered":"2024-04-23T14:32:14.641152+0000","last_clean":"2024-04-23T14:32:14.641152+0000","last_became_active":"2024-04-23T14:32:09.619451+0000","last_became_peered":"2024-04-23T14:32:09.619451+0000","last_unstale":"2024-04-23T14:32:14.641152+0000","last_undegraded":"2024-04-23T14:32:14.641152+0000","last_fullsized":"2024-04-23T14:32:14.641152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:12.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641439+0000","last_change":"2024-04-23T14:32:08.613751+0000","last_active":"2024-04-23T14:32:14.641439+0000","last_peered":"2024-04-23T14:32:14.641439+0000","last_clean":"2024-04-23T14:32:14.641439+0000","last_became_active":"2024-04-23T14:32:08.607415+0000","last_became_peered":"2024-04-23T14:32:08.607415+0000","last_unstale":"2024-04-23T14:32:14.641439+0000","last_undegraded":"2024-04-23T14:32:14.641439+0000","last_fullsized":"2024-04-23T14:32:14.641439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:35.854078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182048+0000","last_change":"2024-04-23T14:32:13.780614+0000","last_active":"2024-04-23T14:32:15.182048+0000","last_peered":"2024-04-23T14:32:15.182048+0000","last_clean":"2024-04-23T14:32:15.182048+0000","last_became_active":"2024-04-23T14:32:13.780135+0000","last_became_peered":"2024-04-23T14:32:13.780135+0000","last_unstale":"2024-04-23T14:32:15.182048+0000","last_undegraded":"2024-04-23T14:32:15.182048+0000","last_fullsized":"2024-04-23T14:32:15.182048+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:16:27.476884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179624+0000","last_change":"2024-04-23T14:32:09.642908+0000","last_active":"2024-04-23T14:32:15.179624+0000","last_peered":"2024-04-23T14:32:15.179624+0000","last_clean":"2024-04-23T14:32:15.179624+0000","last_became_active":"2024-04-23T14:32:09.642670+0000","last_became_peered":"2024-04-23T14:32:09.642670+0000","last_unstale":"2024-04-23T14:32:15.179624+0000","last_undegraded":"2024-04-23T14:32:15.179624+0000","last_fullsized":"2024-04-23T14:32:15.179624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:56.096479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640627+0000","last_change":"2024-04-23T14:32:08.598260+0000","last_active":"2024-04-23T14:32:14.640627+0000","last_peered":"2024-04-23T14:32:14.640627+0000","last_clean":"2024-04-23T14:32:14.640627+0000","last_became_active":"2024-04-23T14:32:08.597931+0000","last_became_peered":"2024-04-23T14:32:08.597931+0000","last_unstale":"2024-04-23T14:32:14.640627+0000","last_undegraded":"2024-04-23T14:32:14.640627+0000","last_fullsized":"2024-04-23T14:32:14.640627+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:51.650822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182382+0000","last_change":"2024-04-23T14:32:13.775555+0000","last_active":"2024-04-23T14:32:15.182382+0000","last_peered":"2024-04-23T14:32:15.182382+0000","last_clean":"2024-04-23T14:32:15.182382+0000","last_became_active":"2024-04-23T14:32:13.774654+0000","last_became_peered":"2024-04-23T14:32:13.774654+0000","last_unstale":"2024-04-23T14:32:15.182382+0000","last_undegraded":"2024-04-23T14:32:15.182382+0000","last_fullsized":"2024-04-23T14:32:15.182382+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:57.743802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639957+0000","last_change":"2024-04-23T14:32:09.626794+0000","last_active":"2024-04-23T14:32:14.639957+0000","last_peered":"2024-04-23T14:32:14.639957+0000","last_clean":"2024-04-23T14:32:14.639957+0000","last_became_active":"2024-04-23T14:32:09.626382+0000","last_became_peered":"2024-04-23T14:32:09.626382+0000","last_unstale":"2024-04-23T14:32:14.639957+0000","last_undegraded":"2024-04-23T14:32:14.639957+0000","last_fullsized":"2024-04-23T14:32:14.639957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:54:08.664672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640520+0000","last_change":"2024-04-23T14:32:08.600425+0000","last_active":"2024-04-23T14:32:14.640520+0000","last_peered":"2024-04-23T14:32:14.640520+0000","last_clean":"2024-04-23T14:32:14.640520+0000","last_became_active":"2024-04-23T14:32:08.600113+0000","last_became_peered":"2024-04-23T14:32:08.600113+0000","last_unstale":"2024-04-23T14:32:14.640520+0000","last_undegraded":"2024-04-23T14:32:14.640520+0000","last_fullsized":"2024-04-23T14:32:14.640520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:07:08.639311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639973+0000","last_change":"2024-04-23T14:32:13.777092+0000","last_active":"2024-04-23T14:32:14.639973+0000","last_peered":"2024-04-23T14:32:14.639973+0000","last_clean":"2024-04-23T14:32:14.639973+0000","last_became_active":"2024-04-23T14:32:13.776681+0000","last_became_peered":"2024-04-23T14:32:13.776681+0000","last_unstale":"2024-04-23T14:32:14.639973+0000","last_undegraded":"2024-04-23T14:32:14.639973+0000","last_fullsized":"2024-04-23T14:32:14.639973+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:21:36.288246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179674+0000","last_change":"2024-04-23T14:32:09.629485+0000","last_active":"2024-04-23T14:32:15.179674+0000","last_peered":"2024-04-23T14:32:15.179674+0000","last_clean":"2024-04-23T14:32:15.179674+0000","last_became_active":"2024-04-23T14:32:09.629157+0000","last_became_peered":"2024-04-23T14:32:09.629157+0000","last_unstale":"2024-04-23T14:32:15.179674+0000","last_undegraded":"2024-04-23T14:32:15.179674+0000","last_fullsized":"2024-04-23T14:32:15.179674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:47:31.805399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:25.916019+0000","last_change":"2024-04-23T14:32:08.604477+0000","last_active":"2024-04-23T14:32:25.916019+0000","last_peered":"2024-04-23T14:32:25.916019+0000","last_clean":"2024-04-23T14:32:25.916019+0000","last_became_active":"2024-04-23T14:32:08.604148+0000","last_became_peered":"2024-04-23T14:32:08.604148+0000","last_unstale":"2024-04-23T14:32:25.916019+0000","last_undegraded":"2024-04-23T14:32:25.916019+0000","last_fullsized":"2024-04-23T14:32:25.916019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:51.650822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181124+0000","last_change":"2024-04-23T14:32:13.660428+0000","last_active":"2024-04-23T14:32:15.181124+0000","last_peered":"2024-04-23T14:32:15.181124+0000","last_clean":"2024-04-23T14:32:15.181124+0000","last_became_active":"2024-04-23T14:32:13.659779+0000","last_became_peered":"2024-04-23T14:32:13.659779+0000","last_unstale":"2024-04-23T14:32:15.181124+0000","last_undegraded":"2024-04-23T14:32:15.181124+0000","last_fullsized":"2024-04-23T14:32:15.181124+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:28.688885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152295+0000","last_change":"2024-04-23T14:32:09.628609+0000","last_active":"2024-04-23T14:32:15.152295+0000","last_peered":"2024-04-23T14:32:15.152295+0000","last_clean":"2024-04-23T14:32:15.152295+0000","last_became_active":"2024-04-23T14:32:09.628357+0000","last_became_peered":"2024-04-23T14:32:09.628357+0000","last_unstale":"2024-04-23T14:32:15.152295+0000","last_undegraded":"2024-04-23T14:32:15.152295+0000","last_fullsized":"2024-04-23T14:32:15.152295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:01.062394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182825+0000","last_change":"2024-04-23T14:32:08.607931+0000","last_active":"2024-04-23T14:32:15.182825+0000","last_peered":"2024-04-23T14:32:15.182825+0000","last_clean":"2024-04-23T14:32:15.182825+0000","last_became_active":"2024-04-23T14:32:08.607268+0000","last_became_peered":"2024-04-23T14:32:08.607268+0000","last_unstale":"2024-04-23T14:32:15.182825+0000","last_undegraded":"2024-04-23T14:32:15.182825+0000","last_fullsized":"2024-04-23T14:32:15.182825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:35.854078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152109+0000","last_change":"2024-04-23T14:32:13.661080+0000","last_active":"2024-04-23T14:32:15.152109+0000","last_peered":"2024-04-23T14:32:15.152109+0000","last_clean":"2024-04-23T14:32:15.152109+0000","last_became_active":"2024-04-23T14:32:13.660376+0000","last_became_peered":"2024-04-23T14:32:13.660376+0000","last_unstale":"2024-04-23T14:32:15.152109+0000","last_undegraded":"2024-04-23T14:32:15.152109+0000","last_fullsized":"2024-04-23T14:32:15.152109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:40:00.147412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640230+0000","last_change":"2024-04-23T14:32:09.621197+0000","last_active":"2024-04-23T14:32:14.640230+0000","last_peered":"2024-04-23T14:32:14.640230+0000","last_clean":"2024-04-23T14:32:14.640230+0000","last_became_active":"2024-04-23T14:32:09.620883+0000","last_became_peered":"2024-04-23T14:32:09.620883+0000","last_unstale":"2024-04-23T14:32:14.640230+0000","last_undegraded":"2024-04-23T14:32:14.640230+0000","last_fullsized":"2024-04-23T14:32:14.640230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:12.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640178+0000","last_change":"2024-04-23T14:32:08.606025+0000","last_active":"2024-04-23T14:32:14.640178+0000","last_peered":"2024-04-23T14:32:14.640178+0000","last_clean":"2024-04-23T14:32:14.640178+0000","last_became_active":"2024-04-23T14:32:08.605727+0000","last_became_peered":"2024-04-23T14:32:08.605727+0000","last_unstale":"2024-04-23T14:32:14.640178+0000","last_undegraded":"2024-04-23T14:32:14.640178+0000","last_fullsized":"2024-04-23T14:32:14.640178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:59:03.639740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181124+0000","last_change":"2024-04-23T14:32:13.669518+0000","last_active":"2024-04-23T14:32:15.181124+0000","last_peered":"2024-04-23T14:32:15.181124+0000","last_clean":"2024-04-23T14:32:15.181124+0000","last_became_active":"2024-04-23T14:32:13.669004+0000","last_became_peered":"2024-04-23T14:32:13.669004+0000","last_unstale":"2024-04-23T14:32:15.181124+0000","last_undegraded":"2024-04-23T14:32:15.181124+0000","last_fullsized":"2024-04-23T14:32:15.181124+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:31:45.244234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640043+0000","last_change":"2024-04-23T14:32:09.625280+0000","last_active":"2024-04-23T14:32:14.640043+0000","last_peered":"2024-04-23T14:32:14.640043+0000","last_clean":"2024-04-23T14:32:14.640043+0000","last_became_active":"2024-04-23T14:32:09.625033+0000","last_became_peered":"2024-04-23T14:32:09.625033+0000","last_unstale":"2024-04-23T14:32:14.640043+0000","last_undegraded":"2024-04-23T14:32:14.640043+0000","last_fullsized":"2024-04-23T14:32:14.640043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:14:12.731737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.642133+0000","last_change":"2024-04-23T14:32:08.605039+0000","last_active":"2024-04-23T14:32:14.642133+0000","last_peered":"2024-04-23T14:32:14.642133+0000","last_clean":"2024-04-23T14:32:14.642133+0000","last_became_active":"2024-04-23T14:32:08.604450+0000","last_became_peered":"2024-04-23T14:32:08.604450+0000","last_unstale":"2024-04-23T14:32:14.642133+0000","last_undegraded":"2024-04-23T14:32:14.642133+0000","last_fullsized":"2024-04-23T14:32:14.642133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:45:45.243948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181369+0000","last_change":"2024-04-23T14:32:13.779606+0000","last_active":"2024-04-23T14:32:15.181369+0000","last_peered":"2024-04-23T14:32:15.181369+0000","last_clean":"2024-04-23T14:32:15.181369+0000","last_became_active":"2024-04-23T14:32:13.779243+0000","last_became_peered":"2024-04-23T14:32:13.779243+0000","last_unstale":"2024-04-23T14:32:15.181369+0000","last_undegraded":"2024-04-23T14:32:15.181369+0000","last_fullsized":"2024-04-23T14:32:15.181369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:58:40.249561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640030+0000","last_change":"2024-04-23T14:32:13.779659+0000","last_active":"2024-04-23T14:32:14.640030+0000","last_peered":"2024-04-23T14:32:14.640030+0000","last_clean":"2024-04-23T14:32:14.640030+0000","last_became_active":"2024-04-23T14:32:13.779312+0000","last_became_peered":"2024-04-23T14:32:13.779312+0000","last_unstale":"2024-04-23T14:32:14.640030+0000","last_undegraded":"2024-04-23T14:32:14.640030+0000","last_fullsized":"2024-04-23T14:32:14.640030+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:28.688885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180700+0000","last_change":"2024-04-23T14:32:09.632498+0000","last_active":"2024-04-23T14:32:15.180700+0000","last_peered":"2024-04-23T14:32:15.180700+0000","last_clean":"2024-04-23T14:32:15.180700+0000","last_became_active":"2024-04-23T14:32:09.632191+0000","last_became_peered":"2024-04-23T14:32:09.632191+0000","last_unstale":"2024-04-23T14:32:15.180700+0000","last_undegraded":"2024-04-23T14:32:15.180700+0000","last_fullsized":"2024-04-23T14:32:15.180700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:53.674486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639631+0000","last_change":"2024-04-23T14:32:08.607308+0000","last_active":"2024-04-23T14:32:14.639631+0000","last_peered":"2024-04-23T14:32:14.639631+0000","last_clean":"2024-04-23T14:32:14.639631+0000","last_became_active":"2024-04-23T14:32:08.606840+0000","last_became_peered":"2024-04-23T14:32:08.606840+0000","last_unstale":"2024-04-23T14:32:14.639631+0000","last_undegraded":"2024-04-23T14:32:14.639631+0000","last_fullsized":"2024-04-23T14:32:14.639631+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:35.854078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.642125+0000","last_change":"2024-04-23T14:32:13.654800+0000","last_active":"2024-04-23T14:32:14.642125+0000","last_peered":"2024-04-23T14:32:14.642125+0000","last_clean":"2024-04-23T14:32:14.642125+0000","last_became_active":"2024-04-23T14:32:13.654191+0000","last_became_peered":"2024-04-23T14:32:13.654191+0000","last_unstale":"2024-04-23T14:32:14.642125+0000","last_undegraded":"2024-04-23T14:32:14.642125+0000","last_fullsized":"2024-04-23T14:32:14.642125+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:28.688885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181062+0000","last_change":"2024-04-23T14:32:09.623316+0000","last_active":"2024-04-23T14:32:15.181062+0000","last_peered":"2024-04-23T14:32:15.181062+0000","last_clean":"2024-04-23T14:32:15.181062+0000","last_became_active":"2024-04-23T14:32:09.621723+0000","last_became_peered":"2024-04-23T14:32:09.621723+0000","last_unstale":"2024-04-23T14:32:15.181062+0000","last_undegraded":"2024-04-23T14:32:15.181062+0000","last_fullsized":"2024-04-23T14:32:15.181062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:12.933947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181611+0000","last_change":"2024-04-23T14:32:08.609985+0000","last_active":"2024-04-23T14:32:15.181611+0000","last_peered":"2024-04-23T14:32:15.181611+0000","last_clean":"2024-04-23T14:32:15.181611+0000","last_became_active":"2024-04-23T14:32:08.609621+0000","last_became_peered":"2024-04-23T14:32:08.609621+0000","last_unstale":"2024-04-23T14:32:15.181611+0000","last_undegraded":"2024-04-23T14:32:15.181611+0000","last_fullsized":"2024-04-23T14:32:15.181611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:11.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641831+0000","last_change":"2024-04-23T14:32:08.609884+0000","last_active":"2024-04-23T14:32:14.641831+0000","last_peered":"2024-04-23T14:32:14.641831+0000","last_clean":"2024-04-23T14:32:14.641831+0000","last_became_active":"2024-04-23T14:32:08.607367+0000","last_became_peered":"2024-04-23T14:32:08.607367+0000","last_unstale":"2024-04-23T14:32:14.641831+0000","last_undegraded":"2024-04-23T14:32:14.641831+0000","last_fullsized":"2024-04-23T14:32:14.641831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:54.495261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640439+0000","last_change":"2024-04-23T14:32:09.632614+0000","last_active":"2024-04-23T14:32:14.640439+0000","last_peered":"2024-04-23T14:32:14.640439+0000","last_clean":"2024-04-23T14:32:14.640439+0000","last_became_active":"2024-04-23T14:32:09.632230+0000","last_became_peered":"2024-04-23T14:32:09.632230+0000","last_unstale":"2024-04-23T14:32:14.640439+0000","last_undegraded":"2024-04-23T14:32:14.640439+0000","last_fullsized":"2024-04-23T14:32:14.640439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:53.674486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181628+0000","last_change":"2024-04-23T14:32:13.671189+0000","last_active":"2024-04-23T14:32:15.181628+0000","last_peered":"2024-04-23T14:32:15.181628+0000","last_clean":"2024-04-23T14:32:15.181628+0000","last_became_active":"2024-04-23T14:32:13.670033+0000","last_became_peered":"2024-04-23T14:32:13.670033+0000","last_unstale":"2024-04-23T14:32:15.181628+0000","last_undegraded":"2024-04-23T14:32:15.181628+0000","last_fullsized":"2024-04-23T14:32:15.181628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:08:35.206423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181132+0000","last_change":"2024-04-23T14:32:08.612716+0000","last_active":"2024-04-23T14:32:15.181132+0000","last_peered":"2024-04-23T14:32:15.181132+0000","last_clean":"2024-04-23T14:32:15.181132+0000","last_became_active":"2024-04-23T14:32:08.612378+0000","last_became_peered":"2024-04-23T14:32:08.612378+0000","last_unstale":"2024-04-23T14:32:15.181132+0000","last_undegraded":"2024-04-23T14:32:15.181132+0000","last_fullsized":"2024-04-23T14:32:15.181132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:43:58.790056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151820+0000","last_change":"2024-04-23T14:32:09.620811+0000","last_active":"2024-04-23T14:32:15.151820+0000","last_peered":"2024-04-23T14:32:15.151820+0000","last_clean":"2024-04-23T14:32:15.151820+0000","last_became_active":"2024-04-23T14:32:09.620393+0000","last_became_peered":"2024-04-23T14:32:09.620393+0000","last_unstale":"2024-04-23T14:32:15.151820+0000","last_undegraded":"2024-04-23T14:32:15.151820+0000","last_fullsized":"2024-04-23T14:32:15.151820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:36.859905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181533+0000","last_change":"2024-04-23T14:32:13.670681+0000","last_active":"2024-04-23T14:32:15.181533+0000","last_peered":"2024-04-23T14:32:15.181533+0000","last_clean":"2024-04-23T14:32:15.181533+0000","last_became_active":"2024-04-23T14:32:13.670304+0000","last_became_peered":"2024-04-23T14:32:13.670304+0000","last_unstale":"2024-04-23T14:32:15.181533+0000","last_undegraded":"2024-04-23T14:32:15.181533+0000","last_fullsized":"2024-04-23T14:32:15.181533+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:35:37.731501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641029+0000","last_change":"2024-04-23T14:32:09.617499+0000","last_active":"2024-04-23T14:32:14.641029+0000","last_peered":"2024-04-23T14:32:14.641029+0000","last_clean":"2024-04-23T14:32:14.641029+0000","last_became_active":"2024-04-23T14:32:09.617154+0000","last_became_peered":"2024-04-23T14:32:09.617154+0000","last_unstale":"2024-04-23T14:32:14.641029+0000","last_undegraded":"2024-04-23T14:32:14.641029+0000","last_fullsized":"2024-04-23T14:32:14.641029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:43:49.773005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181553+0000","last_change":"2024-04-23T14:32:09.619214+0000","last_active":"2024-04-23T14:32:15.181553+0000","last_peered":"2024-04-23T14:32:15.181553+0000","last_clean":"2024-04-23T14:32:15.181553+0000","last_became_active":"2024-04-23T14:32:09.618943+0000","last_became_peered":"2024-04-23T14:32:09.618943+0000","last_unstale":"2024-04-23T14:32:15.181553+0000","last_undegraded":"2024-04-23T14:32:15.181553+0000","last_fullsized":"2024-04-23T14:32:15.181553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:12.536442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640531+0000","last_change":"2024-04-23T14:32:09.626499+0000","last_active":"2024-04-23T14:32:14.640531+0000","last_peered":"2024-04-23T14:32:14.640531+0000","last_clean":"2024-04-23T14:32:14.640531+0000","last_became_active":"2024-04-23T14:32:09.626070+0000","last_became_peered":"2024-04-23T14:32:09.626070+0000","last_unstale":"2024-04-23T14:32:14.640531+0000","last_undegraded":"2024-04-23T14:32:14.640531+0000","last_fullsized":"2024-04-23T14:32:14.640531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:50.876901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640295+0000","last_change":"2024-04-23T14:32:09.625391+0000","last_active":"2024-04-23T14:32:14.640295+0000","last_peered":"2024-04-23T14:32:14.640295+0000","last_clean":"2024-04-23T14:32:14.640295+0000","last_became_active":"2024-04-23T14:32:09.625049+0000","last_became_peered":"2024-04-23T14:32:09.625049+0000","last_unstale":"2024-04-23T14:32:14.640295+0000","last_undegraded":"2024-04-23T14:32:14.640295+0000","last_fullsized":"2024-04-23T14:32:14.640295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:52:42.867377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182436+0000","last_change":"2024-04-23T14:32:09.627850+0000","last_active":"2024-04-23T14:32:15.182436+0000","last_peered":"2024-04-23T14:32:15.182436+0000","last_clean":"2024-04-23T14:32:15.182436+0000","last_became_active":"2024-04-23T14:32:09.627603+0000","last_became_peered":"2024-04-23T14:32:09.627603+0000","last_unstale":"2024-04-23T14:32:15.182436+0000","last_undegraded":"2024-04-23T14:32:15.182436+0000","last_fullsized":"2024-04-23T14:32:15.182436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:09.066755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182767+0000","last_change":"2024-04-23T14:32:09.630409+0000","last_active":"2024-04-23T14:32:15.182767+0000","last_peered":"2024-04-23T14:32:15.182767+0000","last_clean":"2024-04-23T14:32:15.182767+0000","last_became_active":"2024-04-23T14:32:09.630100+0000","last_became_peered":"2024-04-23T14:32:09.630100+0000","last_unstale":"2024-04-23T14:32:15.182767+0000","last_undegraded":"2024-04-23T14:32:15.182767+0000","last_fullsized":"2024-04-23T14:32:15.182767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:03.128923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641996+0000","last_change":"2024-04-23T14:32:09.631521+0000","last_active":"2024-04-23T14:32:14.641996+0000","last_peered":"2024-04-23T14:32:14.641996+0000","last_clean":"2024-04-23T14:32:14.641996+0000","last_became_active":"2024-04-23T14:32:09.631291+0000","last_became_peered":"2024-04-23T14:32:09.631291+0000","last_unstale":"2024-04-23T14:32:14.641996+0000","last_undegraded":"2024-04-23T14:32:14.641996+0000","last_fullsized":"2024-04-23T14:32:14.641996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:12.933947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182097+0000","last_change":"2024-04-23T14:32:09.642262+0000","last_active":"2024-04-23T14:32:15.182097+0000","last_peered":"2024-04-23T14:32:15.182097+0000","last_clean":"2024-04-23T14:32:15.182097+0000","last_became_active":"2024-04-23T14:32:09.641982+0000","last_became_peered":"2024-04-23T14:32:09.641982+0000","last_unstale":"2024-04-23T14:32:15.182097+0000","last_undegraded":"2024-04-23T14:32:15.182097+0000","last_fullsized":"2024-04-23T14:32:15.182097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:52.459061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639502+0000","last_change":"2024-04-23T14:32:09.613189+0000","last_active":"2024-04-23T14:32:14.639502+0000","last_peered":"2024-04-23T14:32:14.639502+0000","last_clean":"2024-04-23T14:32:14.639502+0000","last_became_active":"2024-04-23T14:32:09.612846+0000","last_became_peered":"2024-04-23T14:32:09.612846+0000","last_unstale":"2024-04-23T14:32:14.639502+0000","last_undegraded":"2024-04-23T14:32:14.639502+0000","last_fullsized":"2024-04-23T14:32:14.639502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:03.128923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181523+0000","last_change":"2024-04-23T14:32:09.642268+0000","last_active":"2024-04-23T14:32:15.181523+0000","last_peered":"2024-04-23T14:32:15.181523+0000","last_clean":"2024-04-23T14:32:15.181523+0000","last_became_active":"2024-04-23T14:32:09.642007+0000","last_became_peered":"2024-04-23T14:32:09.642007+0000","last_unstale":"2024-04-23T14:32:15.181523+0000","last_undegraded":"2024-04-23T14:32:15.181523+0000","last_fullsized":"2024-04-23T14:32:15.181523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:50:24.043980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640425+0000","last_change":"2024-04-23T14:32:09.626670+0000","last_active":"2024-04-23T14:32:14.640425+0000","last_peered":"2024-04-23T14:32:14.640425+0000","last_clean":"2024-04-23T14:32:14.640425+0000","last_became_active":"2024-04-23T14:32:09.625568+0000","last_became_peered":"2024-04-23T14:32:09.625568+0000","last_unstale":"2024-04-23T14:32:14.640425+0000","last_undegraded":"2024-04-23T14:32:14.640425+0000","last_fullsized":"2024-04-23T14:32:14.640425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:09.066755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639884+0000","last_change":"2024-04-23T14:32:09.627089+0000","last_active":"2024-04-23T14:32:14.639884+0000","last_peered":"2024-04-23T14:32:14.639884+0000","last_clean":"2024-04-23T14:32:14.639884+0000","last_became_active":"2024-04-23T14:32:09.626429+0000","last_became_peered":"2024-04-23T14:32:09.626429+0000","last_unstale":"2024-04-23T14:32:14.639884+0000","last_undegraded":"2024-04-23T14:32:14.639884+0000","last_fullsized":"2024-04-23T14:32:14.639884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:43:49.773005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641487+0000","last_change":"2024-04-23T14:32:09.643824+0000","last_active":"2024-04-23T14:32:14.641487+0000","last_peered":"2024-04-23T14:32:14.641487+0000","last_clean":"2024-04-23T14:32:14.641487+0000","last_became_active":"2024-04-23T14:32:09.643454+0000","last_became_peered":"2024-04-23T14:32:09.643454+0000","last_unstale":"2024-04-23T14:32:14.641487+0000","last_undegraded":"2024-04-23T14:32:14.641487+0000","last_fullsized":"2024-04-23T14:32:14.641487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:03.128923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179997+0000","last_change":"2024-04-23T14:32:09.627274+0000","last_active":"2024-04-23T14:32:15.179997+0000","last_peered":"2024-04-23T14:32:15.179997+0000","last_clean":"2024-04-23T14:32:15.179997+0000","last_became_active":"2024-04-23T14:32:09.626999+0000","last_became_peered":"2024-04-23T14:32:09.626999+0000","last_unstale":"2024-04-23T14:32:15.179997+0000","last_undegraded":"2024-04-23T14:32:15.179997+0000","last_fullsized":"2024-04-23T14:32:15.179997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:12:19.437109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182768+0000","last_change":"2024-04-23T14:32:09.615554+0000","last_active":"2024-04-23T14:32:15.182768+0000","last_peered":"2024-04-23T14:32:15.182768+0000","last_clean":"2024-04-23T14:32:15.182768+0000","last_became_active":"2024-04-23T14:32:09.614793+0000","last_became_peered":"2024-04-23T14:32:09.614793+0000","last_unstale":"2024-04-23T14:32:15.182768+0000","last_undegraded":"2024-04-23T14:32:15.182768+0000","last_fullsized":"2024-04-23T14:32:15.182768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:01.062394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640359+0000","last_change":"2024-04-23T14:32:09.625443+0000","last_active":"2024-04-23T14:32:14.640359+0000","last_peered":"2024-04-23T14:32:14.640359+0000","last_clean":"2024-04-23T14:32:14.640359+0000","last_became_active":"2024-04-23T14:32:09.625140+0000","last_became_peered":"2024-04-23T14:32:09.625140+0000","last_unstale":"2024-04-23T14:32:14.640359+0000","last_undegraded":"2024-04-23T14:32:14.640359+0000","last_fullsized":"2024-04-23T14:32:14.640359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:12:19.437109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640042+0000","last_change":"2024-04-23T14:32:09.614187+0000","last_active":"2024-04-23T14:32:14.640042+0000","last_peered":"2024-04-23T14:32:14.640042+0000","last_clean":"2024-04-23T14:32:14.640042+0000","last_became_active":"2024-04-23T14:32:09.613813+0000","last_became_peered":"2024-04-23T14:32:09.613813+0000","last_unstale":"2024-04-23T14:32:14.640042+0000","last_undegraded":"2024-04-23T14:32:14.640042+0000","last_fullsized":"2024-04-23T14:32:14.640042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:12.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180552+0000","last_change":"2024-04-23T14:32:09.630660+0000","last_active":"2024-04-23T14:32:15.180552+0000","last_peered":"2024-04-23T14:32:15.180552+0000","last_clean":"2024-04-23T14:32:15.180552+0000","last_became_active":"2024-04-23T14:32:09.630381+0000","last_became_peered":"2024-04-23T14:32:09.630381+0000","last_unstale":"2024-04-23T14:32:15.180552+0000","last_undegraded":"2024-04-23T14:32:15.180552+0000","last_fullsized":"2024-04-23T14:32:15.180552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:12.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639936+0000","last_change":"2024-04-23T14:32:09.613974+0000","last_active":"2024-04-23T14:32:14.639936+0000","last_peered":"2024-04-23T14:32:14.639936+0000","last_clean":"2024-04-23T14:32:14.639936+0000","last_became_active":"2024-04-23T14:32:09.613699+0000","last_became_peered":"2024-04-23T14:32:09.613699+0000","last_unstale":"2024-04-23T14:32:14.639936+0000","last_undegraded":"2024-04-23T14:32:14.639936+0000","last_fullsized":"2024-04-23T14:32:14.639936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:08:31.155490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180930+0000","last_change":"2024-04-23T14:32:09.622174+0000","last_active":"2024-04-23T14:32:15.180930+0000","last_peered":"2024-04-23T14:32:15.180930+0000","last_clean":"2024-04-23T14:32:15.180930+0000","last_became_active":"2024-04-23T14:32:09.621360+0000","last_became_peered":"2024-04-23T14:32:09.621360+0000","last_unstale":"2024-04-23T14:32:15.180930+0000","last_undegraded":"2024-04-23T14:32:15.180930+0000","last_fullsized":"2024-04-23T14:32:15.180930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:12.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152292+0000","last_change":"2024-04-23T14:32:09.625529+0000","last_active":"2024-04-23T14:32:15.152292+0000","last_peered":"2024-04-23T14:32:15.152292+0000","last_clean":"2024-04-23T14:32:15.152292+0000","last_became_active":"2024-04-23T14:32:09.625259+0000","last_became_peered":"2024-04-23T14:32:09.625259+0000","last_unstale":"2024-04-23T14:32:15.152292+0000","last_undegraded":"2024-04-23T14:32:15.152292+0000","last_fullsized":"2024-04-23T14:32:15.152292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:12.536442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641776+0000","last_change":"2024-04-23T14:32:09.634616+0000","last_active":"2024-04-23T14:32:14.641776+0000","last_peered":"2024-04-23T14:32:14.641776+0000","last_clean":"2024-04-23T14:32:14.641776+0000","last_became_active":"2024-04-23T14:32:09.634328+0000","last_became_peered":"2024-04-23T14:32:09.634328+0000","last_unstale":"2024-04-23T14:32:14.641776+0000","last_undegraded":"2024-04-23T14:32:14.641776+0000","last_fullsized":"2024-04-23T14:32:14.641776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:52:42.867377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641980+0000","last_change":"2024-04-23T14:32:09.618704+0000","last_active":"2024-04-23T14:32:14.641980+0000","last_peered":"2024-04-23T14:32:14.641980+0000","last_clean":"2024-04-23T14:32:14.641980+0000","last_became_active":"2024-04-23T14:32:09.618328+0000","last_became_peered":"2024-04-23T14:32:09.618328+0000","last_unstale":"2024-04-23T14:32:14.641980+0000","last_undegraded":"2024-04-23T14:32:14.641980+0000","last_fullsized":"2024-04-23T14:32:14.641980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:50:24.043980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180068+0000","last_change":"2024-04-23T14:32:09.637988+0000","last_active":"2024-04-23T14:32:15.180068+0000","last_peered":"2024-04-23T14:32:15.180068+0000","last_clean":"2024-04-23T14:32:15.180068+0000","last_became_active":"2024-04-23T14:32:09.637766+0000","last_became_peered":"2024-04-23T14:32:09.637766+0000","last_unstale":"2024-04-23T14:32:15.180068+0000","last_undegraded":"2024-04-23T14:32:15.180068+0000","last_fullsized":"2024-04-23T14:32:15.180068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:09:07.994350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180933+0000","last_change":"2024-04-23T14:32:09.623580+0000","last_active":"2024-04-23T14:32:15.180933+0000","last_peered":"2024-04-23T14:32:15.180933+0000","last_clean":"2024-04-23T14:32:15.180933+0000","last_became_active":"2024-04-23T14:32:09.622942+0000","last_became_peered":"2024-04-23T14:32:09.622942+0000","last_unstale":"2024-04-23T14:32:15.180933+0000","last_undegraded":"2024-04-23T14:32:15.180933+0000","last_fullsized":"2024-04-23T14:32:15.180933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:52:42.867377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179379+0000","last_change":"2024-04-23T14:32:09.624523+0000","last_active":"2024-04-23T14:32:15.179379+0000","last_peered":"2024-04-23T14:32:15.179379+0000","last_clean":"2024-04-23T14:32:15.179379+0000","last_became_active":"2024-04-23T14:32:09.624229+0000","last_became_peered":"2024-04-23T14:32:09.624229+0000","last_unstale":"2024-04-23T14:32:15.179379+0000","last_undegraded":"2024-04-23T14:32:15.179379+0000","last_fullsized":"2024-04-23T14:32:15.179379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:09.066755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640175+0000","last_change":"2024-04-23T14:32:09.614929+0000","last_active":"2024-04-23T14:32:14.640175+0000","last_peered":"2024-04-23T14:32:14.640175+0000","last_clean":"2024-04-23T14:32:14.640175+0000","last_became_active":"2024-04-23T14:32:09.614584+0000","last_became_peered":"2024-04-23T14:32:09.614584+0000","last_unstale":"2024-04-23T14:32:14.640175+0000","last_undegraded":"2024-04-23T14:32:14.640175+0000","last_fullsized":"2024-04-23T14:32:14.640175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:22.220286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179498+0000","last_change":"2024-04-23T14:32:09.623580+0000","last_active":"2024-04-23T14:32:15.179498+0000","last_peered":"2024-04-23T14:32:15.179498+0000","last_clean":"2024-04-23T14:32:15.179498+0000","last_became_active":"2024-04-23T14:32:09.623197+0000","last_became_peered":"2024-04-23T14:32:09.623197+0000","last_unstale":"2024-04-23T14:32:15.179498+0000","last_undegraded":"2024-04-23T14:32:15.179498+0000","last_fullsized":"2024-04-23T14:32:15.179498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:52.459061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640154+0000","last_change":"2024-04-23T14:32:09.622573+0000","last_active":"2024-04-23T14:32:14.640154+0000","last_peered":"2024-04-23T14:32:14.640154+0000","last_clean":"2024-04-23T14:32:14.640154+0000","last_became_active":"2024-04-23T14:32:09.622232+0000","last_became_peered":"2024-04-23T14:32:09.622232+0000","last_unstale":"2024-04-23T14:32:14.640154+0000","last_undegraded":"2024-04-23T14:32:14.640154+0000","last_fullsized":"2024-04-23T14:32:14.640154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:52.459061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639881+0000","last_change":"2024-04-23T14:32:09.604425+0000","last_active":"2024-04-23T14:32:14.639881+0000","last_peered":"2024-04-23T14:32:14.639881+0000","last_clean":"2024-04-23T14:32:14.639881+0000","last_became_active":"2024-04-23T14:32:09.604159+0000","last_became_peered":"2024-04-23T14:32:09.604159+0000","last_unstale":"2024-04-23T14:32:14.639881+0000","last_undegraded":"2024-04-23T14:32:14.639881+0000","last_fullsized":"2024-04-23T14:32:14.639881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:50:24.043980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.183079+0000","last_change":"2024-04-23T14:32:09.614842+0000","last_active":"2024-04-23T14:32:15.183079+0000","last_peered":"2024-04-23T14:32:15.183079+0000","last_clean":"2024-04-23T14:32:15.183079+0000","last_became_active":"2024-04-23T14:32:09.614385+0000","last_became_peered":"2024-04-23T14:32:09.614385+0000","last_unstale":"2024-04-23T14:32:15.183079+0000","last_undegraded":"2024-04-23T14:32:15.183079+0000","last_fullsized":"2024-04-23T14:32:15.183079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:22.220286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179991+0000","last_change":"2024-04-23T14:32:09.628001+0000","last_active":"2024-04-23T14:32:15.179991+0000","last_peered":"2024-04-23T14:32:15.179991+0000","last_clean":"2024-04-23T14:32:15.179991+0000","last_became_active":"2024-04-23T14:32:09.627722+0000","last_became_peered":"2024-04-23T14:32:09.627722+0000","last_unstale":"2024-04-23T14:32:15.179991+0000","last_undegraded":"2024-04-23T14:32:15.179991+0000","last_fullsized":"2024-04-23T14:32:15.179991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:14:12.731737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:32:35.757 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:32:35.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.927+0000 7f4b23400640 1 Processor -- start 2024-04-23T14:32:35.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.927+0000 7f4b23400640 1 -- start start 2024-04-23T14:32:35.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.931+0000 7f4b23400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b1c15b640 0x7f4b1c0adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.931+0000 7f4b23400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b1c0af080 con 0x7f4b1c073610 2024-04-23T14:32:35.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.931+0000 7f4b23400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b1c0afbc0 con 0x7f4b1c0b2a10 2024-04-23T14:32:35.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.931+0000 7f4b23400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f4b1c15ba20 con 0x7f4b1c15b640 2024-04-23T14:32:35.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.931+0000 7f4b20a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b1c15b640 0x7f4b1c0adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.931+0000 7f4b20a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b1c15b640 0x7f4b1c0adfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57274/0 (socket says 172.21.15.100:57274) 2024-04-23T14:32:35.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.931+0000 7f4b20a00640 1 -- 172.21.15.100:0/2430786408 learned_addr learned my addr 172.21.15.100:0/2430786408 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:35.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.931+0000 7f4b21400640 1 --1- 172.21.15.100:0/2430786408 >> v1:172.21.15.100:6789/0 conn(0x7f4b1c073610 0x7f4b1c06df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57196/0 (socket says 172.21.15.100:57196) 2024-04-23T14:32:35.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.931+0000 7f4b20a00640 1 -- 172.21.15.100:0/2430786408 >> v1:172.21.15.151:6789/0 conn(0x7f4b1c0b2a10 legacy=0x7f4b1c072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:35.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.931+0000 7f4b20a00640 1 -- 172.21.15.100:0/2430786408 >> v1:172.21.15.100:6789/0 conn(0x7f4b1c073610 legacy=0x7f4b1c06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.931+0000 7f4b20a00640 1 -- 172.21.15.100:0/2430786408 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1c0af320 con 0x7f4b1c15b640 2024-04-23T14:32:35.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.935+0000 7f4b20a00640 1 --2- 172.21.15.100:0/2430786408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b1c15b640 0x7f4b1c0adfc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009870 tx=0x7f4b1001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10afdf5c52233902 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:35.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.935+0000 7f4b1b400640 1 -- 172.21.15.100:0/2430786408 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b100ab660 con 0x7f4b1c15b640 2024-04-23T14:32:35.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.935+0000 7f4b1b400640 1 -- 172.21.15.100:0/2430786408 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b100b0070 con 0x7f4b1c15b640 2024-04-23T14:32:35.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.935+0000 7f4b1b400640 1 -- 172.21.15.100:0/2430786408 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b100aba20 con 0x7f4b1c15b640 2024-04-23T14:32:35.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.935+0000 7f4b23400640 1 -- 172.21.15.100:0/2430786408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b1c15b640 msgr2=0x7f4b1c0adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.935+0000 7f4b23400640 1 --2- 172.21.15.100:0/2430786408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b1c15b640 0x7f4b1c0adfc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009870 tx=0x7f4b1001c460 comp rx=0 tx=0).stop 2024-04-23T14:32:35.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.935+0000 7f4b23400640 1 -- 172.21.15.100:0/2430786408 shutdown_connections 2024-04-23T14:32:35.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.935+0000 7f4b23400640 1 --2- 172.21.15.100:0/2430786408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b1c15b640 0x7f4b1c0adfc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.935+0000 7f4b23400640 1 -- 172.21.15.100:0/2430786408 >> 172.21.15.100:0/2430786408 conn(0x7f4b1c0b1e80 msgr2=0x7f4b1c0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:35.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.935+0000 7f4b23400640 1 -- 172.21.15.100:0/2430786408 shutdown_connections 2024-04-23T14:32:35.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.935+0000 7f4b23400640 1 -- 172.21.15.100:0/2430786408 wait complete. 2024-04-23T14:32:35.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.935+0000 7f4b23400640 1 Processor -- start 2024-04-23T14:32:35.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.935+0000 7f4b23400640 1 -- start start 2024-04-23T14:32:35.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.935+0000 7f4b23400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b1c073610 0x7f4b1c14b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.935+0000 7f4b23400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b1c0b2a10 0x7f4b1c1515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.935+0000 7f4b23400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b1c15b640 0x7f4b1c151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.935+0000 7f4b23400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c1691c0 con 0x7f4b1c0b2a10 2024-04-23T14:32:35.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.935+0000 7f4b23400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c06f160 con 0x7f4b1c15b640 2024-04-23T14:32:35.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.935+0000 7f4b23400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4b1c169340 con 0x7f4b1c073610 2024-04-23T14:32:35.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.935+0000 7f4b1be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b1c0b2a10 0x7f4b1c1515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.939+0000 7f4b1be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b1c0b2a10 0x7f4b1c1515a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57278/0 (socket says 172.21.15.100:57278) 2024-04-23T14:32:35.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.939+0000 7f4b1be00640 1 -- 172.21.15.100:0/1010424486 learned_addr learned my addr 172.21.15.100:0/1010424486 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:35.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.939+0000 7f4b20a00640 1 --2- 172.21.15.100:0/1010424486 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b1c073610 0x7f4b1c14b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.939+0000 7f4b21400640 1 --2- 172.21.15.100:0/1010424486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b1c15b640 0x7f4b1c151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.939+0000 7f4b1be00640 1 -- 172.21.15.100:0/1010424486 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b1c073610 msgr2=0x7f4b1c14b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.939+0000 7f4b1be00640 1 --2- 172.21.15.100:0/1010424486 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b1c073610 0x7f4b1c14b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.939+0000 7f4b1be00640 1 -- 172.21.15.100:0/1010424486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b1c15b640 msgr2=0x7f4b1c151ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.939+0000 7f4b1be00640 1 --2- 172.21.15.100:0/1010424486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b1c15b640 0x7f4b1c151ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.939+0000 7f4b1be00640 1 -- 172.21.15.100:0/1010424486 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1c15e810 con 0x7f4b1c0b2a10 2024-04-23T14:32:35.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.939+0000 7f4b1be00640 1 --2- 172.21.15.100:0/1010424486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b1c0b2a10 0x7f4b1c1515a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4b0800c430 tx=0x7f4b0800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:35.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.939+0000 7f4b18c00640 1 -- 172.21.15.100:0/1010424486 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b08007020 con 0x7f4b1c0b2a10 2024-04-23T14:32:35.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.939+0000 7f4b18c00640 1 -- 172.21.15.100:0/1010424486 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b080071c0 con 0x7f4b1c0b2a10 2024-04-23T14:32:35.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.939+0000 7f4b18c00640 1 -- 172.21.15.100:0/1010424486 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b080075b0 con 0x7f4b1c0b2a10 2024-04-23T14:32:35.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.939+0000 7f4b23400640 1 -- 172.21.15.100:0/1010424486 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b1c14d310 con 0x7f4b1c0b2a10 2024-04-23T14:32:35.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.939+0000 7f4b23400640 1 -- 172.21.15.100:0/1010424486 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b1c0cf8d0 con 0x7f4b1c0b2a10 2024-04-23T14:32:35.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.939+0000 7f4b23400640 1 -- 172.21.15.100:0/1010424486 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b1c068ad0 con 0x7f4b1c0b2a10 2024-04-23T14:32:35.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.939+0000 7f4b18c00640 1 -- 172.21.15.100:0/1010424486 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4b08019020 con 0x7f4b1c0b2a10 2024-04-23T14:32:35.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.939+0000 7f4b18c00640 1 --2- 172.21.15.100:0/1010424486 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4afc068380 0x7f4afc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.939+0000 7f4b18c00640 1 -- 172.21.15.100:0/1010424486 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4b08086370 con 0x7f4b1c0b2a10 2024-04-23T14:32:35.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.939+0000 7f4b20a00640 1 --2- 172.21.15.100:0/1010424486 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4afc068380 0x7f4afc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.943+0000 7f4b20a00640 1 --2- 172.21.15.100:0/1010424486 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4afc068380 0x7f4afc06a840 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4b100af000 tx=0x7f4b100a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:35.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:35.943+0000 7f4b18c00640 1 -- 172.21.15.100:0/1010424486 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4b08056680 con 0x7f4b1c0b2a10 2024-04-23T14:32:36.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.099+0000 7f4b23400640 1 -- 172.21.15.100:0/1010424486 --> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b1c0b0860 con 0x7f4afc068380 2024-04-23T14:32:36.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.107+0000 7f4b18c00640 1 -- 172.21.15.100:0/1010424486 <== mgr.4101 v2:172.21.15.100:6832/2859853957 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332537 (secure 0 0 0) 0x7f4b1c0b0860 con 0x7f4afc068380 2024-04-23T14:32:36.111 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:36.113 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-04-23T14:32:36.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.111+0000 7f4b23400640 1 -- 172.21.15.100:0/1010424486 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4afc068380 msgr2=0x7f4afc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.111+0000 7f4b23400640 1 --2- 172.21.15.100:0/1010424486 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4afc068380 0x7f4afc06a840 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4b100af000 tx=0x7f4b100a7000 comp rx=0 tx=0).stop 2024-04-23T14:32:36.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.111+0000 7f4b23400640 1 -- 172.21.15.100:0/1010424486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b1c0b2a10 msgr2=0x7f4b1c1515a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.111+0000 7f4b23400640 1 --2- 172.21.15.100:0/1010424486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b1c0b2a10 0x7f4b1c1515a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4b0800c430 tx=0x7f4b0800c8f0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.111+0000 7f4b23400640 1 -- 172.21.15.100:0/1010424486 shutdown_connections 2024-04-23T14:32:36.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.111+0000 7f4b23400640 1 --2- 172.21.15.100:0/1010424486 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4afc068380 0x7f4afc06a840 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.111+0000 7f4b23400640 1 --2- 172.21.15.100:0/1010424486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b1c15b640 0x7f4b1c151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.111+0000 7f4b23400640 1 --2- 172.21.15.100:0/1010424486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b1c0b2a10 0x7f4b1c1515a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.111+0000 7f4b23400640 1 --2- 172.21.15.100:0/1010424486 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b1c073610 0x7f4b1c14b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.111+0000 7f4b23400640 1 -- 172.21.15.100:0/1010424486 >> 172.21.15.100:0/1010424486 conn(0x7f4b1c0b1e80 msgr2=0x7f4b1c06c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:36.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.111+0000 7f4b23400640 1 -- 172.21.15.100:0/1010424486 shutdown_connections 2024-04-23T14:32:36.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.111+0000 7f4b23400640 1 -- 172.21.15.100:0/1010424486 wait complete. 2024-04-23T14:32:36.131 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-23T14:32:34.740754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219376,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755344,"statfs":{"total":773094113280,"available":772869472256,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002942"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640233+0000","last_change":"2024-04-23T14:32:08.611525+0000","last_active":"2024-04-23T14:32:14.640233+0000","last_peered":"2024-04-23T14:32:14.640233+0000","last_clean":"2024-04-23T14:32:14.640233+0000","last_became_active":"2024-04-23T14:32:08.611163+0000","last_became_peered":"2024-04-23T14:32:08.611163+0000","last_unstale":"2024-04-23T14:32:14.640233+0000","last_undegraded":"2024-04-23T14:32:14.640233+0000","last_fullsized":"2024-04-23T14:32:14.640233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:08:50.925677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181653+0000","last_change":"2024-04-23T14:32:09.625243+0000","last_active":"2024-04-23T14:32:15.181653+0000","last_peered":"2024-04-23T14:32:15.181653+0000","last_clean":"2024-04-23T14:32:15.181653+0000","last_became_active":"2024-04-23T14:32:09.624765+0000","last_became_peered":"2024-04-23T14:32:09.624765+0000","last_unstale":"2024-04-23T14:32:15.181653+0000","last_undegraded":"2024-04-23T14:32:15.181653+0000","last_fullsized":"2024-04-23T14:32:15.181653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:50.876901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181363+0000","last_change":"2024-04-23T14:32:13.779583+0000","last_active":"2024-04-23T14:32:15.181363+0000","last_peered":"2024-04-23T14:32:15.181363+0000","last_clean":"2024-04-23T14:32:15.181363+0000","last_became_active":"2024-04-23T14:32:13.779224+0000","last_became_peered":"2024-04-23T14:32:13.779224+0000","last_unstale":"2024-04-23T14:32:15.181363+0000","last_undegraded":"2024-04-23T14:32:15.181363+0000","last_fullsized":"2024-04-23T14:32:15.181363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:55.041747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181862+0000","last_change":"2024-04-23T14:32:08.587781+0000","last_active":"2024-04-23T14:32:15.181862+0000","last_peered":"2024-04-23T14:32:15.181862+0000","last_clean":"2024-04-23T14:32:15.181862+0000","last_became_active":"2024-04-23T14:32:08.587435+0000","last_became_peered":"2024-04-23T14:32:08.587435+0000","last_unstale":"2024-04-23T14:32:15.181862+0000","last_undegraded":"2024-04-23T14:32:15.181862+0000","last_fullsized":"2024-04-23T14:32:15.181862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:51.650822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640102+0000","last_change":"2024-04-23T14:32:09.621038+0000","last_active":"2024-04-23T14:32:14.640102+0000","last_peered":"2024-04-23T14:32:14.640102+0000","last_clean":"2024-04-23T14:32:14.640102+0000","last_became_active":"2024-04-23T14:32:09.620776+0000","last_became_peered":"2024-04-23T14:32:09.620776+0000","last_unstale":"2024-04-23T14:32:14.640102+0000","last_undegraded":"2024-04-23T14:32:14.640102+0000","last_fullsized":"2024-04-23T14:32:14.640102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:12.933947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151642+0000","last_change":"2024-04-23T14:32:13.671138+0000","last_active":"2024-04-23T14:32:15.151642+0000","last_peered":"2024-04-23T14:32:15.151642+0000","last_clean":"2024-04-23T14:32:15.151642+0000","last_became_active":"2024-04-23T14:32:13.670514+0000","last_became_peered":"2024-04-23T14:32:13.670514+0000","last_unstale":"2024-04-23T14:32:15.151642+0000","last_undegraded":"2024-04-23T14:32:15.151642+0000","last_fullsized":"2024-04-23T14:32:15.151642+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:26.271219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179964+0000","last_change":"2024-04-23T14:32:08.616316+0000","last_active":"2024-04-23T14:32:15.179964+0000","last_peered":"2024-04-23T14:32:15.179964+0000","last_clean":"2024-04-23T14:32:15.179964+0000","last_became_active":"2024-04-23T14:32:08.616020+0000","last_became_peered":"2024-04-23T14:32:08.616020+0000","last_unstale":"2024-04-23T14:32:15.179964+0000","last_undegraded":"2024-04-23T14:32:15.179964+0000","last_fullsized":"2024-04-23T14:32:15.179964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:53:40.004436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640291+0000","last_change":"2024-04-23T14:32:09.635504+0000","last_active":"2024-04-23T14:32:14.640291+0000","last_peered":"2024-04-23T14:32:14.640291+0000","last_clean":"2024-04-23T14:32:14.640291+0000","last_became_active":"2024-04-23T14:32:09.635179+0000","last_became_peered":"2024-04-23T14:32:09.635179+0000","last_unstale":"2024-04-23T14:32:14.640291+0000","last_undegraded":"2024-04-23T14:32:14.640291+0000","last_fullsized":"2024-04-23T14:32:14.640291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:56.096479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151795+0000","last_change":"2024-04-23T14:32:13.667280+0000","last_active":"2024-04-23T14:32:15.151795+0000","last_peered":"2024-04-23T14:32:15.151795+0000","last_clean":"2024-04-23T14:32:15.151795+0000","last_became_active":"2024-04-23T14:32:13.666861+0000","last_became_peered":"2024-04-23T14:32:13.666861+0000","last_unstale":"2024-04-23T14:32:15.151795+0000","last_undegraded":"2024-04-23T14:32:15.151795+0000","last_fullsized":"2024-04-23T14:32:15.151795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:56.509994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179707+0000","last_change":"2024-04-23T14:32:08.615668+0000","last_active":"2024-04-23T14:32:15.179707+0000","last_peered":"2024-04-23T14:32:15.179707+0000","last_clean":"2024-04-23T14:32:15.179707+0000","last_became_active":"2024-04-23T14:32:08.615384+0000","last_became_peered":"2024-04-23T14:32:08.615384+0000","last_unstale":"2024-04-23T14:32:15.179707+0000","last_undegraded":"2024-04-23T14:32:15.179707+0000","last_fullsized":"2024-04-23T14:32:15.179707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:11.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640191+0000","last_change":"2024-04-23T14:32:09.634014+0000","last_active":"2024-04-23T14:32:14.640191+0000","last_peered":"2024-04-23T14:32:14.640191+0000","last_clean":"2024-04-23T14:32:14.640191+0000","last_became_active":"2024-04-23T14:32:09.633633+0000","last_became_peered":"2024-04-23T14:32:09.633633+0000","last_unstale":"2024-04-23T14:32:14.640191+0000","last_undegraded":"2024-04-23T14:32:14.640191+0000","last_fullsized":"2024-04-23T14:32:14.640191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:47:31.805399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151732+0000","last_change":"2024-04-23T14:32:13.661944+0000","last_active":"2024-04-23T14:32:15.151732+0000","last_peered":"2024-04-23T14:32:15.151732+0000","last_clean":"2024-04-23T14:32:15.151732+0000","last_became_active":"2024-04-23T14:32:13.661584+0000","last_became_peered":"2024-04-23T14:32:13.661584+0000","last_unstale":"2024-04-23T14:32:15.151732+0000","last_undegraded":"2024-04-23T14:32:15.151732+0000","last_fullsized":"2024-04-23T14:32:15.151732+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:54.927834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641604+0000","last_change":"2024-04-23T14:32:09.645728+0000","last_active":"2024-04-23T14:32:14.641604+0000","last_peered":"2024-04-23T14:32:14.641604+0000","last_clean":"2024-04-23T14:32:14.641604+0000","last_became_active":"2024-04-23T14:32:09.641089+0000","last_became_peered":"2024-04-23T14:32:09.641089+0000","last_unstale":"2024-04-23T14:32:14.641604+0000","last_undegraded":"2024-04-23T14:32:14.641604+0000","last_fullsized":"2024-04-23T14:32:14.641604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:53.674486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179794+0000","last_change":"2024-04-23T14:32:08.616537+0000","last_active":"2024-04-23T14:32:15.179794+0000","last_peered":"2024-04-23T14:32:15.179794+0000","last_clean":"2024-04-23T14:32:15.179794+0000","last_became_active":"2024-04-23T14:32:08.616145+0000","last_became_peered":"2024-04-23T14:32:08.616145+0000","last_unstale":"2024-04-23T14:32:15.179794+0000","last_undegraded":"2024-04-23T14:32:15.179794+0000","last_fullsized":"2024-04-23T14:32:15.179794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:00.056567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152246+0000","last_change":"2024-04-23T14:32:13.667224+0000","last_active":"2024-04-23T14:32:15.152246+0000","last_peered":"2024-04-23T14:32:15.152246+0000","last_clean":"2024-04-23T14:32:15.152246+0000","last_became_active":"2024-04-23T14:32:13.666816+0000","last_became_peered":"2024-04-23T14:32:13.666816+0000","last_unstale":"2024-04-23T14:32:15.152246+0000","last_undegraded":"2024-04-23T14:32:15.152246+0000","last_fullsized":"2024-04-23T14:32:15.152246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:22.418010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181037+0000","last_change":"2024-04-23T14:32:09.622489+0000","last_active":"2024-04-23T14:32:15.181037+0000","last_peered":"2024-04-23T14:32:15.181037+0000","last_clean":"2024-04-23T14:32:15.181037+0000","last_became_active":"2024-04-23T14:32:09.617206+0000","last_became_peered":"2024-04-23T14:32:09.617206+0000","last_unstale":"2024-04-23T14:32:15.181037+0000","last_undegraded":"2024-04-23T14:32:15.181037+0000","last_fullsized":"2024-04-23T14:32:15.181037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:03.128923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182693+0000","last_change":"2024-04-23T14:32:08.600628+0000","last_active":"2024-04-23T14:32:15.182693+0000","last_peered":"2024-04-23T14:32:15.182693+0000","last_clean":"2024-04-23T14:32:15.182693+0000","last_became_active":"2024-04-23T14:32:08.600318+0000","last_became_peered":"2024-04-23T14:32:08.600318+0000","last_unstale":"2024-04-23T14:32:15.182693+0000","last_undegraded":"2024-04-23T14:32:15.182693+0000","last_fullsized":"2024-04-23T14:32:15.182693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:54.495261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152192+0000","last_change":"2024-04-23T14:32:13.661586+0000","last_active":"2024-04-23T14:32:15.152192+0000","last_peered":"2024-04-23T14:32:15.152192+0000","last_clean":"2024-04-23T14:32:15.152192+0000","last_became_active":"2024-04-23T14:32:13.660406+0000","last_became_peered":"2024-04-23T14:32:13.660406+0000","last_unstale":"2024-04-23T14:32:15.152192+0000","last_undegraded":"2024-04-23T14:32:15.152192+0000","last_fullsized":"2024-04-23T14:32:15.152192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:47:35.856332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181164+0000","last_change":"2024-04-23T14:32:09.627921+0000","last_active":"2024-04-23T14:32:15.181164+0000","last_peered":"2024-04-23T14:32:15.181164+0000","last_clean":"2024-04-23T14:32:15.181164+0000","last_became_active":"2024-04-23T14:32:09.622420+0000","last_became_peered":"2024-04-23T14:32:09.622420+0000","last_unstale":"2024-04-23T14:32:15.181164+0000","last_undegraded":"2024-04-23T14:32:15.181164+0000","last_fullsized":"2024-04-23T14:32:15.181164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:53.674486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180797+0000","last_change":"2024-04-23T14:32:08.612306+0000","last_active":"2024-04-23T14:32:15.180797+0000","last_peered":"2024-04-23T14:32:15.180797+0000","last_clean":"2024-04-23T14:32:15.180797+0000","last_became_active":"2024-04-23T14:32:08.612018+0000","last_became_peered":"2024-04-23T14:32:08.612018+0000","last_unstale":"2024-04-23T14:32:15.180797+0000","last_undegraded":"2024-04-23T14:32:15.180797+0000","last_fullsized":"2024-04-23T14:32:15.180797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:59:03.639740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.183304+0000","last_change":"2024-04-23T14:32:13.780668+0000","last_active":"2024-04-23T14:32:15.183304+0000","last_peered":"2024-04-23T14:32:15.183304+0000","last_clean":"2024-04-23T14:32:15.183304+0000","last_became_active":"2024-04-23T14:32:13.780323+0000","last_became_peered":"2024-04-23T14:32:13.780323+0000","last_unstale":"2024-04-23T14:32:15.183304+0000","last_undegraded":"2024-04-23T14:32:15.183304+0000","last_fullsized":"2024-04-23T14:32:15.183304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:12:23.488042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151992+0000","last_change":"2024-04-23T14:32:09.627663+0000","last_active":"2024-04-23T14:32:15.151992+0000","last_peered":"2024-04-23T14:32:15.151992+0000","last_clean":"2024-04-23T14:32:15.151992+0000","last_became_active":"2024-04-23T14:32:09.620464+0000","last_became_peered":"2024-04-23T14:32:09.620464+0000","last_unstale":"2024-04-23T14:32:15.151992+0000","last_undegraded":"2024-04-23T14:32:15.151992+0000","last_fullsized":"2024-04-23T14:32:15.151992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:52.656649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179300+0000","last_change":"2024-04-23T14:32:08.612870+0000","last_active":"2024-04-23T14:32:15.179300+0000","last_peered":"2024-04-23T14:32:15.179300+0000","last_clean":"2024-04-23T14:32:15.179300+0000","last_became_active":"2024-04-23T14:32:08.610830+0000","last_became_peered":"2024-04-23T14:32:08.610830+0000","last_unstale":"2024-04-23T14:32:15.179300+0000","last_undegraded":"2024-04-23T14:32:15.179300+0000","last_fullsized":"2024-04-23T14:32:15.179300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:51.650822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152268+0000","last_change":"2024-04-23T14:32:13.660771+0000","last_active":"2024-04-23T14:32:15.152268+0000","last_peered":"2024-04-23T14:32:15.152268+0000","last_clean":"2024-04-23T14:32:15.152268+0000","last_became_active":"2024-04-23T14:32:13.660291+0000","last_became_peered":"2024-04-23T14:32:13.660291+0000","last_unstale":"2024-04-23T14:32:15.152268+0000","last_undegraded":"2024-04-23T14:32:15.152268+0000","last_fullsized":"2024-04-23T14:32:15.152268+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:28.688885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640262+0000","last_change":"2024-04-23T14:32:09.612595+0000","last_active":"2024-04-23T14:32:14.640262+0000","last_peered":"2024-04-23T14:32:14.640262+0000","last_clean":"2024-04-23T14:32:14.640262+0000","last_became_active":"2024-04-23T14:32:09.612168+0000","last_became_peered":"2024-04-23T14:32:09.612168+0000","last_unstale":"2024-04-23T14:32:14.640262+0000","last_undegraded":"2024-04-23T14:32:14.640262+0000","last_fullsized":"2024-04-23T14:32:14.640262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:03.128923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.642236+0000","last_change":"2024-04-23T14:32:08.615099+0000","last_active":"2024-04-23T14:32:14.642236+0000","last_peered":"2024-04-23T14:32:14.642236+0000","last_clean":"2024-04-23T14:32:14.642236+0000","last_became_active":"2024-04-23T14:32:08.614350+0000","last_became_peered":"2024-04-23T14:32:08.614350+0000","last_unstale":"2024-04-23T14:32:14.642236+0000","last_undegraded":"2024-04-23T14:32:14.642236+0000","last_fullsized":"2024-04-23T14:32:14.642236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:59:03.639740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151716+0000","last_change":"2024-04-23T14:32:13.775674+0000","last_active":"2024-04-23T14:32:15.151716+0000","last_peered":"2024-04-23T14:32:15.151716+0000","last_clean":"2024-04-23T14:32:15.151716+0000","last_became_active":"2024-04-23T14:32:13.774695+0000","last_became_peered":"2024-04-23T14:32:13.774695+0000","last_unstale":"2024-04-23T14:32:15.151716+0000","last_undegraded":"2024-04-23T14:32:15.151716+0000","last_fullsized":"2024-04-23T14:32:15.151716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:13.117688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182184+0000","last_change":"2024-04-23T14:32:09.630421+0000","last_active":"2024-04-23T14:32:15.182184+0000","last_peered":"2024-04-23T14:32:15.182184+0000","last_clean":"2024-04-23T14:32:15.182184+0000","last_became_active":"2024-04-23T14:32:09.630104+0000","last_became_peered":"2024-04-23T14:32:09.630104+0000","last_unstale":"2024-04-23T14:32:15.182184+0000","last_undegraded":"2024-04-23T14:32:15.182184+0000","last_fullsized":"2024-04-23T14:32:15.182184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:56.096479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.183107+0000","last_change":"2024-04-23T14:32:08.607576+0000","last_active":"2024-04-23T14:32:15.183107+0000","last_peered":"2024-04-23T14:32:15.183107+0000","last_clean":"2024-04-23T14:32:15.183107+0000","last_became_active":"2024-04-23T14:32:08.606611+0000","last_became_peered":"2024-04-23T14:32:08.606611+0000","last_unstale":"2024-04-23T14:32:15.183107+0000","last_undegraded":"2024-04-23T14:32:15.183107+0000","last_fullsized":"2024-04-23T14:32:15.183107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:00.056567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180827+0000","last_change":"2024-04-23T14:32:13.778406+0000","last_active":"2024-04-23T14:32:15.180827+0000","last_peered":"2024-04-23T14:32:15.180827+0000","last_clean":"2024-04-23T14:32:15.180827+0000","last_became_active":"2024-04-23T14:32:13.778034+0000","last_became_peered":"2024-04-23T14:32:13.778034+0000","last_unstale":"2024-04-23T14:32:15.180827+0000","last_undegraded":"2024-04-23T14:32:15.180827+0000","last_fullsized":"2024-04-23T14:32:15.180827+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:21:36.288246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181961+0000","last_change":"2024-04-23T14:32:09.613694+0000","last_active":"2024-04-23T14:32:15.181961+0000","last_peered":"2024-04-23T14:32:15.181961+0000","last_clean":"2024-04-23T14:32:15.181961+0000","last_became_active":"2024-04-23T14:32:09.613413+0000","last_became_peered":"2024-04-23T14:32:09.613413+0000","last_unstale":"2024-04-23T14:32:15.181961+0000","last_undegraded":"2024-04-23T14:32:15.181961+0000","last_fullsized":"2024-04-23T14:32:15.181961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:12.933947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180126+0000","last_change":"2024-04-23T14:32:08.612923+0000","last_active":"2024-04-23T14:32:15.180126+0000","last_peered":"2024-04-23T14:32:15.180126+0000","last_clean":"2024-04-23T14:32:15.180126+0000","last_became_active":"2024-04-23T14:32:08.610847+0000","last_became_peered":"2024-04-23T14:32:08.610847+0000","last_unstale":"2024-04-23T14:32:15.180126+0000","last_undegraded":"2024-04-23T14:32:15.180126+0000","last_fullsized":"2024-04-23T14:32:15.180126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:40:25.063520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640453+0000","last_change":"2024-04-23T14:32:13.775013+0000","last_active":"2024-04-23T14:32:14.640453+0000","last_peered":"2024-04-23T14:32:14.640453+0000","last_clean":"2024-04-23T14:32:14.640453+0000","last_became_active":"2024-04-23T14:32:13.774565+0000","last_became_peered":"2024-04-23T14:32:13.774565+0000","last_unstale":"2024-04-23T14:32:14.640453+0000","last_undegraded":"2024-04-23T14:32:14.640453+0000","last_fullsized":"2024-04-23T14:32:14.640453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:26:08.003427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640083+0000","last_change":"2024-04-23T14:32:09.633886+0000","last_active":"2024-04-23T14:32:14.640083+0000","last_peered":"2024-04-23T14:32:14.640083+0000","last_clean":"2024-04-23T14:32:14.640083+0000","last_became_active":"2024-04-23T14:32:09.633258+0000","last_became_peered":"2024-04-23T14:32:09.633258+0000","last_unstale":"2024-04-23T14:32:14.640083+0000","last_undegraded":"2024-04-23T14:32:14.640083+0000","last_fullsized":"2024-04-23T14:32:14.640083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:24.637952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182012+0000","last_change":"2024-04-23T14:32:08.602248+0000","last_active":"2024-04-23T14:32:15.182012+0000","last_peered":"2024-04-23T14:32:15.182012+0000","last_clean":"2024-04-23T14:32:15.182012+0000","last_became_active":"2024-04-23T14:32:08.601913+0000","last_became_peered":"2024-04-23T14:32:08.601913+0000","last_unstale":"2024-04-23T14:32:15.182012+0000","last_undegraded":"2024-04-23T14:32:15.182012+0000","last_fullsized":"2024-04-23T14:32:15.182012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:11.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180987+0000","last_change":"2024-04-23T14:32:13.666952+0000","last_active":"2024-04-23T14:32:15.180987+0000","last_peered":"2024-04-23T14:32:15.180987+0000","last_clean":"2024-04-23T14:32:15.180987+0000","last_became_active":"2024-04-23T14:32:13.663335+0000","last_became_peered":"2024-04-23T14:32:13.663335+0000","last_unstale":"2024-04-23T14:32:15.180987+0000","last_undegraded":"2024-04-23T14:32:15.180987+0000","last_fullsized":"2024-04-23T14:32:15.180987+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:22.418010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152000+0000","last_change":"2024-04-23T14:32:09.627852+0000","last_active":"2024-04-23T14:32:15.152000+0000","last_peered":"2024-04-23T14:32:15.152000+0000","last_clean":"2024-04-23T14:32:15.152000+0000","last_became_active":"2024-04-23T14:32:09.627575+0000","last_became_peered":"2024-04-23T14:32:09.627575+0000","last_unstale":"2024-04-23T14:32:15.152000+0000","last_undegraded":"2024-04-23T14:32:15.152000+0000","last_fullsized":"2024-04-23T14:32:15.152000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:55.501089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640731+0000","last_change":"2024-04-23T14:32:08.603753+0000","last_active":"2024-04-23T14:32:14.640731+0000","last_peered":"2024-04-23T14:32:14.640731+0000","last_clean":"2024-04-23T14:32:14.640731+0000","last_became_active":"2024-04-23T14:32:08.602944+0000","last_became_peered":"2024-04-23T14:32:08.602944+0000","last_unstale":"2024-04-23T14:32:14.640731+0000","last_undegraded":"2024-04-23T14:32:14.640731+0000","last_fullsized":"2024-04-23T14:32:14.640731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:54.495261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181918+0000","last_change":"2024-04-23T14:32:13.667268+0000","last_active":"2024-04-23T14:32:15.181918+0000","last_peered":"2024-04-23T14:32:15.181918+0000","last_clean":"2024-04-23T14:32:15.181918+0000","last_became_active":"2024-04-23T14:32:13.666836+0000","last_became_peered":"2024-04-23T14:32:13.666836+0000","last_unstale":"2024-04-23T14:32:15.181918+0000","last_undegraded":"2024-04-23T14:32:15.181918+0000","last_fullsized":"2024-04-23T14:32:15.181918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:14:16.782670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180774+0000","last_change":"2024-04-23T14:32:09.613896+0000","last_active":"2024-04-23T14:32:15.180774+0000","last_peered":"2024-04-23T14:32:15.180774+0000","last_clean":"2024-04-23T14:32:15.180774+0000","last_became_active":"2024-04-23T14:32:09.613535+0000","last_became_peered":"2024-04-23T14:32:09.613535+0000","last_unstale":"2024-04-23T14:32:15.180774+0000","last_undegraded":"2024-04-23T14:32:15.180774+0000","last_fullsized":"2024-04-23T14:32:15.180774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:50:24.043980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181650+0000","last_change":"2024-04-23T14:32:08.606268+0000","last_active":"2024-04-23T14:32:15.181650+0000","last_peered":"2024-04-23T14:32:15.181650+0000","last_clean":"2024-04-23T14:32:15.181650+0000","last_became_active":"2024-04-23T14:32:08.605958+0000","last_became_peered":"2024-04-23T14:32:08.605958+0000","last_unstale":"2024-04-23T14:32:15.181650+0000","last_undegraded":"2024-04-23T14:32:15.181650+0000","last_fullsized":"2024-04-23T14:32:15.181650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:45:45.243948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.183441+0000","last_change":"2024-04-23T14:32:13.664183+0000","last_active":"2024-04-23T14:32:15.183441+0000","last_peered":"2024-04-23T14:32:15.183441+0000","last_clean":"2024-04-23T14:32:15.183441+0000","last_became_active":"2024-04-23T14:32:13.663836+0000","last_became_peered":"2024-04-23T14:32:13.663836+0000","last_unstale":"2024-04-23T14:32:15.183441+0000","last_undegraded":"2024-04-23T14:32:15.183441+0000","last_fullsized":"2024-04-23T14:32:15.183441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:09:12.045283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639679+0000","last_change":"2024-04-23T14:32:09.633686+0000","last_active":"2024-04-23T14:32:14.639679+0000","last_peered":"2024-04-23T14:32:14.639679+0000","last_clean":"2024-04-23T14:32:14.639679+0000","last_became_active":"2024-04-23T14:32:09.631825+0000","last_became_peered":"2024-04-23T14:32:09.631825+0000","last_unstale":"2024-04-23T14:32:14.639679+0000","last_undegraded":"2024-04-23T14:32:14.639679+0000","last_fullsized":"2024-04-23T14:32:14.639679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:23:57.869625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179303+0000","last_change":"2024-04-23T14:32:08.613375+0000","last_active":"2024-04-23T14:32:15.179303+0000","last_peered":"2024-04-23T14:32:15.179303+0000","last_clean":"2024-04-23T14:32:15.179303+0000","last_became_active":"2024-04-23T14:32:08.613129+0000","last_became_peered":"2024-04-23T14:32:08.613129+0000","last_unstale":"2024-04-23T14:32:15.179303+0000","last_undegraded":"2024-04-23T14:32:15.179303+0000","last_fullsized":"2024-04-23T14:32:15.179303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:54.495261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640202+0000","last_change":"2024-04-23T14:32:13.779707+0000","last_active":"2024-04-23T14:32:14.640202+0000","last_peered":"2024-04-23T14:32:14.640202+0000","last_clean":"2024-04-23T14:32:14.640202+0000","last_became_active":"2024-04-23T14:32:13.779357+0000","last_became_peered":"2024-04-23T14:32:13.779357+0000","last_unstale":"2024-04-23T14:32:14.640202+0000","last_undegraded":"2024-04-23T14:32:14.640202+0000","last_fullsized":"2024-04-23T14:32:14.640202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:12:23.488042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181182+0000","last_change":"2024-04-23T14:32:09.621797+0000","last_active":"2024-04-23T14:32:15.181182+0000","last_peered":"2024-04-23T14:32:15.181182+0000","last_clean":"2024-04-23T14:32:15.181182+0000","last_became_active":"2024-04-23T14:32:09.618825+0000","last_became_peered":"2024-04-23T14:32:09.618825+0000","last_unstale":"2024-04-23T14:32:15.181182+0000","last_undegraded":"2024-04-23T14:32:15.181182+0000","last_fullsized":"2024-04-23T14:32:15.181182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:22.220286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639965+0000","last_change":"2024-04-23T14:32:08.603338+0000","last_active":"2024-04-23T14:32:14.639965+0000","last_peered":"2024-04-23T14:32:14.639965+0000","last_clean":"2024-04-23T14:32:14.639965+0000","last_became_active":"2024-04-23T14:32:08.602563+0000","last_became_peered":"2024-04-23T14:32:08.602563+0000","last_unstale":"2024-04-23T14:32:14.639965+0000","last_undegraded":"2024-04-23T14:32:14.639965+0000","last_fullsized":"2024-04-23T14:32:14.639965+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:45:45.243948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179439+0000","last_change":"2024-04-23T14:31:58.314235+0000","last_active":"2024-04-23T14:32:15.179439+0000","last_peered":"2024-04-23T14:32:15.179439+0000","last_clean":"2024-04-23T14:32:15.179439+0000","last_became_active":"2024-04-23T14:31:58.313165+0000","last_became_peered":"2024-04-23T14:31:58.313165+0000","last_unstale":"2024-04-23T14:32:15.179439+0000","last_undegraded":"2024-04-23T14:32:15.179439+0000","last_fullsized":"2024-04-23T14:32:15.179439+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_clean_scrub_stamp":"2024-04-23T14:31:57.140712+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:44.170436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641636+0000","last_change":"2024-04-23T14:32:13.777292+0000","last_active":"2024-04-23T14:32:14.641636+0000","last_peered":"2024-04-23T14:32:14.641636+0000","last_clean":"2024-04-23T14:32:14.641636+0000","last_became_active":"2024-04-23T14:32:13.776933+0000","last_became_peered":"2024-04-23T14:32:13.776933+0000","last_unstale":"2024-04-23T14:32:14.641636+0000","last_undegraded":"2024-04-23T14:32:14.641636+0000","last_fullsized":"2024-04-23T14:32:14.641636+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:47:35.856332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179793+0000","last_change":"2024-04-23T14:32:09.639015+0000","last_active":"2024-04-23T14:32:15.179793+0000","last_peered":"2024-04-23T14:32:15.179793+0000","last_clean":"2024-04-23T14:32:15.179793+0000","last_became_active":"2024-04-23T14:32:09.638622+0000","last_became_peered":"2024-04-23T14:32:09.638622+0000","last_unstale":"2024-04-23T14:32:15.179793+0000","last_undegraded":"2024-04-23T14:32:15.179793+0000","last_fullsized":"2024-04-23T14:32:15.179793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:18.367077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181794+0000","last_change":"2024-04-23T14:32:08.607926+0000","last_active":"2024-04-23T14:32:15.181794+0000","last_peered":"2024-04-23T14:32:15.181794+0000","last_clean":"2024-04-23T14:32:15.181794+0000","last_became_active":"2024-04-23T14:32:08.607649+0000","last_became_peered":"2024-04-23T14:32:08.607649+0000","last_unstale":"2024-04-23T14:32:15.181794+0000","last_undegraded":"2024-04-23T14:32:15.181794+0000","last_fullsized":"2024-04-23T14:32:15.181794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:59:03.639740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641732+0000","last_change":"2024-04-23T14:32:13.774449+0000","last_active":"2024-04-23T14:32:14.641732+0000","last_peered":"2024-04-23T14:32:14.641732+0000","last_clean":"2024-04-23T14:32:14.641732+0000","last_became_active":"2024-04-23T14:32:13.773937+0000","last_became_peered":"2024-04-23T14:32:13.773937+0000","last_unstale":"2024-04-23T14:32:14.641732+0000","last_undegraded":"2024-04-23T14:32:14.641732+0000","last_fullsized":"2024-04-23T14:32:14.641732+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:22.418010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640399+0000","last_change":"2024-04-23T14:32:09.626405+0000","last_active":"2024-04-23T14:32:14.640399+0000","last_peered":"2024-04-23T14:32:14.640399+0000","last_clean":"2024-04-23T14:32:14.640399+0000","last_became_active":"2024-04-23T14:32:09.626056+0000","last_became_peered":"2024-04-23T14:32:09.626056+0000","last_unstale":"2024-04-23T14:32:14.640399+0000","last_undegraded":"2024-04-23T14:32:14.640399+0000","last_fullsized":"2024-04-23T14:32:14.640399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:18.367077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182898+0000","last_change":"2024-04-23T14:32:08.608148+0000","last_active":"2024-04-23T14:32:15.182898+0000","last_peered":"2024-04-23T14:32:15.182898+0000","last_clean":"2024-04-23T14:32:15.182898+0000","last_became_active":"2024-04-23T14:32:08.607623+0000","last_became_peered":"2024-04-23T14:32:08.607623+0000","last_unstale":"2024-04-23T14:32:15.182898+0000","last_undegraded":"2024-04-23T14:32:15.182898+0000","last_fullsized":"2024-04-23T14:32:15.182898+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:51.650822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640246+0000","last_change":"2024-04-23T14:32:13.663295+0000","last_active":"2024-04-23T14:32:14.640246+0000","last_peered":"2024-04-23T14:32:14.640246+0000","last_clean":"2024-04-23T14:32:14.640246+0000","last_became_active":"2024-04-23T14:32:13.662900+0000","last_became_peered":"2024-04-23T14:32:13.662900+0000","last_unstale":"2024-04-23T14:32:14.640246+0000","last_undegraded":"2024-04-23T14:32:14.640246+0000","last_fullsized":"2024-04-23T14:32:14.640246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:09:12.045283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180115+0000","last_change":"2024-04-23T14:32:09.641393+0000","last_active":"2024-04-23T14:32:15.180115+0000","last_peered":"2024-04-23T14:32:15.180115+0000","last_clean":"2024-04-23T14:32:15.180115+0000","last_became_active":"2024-04-23T14:32:09.641075+0000","last_became_peered":"2024-04-23T14:32:09.641075+0000","last_unstale":"2024-04-23T14:32:15.180115+0000","last_undegraded":"2024-04-23T14:32:15.180115+0000","last_fullsized":"2024-04-23T14:32:15.180115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:26:03.952494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182813+0000","last_change":"2024-04-23T14:32:08.607178+0000","last_active":"2024-04-23T14:32:15.182813+0000","last_peered":"2024-04-23T14:32:15.182813+0000","last_clean":"2024-04-23T14:32:15.182813+0000","last_became_active":"2024-04-23T14:32:08.606863+0000","last_became_peered":"2024-04-23T14:32:08.606863+0000","last_unstale":"2024-04-23T14:32:15.182813+0000","last_undegraded":"2024-04-23T14:32:15.182813+0000","last_fullsized":"2024-04-23T14:32:15.182813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:35.854078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639398+0000","last_change":"2024-04-23T14:32:13.773533+0000","last_active":"2024-04-23T14:32:14.639398+0000","last_peered":"2024-04-23T14:32:14.639398+0000","last_clean":"2024-04-23T14:32:14.639398+0000","last_became_active":"2024-04-23T14:32:13.773057+0000","last_became_peered":"2024-04-23T14:32:13.773057+0000","last_unstale":"2024-04-23T14:32:14.639398+0000","last_undegraded":"2024-04-23T14:32:14.639398+0000","last_fullsized":"2024-04-23T14:32:14.639398+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:08.115745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180012+0000","last_change":"2024-04-23T14:32:09.641317+0000","last_active":"2024-04-23T14:32:15.180012+0000","last_peered":"2024-04-23T14:32:15.180012+0000","last_clean":"2024-04-23T14:32:15.180012+0000","last_became_active":"2024-04-23T14:32:09.640930+0000","last_became_peered":"2024-04-23T14:32:09.640930+0000","last_unstale":"2024-04-23T14:32:15.180012+0000","last_undegraded":"2024-04-23T14:32:15.180012+0000","last_fullsized":"2024-04-23T14:32:15.180012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:24.637952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152438+0000","last_change":"2024-04-23T14:32:08.601936+0000","last_active":"2024-04-23T14:32:15.152438+0000","last_peered":"2024-04-23T14:32:15.152438+0000","last_clean":"2024-04-23T14:32:15.152438+0000","last_became_active":"2024-04-23T14:32:08.601653+0000","last_became_peered":"2024-04-23T14:32:08.601653+0000","last_unstale":"2024-04-23T14:32:15.152438+0000","last_undegraded":"2024-04-23T14:32:15.152438+0000","last_fullsized":"2024-04-23T14:32:15.152438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:07:08.639311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151588+0000","last_change":"2024-04-23T14:32:13.664132+0000","last_active":"2024-04-23T14:32:15.151588+0000","last_peered":"2024-04-23T14:32:15.151588+0000","last_clean":"2024-04-23T14:32:15.151588+0000","last_became_active":"2024-04-23T14:32:13.663741+0000","last_became_peered":"2024-04-23T14:32:13.663741+0000","last_unstale":"2024-04-23T14:32:15.151588+0000","last_undegraded":"2024-04-23T14:32:15.151588+0000","last_fullsized":"2024-04-23T14:32:15.151588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:07.179856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179517+0000","last_change":"2024-04-23T14:32:09.641942+0000","last_active":"2024-04-23T14:32:15.179517+0000","last_peered":"2024-04-23T14:32:15.179517+0000","last_clean":"2024-04-23T14:32:15.179517+0000","last_became_active":"2024-04-23T14:32:09.641680+0000","last_became_peered":"2024-04-23T14:32:09.641680+0000","last_unstale":"2024-04-23T14:32:15.179517+0000","last_undegraded":"2024-04-23T14:32:15.179517+0000","last_fullsized":"2024-04-23T14:32:15.179517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:50.876901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181283+0000","last_change":"2024-04-23T14:32:08.603405+0000","last_active":"2024-04-23T14:32:15.181283+0000","last_peered":"2024-04-23T14:32:15.181283+0000","last_clean":"2024-04-23T14:32:15.181283+0000","last_became_active":"2024-04-23T14:32:08.603116+0000","last_became_peered":"2024-04-23T14:32:08.603116+0000","last_unstale":"2024-04-23T14:32:15.181283+0000","last_undegraded":"2024-04-23T14:32:15.181283+0000","last_fullsized":"2024-04-23T14:32:15.181283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:43:58.790056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640138+0000","last_change":"2024-04-23T14:32:13.660242+0000","last_active":"2024-04-23T14:32:14.640138+0000","last_peered":"2024-04-23T14:32:14.640138+0000","last_clean":"2024-04-23T14:32:14.640138+0000","last_became_active":"2024-04-23T14:32:13.659712+0000","last_became_peered":"2024-04-23T14:32:13.659712+0000","last_unstale":"2024-04-23T14:32:14.640138+0000","last_undegraded":"2024-04-23T14:32:14.640138+0000","last_fullsized":"2024-04-23T14:32:14.640138+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:14:16.782670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641152+0000","last_change":"2024-04-23T14:32:09.619773+0000","last_active":"2024-04-23T14:32:14.641152+0000","last_peered":"2024-04-23T14:32:14.641152+0000","last_clean":"2024-04-23T14:32:14.641152+0000","last_became_active":"2024-04-23T14:32:09.619451+0000","last_became_peered":"2024-04-23T14:32:09.619451+0000","last_unstale":"2024-04-23T14:32:14.641152+0000","last_undegraded":"2024-04-23T14:32:14.641152+0000","last_fullsized":"2024-04-23T14:32:14.641152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:12.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641439+0000","last_change":"2024-04-23T14:32:08.613751+0000","last_active":"2024-04-23T14:32:14.641439+0000","last_peered":"2024-04-23T14:32:14.641439+0000","last_clean":"2024-04-23T14:32:14.641439+0000","last_became_active":"2024-04-23T14:32:08.607415+0000","last_became_peered":"2024-04-23T14:32:08.607415+0000","last_unstale":"2024-04-23T14:32:14.641439+0000","last_undegraded":"2024-04-23T14:32:14.641439+0000","last_fullsized":"2024-04-23T14:32:14.641439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:35.854078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182048+0000","last_change":"2024-04-23T14:32:13.780614+0000","last_active":"2024-04-23T14:32:15.182048+0000","last_peered":"2024-04-23T14:32:15.182048+0000","last_clean":"2024-04-23T14:32:15.182048+0000","last_became_active":"2024-04-23T14:32:13.780135+0000","last_became_peered":"2024-04-23T14:32:13.780135+0000","last_unstale":"2024-04-23T14:32:15.182048+0000","last_undegraded":"2024-04-23T14:32:15.182048+0000","last_fullsized":"2024-04-23T14:32:15.182048+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:16:27.476884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179624+0000","last_change":"2024-04-23T14:32:09.642908+0000","last_active":"2024-04-23T14:32:15.179624+0000","last_peered":"2024-04-23T14:32:15.179624+0000","last_clean":"2024-04-23T14:32:15.179624+0000","last_became_active":"2024-04-23T14:32:09.642670+0000","last_became_peered":"2024-04-23T14:32:09.642670+0000","last_unstale":"2024-04-23T14:32:15.179624+0000","last_undegraded":"2024-04-23T14:32:15.179624+0000","last_fullsized":"2024-04-23T14:32:15.179624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:56.096479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640627+0000","last_change":"2024-04-23T14:32:08.598260+0000","last_active":"2024-04-23T14:32:14.640627+0000","last_peered":"2024-04-23T14:32:14.640627+0000","last_clean":"2024-04-23T14:32:14.640627+0000","last_became_active":"2024-04-23T14:32:08.597931+0000","last_became_peered":"2024-04-23T14:32:08.597931+0000","last_unstale":"2024-04-23T14:32:14.640627+0000","last_undegraded":"2024-04-23T14:32:14.640627+0000","last_fullsized":"2024-04-23T14:32:14.640627+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:51.650822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182382+0000","last_change":"2024-04-23T14:32:13.775555+0000","last_active":"2024-04-23T14:32:15.182382+0000","last_peered":"2024-04-23T14:32:15.182382+0000","last_clean":"2024-04-23T14:32:15.182382+0000","last_became_active":"2024-04-23T14:32:13.774654+0000","last_became_peered":"2024-04-23T14:32:13.774654+0000","last_unstale":"2024-04-23T14:32:15.182382+0000","last_undegraded":"2024-04-23T14:32:15.182382+0000","last_fullsized":"2024-04-23T14:32:15.182382+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:57.743802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639957+0000","last_change":"2024-04-23T14:32:09.626794+0000","last_active":"2024-04-23T14:32:14.639957+0000","last_peered":"2024-04-23T14:32:14.639957+0000","last_clean":"2024-04-23T14:32:14.639957+0000","last_became_active":"2024-04-23T14:32:09.626382+0000","last_became_peered":"2024-04-23T14:32:09.626382+0000","last_unstale":"2024-04-23T14:32:14.639957+0000","last_undegraded":"2024-04-23T14:32:14.639957+0000","last_fullsized":"2024-04-23T14:32:14.639957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:54:08.664672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640520+0000","last_change":"2024-04-23T14:32:08.600425+0000","last_active":"2024-04-23T14:32:14.640520+0000","last_peered":"2024-04-23T14:32:14.640520+0000","last_clean":"2024-04-23T14:32:14.640520+0000","last_became_active":"2024-04-23T14:32:08.600113+0000","last_became_peered":"2024-04-23T14:32:08.600113+0000","last_unstale":"2024-04-23T14:32:14.640520+0000","last_undegraded":"2024-04-23T14:32:14.640520+0000","last_fullsized":"2024-04-23T14:32:14.640520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:07:08.639311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639973+0000","last_change":"2024-04-23T14:32:13.777092+0000","last_active":"2024-04-23T14:32:14.639973+0000","last_peered":"2024-04-23T14:32:14.639973+0000","last_clean":"2024-04-23T14:32:14.639973+0000","last_became_active":"2024-04-23T14:32:13.776681+0000","last_became_peered":"2024-04-23T14:32:13.776681+0000","last_unstale":"2024-04-23T14:32:14.639973+0000","last_undegraded":"2024-04-23T14:32:14.639973+0000","last_fullsized":"2024-04-23T14:32:14.639973+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:21:36.288246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179674+0000","last_change":"2024-04-23T14:32:09.629485+0000","last_active":"2024-04-23T14:32:15.179674+0000","last_peered":"2024-04-23T14:32:15.179674+0000","last_clean":"2024-04-23T14:32:15.179674+0000","last_became_active":"2024-04-23T14:32:09.629157+0000","last_became_peered":"2024-04-23T14:32:09.629157+0000","last_unstale":"2024-04-23T14:32:15.179674+0000","last_undegraded":"2024-04-23T14:32:15.179674+0000","last_fullsized":"2024-04-23T14:32:15.179674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:47:31.805399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:25.916019+0000","last_change":"2024-04-23T14:32:08.604477+0000","last_active":"2024-04-23T14:32:25.916019+0000","last_peered":"2024-04-23T14:32:25.916019+0000","last_clean":"2024-04-23T14:32:25.916019+0000","last_became_active":"2024-04-23T14:32:08.604148+0000","last_became_peered":"2024-04-23T14:32:08.604148+0000","last_unstale":"2024-04-23T14:32:25.916019+0000","last_undegraded":"2024-04-23T14:32:25.916019+0000","last_fullsized":"2024-04-23T14:32:25.916019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:51.650822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181124+0000","last_change":"2024-04-23T14:32:13.660428+0000","last_active":"2024-04-23T14:32:15.181124+0000","last_peered":"2024-04-23T14:32:15.181124+0000","last_clean":"2024-04-23T14:32:15.181124+0000","last_became_active":"2024-04-23T14:32:13.659779+0000","last_became_peered":"2024-04-23T14:32:13.659779+0000","last_unstale":"2024-04-23T14:32:15.181124+0000","last_undegraded":"2024-04-23T14:32:15.181124+0000","last_fullsized":"2024-04-23T14:32:15.181124+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:28.688885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152295+0000","last_change":"2024-04-23T14:32:09.628609+0000","last_active":"2024-04-23T14:32:15.152295+0000","last_peered":"2024-04-23T14:32:15.152295+0000","last_clean":"2024-04-23T14:32:15.152295+0000","last_became_active":"2024-04-23T14:32:09.628357+0000","last_became_peered":"2024-04-23T14:32:09.628357+0000","last_unstale":"2024-04-23T14:32:15.152295+0000","last_undegraded":"2024-04-23T14:32:15.152295+0000","last_fullsized":"2024-04-23T14:32:15.152295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:01.062394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182825+0000","last_change":"2024-04-23T14:32:08.607931+0000","last_active":"2024-04-23T14:32:15.182825+0000","last_peered":"2024-04-23T14:32:15.182825+0000","last_clean":"2024-04-23T14:32:15.182825+0000","last_became_active":"2024-04-23T14:32:08.607268+0000","last_became_peered":"2024-04-23T14:32:08.607268+0000","last_unstale":"2024-04-23T14:32:15.182825+0000","last_undegraded":"2024-04-23T14:32:15.182825+0000","last_fullsized":"2024-04-23T14:32:15.182825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:35.854078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152109+0000","last_change":"2024-04-23T14:32:13.661080+0000","last_active":"2024-04-23T14:32:15.152109+0000","last_peered":"2024-04-23T14:32:15.152109+0000","last_clean":"2024-04-23T14:32:15.152109+0000","last_became_active":"2024-04-23T14:32:13.660376+0000","last_became_peered":"2024-04-23T14:32:13.660376+0000","last_unstale":"2024-04-23T14:32:15.152109+0000","last_undegraded":"2024-04-23T14:32:15.152109+0000","last_fullsized":"2024-04-23T14:32:15.152109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:40:00.147412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640230+0000","last_change":"2024-04-23T14:32:09.621197+0000","last_active":"2024-04-23T14:32:14.640230+0000","last_peered":"2024-04-23T14:32:14.640230+0000","last_clean":"2024-04-23T14:32:14.640230+0000","last_became_active":"2024-04-23T14:32:09.620883+0000","last_became_peered":"2024-04-23T14:32:09.620883+0000","last_unstale":"2024-04-23T14:32:14.640230+0000","last_undegraded":"2024-04-23T14:32:14.640230+0000","last_fullsized":"2024-04-23T14:32:14.640230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:12.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640178+0000","last_change":"2024-04-23T14:32:08.606025+0000","last_active":"2024-04-23T14:32:14.640178+0000","last_peered":"2024-04-23T14:32:14.640178+0000","last_clean":"2024-04-23T14:32:14.640178+0000","last_became_active":"2024-04-23T14:32:08.605727+0000","last_became_peered":"2024-04-23T14:32:08.605727+0000","last_unstale":"2024-04-23T14:32:14.640178+0000","last_undegraded":"2024-04-23T14:32:14.640178+0000","last_fullsized":"2024-04-23T14:32:14.640178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:59:03.639740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181124+0000","last_change":"2024-04-23T14:32:13.669518+0000","last_active":"2024-04-23T14:32:15.181124+0000","last_peered":"2024-04-23T14:32:15.181124+0000","last_clean":"2024-04-23T14:32:15.181124+0000","last_became_active":"2024-04-23T14:32:13.669004+0000","last_became_peered":"2024-04-23T14:32:13.669004+0000","last_unstale":"2024-04-23T14:32:15.181124+0000","last_undegraded":"2024-04-23T14:32:15.181124+0000","last_fullsized":"2024-04-23T14:32:15.181124+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:31:45.244234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640043+0000","last_change":"2024-04-23T14:32:09.625280+0000","last_active":"2024-04-23T14:32:14.640043+0000","last_peered":"2024-04-23T14:32:14.640043+0000","last_clean":"2024-04-23T14:32:14.640043+0000","last_became_active":"2024-04-23T14:32:09.625033+0000","last_became_peered":"2024-04-23T14:32:09.625033+0000","last_unstale":"2024-04-23T14:32:14.640043+0000","last_undegraded":"2024-04-23T14:32:14.640043+0000","last_fullsized":"2024-04-23T14:32:14.640043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:14:12.731737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.642133+0000","last_change":"2024-04-23T14:32:08.605039+0000","last_active":"2024-04-23T14:32:14.642133+0000","last_peered":"2024-04-23T14:32:14.642133+0000","last_clean":"2024-04-23T14:32:14.642133+0000","last_became_active":"2024-04-23T14:32:08.604450+0000","last_became_peered":"2024-04-23T14:32:08.604450+0000","last_unstale":"2024-04-23T14:32:14.642133+0000","last_undegraded":"2024-04-23T14:32:14.642133+0000","last_fullsized":"2024-04-23T14:32:14.642133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:45:45.243948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181369+0000","last_change":"2024-04-23T14:32:13.779606+0000","last_active":"2024-04-23T14:32:15.181369+0000","last_peered":"2024-04-23T14:32:15.181369+0000","last_clean":"2024-04-23T14:32:15.181369+0000","last_became_active":"2024-04-23T14:32:13.779243+0000","last_became_peered":"2024-04-23T14:32:13.779243+0000","last_unstale":"2024-04-23T14:32:15.181369+0000","last_undegraded":"2024-04-23T14:32:15.181369+0000","last_fullsized":"2024-04-23T14:32:15.181369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:58:40.249561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640030+0000","last_change":"2024-04-23T14:32:13.779659+0000","last_active":"2024-04-23T14:32:14.640030+0000","last_peered":"2024-04-23T14:32:14.640030+0000","last_clean":"2024-04-23T14:32:14.640030+0000","last_became_active":"2024-04-23T14:32:13.779312+0000","last_became_peered":"2024-04-23T14:32:13.779312+0000","last_unstale":"2024-04-23T14:32:14.640030+0000","last_undegraded":"2024-04-23T14:32:14.640030+0000","last_fullsized":"2024-04-23T14:32:14.640030+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:28.688885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180700+0000","last_change":"2024-04-23T14:32:09.632498+0000","last_active":"2024-04-23T14:32:15.180700+0000","last_peered":"2024-04-23T14:32:15.180700+0000","last_clean":"2024-04-23T14:32:15.180700+0000","last_became_active":"2024-04-23T14:32:09.632191+0000","last_became_peered":"2024-04-23T14:32:09.632191+0000","last_unstale":"2024-04-23T14:32:15.180700+0000","last_undegraded":"2024-04-23T14:32:15.180700+0000","last_fullsized":"2024-04-23T14:32:15.180700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:53.674486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639631+0000","last_change":"2024-04-23T14:32:08.607308+0000","last_active":"2024-04-23T14:32:14.639631+0000","last_peered":"2024-04-23T14:32:14.639631+0000","last_clean":"2024-04-23T14:32:14.639631+0000","last_became_active":"2024-04-23T14:32:08.606840+0000","last_became_peered":"2024-04-23T14:32:08.606840+0000","last_unstale":"2024-04-23T14:32:14.639631+0000","last_undegraded":"2024-04-23T14:32:14.639631+0000","last_fullsized":"2024-04-23T14:32:14.639631+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:35.854078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.642125+0000","last_change":"2024-04-23T14:32:13.654800+0000","last_active":"2024-04-23T14:32:14.642125+0000","last_peered":"2024-04-23T14:32:14.642125+0000","last_clean":"2024-04-23T14:32:14.642125+0000","last_became_active":"2024-04-23T14:32:13.654191+0000","last_became_peered":"2024-04-23T14:32:13.654191+0000","last_unstale":"2024-04-23T14:32:14.642125+0000","last_undegraded":"2024-04-23T14:32:14.642125+0000","last_fullsized":"2024-04-23T14:32:14.642125+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:28.688885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181062+0000","last_change":"2024-04-23T14:32:09.623316+0000","last_active":"2024-04-23T14:32:15.181062+0000","last_peered":"2024-04-23T14:32:15.181062+0000","last_clean":"2024-04-23T14:32:15.181062+0000","last_became_active":"2024-04-23T14:32:09.621723+0000","last_became_peered":"2024-04-23T14:32:09.621723+0000","last_unstale":"2024-04-23T14:32:15.181062+0000","last_undegraded":"2024-04-23T14:32:15.181062+0000","last_fullsized":"2024-04-23T14:32:15.181062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:12.933947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181611+0000","last_change":"2024-04-23T14:32:08.609985+0000","last_active":"2024-04-23T14:32:15.181611+0000","last_peered":"2024-04-23T14:32:15.181611+0000","last_clean":"2024-04-23T14:32:15.181611+0000","last_became_active":"2024-04-23T14:32:08.609621+0000","last_became_peered":"2024-04-23T14:32:08.609621+0000","last_unstale":"2024-04-23T14:32:15.181611+0000","last_undegraded":"2024-04-23T14:32:15.181611+0000","last_fullsized":"2024-04-23T14:32:15.181611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:11.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641831+0000","last_change":"2024-04-23T14:32:08.609884+0000","last_active":"2024-04-23T14:32:14.641831+0000","last_peered":"2024-04-23T14:32:14.641831+0000","last_clean":"2024-04-23T14:32:14.641831+0000","last_became_active":"2024-04-23T14:32:08.607367+0000","last_became_peered":"2024-04-23T14:32:08.607367+0000","last_unstale":"2024-04-23T14:32:14.641831+0000","last_undegraded":"2024-04-23T14:32:14.641831+0000","last_fullsized":"2024-04-23T14:32:14.641831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:54.495261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640439+0000","last_change":"2024-04-23T14:32:09.632614+0000","last_active":"2024-04-23T14:32:14.640439+0000","last_peered":"2024-04-23T14:32:14.640439+0000","last_clean":"2024-04-23T14:32:14.640439+0000","last_became_active":"2024-04-23T14:32:09.632230+0000","last_became_peered":"2024-04-23T14:32:09.632230+0000","last_unstale":"2024-04-23T14:32:14.640439+0000","last_undegraded":"2024-04-23T14:32:14.640439+0000","last_fullsized":"2024-04-23T14:32:14.640439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:53.674486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181628+0000","last_change":"2024-04-23T14:32:13.671189+0000","last_active":"2024-04-23T14:32:15.181628+0000","last_peered":"2024-04-23T14:32:15.181628+0000","last_clean":"2024-04-23T14:32:15.181628+0000","last_became_active":"2024-04-23T14:32:13.670033+0000","last_became_peered":"2024-04-23T14:32:13.670033+0000","last_unstale":"2024-04-23T14:32:15.181628+0000","last_undegraded":"2024-04-23T14:32:15.181628+0000","last_fullsized":"2024-04-23T14:32:15.181628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:08:35.206423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181132+0000","last_change":"2024-04-23T14:32:08.612716+0000","last_active":"2024-04-23T14:32:15.181132+0000","last_peered":"2024-04-23T14:32:15.181132+0000","last_clean":"2024-04-23T14:32:15.181132+0000","last_became_active":"2024-04-23T14:32:08.612378+0000","last_became_peered":"2024-04-23T14:32:08.612378+0000","last_unstale":"2024-04-23T14:32:15.181132+0000","last_undegraded":"2024-04-23T14:32:15.181132+0000","last_fullsized":"2024-04-23T14:32:15.181132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:43:58.790056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151820+0000","last_change":"2024-04-23T14:32:09.620811+0000","last_active":"2024-04-23T14:32:15.151820+0000","last_peered":"2024-04-23T14:32:15.151820+0000","last_clean":"2024-04-23T14:32:15.151820+0000","last_became_active":"2024-04-23T14:32:09.620393+0000","last_became_peered":"2024-04-23T14:32:09.620393+0000","last_unstale":"2024-04-23T14:32:15.151820+0000","last_undegraded":"2024-04-23T14:32:15.151820+0000","last_fullsized":"2024-04-23T14:32:15.151820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:36.859905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181533+0000","last_change":"2024-04-23T14:32:13.670681+0000","last_active":"2024-04-23T14:32:15.181533+0000","last_peered":"2024-04-23T14:32:15.181533+0000","last_clean":"2024-04-23T14:32:15.181533+0000","last_became_active":"2024-04-23T14:32:13.670304+0000","last_became_peered":"2024-04-23T14:32:13.670304+0000","last_unstale":"2024-04-23T14:32:15.181533+0000","last_undegraded":"2024-04-23T14:32:15.181533+0000","last_fullsized":"2024-04-23T14:32:15.181533+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:35:37.731501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641029+0000","last_change":"2024-04-23T14:32:09.617499+0000","last_active":"2024-04-23T14:32:14.641029+0000","last_peered":"2024-04-23T14:32:14.641029+0000","last_clean":"2024-04-23T14:32:14.641029+0000","last_became_active":"2024-04-23T14:32:09.617154+0000","last_became_peered":"2024-04-23T14:32:09.617154+0000","last_unstale":"2024-04-23T14:32:14.641029+0000","last_undegraded":"2024-04-23T14:32:14.641029+0000","last_fullsized":"2024-04-23T14:32:14.641029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:43:49.773005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181553+0000","last_change":"2024-04-23T14:32:09.619214+0000","last_active":"2024-04-23T14:32:15.181553+0000","last_peered":"2024-04-23T14:32:15.181553+0000","last_clean":"2024-04-23T14:32:15.181553+0000","last_became_active":"2024-04-23T14:32:09.618943+0000","last_became_peered":"2024-04-23T14:32:09.618943+0000","last_unstale":"2024-04-23T14:32:15.181553+0000","last_undegraded":"2024-04-23T14:32:15.181553+0000","last_fullsized":"2024-04-23T14:32:15.181553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:12.536442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640531+0000","last_change":"2024-04-23T14:32:09.626499+0000","last_active":"2024-04-23T14:32:14.640531+0000","last_peered":"2024-04-23T14:32:14.640531+0000","last_clean":"2024-04-23T14:32:14.640531+0000","last_became_active":"2024-04-23T14:32:09.626070+0000","last_became_peered":"2024-04-23T14:32:09.626070+0000","last_unstale":"2024-04-23T14:32:14.640531+0000","last_undegraded":"2024-04-23T14:32:14.640531+0000","last_fullsized":"2024-04-23T14:32:14.640531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:50.876901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640295+0000","last_change":"2024-04-23T14:32:09.625391+0000","last_active":"2024-04-23T14:32:14.640295+0000","last_peered":"2024-04-23T14:32:14.640295+0000","last_clean":"2024-04-23T14:32:14.640295+0000","last_became_active":"2024-04-23T14:32:09.625049+0000","last_became_peered":"2024-04-23T14:32:09.625049+0000","last_unstale":"2024-04-23T14:32:14.640295+0000","last_undegraded":"2024-04-23T14:32:14.640295+0000","last_fullsized":"2024-04-23T14:32:14.640295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:52:42.867377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182436+0000","last_change":"2024-04-23T14:32:09.627850+0000","last_active":"2024-04-23T14:32:15.182436+0000","last_peered":"2024-04-23T14:32:15.182436+0000","last_clean":"2024-04-23T14:32:15.182436+0000","last_became_active":"2024-04-23T14:32:09.627603+0000","last_became_peered":"2024-04-23T14:32:09.627603+0000","last_unstale":"2024-04-23T14:32:15.182436+0000","last_undegraded":"2024-04-23T14:32:15.182436+0000","last_fullsized":"2024-04-23T14:32:15.182436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:09.066755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182767+0000","last_change":"2024-04-23T14:32:09.630409+0000","last_active":"2024-04-23T14:32:15.182767+0000","last_peered":"2024-04-23T14:32:15.182767+0000","last_clean":"2024-04-23T14:32:15.182767+0000","last_became_active":"2024-04-23T14:32:09.630100+0000","last_became_peered":"2024-04-23T14:32:09.630100+0000","last_unstale":"2024-04-23T14:32:15.182767+0000","last_undegraded":"2024-04-23T14:32:15.182767+0000","last_fullsized":"2024-04-23T14:32:15.182767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:03.128923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641996+0000","last_change":"2024-04-23T14:32:09.631521+0000","last_active":"2024-04-23T14:32:14.641996+0000","last_peered":"2024-04-23T14:32:14.641996+0000","last_clean":"2024-04-23T14:32:14.641996+0000","last_became_active":"2024-04-23T14:32:09.631291+0000","last_became_peered":"2024-04-23T14:32:09.631291+0000","last_unstale":"2024-04-23T14:32:14.641996+0000","last_undegraded":"2024-04-23T14:32:14.641996+0000","last_fullsized":"2024-04-23T14:32:14.641996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:12.933947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182097+0000","last_change":"2024-04-23T14:32:09.642262+0000","last_active":"2024-04-23T14:32:15.182097+0000","last_peered":"2024-04-23T14:32:15.182097+0000","last_clean":"2024-04-23T14:32:15.182097+0000","last_became_active":"2024-04-23T14:32:09.641982+0000","last_became_peered":"2024-04-23T14:32:09.641982+0000","last_unstale":"2024-04-23T14:32:15.182097+0000","last_undegraded":"2024-04-23T14:32:15.182097+0000","last_fullsized":"2024-04-23T14:32:15.182097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:52.459061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639502+0000","last_change":"2024-04-23T14:32:09.613189+0000","last_active":"2024-04-23T14:32:14.639502+0000","last_peered":"2024-04-23T14:32:14.639502+0000","last_clean":"2024-04-23T14:32:14.639502+0000","last_became_active":"2024-04-23T14:32:09.612846+0000","last_became_peered":"2024-04-23T14:32:09.612846+0000","last_unstale":"2024-04-23T14:32:14.639502+0000","last_undegraded":"2024-04-23T14:32:14.639502+0000","last_fullsized":"2024-04-23T14:32:14.639502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:03.128923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181523+0000","last_change":"2024-04-23T14:32:09.642268+0000","last_active":"2024-04-23T14:32:15.181523+0000","last_peered":"2024-04-23T14:32:15.181523+0000","last_clean":"2024-04-23T14:32:15.181523+0000","last_became_active":"2024-04-23T14:32:09.642007+0000","last_became_peered":"2024-04-23T14:32:09.642007+0000","last_unstale":"2024-04-23T14:32:15.181523+0000","last_undegraded":"2024-04-23T14:32:15.181523+0000","last_fullsized":"2024-04-23T14:32:15.181523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:50:24.043980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640425+0000","last_change":"2024-04-23T14:32:09.626670+0000","last_active":"2024-04-23T14:32:14.640425+0000","last_peered":"2024-04-23T14:32:14.640425+0000","last_clean":"2024-04-23T14:32:14.640425+0000","last_became_active":"2024-04-23T14:32:09.625568+0000","last_became_peered":"2024-04-23T14:32:09.625568+0000","last_unstale":"2024-04-23T14:32:14.640425+0000","last_undegraded":"2024-04-23T14:32:14.640425+0000","last_fullsized":"2024-04-23T14:32:14.640425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:09.066755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639884+0000","last_change":"2024-04-23T14:32:09.627089+0000","last_active":"2024-04-23T14:32:14.639884+0000","last_peered":"2024-04-23T14:32:14.639884+0000","last_clean":"2024-04-23T14:32:14.639884+0000","last_became_active":"2024-04-23T14:32:09.626429+0000","last_became_peered":"2024-04-23T14:32:09.626429+0000","last_unstale":"2024-04-23T14:32:14.639884+0000","last_undegraded":"2024-04-23T14:32:14.639884+0000","last_fullsized":"2024-04-23T14:32:14.639884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:43:49.773005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641487+0000","last_change":"2024-04-23T14:32:09.643824+0000","last_active":"2024-04-23T14:32:14.641487+0000","last_peered":"2024-04-23T14:32:14.641487+0000","last_clean":"2024-04-23T14:32:14.641487+0000","last_became_active":"2024-04-23T14:32:09.643454+0000","last_became_peered":"2024-04-23T14:32:09.643454+0000","last_unstale":"2024-04-23T14:32:14.641487+0000","last_undegraded":"2024-04-23T14:32:14.641487+0000","last_fullsized":"2024-04-23T14:32:14.641487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:03.128923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179997+0000","last_change":"2024-04-23T14:32:09.627274+0000","last_active":"2024-04-23T14:32:15.179997+0000","last_peered":"2024-04-23T14:32:15.179997+0000","last_clean":"2024-04-23T14:32:15.179997+0000","last_became_active":"2024-04-23T14:32:09.626999+0000","last_became_peered":"2024-04-23T14:32:09.626999+0000","last_unstale":"2024-04-23T14:32:15.179997+0000","last_undegraded":"2024-04-23T14:32:15.179997+0000","last_fullsized":"2024-04-23T14:32:15.179997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:12:19.437109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182768+0000","last_change":"2024-04-23T14:32:09.615554+0000","last_active":"2024-04-23T14:32:15.182768+0000","last_peered":"2024-04-23T14:32:15.182768+0000","last_clean":"2024-04-23T14:32:15.182768+0000","last_became_active":"2024-04-23T14:32:09.614793+0000","last_became_peered":"2024-04-23T14:32:09.614793+0000","last_unstale":"2024-04-23T14:32:15.182768+0000","last_undegraded":"2024-04-23T14:32:15.182768+0000","last_fullsized":"2024-04-23T14:32:15.182768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:01.062394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640359+0000","last_change":"2024-04-23T14:32:09.625443+0000","last_active":"2024-04-23T14:32:14.640359+0000","last_peered":"2024-04-23T14:32:14.640359+0000","last_clean":"2024-04-23T14:32:14.640359+0000","last_became_active":"2024-04-23T14:32:09.625140+0000","last_became_peered":"2024-04-23T14:32:09.625140+0000","last_unstale":"2024-04-23T14:32:14.640359+0000","last_undegraded":"2024-04-23T14:32:14.640359+0000","last_fullsized":"2024-04-23T14:32:14.640359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:12:19.437109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640042+0000","last_change":"2024-04-23T14:32:09.614187+0000","last_active":"2024-04-23T14:32:14.640042+0000","last_peered":"2024-04-23T14:32:14.640042+0000","last_clean":"2024-04-23T14:32:14.640042+0000","last_became_active":"2024-04-23T14:32:09.613813+0000","last_became_peered":"2024-04-23T14:32:09.613813+0000","last_unstale":"2024-04-23T14:32:14.640042+0000","last_undegraded":"2024-04-23T14:32:14.640042+0000","last_fullsized":"2024-04-23T14:32:14.640042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:12.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180552+0000","last_change":"2024-04-23T14:32:09.630660+0000","last_active":"2024-04-23T14:32:15.180552+0000","last_peered":"2024-04-23T14:32:15.180552+0000","last_clean":"2024-04-23T14:32:15.180552+0000","last_became_active":"2024-04-23T14:32:09.630381+0000","last_became_peered":"2024-04-23T14:32:09.630381+0000","last_unstale":"2024-04-23T14:32:15.180552+0000","last_undegraded":"2024-04-23T14:32:15.180552+0000","last_fullsized":"2024-04-23T14:32:15.180552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:12.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639936+0000","last_change":"2024-04-23T14:32:09.613974+0000","last_active":"2024-04-23T14:32:14.639936+0000","last_peered":"2024-04-23T14:32:14.639936+0000","last_clean":"2024-04-23T14:32:14.639936+0000","last_became_active":"2024-04-23T14:32:09.613699+0000","last_became_peered":"2024-04-23T14:32:09.613699+0000","last_unstale":"2024-04-23T14:32:14.639936+0000","last_undegraded":"2024-04-23T14:32:14.639936+0000","last_fullsized":"2024-04-23T14:32:14.639936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:08:31.155490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180930+0000","last_change":"2024-04-23T14:32:09.622174+0000","last_active":"2024-04-23T14:32:15.180930+0000","last_peered":"2024-04-23T14:32:15.180930+0000","last_clean":"2024-04-23T14:32:15.180930+0000","last_became_active":"2024-04-23T14:32:09.621360+0000","last_became_peered":"2024-04-23T14:32:09.621360+0000","last_unstale":"2024-04-23T14:32:15.180930+0000","last_undegraded":"2024-04-23T14:32:15.180930+0000","last_fullsized":"2024-04-23T14:32:15.180930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:12.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152292+0000","last_change":"2024-04-23T14:32:09.625529+0000","last_active":"2024-04-23T14:32:15.152292+0000","last_peered":"2024-04-23T14:32:15.152292+0000","last_clean":"2024-04-23T14:32:15.152292+0000","last_became_active":"2024-04-23T14:32:09.625259+0000","last_became_peered":"2024-04-23T14:32:09.625259+0000","last_unstale":"2024-04-23T14:32:15.152292+0000","last_undegraded":"2024-04-23T14:32:15.152292+0000","last_fullsized":"2024-04-23T14:32:15.152292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:12.536442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641776+0000","last_change":"2024-04-23T14:32:09.634616+0000","last_active":"2024-04-23T14:32:14.641776+0000","last_peered":"2024-04-23T14:32:14.641776+0000","last_clean":"2024-04-23T14:32:14.641776+0000","last_became_active":"2024-04-23T14:32:09.634328+0000","last_became_peered":"2024-04-23T14:32:09.634328+0000","last_unstale":"2024-04-23T14:32:14.641776+0000","last_undegraded":"2024-04-23T14:32:14.641776+0000","last_fullsized":"2024-04-23T14:32:14.641776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:52:42.867377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641980+0000","last_change":"2024-04-23T14:32:09.618704+0000","last_active":"2024-04-23T14:32:14.641980+0000","last_peered":"2024-04-23T14:32:14.641980+0000","last_clean":"2024-04-23T14:32:14.641980+0000","last_became_active":"2024-04-23T14:32:09.618328+0000","last_became_peered":"2024-04-23T14:32:09.618328+0000","last_unstale":"2024-04-23T14:32:14.641980+0000","last_undegraded":"2024-04-23T14:32:14.641980+0000","last_fullsized":"2024-04-23T14:32:14.641980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:50:24.043980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180068+0000","last_change":"2024-04-23T14:32:09.637988+0000","last_active":"2024-04-23T14:32:15.180068+0000","last_peered":"2024-04-23T14:32:15.180068+0000","last_clean":"2024-04-23T14:32:15.180068+0000","last_became_active":"2024-04-23T14:32:09.637766+0000","last_became_peered":"2024-04-23T14:32:09.637766+0000","last_unstale":"2024-04-23T14:32:15.180068+0000","last_undegraded":"2024-04-23T14:32:15.180068+0000","last_fullsized":"2024-04-23T14:32:15.180068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:09:07.994350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180933+0000","last_change":"2024-04-23T14:32:09.623580+0000","last_active":"2024-04-23T14:32:15.180933+0000","last_peered":"2024-04-23T14:32:15.180933+0000","last_clean":"2024-04-23T14:32:15.180933+0000","last_became_active":"2024-04-23T14:32:09.622942+0000","last_became_peered":"2024-04-23T14:32:09.622942+0000","last_unstale":"2024-04-23T14:32:15.180933+0000","last_undegraded":"2024-04-23T14:32:15.180933+0000","last_fullsized":"2024-04-23T14:32:15.180933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:52:42.867377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179379+0000","last_change":"2024-04-23T14:32:09.624523+0000","last_active":"2024-04-23T14:32:15.179379+0000","last_peered":"2024-04-23T14:32:15.179379+0000","last_clean":"2024-04-23T14:32:15.179379+0000","last_became_active":"2024-04-23T14:32:09.624229+0000","last_became_peered":"2024-04-23T14:32:09.624229+0000","last_unstale":"2024-04-23T14:32:15.179379+0000","last_undegraded":"2024-04-23T14:32:15.179379+0000","last_fullsized":"2024-04-23T14:32:15.179379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:09.066755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640175+0000","last_change":"2024-04-23T14:32:09.614929+0000","last_active":"2024-04-23T14:32:14.640175+0000","last_peered":"2024-04-23T14:32:14.640175+0000","last_clean":"2024-04-23T14:32:14.640175+0000","last_became_active":"2024-04-23T14:32:09.614584+0000","last_became_peered":"2024-04-23T14:32:09.614584+0000","last_unstale":"2024-04-23T14:32:14.640175+0000","last_undegraded":"2024-04-23T14:32:14.640175+0000","last_fullsized":"2024-04-23T14:32:14.640175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:22.220286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179498+0000","last_change":"2024-04-23T14:32:09.623580+0000","last_active":"2024-04-23T14:32:15.179498+0000","last_peered":"2024-04-23T14:32:15.179498+0000","last_clean":"2024-04-23T14:32:15.179498+0000","last_became_active":"2024-04-23T14:32:09.623197+0000","last_became_peered":"2024-04-23T14:32:09.623197+0000","last_unstale":"2024-04-23T14:32:15.179498+0000","last_undegraded":"2024-04-23T14:32:15.179498+0000","last_fullsized":"2024-04-23T14:32:15.179498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:52.459061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640154+0000","last_change":"2024-04-23T14:32:09.622573+0000","last_active":"2024-04-23T14:32:14.640154+0000","last_peered":"2024-04-23T14:32:14.640154+0000","last_clean":"2024-04-23T14:32:14.640154+0000","last_became_active":"2024-04-23T14:32:09.622232+0000","last_became_peered":"2024-04-23T14:32:09.622232+0000","last_unstale":"2024-04-23T14:32:14.640154+0000","last_undegraded":"2024-04-23T14:32:14.640154+0000","last_fullsized":"2024-04-23T14:32:14.640154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:52.459061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639881+0000","last_change":"2024-04-23T14:32:09.604425+0000","last_active":"2024-04-23T14:32:14.639881+0000","last_peered":"2024-04-23T14:32:14.639881+0000","last_clean":"2024-04-23T14:32:14.639881+0000","last_became_active":"2024-04-23T14:32:09.604159+0000","last_became_peered":"2024-04-23T14:32:09.604159+0000","last_unstale":"2024-04-23T14:32:14.639881+0000","last_undegraded":"2024-04-23T14:32:14.639881+0000","last_fullsized":"2024-04-23T14:32:14.639881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:50:24.043980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.183079+0000","last_change":"2024-04-23T14:32:09.614842+0000","last_active":"2024-04-23T14:32:15.183079+0000","last_peered":"2024-04-23T14:32:15.183079+0000","last_clean":"2024-04-23T14:32:15.183079+0000","last_became_active":"2024-04-23T14:32:09.614385+0000","last_became_peered":"2024-04-23T14:32:09.614385+0000","last_unstale":"2024-04-23T14:32:15.183079+0000","last_undegraded":"2024-04-23T14:32:15.183079+0000","last_fullsized":"2024-04-23T14:32:15.183079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:22.220286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179991+0000","last_change":"2024-04-23T14:32:09.628001+0000","last_active":"2024-04-23T14:32:15.179991+0000","last_peered":"2024-04-23T14:32:15.179991+0000","last_clean":"2024-04-23T14:32:15.179991+0000","last_became_active":"2024-04-23T14:32:09.627722+0000","last_became_peered":"2024-04-23T14:32:09.627722+0000","last_unstale":"2024-04-23T14:32:15.179991+0000","last_undegraded":"2024-04-23T14:32:15.179991+0000","last_fullsized":"2024-04-23T14:32:15.179991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:14:12.731737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:32:36.136 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-23T14:32:36.136 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-23T14:32:36.136 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-23T14:32:36.136 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-23T14:32:36.137 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-23T14:32:36.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.311+0000 7fa5f0e00640 1 Processor -- start 2024-04-23T14:32:36.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.311+0000 7fa5f0e00640 1 -- start start 2024-04-23T14:32:36.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.311+0000 7fa5f0e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fa5ec0cff50 0x7fa5ec06d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.311+0000 7fa5f0e00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5ec0b5820 con 0x7fa5ec15e820 2024-04-23T14:32:36.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.311+0000 7fa5f0e00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5ec0b5b00 con 0x7fa5ec0cfaf0 2024-04-23T14:32:36.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.311+0000 7fa5f0e00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fa5ec15ec00 con 0x7fa5ec0cff50 2024-04-23T14:32:36.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.315+0000 7fa5e9e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fa5ec15e820 0x7fa5ec0b4d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57210/0 (socket says 172.21.15.100:57210) 2024-04-23T14:32:36.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.315+0000 7fa5e9e00640 1 -- 172.21.15.100:0/220676408 learned_addr learned my addr 172.21.15.100:0/220676408 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:36.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.315+0000 7fa5ea800640 1 --2- 172.21.15.100:0/220676408 >> v2:172.21.15.151:3300/0 conn(0x7fa5ec0cff50 0x7fa5ec06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:36.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.315+0000 7fa5e9400640 1 -- 172.21.15.100:0/220676408 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 668121035 0 0) 0x7fa5ec0b5820 con 0x7fa5ec15e820 2024-04-23T14:32:36.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.315+0000 7fa5e9400640 1 -- 172.21.15.100:0/220676408 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5d0003530 con 0x7fa5ec15e820 2024-04-23T14:32:36.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.315+0000 7fa5ea800640 1 -- 172.21.15.100:0/220676408 >> v1:172.21.15.151:6789/0 conn(0x7fa5ec0cfaf0 legacy=0x7fa5ec06cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.315+0000 7fa5ea800640 1 -- 172.21.15.100:0/220676408 >> v1:172.21.15.100:6789/0 conn(0x7fa5ec15e820 legacy=0x7fa5ec0b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.315+0000 7fa5ea800640 1 -- 172.21.15.100:0/220676408 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ec0b5de0 con 0x7fa5ec0cff50 2024-04-23T14:32:36.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.315+0000 7fa5ea800640 1 --2- 172.21.15.100:0/220676408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5ec0cff50 0x7fa5ec06d250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009d90 tx=0x7fa5e401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3011ba5f777f72b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:36.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.315+0000 7fa5e9400640 1 -- 172.21.15.100:0/220676408 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e40b2070 con 0x7fa5ec0cff50 2024-04-23T14:32:36.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.315+0000 7fa5e9400640 1 -- 172.21.15.100:0/220676408 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5e40a5a00 con 0x7fa5ec0cff50 2024-04-23T14:32:36.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.315+0000 7fa5e9400640 1 -- 172.21.15.100:0/220676408 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e40a5dd0 con 0x7fa5ec0cff50 2024-04-23T14:32:36.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5f0e00640 1 -- 172.21.15.100:0/220676408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5ec0cff50 msgr2=0x7fa5ec06d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5f0e00640 1 --2- 172.21.15.100:0/220676408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5ec0cff50 0x7fa5ec06d250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009d90 tx=0x7fa5e401c940 comp rx=0 tx=0).stop 2024-04-23T14:32:36.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5f0e00640 1 -- 172.21.15.100:0/220676408 shutdown_connections 2024-04-23T14:32:36.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5f0e00640 1 --2- 172.21.15.100:0/220676408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5ec0cff50 0x7fa5ec06d250 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5f0e00640 1 -- 172.21.15.100:0/220676408 >> 172.21.15.100:0/220676408 conn(0x7fa5ec070690 msgr2=0x7fa5ec070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:36.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5f0e00640 1 -- 172.21.15.100:0/220676408 shutdown_connections 2024-04-23T14:32:36.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5f0e00640 1 -- 172.21.15.100:0/220676408 wait complete. 2024-04-23T14:32:36.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5f0e00640 1 Processor -- start 2024-04-23T14:32:36.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5f0e00640 1 -- start start 2024-04-23T14:32:36.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5f0e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5ec0cfaf0 0x7fa5ec150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5f0e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5ec0cff50 0x7fa5ec14ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5f0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5ec15e820 0x7fa5ec146c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5f0e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec0cf6e0 con 0x7fa5ec15e820 2024-04-23T14:32:36.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5f0e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec0b5490 con 0x7fa5ec0cff50 2024-04-23T14:32:36.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5f0e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa5ec0cf860 con 0x7fa5ec0cfaf0 2024-04-23T14:32:36.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5ea800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5ec15e820 0x7fa5ec146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:36.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5ea800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5ec15e820 0x7fa5ec146c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57288/0 (socket says 172.21.15.100:57288) 2024-04-23T14:32:36.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5ea800640 1 -- 172.21.15.100:0/119394286 learned_addr learned my addr 172.21.15.100:0/119394286 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:36.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5e9e00640 1 --2- 172.21.15.100:0/119394286 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5ec0cfaf0 0x7fa5ec150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:36.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5e3e00640 1 --2- 172.21.15.100:0/119394286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5ec0cff50 0x7fa5ec14ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:36.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5ea800640 1 -- 172.21.15.100:0/119394286 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5ec0cfaf0 msgr2=0x7fa5ec150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5ea800640 1 --2- 172.21.15.100:0/119394286 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5ec0cfaf0 0x7fa5ec150290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5ea800640 1 -- 172.21.15.100:0/119394286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5ec0cff50 msgr2=0x7fa5ec14ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5ea800640 1 --2- 172.21.15.100:0/119394286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5ec0cff50 0x7fa5ec14ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.319+0000 7fa5ea800640 1 -- 172.21.15.100:0/119394286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ec14b660 con 0x7fa5ec15e820 2024-04-23T14:32:36.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.323+0000 7fa5ea800640 1 --2- 172.21.15.100:0/119394286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5ec15e820 0x7fa5ec146c80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa5e40b0040 tx=0x7fa5ec06dfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:36.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.323+0000 7fa5e2000640 1 -- 172.21.15.100:0/119394286 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e40bd020 con 0x7fa5ec15e820 2024-04-23T14:32:36.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.323+0000 7fa5e2000640 1 -- 172.21.15.100:0/119394286 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5e40ad6c0 con 0x7fa5ec15e820 2024-04-23T14:32:36.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.323+0000 7fa5f0e00640 1 -- 172.21.15.100:0/119394286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5ec14b940 con 0x7fa5ec15e820 2024-04-23T14:32:36.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.323+0000 7fa5e2000640 1 -- 172.21.15.100:0/119394286 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e40b2040 con 0x7fa5ec15e820 2024-04-23T14:32:36.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.323+0000 7fa5f0e00640 1 -- 172.21.15.100:0/119394286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5ec14bf00 con 0x7fa5ec15e820 2024-04-23T14:32:36.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.323+0000 7fa5f0e00640 1 -- 172.21.15.100:0/119394286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5ec068ad0 con 0x7fa5ec15e820 2024-04-23T14:32:36.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.323+0000 7fa5e2000640 1 -- 172.21.15.100:0/119394286 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa5e40adab0 con 0x7fa5ec15e820 2024-04-23T14:32:36.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.323+0000 7fa5e2000640 1 --2- 172.21.15.100:0/119394286 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa5c80683b0 0x7fa5c806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.323+0000 7fa5e2000640 1 -- 172.21.15.100:0/119394286 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa5e41211f0 con 0x7fa5ec15e820 2024-04-23T14:32:36.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.323+0000 7fa5e9e00640 1 --2- 172.21.15.100:0/119394286 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa5c80683b0 0x7fa5c806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:36.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.327+0000 7fa5e9e00640 1 --2- 172.21.15.100:0/119394286 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa5c80683b0 0x7fa5c806a870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc0011f0 tx=0x7fa5dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:36.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.327+0000 7fa5e2000640 1 -- 172.21.15.100:0/119394286 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa5e4016560 con 0x7fa5ec15e820 2024-04-23T14:32:36.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.523+0000 7fa5f0e00640 1 -- 172.21.15.100:0/119394286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fa5ec15de40 con 0x7fa5ec15e820 2024-04-23T14:32:36.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.527+0000 7fa5e2000640 1 -- 172.21.15.100:0/119394286 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+168089 (secure 0 0 0) 0x7fa5e40b5350 con 0x7fa5ec15e820 2024-04-23T14:32:36.528 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:36.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.527+0000 7fa5f0e00640 1 -- 172.21.15.100:0/119394286 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa5c80683b0 msgr2=0x7fa5c806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.527+0000 7fa5f0e00640 1 --2- 172.21.15.100:0/119394286 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa5c80683b0 0x7fa5c806a870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc0011f0 tx=0x7fa5dc009040 comp rx=0 tx=0).stop 2024-04-23T14:32:36.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.527+0000 7fa5f0e00640 1 -- 172.21.15.100:0/119394286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5ec15e820 msgr2=0x7fa5ec146c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.527+0000 7fa5f0e00640 1 --2- 172.21.15.100:0/119394286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5ec15e820 0x7fa5ec146c80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa5e40b0040 tx=0x7fa5ec06dfa0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.527+0000 7fa5f0e00640 1 -- 172.21.15.100:0/119394286 shutdown_connections 2024-04-23T14:32:36.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.531+0000 7fa5f0e00640 1 --2- 172.21.15.100:0/119394286 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa5c80683b0 0x7fa5c806a870 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.531+0000 7fa5f0e00640 1 --2- 172.21.15.100:0/119394286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5ec15e820 0x7fa5ec146c80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.531+0000 7fa5f0e00640 1 --2- 172.21.15.100:0/119394286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5ec0cff50 0x7fa5ec14ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.531+0000 7fa5f0e00640 1 --2- 172.21.15.100:0/119394286 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5ec0cfaf0 0x7fa5ec150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.531+0000 7fa5f0e00640 1 -- 172.21.15.100:0/119394286 >> 172.21.15.100:0/119394286 conn(0x7fa5ec070690 msgr2=0x7fa5ec0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:36.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.531+0000 7fa5f0e00640 1 -- 172.21.15.100:0/119394286 shutdown_connections 2024-04-23T14:32:36.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.531+0000 7fa5f0e00640 1 -- 172.21.15.100:0/119394286 wait complete. 2024-04-23T14:32:36.549 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2859853957},{"type":"v1","addr":"172.21.15.100:6833","nonce":2859853957}]},"active_addr":"172.21.15.100:6833/2859853957","active_change":"2024-04-23T14:31:48.723461+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4103,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":909346773}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":31996296}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":1606491443}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":2439670853}]}]} 2024-04-23T14:32:36.552 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-23T14:32:36.552 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-23T14:32:36.553 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:36.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.687+0000 7fc0cc000640 1 Processor -- start 2024-04-23T14:32:36.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.687+0000 7fc0cc000640 1 -- start start 2024-04-23T14:32:36.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.687+0000 7fc0cc000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc0c415c470 0x7fc0c40b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.687+0000 7fc0cc000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0c4160de0 0x7fc0c40ad160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.687+0000 7fc0cc000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc0c40ad6a0 0x7fc0c406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.687+0000 7fc0cc000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc0c40adad0 con 0x7fc0c40ad6a0 2024-04-23T14:32:36.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.687+0000 7fc0cc000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fc0c4160320 con 0x7fc0c415c470 2024-04-23T14:32:36.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.687+0000 7fc0cc000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c40cc7e0 con 0x7fc0c4160de0 2024-04-23T14:32:36.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.687+0000 7fc0ca000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc0c40ad6a0 0x7fc0c406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:36.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.687+0000 7fc0ca000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc0c40ad6a0 0x7fc0c406bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57302/0 (socket says 172.21.15.100:57302) 2024-04-23T14:32:36.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.687+0000 7fc0ca000640 1 -- 172.21.15.100:0/3454513436 learned_addr learned my addr 172.21.15.100:0/3454513436 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:36.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.687+0000 7fc0c8c00640 1 --2- 172.21.15.100:0/3454513436 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0c4160de0 0x7fc0c40ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:36.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.687+0000 7fc0c9600640 1 --2- 172.21.15.100:0/3454513436 >> v2:172.21.15.151:3300/0 conn(0x7fc0c415c470 0x7fc0c40b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:36.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.687+0000 7fc0ca000640 1 -- 172.21.15.100:0/3454513436 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0c4160de0 msgr2=0x7fc0c40ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:36.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.687+0000 7fc0ca000640 1 --2- 172.21.15.100:0/3454513436 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0c4160de0 0x7fc0c40ad160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.687+0000 7fc0ca000640 1 -- 172.21.15.100:0/3454513436 >> v2:172.21.15.151:3300/0 conn(0x7fc0c415c470 msgr2=0x7fc0c40b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:36.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.687+0000 7fc0ca000640 1 --2- 172.21.15.100:0/3454513436 >> v2:172.21.15.151:3300/0 conn(0x7fc0c415c470 0x7fc0c40b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.687+0000 7fc0ca000640 1 -- 172.21.15.100:0/3454513436 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c41605e0 con 0x7fc0c40ad6a0 2024-04-23T14:32:36.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.691+0000 7fc0ca000640 1 --2- 172.21.15.100:0/3454513436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0c40ad6a0 0x7fc0c406bab0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4074300 tx=0x7fc0bc01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49a6261b4fca51 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:36.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.691+0000 7fc0c3e00640 1 -- 172.21.15.100:0/3454513436 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc0a56f0 con 0x7fc0c40ad6a0 2024-04-23T14:32:36.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.691+0000 7fc0c3e00640 1 -- 172.21.15.100:0/3454513436 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0bc0a5890 con 0x7fc0c40ad6a0 2024-04-23T14:32:36.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.691+0000 7fc0c3e00640 1 -- 172.21.15.100:0/3454513436 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc0a5c60 con 0x7fc0c40ad6a0 2024-04-23T14:32:36.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.691+0000 7fc0cc000640 1 -- 172.21.15.100:0/3454513436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0c40ad6a0 msgr2=0x7fc0c406bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.691+0000 7fc0cc000640 1 --2- 172.21.15.100:0/3454513436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0c40ad6a0 0x7fc0c406bab0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4074300 tx=0x7fc0bc01c960 comp rx=0 tx=0).stop 2024-04-23T14:32:36.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.691+0000 7fc0cc000640 1 -- 172.21.15.100:0/3454513436 shutdown_connections 2024-04-23T14:32:36.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.691+0000 7fc0cc000640 1 --2- 172.21.15.100:0/3454513436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0c40ad6a0 0x7fc0c406bab0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.691+0000 7fc0cc000640 1 --2- 172.21.15.100:0/3454513436 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0c4160de0 0x7fc0c40ad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.691+0000 7fc0cc000640 1 --2- 172.21.15.100:0/3454513436 >> v2:172.21.15.151:3300/0 conn(0x7fc0c415c470 0x7fc0c40b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.691+0000 7fc0cc000640 1 -- 172.21.15.100:0/3454513436 >> 172.21.15.100:0/3454513436 conn(0x7fc0c40b1e80 msgr2=0x7fc0c40b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:36.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.691+0000 7fc0cc000640 1 -- 172.21.15.100:0/3454513436 shutdown_connections 2024-04-23T14:32:36.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.691+0000 7fc0cc000640 1 -- 172.21.15.100:0/3454513436 wait complete. 2024-04-23T14:32:36.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0cc000640 1 Processor -- start 2024-04-23T14:32:36.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0cc000640 1 -- start start 2024-04-23T14:32:36.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0cc000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0c40ad6a0 0x7fc0c4156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0cc000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0c415c470 0x7fc0c4153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0cc000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0c4160de0 0x7fc0c40b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0cc000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c40cc370 con 0x7fc0c415c470 2024-04-23T14:32:36.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0cc000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c415caa0 con 0x7fc0c40ad6a0 2024-04-23T14:32:36.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0cc000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c40cc4f0 con 0x7fc0c4160de0 2024-04-23T14:32:36.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0c8c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0c415c470 0x7fc0c4153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:36.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0c8c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0c415c470 0x7fc0c4153e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57304/0 (socket says 172.21.15.100:57304) 2024-04-23T14:32:36.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0c8c00640 1 -- 172.21.15.100:0/1221994078 learned_addr learned my addr 172.21.15.100:0/1221994078 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:36.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0ca000640 1 --2- 172.21.15.100:0/1221994078 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0c4160de0 0x7fc0c40b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:36.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0c9600640 1 --2- 172.21.15.100:0/1221994078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0c40ad6a0 0x7fc0c4156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:36.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0c8c00640 1 -- 172.21.15.100:0/1221994078 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0c4160de0 msgr2=0x7fc0c40b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0c8c00640 1 --2- 172.21.15.100:0/1221994078 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0c4160de0 0x7fc0c40b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0c8c00640 1 -- 172.21.15.100:0/1221994078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0c40ad6a0 msgr2=0x7fc0c4156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0c8c00640 1 --2- 172.21.15.100:0/1221994078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0c40ad6a0 0x7fc0c4156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0c8c00640 1 -- 172.21.15.100:0/1221994078 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c4157380 con 0x7fc0c415c470 2024-04-23T14:32:36.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0c8c00640 1 --2- 172.21.15.100:0/1221994078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0c415c470 0x7fc0c4153e90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc0b000e960 tx=0x7fc0b000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:36.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0c1600640 1 -- 172.21.15.100:0/1221994078 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0017070 con 0x7fc0c415c470 2024-04-23T14:32:36.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0c1600640 1 -- 172.21.15.100:0/1221994078 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b000ca00 con 0x7fc0c415c470 2024-04-23T14:32:36.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0cc000640 1 -- 172.21.15.100:0/1221994078 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c4157660 con 0x7fc0c415c470 2024-04-23T14:32:36.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0c1600640 1 -- 172.21.15.100:0/1221994078 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b000cdb0 con 0x7fc0c415c470 2024-04-23T14:32:36.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.695+0000 7fc0cc000640 1 -- 172.21.15.100:0/1221994078 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c4157c20 con 0x7fc0c415c470 2024-04-23T14:32:36.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.699+0000 7fc0cc000640 1 -- 172.21.15.100:0/1221994078 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c4068ad0 con 0x7fc0c415c470 2024-04-23T14:32:36.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.699+0000 7fc0c1600640 1 -- 172.21.15.100:0/1221994078 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc0b001a070 con 0x7fc0c415c470 2024-04-23T14:32:36.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.699+0000 7fc0c1600640 1 --2- 172.21.15.100:0/1221994078 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc0a4068380 0x7fc0a406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.699+0000 7fc0c1600640 1 -- 172.21.15.100:0/1221994078 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc0b0086d10 con 0x7fc0c415c470 2024-04-23T14:32:36.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.699+0000 7fc0c9600640 1 --2- 172.21.15.100:0/1221994078 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc0a4068380 0x7fc0a406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:36.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.699+0000 7fc0c9600640 1 --2- 172.21.15.100:0/1221994078 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc0a4068380 0x7fc0a406a840 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc0c40b1ac0 tx=0x7fc0b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:36.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.703+0000 7fc0c1600640 1 -- 172.21.15.100:0/1221994078 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc0b008c020 con 0x7fc0c415c470 2024-04-23T14:32:36.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.863+0000 7fc0cc000640 1 -- 172.21.15.100:0/1221994078 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0c40b5460 con 0x7fc0c415c470 2024-04-23T14:32:36.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.863+0000 7fc0c1600640 1 -- 172.21.15.100:0/1221994078 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7fc0b0056f80 con 0x7fc0c415c470 2024-04-23T14:32:36.866 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:36.866 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":24,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:14.037027+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:36.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.867+0000 7fc0cc000640 1 -- 172.21.15.100:0/1221994078 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc0a4068380 msgr2=0x7fc0a406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.867+0000 7fc0cc000640 1 --2- 172.21.15.100:0/1221994078 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc0a4068380 0x7fc0a406a840 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc0c40b1ac0 tx=0x7fc0b8008040 comp rx=0 tx=0).stop 2024-04-23T14:32:36.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.867+0000 7fc0cc000640 1 -- 172.21.15.100:0/1221994078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0c415c470 msgr2=0x7fc0c4153e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.867+0000 7fc0cc000640 1 --2- 172.21.15.100:0/1221994078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0c415c470 0x7fc0c4153e90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc0b000e960 tx=0x7fc0b000ee20 comp rx=0 tx=0).stop 2024-04-23T14:32:36.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.867+0000 7fc0cc000640 1 -- 172.21.15.100:0/1221994078 shutdown_connections 2024-04-23T14:32:36.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.867+0000 7fc0cc000640 1 --2- 172.21.15.100:0/1221994078 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc0a4068380 0x7fc0a406a840 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.867+0000 7fc0cc000640 1 --2- 172.21.15.100:0/1221994078 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0c4160de0 0x7fc0c40b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.867+0000 7fc0cc000640 1 --2- 172.21.15.100:0/1221994078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0c415c470 0x7fc0c4153e90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.867+0000 7fc0cc000640 1 --2- 172.21.15.100:0/1221994078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0c40ad6a0 0x7fc0c4156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.867+0000 7fc0cc000640 1 -- 172.21.15.100:0/1221994078 >> 172.21.15.100:0/1221994078 conn(0x7fc0c40b1e80 msgr2=0x7fc0c40b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:36.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.867+0000 7fc0cc000640 1 -- 172.21.15.100:0/1221994078 shutdown_connections 2024-04-23T14:32:36.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:36.867+0000 7fc0cc000640 1 -- 172.21.15.100:0/1221994078 wait complete. 2024-04-23T14:32:36.887 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-23T14:32:36.887 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:37.014 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:32:37.019 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:32:37.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.023+0000 7f9cba600640 1 Processor -- start 2024-04-23T14:32:37.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.023+0000 7f9cba600640 1 -- start start 2024-04-23T14:32:37.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.023+0000 7f9cba600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9cb40d3e40 0x7f9cb40d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.023+0000 7f9cba600640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f9cb40d4c20 0x7f9cb40d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.023+0000 7f9cba600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cb40cfc40 con 0x7f9cb415eb10 2024-04-23T14:32:37.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.023+0000 7f9cba600640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f9cb40cfee0 con 0x7f9cb40d3e40 2024-04-23T14:32:37.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.023+0000 7f9cba600640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f9cb40d01c0 con 0x7f9cb40d4c20 2024-04-23T14:32:37.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.023+0000 7f9cb2a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9cb40d3e40 0x7f9cb40d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.023+0000 7f9cb2a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9cb40d3e40 0x7f9cb40d0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57316/0 (socket says 172.21.15.100:57316) 2024-04-23T14:32:37.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.023+0000 7f9cb2a00640 1 -- 172.21.15.100:0/2959734338 learned_addr learned my addr 172.21.15.100:0/2959734338 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.023+0000 7f9cb3e00640 1 --2- 172.21.15.100:0/2959734338 >> v2:172.21.15.151:3300/0 conn(0x7f9cb40d4c20 0x7f9cb40d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.023+0000 7f9cb2a00640 1 -- 172.21.15.100:0/2959734338 >> v2:172.21.15.151:3300/0 conn(0x7f9cb40d4c20 msgr2=0x7f9cb40d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.023+0000 7f9cb2a00640 1 --2- 172.21.15.100:0/2959734338 >> v2:172.21.15.151:3300/0 conn(0x7f9cb40d4c20 0x7f9cb40d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.023+0000 7f9cb2a00640 1 -- 172.21.15.100:0/2959734338 >> v1:172.21.15.100:6789/0 conn(0x7f9cb415eb10 legacy=0x7f9cb40d4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.023+0000 7f9cb2a00640 1 -- 172.21.15.100:0/2959734338 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb40d04a0 con 0x7f9cb40d3e40 2024-04-23T14:32:37.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.023+0000 7f9cb2000640 1 -- 172.21.15.100:0/2959734338 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 371490618 0 0) 0x7f9cb40cfc40 con 0x7f9cb415eb10 2024-04-23T14:32:37.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.027+0000 7f9cb2a00640 1 --2- 172.21.15.100:0/2959734338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cb40d3e40 0x7f9cb40d0f80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9cb408e520 tx=0x7f9ca001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de152ab306fb6508 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.027+0000 7f9cb2000640 1 -- 172.21.15.100:0/2959734338 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca00aa670 con 0x7f9cb40d3e40 2024-04-23T14:32:37.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.027+0000 7f9cb2000640 1 -- 172.21.15.100:0/2959734338 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca00af070 con 0x7f9cb40d3e40 2024-04-23T14:32:37.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.027+0000 7f9cb2000640 1 -- 172.21.15.100:0/2959734338 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca00aaa30 con 0x7f9cb40d3e40 2024-04-23T14:32:37.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.027+0000 7f9cba600640 1 -- 172.21.15.100:0/2959734338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cb40d3e40 msgr2=0x7f9cb40d0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.027+0000 7f9cba600640 1 --2- 172.21.15.100:0/2959734338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cb40d3e40 0x7f9cb40d0f80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9cb408e520 tx=0x7f9ca001c4f0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.027+0000 7f9cba600640 1 -- 172.21.15.100:0/2959734338 shutdown_connections 2024-04-23T14:32:37.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.027+0000 7f9cba600640 1 --2- 172.21.15.100:0/2959734338 >> v2:172.21.15.151:3300/0 conn(0x7f9cb40d4c20 0x7f9cb40d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.027+0000 7f9cba600640 1 --2- 172.21.15.100:0/2959734338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cb40d3e40 0x7f9cb40d0f80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.027+0000 7f9cba600640 1 -- 172.21.15.100:0/2959734338 >> 172.21.15.100:0/2959734338 conn(0x7f9cb40b9a30 msgr2=0x7f9cb40b75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.027+0000 7f9cba600640 1 -- 172.21.15.100:0/2959734338 shutdown_connections 2024-04-23T14:32:37.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.027+0000 7f9cba600640 1 -- 172.21.15.100:0/2959734338 wait complete. 2024-04-23T14:32:37.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cba600640 1 Processor -- start 2024-04-23T14:32:37.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cba600640 1 -- start start 2024-04-23T14:32:37.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cba600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cb40d4c20 0x7f9cb414c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cba600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cb415eb10 0x7f9cb4150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cba600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9cb415c400 0x7f9cb40b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cba600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb40d55a0 con 0x7f9cb415eb10 2024-04-23T14:32:37.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cba600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb40d5420 con 0x7f9cb40d4c20 2024-04-23T14:32:37.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cba600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb40b12c0 con 0x7f9cb415c400 2024-04-23T14:32:37.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cb2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cb415eb10 0x7f9cb4150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cb2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cb415eb10 0x7f9cb4150d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57320/0 (socket says 172.21.15.100:57320) 2024-04-23T14:32:37.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cb2a00640 1 -- 172.21.15.100:0/1145880265 learned_addr learned my addr 172.21.15.100:0/1145880265 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cb3400640 1 --2- 172.21.15.100:0/1145880265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cb40d4c20 0x7f9cb414c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cb3e00640 1 --2- 172.21.15.100:0/1145880265 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9cb415c400 0x7f9cb40b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cb2a00640 1 -- 172.21.15.100:0/1145880265 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9cb415c400 msgr2=0x7f9cb40b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cb2a00640 1 --2- 172.21.15.100:0/1145880265 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9cb415c400 0x7f9cb40b2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cb2a00640 1 -- 172.21.15.100:0/1145880265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cb40d4c20 msgr2=0x7f9cb414c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cb2a00640 1 --2- 172.21.15.100:0/1145880265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cb40d4c20 0x7f9cb414c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cb2a00640 1 -- 172.21.15.100:0/1145880265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb414f8c0 con 0x7f9cb415eb10 2024-04-23T14:32:37.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cb3e00640 1 --2- 172.21.15.100:0/1145880265 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9cb415c400 0x7f9cb40b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:37.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cb2a00640 1 --2- 172.21.15.100:0/1145880265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cb415eb10 0x7f9cb4150d20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9ca001a120 tx=0x7f9ca001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cab400640 1 -- 172.21.15.100:0/1145880265 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca00bd5f0 con 0x7f9cb415eb10 2024-04-23T14:32:37.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cab400640 1 -- 172.21.15.100:0/1145880265 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca00bd7b0 con 0x7f9cb415eb10 2024-04-23T14:32:37.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cba600640 1 -- 172.21.15.100:0/1145880265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb414e150 con 0x7f9cb415eb10 2024-04-23T14:32:37.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.031+0000 7f9cab400640 1 -- 172.21.15.100:0/1145880265 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca00bdba0 con 0x7f9cb415eb10 2024-04-23T14:32:37.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.035+0000 7f9cba600640 1 -- 172.21.15.100:0/1145880265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb414f5e0 con 0x7f9cb415eb10 2024-04-23T14:32:37.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.035+0000 7f9cba600640 1 -- 172.21.15.100:0/1145880265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cb4068ad0 con 0x7f9cb415eb10 2024-04-23T14:32:37.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.035+0000 7f9cab400640 1 -- 172.21.15.100:0/1145880265 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9ca00b33f0 con 0x7f9cb415eb10 2024-04-23T14:32:37.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.035+0000 7f9cab400640 1 --2- 172.21.15.100:0/1145880265 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9c94068380 0x7f9c9406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.035+0000 7f9cab400640 1 -- 172.21.15.100:0/1145880265 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9ca0121af0 con 0x7f9cb415eb10 2024-04-23T14:32:37.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.035+0000 7f9cb3400640 1 --2- 172.21.15.100:0/1145880265 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9c94068380 0x7f9c9406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.035+0000 7f9cb3400640 1 --2- 172.21.15.100:0/1145880265 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9c94068380 0x7f9c9406a840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4000f70 tx=0x7f9ca4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.039+0000 7f9cab400640 1 -- 172.21.15.100:0/1145880265 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9ca00c3060 con 0x7f9cb415eb10 2024-04-23T14:32:37.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.199+0000 7f9cba600640 1 -- 172.21.15.100:0/1145880265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9cb40b47e0 con 0x7f9cb415eb10 2024-04-23T14:32:37.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.199+0000 7f9cab400640 1 -- 172.21.15.100:0/1145880265 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f9cb414f8c0 con 0x7f9cb415eb10 2024-04-23T14:32:37.200 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:37.200 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":24,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:14.037027+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:37.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.203+0000 7f9cba600640 1 -- 172.21.15.100:0/1145880265 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9c94068380 msgr2=0x7f9c9406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.203+0000 7f9cba600640 1 --2- 172.21.15.100:0/1145880265 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9c94068380 0x7f9c9406a840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4000f70 tx=0x7f9ca4009040 comp rx=0 tx=0).stop 2024-04-23T14:32:37.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.203+0000 7f9cba600640 1 -- 172.21.15.100:0/1145880265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cb415eb10 msgr2=0x7f9cb4150d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.203+0000 7f9cba600640 1 --2- 172.21.15.100:0/1145880265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cb415eb10 0x7f9cb4150d20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9ca001a120 tx=0x7f9ca001ad30 comp rx=0 tx=0).stop 2024-04-23T14:32:37.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.203+0000 7f9cba600640 1 -- 172.21.15.100:0/1145880265 shutdown_connections 2024-04-23T14:32:37.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.203+0000 7f9cba600640 1 --2- 172.21.15.100:0/1145880265 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9c94068380 0x7f9c9406a840 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.203+0000 7f9cba600640 1 --2- 172.21.15.100:0/1145880265 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9cb415c400 0x7f9cb40b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.203+0000 7f9cba600640 1 --2- 172.21.15.100:0/1145880265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cb415eb10 0x7f9cb4150d20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.203+0000 7f9cba600640 1 --2- 172.21.15.100:0/1145880265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cb40d4c20 0x7f9cb414c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.203+0000 7f9cba600640 1 -- 172.21.15.100:0/1145880265 >> 172.21.15.100:0/1145880265 conn(0x7f9cb40b9a30 msgr2=0x7f9cb40d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.203+0000 7f9cba600640 1 -- 172.21.15.100:0/1145880265 shutdown_connections 2024-04-23T14:32:37.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.203+0000 7f9cba600640 1 -- 172.21.15.100:0/1145880265 wait complete. 2024-04-23T14:32:37.222 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-23T14:32:37.222 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-23T14:32:37.223 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-23T14:32:37.223 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-23T14:32:37.224 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-23T14:32:37.224 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-23T14:32:37.224 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-23T14:32:37.225 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-23T14:32:37.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.463+0000 7fd10a000640 1 Processor -- start 2024-04-23T14:32:37.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.463+0000 7feba3200640 1 Processor -- start 2024-04-23T14:32:37.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.463+0000 7fd10a000640 1 -- start start 2024-04-23T14:32:37.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.463+0000 7feba3200640 1 -- start start 2024-04-23T14:32:37.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.463+0000 7fd10a000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd104073c30 0x7fd10415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.463+0000 7fd10a000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd1040cde20 0x7fd1040d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.463+0000 7fd10a000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd1040b5c40 0x7fd1040c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.463+0000 7fd10a000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fd1040d1190 con 0x7fd104073c30 2024-04-23T14:32:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7feba3200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7feb9c04fa80 0x7feb9c04fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7feba3200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7feb9c0df390 0x7feb9c007190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.463+0000 7fd10a000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fd10406c010 con 0x7fd1040b5c40 2024-04-23T14:32:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.463+0000 7fd10a000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd1040d0640 con 0x7fd1040cde20 2024-04-23T14:32:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7fd103400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd104073c30 0x7fd10415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7fd103e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd1040b5c40 0x7fd1040c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7fd103400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd104073c30 0x7fd10415aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57328/0 (socket says 172.21.15.100:57328) 2024-04-23T14:32:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7fd103400640 1 -- 172.21.15.100:0/2051355884 learned_addr learned my addr 172.21.15.100:0/2051355884 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7fd102a00640 1 --2- 172.21.15.100:0/2051355884 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd1040cde20 0x7fd1040d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7feba3200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb9c054390 con 0x7feb9c04f5c0 2024-04-23T14:32:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7feba3200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7feb9c005ea0 con 0x7feb9c04fa80 2024-04-23T14:32:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7feba3200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7feb9c006d00 con 0x7feb9c0df390 2024-04-23T14:32:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7fd103400640 1 -- 172.21.15.100:0/2051355884 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd1040cde20 msgr2=0x7fd1040d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7fd103400640 1 --2- 172.21.15.100:0/2051355884 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd1040cde20 0x7fd1040d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7fd103400640 1 -- 172.21.15.100:0/2051355884 >> v2:172.21.15.151:3300/0 conn(0x7fd1040b5c40 msgr2=0x7fd1040c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7fd103400640 1 --2- 172.21.15.100:0/2051355884 >> v2:172.21.15.151:3300/0 conn(0x7fd1040b5c40 0x7fd1040c62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7feba1e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7feb9c04fa80 0x7feb9c04fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7feba1e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7feb9c04fa80 0x7feb9c04fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57340/0 (socket says 172.21.15.100:57340) 2024-04-23T14:32:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7feba1e00640 1 -- 172.21.15.100:0/1837771626 learned_addr learned my addr 172.21.15.100:0/1837771626 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7feba1400640 1 --2- 172.21.15.100:0/1837771626 >> v2:172.21.15.151:3300/0 conn(0x7feb9c0df390 0x7feb9c007190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7fd103400640 1 -- 172.21.15.100:0/2051355884 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1040d08f0 con 0x7fd104073c30 2024-04-23T14:32:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7fd103e00640 1 --2- 172.21.15.100:0/2051355884 >> v2:172.21.15.151:3300/0 conn(0x7fd1040b5c40 0x7fd1040c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7fd102a00640 1 --2- 172.21.15.100:0/2051355884 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd1040cde20 0x7fd1040d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7feba1e00640 1 -- 172.21.15.100:0/1837771626 >> v2:172.21.15.151:3300/0 conn(0x7feb9c0df390 msgr2=0x7feb9c007190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7feba1e00640 1 --2- 172.21.15.100:0/1837771626 >> v2:172.21.15.151:3300/0 conn(0x7feb9c0df390 0x7feb9c007190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7feba1e00640 1 -- 172.21.15.100:0/1837771626 >> v1:172.21.15.100:6789/0 conn(0x7feb9c04f5c0 legacy=0x7feb9c038020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7feba1e00640 1 -- 172.21.15.100:0/1837771626 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9c0503a0 con 0x7feb9c04fa80 2024-04-23T14:32:37.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7feba1400640 1 --2- 172.21.15.100:0/1837771626 >> v2:172.21.15.151:3300/0 conn(0x7feb9c0df390 0x7feb9c007190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:37.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.467+0000 7fd361200640 1 Processor -- start 2024-04-23T14:32:37.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.471+0000 7feba1e00640 1 --2- 172.21.15.100:0/1837771626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb9c04fa80 0x7feb9c04fe60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7feb9c050ca0 tx=0x7feb9401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54d7e67f0197f68 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.471+0000 7fd103400640 1 --2- 172.21.15.100:0/2051355884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd104073c30 0x7fd10415aa80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd1040b3230 tx=0x7fd0f401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ee2ecf10cecf79 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.471+0000 7feba0a00640 1 -- 172.21.15.100:0/1837771626 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb940aa610 con 0x7feb9c04fa80 2024-04-23T14:32:37.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.471+0000 7fd361200640 1 -- start start 2024-04-23T14:32:37.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.471+0000 7fd361200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd35c166010 0x7fd35c072480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.471+0000 7fd361200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd35c0cbb60 0x7fd35c0cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.471+0000 7fd361200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd35c0cdfc0 con 0x7fd35c06cac0 2024-04-23T14:32:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.471+0000 7fd361200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fd35c0ca920 con 0x7fd35c166010 2024-04-23T14:32:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.471+0000 7fd361200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd35c0ce2a0 con 0x7fd35c0cbb60 2024-04-23T14:32:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.471+0000 7fec33c00640 1 Processor -- start 2024-04-23T14:32:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.471+0000 7fec33c00640 1 -- start start 2024-04-23T14:32:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.471+0000 7fd35be00640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7fd35c06cac0 0x7fd35c0cadb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:58448/0 (socket says 172.21.15.100:58448) 2024-04-23T14:32:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.471+0000 7fd35be00640 1 -- 172.21.15.100:0/1314325898 learned_addr learned my addr 172.21.15.100:0/1314325898 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.471+0000 7fd102000640 1 -- 172.21.15.100:0/2051355884 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f40b0070 con 0x7fd104073c30 2024-04-23T14:32:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.471+0000 7feba0a00640 1 -- 172.21.15.100:0/1837771626 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb940af070 con 0x7feb9c04fa80 2024-04-23T14:32:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.471+0000 7feba0a00640 1 -- 172.21.15.100:0/1837771626 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb940aa9d0 con 0x7feb9c04fa80 2024-04-23T14:32:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.471+0000 7fd35b400640 1 --2- 172.21.15.100:0/1314325898 >> v2:172.21.15.100:3300/0 conn(0x7fd35c166010 0x7fd35c072480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.471+0000 7fd102000640 1 -- 172.21.15.100:0/2051355884 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0f40ab600 con 0x7fd104073c30 2024-04-23T14:32:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.471+0000 7fd102000640 1 -- 172.21.15.100:0/2051355884 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f40ab9d0 con 0x7fd104073c30 2024-04-23T14:32:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.471+0000 7fd35aa00640 1 --2- 172.21.15.100:0/1314325898 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd35c0cbb60 0x7fd35c0cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.471+0000 7fd35a000640 1 -- 172.21.15.100:0/1314325898 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2557988388 0 0) 0x7fd35c0cdfc0 con 0x7fd35c06cac0 2024-04-23T14:32:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.471+0000 7fd35a000640 1 -- 172.21.15.100:0/1314325898 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd340003530 con 0x7fd35c06cac0 2024-04-23T14:32:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd10a000640 1 -- 172.21.15.100:0/2051355884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd104073c30 msgr2=0x7fd10415aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd10a000640 1 --2- 172.21.15.100:0/2051355884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd104073c30 0x7fd10415aa80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd1040b3230 tx=0x7fd0f401c460 comp rx=0 tx=0).stop 2024-04-23T14:32:37.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd10a000640 1 -- 172.21.15.100:0/2051355884 shutdown_connections 2024-04-23T14:32:37.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd10a000640 1 --2- 172.21.15.100:0/2051355884 >> v2:172.21.15.151:3300/0 conn(0x7fd1040b5c40 0x7fd1040c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd10a000640 1 --2- 172.21.15.100:0/2051355884 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd1040cde20 0x7fd1040d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd10a000640 1 --2- 172.21.15.100:0/2051355884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd104073c30 0x7fd10415aa80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd10a000640 1 -- 172.21.15.100:0/2051355884 >> 172.21.15.100:0/2051355884 conn(0x7fd104073110 msgr2=0x7fd104073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7feba3200640 1 -- 172.21.15.100:0/1837771626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb9c04fa80 msgr2=0x7feb9c04fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7feba3200640 1 --2- 172.21.15.100:0/1837771626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb9c04fa80 0x7feb9c04fe60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7feb9c050ca0 tx=0x7feb9401c4b0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35a000640 1 -- 172.21.15.100:0/1314325898 <== mon.1 v1:172.21.15.151:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3718534972 0 0) 0x7fd340003530 con 0x7fd35c06cac0 2024-04-23T14:32:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35a000640 1 -- 172.21.15.100:0/1314325898 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd35c0cbb60 msgr2=0x7fd35c0cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35a000640 1 --2- 172.21.15.100:0/1314325898 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd35c0cbb60 0x7fd35c0cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35a000640 1 -- 172.21.15.100:0/1314325898 >> v2:172.21.15.100:3300/0 conn(0x7fd35c166010 msgr2=0x7fd35c072480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd10a000640 1 -- 172.21.15.100:0/2051355884 shutdown_connections 2024-04-23T14:32:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd10a000640 1 -- 172.21.15.100:0/2051355884 wait complete. 2024-04-23T14:32:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7feba3200640 1 -- 172.21.15.100:0/1837771626 shutdown_connections 2024-04-23T14:32:37.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7feba3200640 1 --2- 172.21.15.100:0/1837771626 >> v2:172.21.15.151:3300/0 conn(0x7feb9c0df390 0x7feb9c007190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7feba3200640 1 --2- 172.21.15.100:0/1837771626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb9c04fa80 0x7feb9c04fe60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7feba3200640 1 -- 172.21.15.100:0/1837771626 >> 172.21.15.100:0/1837771626 conn(0x7feb9c009fc0 msgr2=0x7feb9c00a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7feba3200640 1 -- 172.21.15.100:0/1837771626 shutdown_connections 2024-04-23T14:32:37.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7feba3200640 1 -- 172.21.15.100:0/1837771626 wait complete. 2024-04-23T14:32:37.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35a000640 1 --2- 172.21.15.100:0/1314325898 >> v2:172.21.15.100:3300/0 conn(0x7fd35c166010 0x7fd35c072480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35a000640 1 -- 172.21.15.100:0/1314325898 --> v1:172.21.15.151:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35c0ce580 con 0x7fd35c06cac0 2024-04-23T14:32:37.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35a000640 1 -- 172.21.15.100:0/1314325898 <== mon.1 v1:172.21.15.151:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7fd34c001660 con 0x7fd35c06cac0 2024-04-23T14:32:37.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35a000640 1 -- 172.21.15.100:0/1314325898 >> v1:172.21.15.151:6789/0 conn(0x7fd35c06cac0 legacy=0x7fd35c0cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35a000640 1 --2- 172.21.15.100:0/1314325898 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd35c0cbb60 0x7fd3400039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35a000640 1 --2- 172.21.15.100:0/1314325898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd340003f60 0x7fd340004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35a000640 1 --2- 172.21.15.100:0/1314325898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd340004d50 0x7fd3400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35a000640 1 -- 172.21.15.100:0/1314325898 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c0ca920 con 0x7fd340003f60 2024-04-23T14:32:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35a000640 1 -- 172.21.15.100:0/1314325898 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c0ce2a0 con 0x7fd340004d50 2024-04-23T14:32:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35a000640 1 -- 172.21.15.100:0/1314325898 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd34000ba20 con 0x7fd35c0cbb60 2024-04-23T14:32:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35b400640 1 --2- 172.21.15.100:0/1314325898 >> v2:172.21.15.100:3300/0 conn(0x7fd35c166010 0x7fd35c072480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35be00640 1 --2- 172.21.15.100:0/1314325898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd340004d50 0x7fd3400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35b400640 1 --2- 172.21.15.100:0/1314325898 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd35c0cbb60 0x7fd3400039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec33c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec2c15c610 0x7fec2c06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec33c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fec2c155c50 0x7fec2c15f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec33c00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec2c06f580 con 0x7fec2c1557f0 2024-04-23T14:32:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec33c00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fec2c06f860 con 0x7fec2c155c50 2024-04-23T14:32:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec33c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fec2c06fb40 con 0x7fec2c15c610 2024-04-23T14:32:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec2be00640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7fec2c1557f0 0x7fec2c15eff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:58458/0 (socket says 172.21.15.100:58458) 2024-04-23T14:32:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec2be00640 1 -- 172.21.15.100:0/1917850398 learned_addr learned my addr 172.21.15.100:0/1917850398 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec31200640 1 --2- 172.21.15.100:0/1917850398 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec2c15c610 0x7fec2c06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35aa00640 1 --2- 172.21.15.100:0/1314325898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd340003f60 0x7fd340004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec2b400640 1 -- 172.21.15.100:0/1917850398 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2369852429 0 0) 0x7fec2c06f580 con 0x7fec2c1557f0 2024-04-23T14:32:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec2b400640 1 -- 172.21.15.100:0/1917850398 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec14003530 con 0x7fec2c1557f0 2024-04-23T14:32:37.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35be00640 1 -- 172.21.15.100:0/1314325898 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd35c0cbb60 msgr2=0x7fd3400039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35be00640 1 --2- 172.21.15.100:0/1314325898 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd35c0cbb60 0x7fd3400039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35be00640 1 -- 172.21.15.100:0/1314325898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd340003f60 msgr2=0x7fd340004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35be00640 1 --2- 172.21.15.100:0/1314325898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd340003f60 0x7fd340004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35be00640 1 -- 172.21.15.100:0/1314325898 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd340003530 con 0x7fd340004d50 2024-04-23T14:32:37.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fd35aa00640 1 --2- 172.21.15.100:0/1314325898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd340003f60 0x7fd340004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:37.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec2b400640 1 -- 172.21.15.100:0/1917850398 <== mon.1 v1:172.21.15.151:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 786403725 0 0) 0x7fec14003530 con 0x7fec2c1557f0 2024-04-23T14:32:37.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec2b400640 1 -- 172.21.15.100:0/1917850398 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec2c15c610 msgr2=0x7fec2c06ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec2b400640 1 --2- 172.21.15.100:0/1917850398 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec2c15c610 0x7fec2c06ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec2b400640 1 -- 172.21.15.100:0/1917850398 >> v2:172.21.15.100:3300/0 conn(0x7fec2c155c50 msgr2=0x7fec2c15f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec2b400640 1 --2- 172.21.15.100:0/1917850398 >> v2:172.21.15.100:3300/0 conn(0x7fec2c155c50 0x7fec2c15f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec2b400640 1 -- 172.21.15.100:0/1917850398 --> v1:172.21.15.151:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2c0b43c0 con 0x7fec2c1557f0 2024-04-23T14:32:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7f1e3d400640 1 Processor -- start 2024-04-23T14:32:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec2b400640 1 -- 172.21.15.100:0/1917850398 <== mon.1 v1:172.21.15.151:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7fec18001660 con 0x7fec2c1557f0 2024-04-23T14:32:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec2b400640 1 -- 172.21.15.100:0/1917850398 >> v1:172.21.15.151:6789/0 conn(0x7fec2c1557f0 legacy=0x7fec2c15eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec2b400640 1 --2- 172.21.15.100:0/1917850398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec140039c0 0x7fec14003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec2b400640 1 --2- 172.21.15.100:0/1917850398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec14004810 0x7fec14004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec2b400640 1 --2- 172.21.15.100:0/1917850398 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec140051d0 0x7fec1400fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec2b400640 1 -- 172.21.15.100:0/1917850398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c06f860 con 0x7fec14004810 2024-04-23T14:32:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec2b400640 1 -- 172.21.15.100:0/1917850398 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c06fb40 con 0x7fec140039c0 2024-04-23T14:32:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec2b400640 1 -- 172.21.15.100:0/1917850398 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fec14005a20 con 0x7fec140051d0 2024-04-23T14:32:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7ffa5fa00640 1 Processor -- start 2024-04-23T14:32:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec2be00640 1 --2- 172.21.15.100:0/1917850398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec14004810 0x7fec14004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7fec31c00640 1 --2- 172.21.15.100:0/1917850398 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec140051d0 0x7fec1400fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.475+0000 7ffa5fa00640 1 -- start start 2024-04-23T14:32:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7fd10a000640 1 Processor -- start 2024-04-23T14:32:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7fd10a000640 1 -- start start 2024-04-23T14:32:37.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7feba3200640 1 Processor -- start 2024-04-23T14:32:37.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7fec31200640 1 --2- 172.21.15.100:0/1917850398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec140039c0 0x7fec14003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7f1e3d400640 1 -- start start 2024-04-23T14:32:37.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7fd10a000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd1040b5c40 0x7fd1041551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7fd10a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1040cde20 0x7fd1040c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7feba3200640 1 -- start start 2024-04-23T14:32:37.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7feba3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb9c04f5c0 0x7feb9c0d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7feba3200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb9c04fa80 0x7feb9c037080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7feba3200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb9c0df390 0x7feb9c0d5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7feba3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c006990 con 0x7feb9c04f5c0 2024-04-23T14:32:37.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7feba3200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c0076d0 con 0x7feb9c0df390 2024-04-23T14:32:37.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7feba3200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7feb9c0cfaf0 con 0x7feb9c04fa80 2024-04-23T14:32:37.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7feba1400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb9c04fa80 0x7feb9c037080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7fbd52e00640 1 Processor -- start 2024-04-23T14:32:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7fbd52e00640 1 -- start start 2024-04-23T14:32:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7ffa5fa00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ffa5815d050 0x7ffa58163b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7ffa5fa00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ffa580afb80 0x7ffa5806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7ffa5fa00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ffa5806ef60 0x7ffa580b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7ffa5fa00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ffa5806bf80 con 0x7ffa5806ef60 2024-04-23T14:32:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7ffa5fa00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7ffa58164070 con 0x7ffa5815d050 2024-04-23T14:32:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7ffa5fa00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ffa5806bc30 con 0x7ffa580afb80 2024-04-23T14:32:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7ffa57e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ffa580afb80 0x7ffa5806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7ffa57e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ffa580afb80 0x7ffa5806ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:49640/0 (socket says 172.21.15.100:49640) 2024-04-23T14:32:37.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7ffa57e00640 1 -- 172.21.15.100:0/1779698298 learned_addr learned my addr 172.21.15.100:0/1779698298 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7ffa5d000640 1 --2- 172.21.15.100:0/1779698298 >> v2:172.21.15.151:3300/0 conn(0x7ffa5815d050 0x7ffa58163b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7feba1400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb9c04fa80 0x7feb9c037080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:49652/0 (socket says 172.21.15.100:49652) 2024-04-23T14:32:37.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7feba1400640 1 -- 172.21.15.100:0/13197564 learned_addr learned my addr 172.21.15.100:0/13197564 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7feba2800640 1 --2- 172.21.15.100:0/13197564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb9c0df390 0x7feb9c0d5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7fd10a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd104099810 0x7fd104204bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7fec31200640 1 -- 172.21.15.100:0/1917850398 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec140051d0 msgr2=0x7fec1400fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7fec31200640 1 --2- 172.21.15.100:0/1917850398 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec140051d0 0x7fec1400fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7fec31200640 1 -- 172.21.15.100:0/1917850398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec14004810 msgr2=0x7fec14004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7fec31200640 1 --2- 172.21.15.100:0/1917850398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec14004810 0x7fec14004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7fec31200640 1 -- 172.21.15.100:0/1917850398 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec14003530 con 0x7fec140039c0 2024-04-23T14:32:37.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7fd10a000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd1040cd4f0 con 0x7fd1040cde20 2024-04-23T14:32:37.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7fd10a000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd1040cd370 con 0x7fd104099810 2024-04-23T14:32:37.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7fd10a000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd1040cd670 con 0x7fd1040b5c40 2024-04-23T14:32:37.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7f1e3d400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f1e380b5c40 0x7f1e3815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7fd103400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd1040b5c40 0x7fd1041551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7fd103400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd1040b5c40 0x7fd1041551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:49668/0 (socket says 172.21.15.100:49668) 2024-04-23T14:32:37.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7fd103400640 1 -- 172.21.15.100:0/3005792584 learned_addr learned my addr 172.21.15.100:0/3005792584 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7f1e3d400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e380d1190 con 0x7f1e380cde20 2024-04-23T14:32:37.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7f1e3d400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e3806c010 con 0x7f1e38073c30 2024-04-23T14:32:37.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7f1e3d400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f1e380d08f0 con 0x7f1e380b5c40 2024-04-23T14:32:37.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7ffa57e00640 1 -- 172.21.15.100:0/1779698298 >> v2:172.21.15.151:3300/0 conn(0x7ffa5815d050 msgr2=0x7ffa58163b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7ffa57e00640 1 --2- 172.21.15.100:0/1779698298 >> v2:172.21.15.151:3300/0 conn(0x7ffa5815d050 0x7ffa58163b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7ffa57e00640 1 -- 172.21.15.100:0/1779698298 >> v2:172.21.15.100:3300/0 conn(0x7ffa5806ef60 msgr2=0x7ffa580b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7ffa57e00640 1 --2- 172.21.15.100:0/1779698298 >> v2:172.21.15.100:3300/0 conn(0x7ffa5806ef60 0x7ffa580b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7ffa57e00640 1 -- 172.21.15.100:0/1779698298 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5806e1d0 con 0x7ffa580afb80 2024-04-23T14:32:37.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fd35be00640 1 --2- 172.21.15.100:0/1314325898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd340004d50 0x7fd3400051d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00c190 tx=0x7fd34c01cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bca17dd72a2a347 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7f1e36800640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f1e38073c30 0x7f1e380d0100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:58468/0 (socket says 172.21.15.100:58468) 2024-04-23T14:32:37.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7f1e36800640 1 -- 172.21.15.100:0/2113226807 learned_addr learned my addr 172.21.15.100:0/2113226807 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:37.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7fd103e00640 1 --2- 172.21.15.100:0/3005792584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd104099810 0x7fd104204bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fbd52e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbd4c15aa80 0x7fbd4c0d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fd103e00640 1 -- 172.21.15.100:0/3005792584 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd1040b5c40 msgr2=0x7fd1041551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fd103e00640 1 --2- 172.21.15.100:0/3005792584 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd1040b5c40 0x7fd1041551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fd103e00640 1 -- 172.21.15.100:0/3005792584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1040cde20 msgr2=0x7fd1040c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fd103e00640 1 --2- 172.21.15.100:0/3005792584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1040cde20 0x7fd1040c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fd103e00640 1 -- 172.21.15.100:0/3005792584 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1040c9d00 con 0x7fd104099810 2024-04-23T14:32:37.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fd103e00640 1 --2- 172.21.15.100:0/3005792584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd104099810 0x7fd104204bd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00eed0 tx=0x7fd0fc00b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7feba1e00640 1 --2- 172.21.15.100:0/13197564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb9c04f5c0 0x7feb9c0d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7feba2800640 1 -- 172.21.15.100:0/13197564 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb9c04fa80 msgr2=0x7feb9c037080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7f1e37200640 1 --2- 172.21.15.100:0/2113226807 >> v2:172.21.15.151:3300/0 conn(0x7f1e380b5c40 0x7f1e3815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fbd52e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbd4c0b5c40 0x7fbd4c06c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fbd52e00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd4c0d1190 con 0x7fbd4c0cde20 2024-04-23T14:32:37.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fbd52e00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fbd4c06c010 con 0x7fbd4c15aa80 2024-04-23T14:32:37.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fbd52e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fbd4c0d0640 con 0x7fbd4c0b5c40 2024-04-23T14:32:37.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fbd50e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbd4c0b5c40 0x7fbd4c06c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fbd50e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbd4c0b5c40 0x7fbd4c06c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:49674/0 (socket says 172.21.15.100:49674) 2024-04-23T14:32:37.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fbd50e00640 1 -- 172.21.15.100:0/2874945264 learned_addr learned my addr 172.21.15.100:0/2874945264 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.479+0000 7fabba200640 1 Processor -- start 2024-04-23T14:32:37.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fabba200640 1 -- start start 2024-04-23T14:32:37.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fd35a000640 1 -- 172.21.15.100:0/1314325898 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c007d00 con 0x7fd340004d50 2024-04-23T14:32:37.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fd35a000640 1 -- 172.21.15.100:0/1314325898 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd34c0a5b40 con 0x7fd340004d50 2024-04-23T14:32:37.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fd35a000640 1 -- 172.21.15.100:0/1314325898 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c0071b0 con 0x7fd340004d50 2024-04-23T14:32:37.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fd0fb400640 1 -- 172.21.15.100:0/3005792584 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc018030 con 0x7fd104099810 2024-04-23T14:32:37.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fd0fb400640 1 -- 172.21.15.100:0/3005792584 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0fc00bd20 con 0x7fd104099810 2024-04-23T14:32:37.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fd0fb400640 1 -- 172.21.15.100:0/3005792584 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc003be0 con 0x7fd104099810 2024-04-23T14:32:37.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fd10a000640 1 -- 172.21.15.100:0/3005792584 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1040c9fe0 con 0x7fd104099810 2024-04-23T14:32:37.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7feba2800640 1 --2- 172.21.15.100:0/13197564 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb9c04fa80 0x7feb9c037080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7feba2800640 1 -- 172.21.15.100:0/13197564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb9c04f5c0 msgr2=0x7feb9c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7feba2800640 1 --2- 172.21.15.100:0/13197564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb9c04f5c0 0x7feb9c0d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7feba2800640 1 -- 172.21.15.100:0/13197564 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9c04db60 con 0x7feb9c0df390 2024-04-23T14:32:37.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7feba1400640 1 --2- 172.21.15.100:0/13197564 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb9c04fa80 0x7feb9c037080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:37.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7feba1e00640 1 --2- 172.21.15.100:0/13197564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb9c04f5c0 0x7feb9c0d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:37.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fbd50e00640 1 -- 172.21.15.100:0/2874945264 >> v2:172.21.15.100:3300/0 conn(0x7fbd4c15aa80 msgr2=0x7fbd4c0d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fbd50e00640 1 --2- 172.21.15.100:0/2874945264 >> v2:172.21.15.100:3300/0 conn(0x7fbd4c15aa80 0x7fbd4c0d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fbd50e00640 1 -- 172.21.15.100:0/2874945264 >> v1:172.21.15.151:6789/0 conn(0x7fbd4c0cde20 legacy=0x7fbd4c0c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fbd50e00640 1 -- 172.21.15.100:0/2874945264 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4c15ae60 con 0x7fbd4c0b5c40 2024-04-23T14:32:37.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7f1e35400640 1 -- 172.21.15.100:0/2113226807 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 392935940 0 0) 0x7f1e3806c010 con 0x7f1e38073c30 2024-04-23T14:32:37.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7f1e35400640 1 -- 172.21.15.100:0/2113226807 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e1c003530 con 0x7f1e38073c30 2024-04-23T14:32:37.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fabba200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fabac033e40 0x7fabac0eb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fd10a000640 1 -- 172.21.15.100:0/3005792584 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1040b0ea0 con 0x7fd104099810 2024-04-23T14:32:37.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fec31200640 1 --2- 172.21.15.100:0/1917850398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec140039c0 0x7fec14003e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fec200099a0 tx=0x7fec2001c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf984f6c52ae4c95 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7f1e37200640 1 -- 172.21.15.100:0/2113226807 >> v1:172.21.15.151:6789/0 conn(0x7f1e38073c30 legacy=0x7f1e380d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7f1e37200640 1 -- 172.21.15.100:0/2113226807 >> v1:172.21.15.100:6789/0 conn(0x7f1e380cde20 legacy=0x7f1e380c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7f1e37200640 1 -- 172.21.15.100:0/2113226807 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e380d0bd0 con 0x7f1e380b5c40 2024-04-23T14:32:37.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fec2b400640 1 -- 172.21.15.100:0/1917850398 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec200b2070 con 0x7fec140039c0 2024-04-23T14:32:37.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fec2b400640 1 -- 172.21.15.100:0/1917850398 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec200a5520 con 0x7fec140039c0 2024-04-23T14:32:37.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fec2b400640 1 -- 172.21.15.100:0/1917850398 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20009670 con 0x7fec140039c0 2024-04-23T14:32:37.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7feba2800640 1 --2- 172.21.15.100:0/13197564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb9c0df390 0x7feb9c0d5650 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feba406a400 tx=0x7feba40747d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7feb9a000640 1 -- 172.21.15.100:0/13197564 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba4075470 con 0x7feb9c0df390 2024-04-23T14:32:37.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7fec33c00640 1 -- 172.21.15.100:0/1917850398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec140039c0 msgr2=0x7fec14003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7fec33c00640 1 --2- 172.21.15.100:0/1917850398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec140039c0 0x7fec14003e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fec200099a0 tx=0x7fec2001c930 comp rx=0 tx=0).stop 2024-04-23T14:32:37.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7fec31c00640 1 -- 172.21.15.100:0/1917850398 reap_dead start 2024-04-23T14:32:37.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fabba200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fabac047c20 0x7fabac0eb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7feb9a000640 1 -- 172.21.15.100:0/13197564 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feba4074900 con 0x7feb9c0df390 2024-04-23T14:32:37.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7feb9a000640 1 -- 172.21.15.100:0/13197564 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba4093640 con 0x7feb9c0df390 2024-04-23T14:32:37.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fabba200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabac034220 con 0x7fabac0ebc00 2024-04-23T14:32:37.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fabba200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fabac050210 con 0x7fabac047c20 2024-04-23T14:32:37.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fabba200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fabac037bc0 con 0x7fabac033e40 2024-04-23T14:32:37.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fabb8e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fabac033e40 0x7fabac0eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fabb8e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fabac033e40 0x7fabac0eb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:50790/0 (socket says 172.21.15.100:50790) 2024-04-23T14:32:37.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fabb8e00640 1 -- 172.21.15.100:0/2665440646 learned_addr learned my addr 172.21.15.100:0/2665440646 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fabb8e00640 1 -- 172.21.15.100:0/2665440646 >> v2:172.21.15.100:3300/0 conn(0x7fabac047c20 msgr2=0x7fabac0eb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fabb8e00640 1 --2- 172.21.15.100:0/2665440646 >> v2:172.21.15.100:3300/0 conn(0x7fabac047c20 0x7fabac0eb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7fabb8e00640 1 -- 172.21.15.100:0/2665440646 >> v1:172.21.15.151:6789/0 conn(0x7fabac0ebc00 legacy=0x7fabac052f60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:32:37.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7fabb8e00640 1 -- 172.21.15.100:0/2665440646 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabac04fa30 con 0x7fabac033e40 2024-04-23T14:32:37.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7fec33c00640 1 -- 172.21.15.100:0/1917850398 shutdown_connections 2024-04-23T14:32:37.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7fec33c00640 1 -- 172.21.15.100:0/1917850398 >> 172.21.15.100:0/1917850398 conn(0x7fec2c0d1310 msgr2=0x7fec2c0d1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7feba3200640 1 -- 172.21.15.100:0/13197564 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb9c04de40 con 0x7feb9c0df390 2024-04-23T14:32:37.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7feba3200640 1 -- 172.21.15.100:0/13197564 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb9c04e400 con 0x7feb9c0df390 2024-04-23T14:32:37.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7fd0fb400640 1 -- 172.21.15.100:0/3005792584 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd0fc01b070 con 0x7fd104099810 2024-04-23T14:32:37.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fd361200640 1 -- 172.21.15.100:0/1314325898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd340004d50 msgr2=0x7fd3400051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fd361200640 1 --2- 172.21.15.100:0/1314325898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd340004d50 0x7fd3400051d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00c190 tx=0x7fd34c01cc40 comp rx=0 tx=0).stop 2024-04-23T14:32:37.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fd35be00640 1 -- 172.21.15.100:0/1314325898 reap_dead start 2024-04-23T14:32:37.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fd361200640 1 -- 172.21.15.100:0/1314325898 shutdown_connections 2024-04-23T14:32:37.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fd361200640 1 -- 172.21.15.100:0/1314325898 >> 172.21.15.100:0/1314325898 conn(0x7fd35c0b1b20 msgr2=0x7fd35c0b1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fd361200640 1 -- 172.21.15.100:0/1314325898 shutdown_connections 2024-04-23T14:32:37.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.483+0000 7fd361200640 1 -- 172.21.15.100:0/1314325898 wait complete. 2024-04-23T14:32:37.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7fd361200640 1 Processor -- start 2024-04-23T14:32:37.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7fec33c00640 1 -- 172.21.15.100:0/1917850398 shutdown_connections 2024-04-23T14:32:37.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7fd361200640 1 -- start start 2024-04-23T14:32:37.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7fd10a000640 1 -- 172.21.15.100:0/3005792584 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd104068ad0 con 0x7fd104099810 2024-04-23T14:32:37.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7fec33c00640 1 -- 172.21.15.100:0/1917850398 wait complete. 2024-04-23T14:32:37.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7fbd50e00640 1 --2- 172.21.15.100:0/2874945264 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbd4c0b5c40 0x7fbd4c06c4f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009e50 tx=0x7fbd4401c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dbc67b8418b2e0d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7feba3200640 1 -- 172.21.15.100:0/13197564 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feb9c0053a0 con 0x7feb9c0df390 2024-04-23T14:32:37.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7fd0fb400640 1 --2- 172.21.15.100:0/3005792584 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd0e4068380 0x7fd0e406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7fd0fb400640 1 -- 172.21.15.100:0/3005792584 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd0fc08b060 con 0x7fd104099810 2024-04-23T14:32:37.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7fd361200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd340004d50 0x7fd35c1587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7fd361200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd35c158d30 0x7fd35c0c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7fbd4aa00640 1 -- 172.21.15.100:0/2874945264 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd440a5830 con 0x7fbd4c0b5c40 2024-04-23T14:32:37.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7fbd4aa00640 1 -- 172.21.15.100:0/2874945264 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd440a59d0 con 0x7fbd4c0b5c40 2024-04-23T14:32:37.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7fbd4aa00640 1 -- 172.21.15.100:0/2874945264 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd440a5da0 con 0x7fbd4c0b5c40 2024-04-23T14:32:37.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7ffa57e00640 1 --2- 172.21.15.100:0/1779698298 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ffa580afb80 0x7ffa5806ea20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffa580688c0 tx=0x7ffa4401c260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a76e9df8011f3a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7ffa57400640 1 -- 172.21.15.100:0/1779698298 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa440b0070 con 0x7ffa580afb80 2024-04-23T14:32:37.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7ffa57400640 1 -- 172.21.15.100:0/1779698298 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa440ab600 con 0x7ffa580afb80 2024-04-23T14:32:37.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7ffa57400640 1 -- 172.21.15.100:0/1779698298 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa440ab9d0 con 0x7ffa580afb80 2024-04-23T14:32:37.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.487+0000 7fd361200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd35c0c9280 0x7fd35c1560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd361200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c0cdd80 con 0x7fd340004d50 2024-04-23T14:32:37.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd361200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c0cdc00 con 0x7fd35c158d30 2024-04-23T14:32:37.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd361200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd35c0729c0 con 0x7fd35c0c9280 2024-04-23T14:32:37.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd0fb400640 1 --2- 172.21.15.100:0/3005792584 >> [v2:172.21.15.100:6800/43918908,v1:172.21.15.100:6801/43918908] conn(0x7fd0e4071a80 0x7fd0e4073ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd103400640 1 --2- 172.21.15.100:0/3005792584 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd0e4068380 0x7fd0e406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd0fb400640 1 -- 172.21.15.100:0/3005792584 --> [v2:172.21.15.100:6800/43918908,v1:172.21.15.100:6801/43918908] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd0fc009200 con 0x7fd0e4071a80 2024-04-23T14:32:37.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd0fb400640 1 -- 172.21.15.100:0/3005792584 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd0fc0868a0 con 0x7fd104099810 2024-04-23T14:32:37.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd102a00640 1 --2- 172.21.15.100:0/3005792584 >> [v2:172.21.15.100:6800/43918908,v1:172.21.15.100:6801/43918908] conn(0x7fd0e4071a80 0x7fd0e4073ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd35b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd340004d50 0x7fd35c1587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd35b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd340004d50 0x7fd35c1587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57442/0 (socket says 172.21.15.100:57442) 2024-04-23T14:32:37.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd35b400640 1 -- 172.21.15.100:0/249872035 learned_addr learned my addr 172.21.15.100:0/249872035 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd35be00640 1 --2- 172.21.15.100:0/249872035 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd35c0c9280 0x7fd35c1560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fbd52e00640 1 -- 172.21.15.100:0/2874945264 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbd4c0b5c40 msgr2=0x7fbd4c06c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fbd52e00640 1 --2- 172.21.15.100:0/2874945264 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbd4c0b5c40 0x7fbd4c06c4f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009e50 tx=0x7fbd4401c990 comp rx=0 tx=0).stop 2024-04-23T14:32:37.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fbd52e00640 1 -- 172.21.15.100:0/2874945264 shutdown_connections 2024-04-23T14:32:37.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fbd52e00640 1 --2- 172.21.15.100:0/2874945264 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbd4c0b5c40 0x7fbd4c06c4f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fbd52e00640 1 --2- 172.21.15.100:0/2874945264 >> v2:172.21.15.100:3300/0 conn(0x7fbd4c15aa80 0x7fbd4c0d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fbd52e00640 1 -- 172.21.15.100:0/2874945264 >> 172.21.15.100:0/2874945264 conn(0x7fbd4c073110 msgr2=0x7fbd4c073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fbd52e00640 1 -- 172.21.15.100:0/2874945264 shutdown_connections 2024-04-23T14:32:37.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fbd52e00640 1 -- 172.21.15.100:0/2874945264 wait complete. 2024-04-23T14:32:37.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7ffa5fa00640 1 -- 172.21.15.100:0/1779698298 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ffa580afb80 msgr2=0x7ffa5806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7ffa5fa00640 1 --2- 172.21.15.100:0/1779698298 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ffa580afb80 0x7ffa5806ea20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffa580688c0 tx=0x7ffa4401c260 comp rx=0 tx=0).stop 2024-04-23T14:32:37.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fabb8e00640 1 --2- 172.21.15.100:0/2665440646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabac033e40 0x7fabac0eb180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fabac054a30 tx=0x7faba001ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fefe9e4a00b449d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fabb3400640 1 -- 172.21.15.100:0/2665440646 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba00b5070 con 0x7fabac033e40 2024-04-23T14:32:37.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd103400640 1 --2- 172.21.15.100:0/3005792584 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd0e4068380 0x7fd0e406a840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd0f401c4a0 tx=0x7fd0f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fabb3400640 1 -- 172.21.15.100:0/2665440646 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba00a89f0 con 0x7fabac033e40 2024-04-23T14:32:37.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fabb3400640 1 -- 172.21.15.100:0/2665440646 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba00b5070 con 0x7fabac033e40 2024-04-23T14:32:37.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd35b400640 1 -- 172.21.15.100:0/249872035 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd35c0c9280 msgr2=0x7fd35c1560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd35b400640 1 --2- 172.21.15.100:0/249872035 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd35c0c9280 0x7fd35c1560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd35b400640 1 -- 172.21.15.100:0/249872035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd35c158d30 msgr2=0x7fd35c0c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd35b400640 1 --2- 172.21.15.100:0/249872035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd35c158d30 0x7fd35c0c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd35b400640 1 -- 172.21.15.100:0/249872035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35c1574e0 con 0x7fd340004d50 2024-04-23T14:32:37.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd102a00640 1 --2- 172.21.15.100:0/3005792584 >> [v2:172.21.15.100:6800/43918908,v1:172.21.15.100:6801/43918908] conn(0x7fd0e4071a80 0x7fd0e4073ec0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd35b400640 1 --2- 172.21.15.100:0/249872035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd340004d50 0x7fd35c1587f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd348011d30 tx=0x7fd34800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7feb9a000640 1 -- 172.21.15.100:0/13197564 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7feba40937e0 con 0x7feb9c0df390 2024-04-23T14:32:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7feb9a000640 1 --2- 172.21.15.100:0/13197564 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7feb84068450 0x7feb8406a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7feb9a000640 1 -- 172.21.15.100:0/13197564 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7feba4102030 con 0x7feb9c0df390 2024-04-23T14:32:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7feb9a000640 1 --2- 172.21.15.100:0/13197564 >> [v2:172.21.15.100:6808/3988727915,v1:172.21.15.100:6809/3988727915] conn(0x7feb84071b40 0x7feb84073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7feb9a000640 1 -- 172.21.15.100:0/13197564 --> [v2:172.21.15.100:6808/3988727915,v1:172.21.15.100:6809/3988727915] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feba4070e50 con 0x7feb84071b40 2024-04-23T14:32:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7feba1e00640 1 --2- 172.21.15.100:0/13197564 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7feb84068450 0x7feb8406a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fec33c00640 1 Processor -- start 2024-04-23T14:32:37.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7feb9a000640 1 -- 172.21.15.100:0/13197564 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7feba4109050 con 0x7feb9c0df390 2024-04-23T14:32:37.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fabba200640 1 -- 172.21.15.100:0/2665440646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabac033e40 msgr2=0x7fabac0eb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fabba200640 1 --2- 172.21.15.100:0/2665440646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabac033e40 0x7fabac0eb180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fabac054a30 tx=0x7faba001ecf0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd353400640 1 -- 172.21.15.100:0/249872035 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34800fcc0 con 0x7fd340004d50 2024-04-23T14:32:37.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7ffa5fa00640 1 -- 172.21.15.100:0/1779698298 shutdown_connections 2024-04-23T14:32:37.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7ffa5fa00640 1 --2- 172.21.15.100:0/1779698298 >> v2:172.21.15.100:3300/0 conn(0x7ffa5806ef60 0x7ffa580b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7ffa5fa00640 1 --2- 172.21.15.100:0/1779698298 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ffa580afb80 0x7ffa5806ea20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7ffa5fa00640 1 --2- 172.21.15.100:0/1779698298 >> v2:172.21.15.151:3300/0 conn(0x7ffa5815d050 0x7ffa58163b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7ffa5fa00640 1 -- 172.21.15.100:0/1779698298 >> 172.21.15.100:0/1779698298 conn(0x7ffa580d07e0 msgr2=0x7ffa580d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fabba200640 1 -- 172.21.15.100:0/2665440646 shutdown_connections 2024-04-23T14:32:37.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fabba200640 1 --2- 172.21.15.100:0/2665440646 >> v2:172.21.15.100:3300/0 conn(0x7fabac047c20 0x7fabac0eb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fabba200640 1 --2- 172.21.15.100:0/2665440646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabac033e40 0x7fabac0eb180 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fabba200640 1 -- 172.21.15.100:0/2665440646 >> 172.21.15.100:0/2665440646 conn(0x7fabac009fc0 msgr2=0x7fabac00a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd353400640 1 -- 172.21.15.100:0/249872035 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd34800fe60 con 0x7fd340004d50 2024-04-23T14:32:37.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd361200640 1 -- 172.21.15.100:0/249872035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd35c1577c0 con 0x7fd340004d50 2024-04-23T14:32:37.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd353400640 1 -- 172.21.15.100:0/249872035 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348004270 con 0x7fd340004d50 2024-04-23T14:32:37.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd361200640 1 -- 172.21.15.100:0/249872035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd35c157d80 con 0x7fd340004d50 2024-04-23T14:32:37.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fec33c00640 1 -- start start 2024-04-23T14:32:37.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fabba200640 1 -- 172.21.15.100:0/2665440646 shutdown_connections 2024-04-23T14:32:37.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fabba200640 1 -- 172.21.15.100:0/2665440646 wait complete. 2024-04-23T14:32:37.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fd0fb400640 1 -- 172.21.15.100:0/3005792584 <== osd.0 v2:172.21.15.100:6800/43918908 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd0fc009200 con 0x7fd0e4071a80 2024-04-23T14:32:37.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7feba1400640 1 --2- 172.21.15.100:0/13197564 >> [v2:172.21.15.100:6808/3988727915,v1:172.21.15.100:6809/3988727915] conn(0x7feb84071b40 0x7feb84073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7feba1e00640 1 --2- 172.21.15.100:0/13197564 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7feb84068450 0x7feb8406a910 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feb940ae000 tx=0x7feb940a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7fbd52e00640 1 Processor -- start 2024-04-23T14:32:37.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7ffa5fa00640 1 -- 172.21.15.100:0/1779698298 shutdown_connections 2024-04-23T14:32:37.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.491+0000 7ffa5fa00640 1 -- 172.21.15.100:0/1779698298 wait complete. 2024-04-23T14:32:37.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fbd52e00640 1 -- start start 2024-04-23T14:32:37.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7f1e37200640 1 --2- 172.21.15.100:0/2113226807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e380b5c40 0x7f1e3815aa80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e300097c0 tx=0x7f1e3001e5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33b51b1c9f3b6c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fd361200640 1 -- 172.21.15.100:0/249872035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd334001040 con 0x7fd340004d50 2024-04-23T14:32:37.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7feba1400640 1 --2- 172.21.15.100:0/13197564 >> [v2:172.21.15.100:6808/3988727915,v1:172.21.15.100:6809/3988727915] conn(0x7feb84071b40 0x7feb84073f80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fec33c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec140039c0 0x7fec2c1545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fec33c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec2c154b20 0x7fec2c1435e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fec33c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec2c143b20 0x7fec2c143fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fec33c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c15fdc0 con 0x7fec140039c0 2024-04-23T14:32:37.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fec33c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c15fc40 con 0x7fec2c143b20 2024-04-23T14:32:37.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fec33c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fec2c0b5bd0 con 0x7fec2c154b20 2024-04-23T14:32:37.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7feb9a000640 1 -- 172.21.15.100:0/13197564 <== osd.1 v2:172.21.15.100:6808/3988727915 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feba4070e50 con 0x7feb84071b40 2024-04-23T14:32:37.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fec31c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec2c143b20 0x7fec2c143fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7f1e35400640 1 -- 172.21.15.100:0/2113226807 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e300ad600 con 0x7f1e380b5c40 2024-04-23T14:32:37.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7f1e35400640 1 -- 172.21.15.100:0/2113226807 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e300b2070 con 0x7f1e380b5c40 2024-04-23T14:32:37.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7f1e35400640 1 -- 172.21.15.100:0/2113226807 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e300ad9c0 con 0x7f1e380b5c40 2024-04-23T14:32:37.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fbd52e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd4c0b5c40 0x7fbd4c0cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fbd52e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd4c0cde20 0x7fbd4c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fbd52e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbd4c15aa80 0x7fbd4c1570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fbd52e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c06d3c0 con 0x7fbd4c0cde20 2024-04-23T14:32:37.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fbd52e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c06d240 con 0x7fbd4c0b5c40 2024-04-23T14:32:37.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fbd52e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fbd4c1655d0 con 0x7fbd4c15aa80 2024-04-23T14:32:37.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fbd50e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbd4c15aa80 0x7fbd4c1570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fbd50e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbd4c15aa80 0x7fbd4c1570a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:49708/0 (socket says 172.21.15.100:49708) 2024-04-23T14:32:37.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fbd4b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd4c0cde20 0x7fbd4c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fbd50e00640 1 -- 172.21.15.100:0/4218029250 learned_addr learned my addr 172.21.15.100:0/4218029250 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fbd50e00640 1 -- 172.21.15.100:0/4218029250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd4c0b5c40 msgr2=0x7fbd4c0cb830 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-23T14:32:37.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fbd50e00640 1 --2- 172.21.15.100:0/4218029250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd4c0b5c40 0x7fbd4c0cb830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fbd50e00640 1 -- 172.21.15.100:0/4218029250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd4c0cde20 msgr2=0x7fbd4c0ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fbd50e00640 1 --2- 172.21.15.100:0/4218029250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd4c0cde20 0x7fbd4c0ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fbd50e00640 1 -- 172.21.15.100:0/4218029250 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4c15dab0 con 0x7fbd4c15aa80 2024-04-23T14:32:37.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7ffa5fa00640 1 Processor -- start 2024-04-23T14:32:37.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7ffa5fa00640 1 -- start start 2024-04-23T14:32:37.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa5fa00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ffa5806ef60 0x7ffa58149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fabba200640 1 Processor -- start 2024-04-23T14:32:37.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fabba200640 1 -- start start 2024-04-23T14:32:37.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fabba200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabac033e40 0x7fabac0363e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fabba200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fabac047c20 0x7fabac0e9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fabba200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabac0ebc00 0x7fabac0e9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fabba200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fabac04fd80 con 0x7fabac033e40 2024-04-23T14:32:37.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fabba200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fabac04f090 con 0x7fabac0ebc00 2024-04-23T14:32:37.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fabba200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fabac04f5c0 con 0x7fabac047c20 2024-04-23T14:32:37.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fabb9800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabac0ebc00 0x7fabac0e9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fabb9800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabac0ebc00 0x7fabac0e9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:50820/0 (socket says 172.21.15.100:50820) 2024-04-23T14:32:37.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fabb9800640 1 -- 172.21.15.100:0/1291404021 learned_addr learned my addr 172.21.15.100:0/1291404021 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fabb3e00640 1 --2- 172.21.15.100:0/1291404021 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fabac047c20 0x7fabac0e9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa5fa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa580afb80 0x7ffa5814bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7f1e3d400640 1 -- 172.21.15.100:0/2113226807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e380b5c40 msgr2=0x7f1e3815aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7f1e3d400640 1 --2- 172.21.15.100:0/2113226807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e380b5c40 0x7f1e3815aa80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e300097c0 tx=0x7f1e3001e5f0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fabb9800640 1 -- 172.21.15.100:0/1291404021 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fabac047c20 msgr2=0x7fabac0e9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fabb9800640 1 --2- 172.21.15.100:0/1291404021 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fabac047c20 0x7fabac0e9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fabb9800640 1 -- 172.21.15.100:0/1291404021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabac033e40 msgr2=0x7fabac0363e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fabb9800640 1 --2- 172.21.15.100:0/1291404021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabac033e40 0x7fabac0363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fabb9800640 1 -- 172.21.15.100:0/1291404021 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabac172630 con 0x7fabac0ebc00 2024-04-23T14:32:37.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa5fa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa5815d050 0x7ffa58148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa5fa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5806c3f0 con 0x7ffa5815d050 2024-04-23T14:32:37.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa5fa00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5806f3d0 con 0x7ffa580afb80 2024-04-23T14:32:37.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa5fa00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ffa5806c570 con 0x7ffa5806ef60 2024-04-23T14:32:37.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fbd50e00640 1 --2- 172.21.15.100:0/4218029250 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbd4c15aa80 0x7fbd4c1570a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbd440af040 tx=0x7fbd44019730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fd353400640 1 -- 172.21.15.100:0/249872035 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd348012040 con 0x7fd340004d50 2024-04-23T14:32:37.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fec31c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec2c143b20 0x7fec2c143fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:50808/0 (socket says 172.21.15.100:50808) 2024-04-23T14:32:37.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fec31c00640 1 -- 172.21.15.100:0/2843672516 learned_addr learned my addr 172.21.15.100:0/2843672516 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fec31c00640 1 -- 172.21.15.100:0/2843672516 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec2c154b20 msgr2=0x7fec2c1435e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-23T14:32:37.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fec31c00640 1 --2- 172.21.15.100:0/2843672516 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec2c154b20 0x7fec2c1435e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.495+0000 7fec31c00640 1 -- 172.21.15.100:0/2843672516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec140039c0 msgr2=0x7fec2c1545e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T14:32:37.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fec31c00640 1 --2- 172.21.15.100:0/2843672516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec140039c0 0x7fec2c1545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fec31c00640 1 -- 172.21.15.100:0/2843672516 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2c1444e0 con 0x7fec2c143b20 2024-04-23T14:32:37.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fec31c00640 1 --2- 172.21.15.100:0/2843672516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec2c143b20 0x7fec2c143fa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0ce540 tx=0x7fec2400ede0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7f1e3d400640 1 -- 172.21.15.100:0/2113226807 shutdown_connections 2024-04-23T14:32:37.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7f1e3d400640 1 --2- 172.21.15.100:0/2113226807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e380b5c40 0x7f1e3815aa80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7f1e3d400640 1 -- 172.21.15.100:0/2113226807 >> 172.21.15.100:0/2113226807 conn(0x7f1e38073110 msgr2=0x7f1e38073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fbd43e00640 1 -- 172.21.15.100:0/4218029250 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd440bc020 con 0x7fbd4c15aa80 2024-04-23T14:32:37.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fbd43e00640 1 -- 172.21.15.100:0/4218029250 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd440b8880 con 0x7fbd4c15aa80 2024-04-23T14:32:37.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fbd43e00640 1 -- 172.21.15.100:0/4218029250 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd440b8c50 con 0x7fbd4c15aa80 2024-04-23T14:32:37.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa5d000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ffa5806ef60 0x7ffa58149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa57e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa580afb80 0x7ffa5814bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa57e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa580afb80 0x7ffa5814bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:50828/0 (socket says 172.21.15.100:50828) 2024-04-23T14:32:37.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa57e00640 1 -- 172.21.15.100:0/2870348827 learned_addr learned my addr 172.21.15.100:0/2870348827 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa5da00640 1 --2- 172.21.15.100:0/2870348827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa5815d050 0x7ffa58148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa5d000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ffa5806ef60 0x7ffa58149360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:49724/0 (socket says 172.21.15.100:49724) 2024-04-23T14:32:37.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa57e00640 1 -- 172.21.15.100:0/2870348827 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ffa5806ef60 msgr2=0x7ffa58149360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa57e00640 1 --2- 172.21.15.100:0/2870348827 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ffa5806ef60 0x7ffa58149360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa57e00640 1 -- 172.21.15.100:0/2870348827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa5815d050 msgr2=0x7ffa58148be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa57e00640 1 --2- 172.21.15.100:0/2870348827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa5815d050 0x7ffa58148be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa57e00640 1 -- 172.21.15.100:0/2870348827 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa58149120 con 0x7ffa580afb80 2024-04-23T14:32:37.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fabb9800640 1 --2- 172.21.15.100:0/1291404021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabac0ebc00 0x7fabac0e9f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faba800b410 tx=0x7faba800b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fabb0c00640 1 -- 172.21.15.100:0/1291404021 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8015020 con 0x7fabac0ebc00 2024-04-23T14:32:37.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fabb0c00640 1 -- 172.21.15.100:0/1291404021 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba8011830 con 0x7fabac0ebc00 2024-04-23T14:32:37.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fabb0c00640 1 -- 172.21.15.100:0/1291404021 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8011c20 con 0x7fabac0ebc00 2024-04-23T14:32:37.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fabba200640 1 -- 172.21.15.100:0/1291404021 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabac172910 con 0x7fabac0ebc00 2024-04-23T14:32:37.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fabba200640 1 -- 172.21.15.100:0/1291404021 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabac172ed0 con 0x7fabac0ebc00 2024-04-23T14:32:37.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fbd52e00640 1 -- 172.21.15.100:0/4218029250 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4c15dd90 con 0x7fbd4c15aa80 2024-04-23T14:32:37.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fbd52e00640 1 -- 172.21.15.100:0/4218029250 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd4c0c9c70 con 0x7fbd4c15aa80 2024-04-23T14:32:37.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa57e00640 1 --2- 172.21.15.100:0/2870348827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa580afb80 0x7ffa5814bed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffa44019e90 tx=0x7ffa44019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa54c00640 1 -- 172.21.15.100:0/2870348827 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa440b9020 con 0x7ffa580afb80 2024-04-23T14:32:37.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa54c00640 1 -- 172.21.15.100:0/2870348827 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa440b1040 con 0x7ffa580afb80 2024-04-23T14:32:37.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa54c00640 1 -- 172.21.15.100:0/2870348827 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa440b0050 con 0x7ffa580afb80 2024-04-23T14:32:37.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa5fa00640 1 -- 172.21.15.100:0/2870348827 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa581499f0 con 0x7ffa580afb80 2024-04-23T14:32:37.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fabba200640 1 -- 172.21.15.100:0/1291404021 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fabac0053a0 con 0x7fabac0ebc00 2024-04-23T14:32:37.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fd353400640 1 --2- 172.21.15.100:0/249872035 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd33c068450 0x7fd33c06a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7ffa5fa00640 1 -- 172.21.15.100:0/2870348827 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa58149fb0 con 0x7ffa580afb80 2024-04-23T14:32:37.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fbd41600640 1 -- 172.21.15.100:0/4218029250 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbd4c068ad0 con 0x7fbd4c15aa80 2024-04-23T14:32:37.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7fd35aa00640 1 --2- 172.21.15.100:0/249872035 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd33c068450 0x7fd33c06a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fec28c00640 1 -- 172.21.15.100:0/2843672516 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec24006e60 con 0x7fec2c143b20 2024-04-23T14:32:37.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fec28c00640 1 -- 172.21.15.100:0/2843672516 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec2400cda0 con 0x7fec2c143b20 2024-04-23T14:32:37.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fec28c00640 1 -- 172.21.15.100:0/2843672516 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec24016660 con 0x7fec2c143b20 2024-04-23T14:32:37.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7f1e3d400640 1 -- 172.21.15.100:0/2113226807 shutdown_connections 2024-04-23T14:32:37.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.499+0000 7f1e3d400640 1 -- 172.21.15.100:0/2113226807 wait complete. 2024-04-23T14:32:37.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7f1e3d400640 1 Processor -- start 2024-04-23T14:32:37.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fbd43e00640 1 -- 172.21.15.100:0/4218029250 <== mon.2 v2:172.21.15.151:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbd440bf020 con 0x7fbd4c15aa80 2024-04-23T14:32:37.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fd353400640 1 -- 172.21.15.100:0/249872035 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd3480879d0 con 0x7fd340004d50 2024-04-23T14:32:37.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fd35aa00640 1 --2- 172.21.15.100:0/249872035 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd33c068450 0x7fd33c06a910 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0c48b0 tx=0x7fd354009750 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fabb0c00640 1 -- 172.21.15.100:0/1291404021 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7faba8018020 con 0x7fabac0ebc00 2024-04-23T14:32:37.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fd353400640 1 --2- 172.21.15.100:0/249872035 >> [v2:172.21.15.100:6816/1257826180,v1:172.21.15.100:6817/1257826180] conn(0x7fd33c071bc0 0x7fd33c074000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fd353400640 1 -- 172.21.15.100:0/249872035 --> [v2:172.21.15.100:6816/1257826180,v1:172.21.15.100:6817/1257826180] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd34801bec0 con 0x7fd33c071bc0 2024-04-23T14:32:37.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fd353400640 1 -- 172.21.15.100:0/249872035 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd348087dc0 con 0x7fd340004d50 2024-04-23T14:32:37.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7ffa5fa00640 1 -- 172.21.15.100:0/2870348827 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa580691e0 con 0x7ffa580afb80 2024-04-23T14:32:37.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7ffa54c00640 1 -- 172.21.15.100:0/2870348827 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ffa440bc020 con 0x7ffa580afb80 2024-04-23T14:32:37.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fec33c00640 1 -- 172.21.15.100:0/2843672516 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2c157620 con 0x7fec2c143b20 2024-04-23T14:32:37.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fec33c00640 1 -- 172.21.15.100:0/2843672516 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec2c0b1c60 con 0x7fec2c143b20 2024-04-23T14:32:37.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7f1e3d400640 1 -- start start 2024-04-23T14:32:37.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fabb0c00640 1 --2- 172.21.15.100:0/1291404021 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fab94068450 0x7fab9406a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fd35be00640 1 --2- 172.21.15.100:0/249872035 >> [v2:172.21.15.100:6816/1257826180,v1:172.21.15.100:6817/1257826180] conn(0x7fd33c071bc0 0x7fd33c074000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fd35be00640 1 --2- 172.21.15.100:0/249872035 >> [v2:172.21.15.100:6816/1257826180,v1:172.21.15.100:6817/1257826180] conn(0x7fd33c071bc0 0x7fd33c074000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fbd43e00640 1 --2- 172.21.15.100:0/4218029250 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fbd2c0683d0 0x7fbd2c06a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fbd43e00640 1 -- 172.21.15.100:0/4218029250 <== mon.2 v2:172.21.15.151:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbd44122340 con 0x7fbd4c15aa80 2024-04-23T14:32:37.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fbd43e00640 1 --2- 172.21.15.100:0/4218029250 >> [v2:172.21.15.151:6808/4283546311,v1:172.21.15.151:6809/4283546311] conn(0x7fbd2c071b50 0x7fbd2c073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fbd43e00640 1 -- 172.21.15.100:0/4218029250 --> [v2:172.21.15.151:6808/4283546311,v1:172.21.15.151:6809/4283546311] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbd440b6ea0 con 0x7fbd2c071b50 2024-04-23T14:32:37.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fbd43e00640 1 -- 172.21.15.100:0/4218029250 <== mon.2 v2:172.21.15.151:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbd441226f0 con 0x7fbd4c15aa80 2024-04-23T14:32:37.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fabb0c00640 1 -- 172.21.15.100:0/1291404021 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7faba8087250 con 0x7fabac0ebc00 2024-04-23T14:32:37.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fabb0c00640 1 --2- 172.21.15.100:0/1291404021 >> [v2:172.21.15.151:6816/389311850,v1:172.21.15.151:6817/389311850] conn(0x7fab94071bc0 0x7fab94074000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fabb0c00640 1 -- 172.21.15.100:0/1291404021 --> [v2:172.21.15.151:6816/389311850,v1:172.21.15.151:6817/389311850] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faba8014e70 con 0x7fab94071bc0 2024-04-23T14:32:37.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7fabb0c00640 1 -- 172.21.15.100:0/1291404021 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faba8087640 con 0x7fabac0ebc00 2024-04-23T14:32:37.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7ffa54c00640 1 --2- 172.21.15.100:0/2870348827 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ffa38068400 0x7ffa3806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.503+0000 7ffa54c00640 1 -- 172.21.15.100:0/2870348827 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ffa44120560 con 0x7ffa580afb80 2024-04-23T14:32:37.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.507+0000 7ffa54c00640 1 --2- 172.21.15.100:0/2870348827 >> [v2:172.21.15.151:6824/547449762,v1:172.21.15.151:6825/547449762] conn(0x7ffa38071b70 0x7ffa38073fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.507+0000 7ffa54c00640 1 -- 172.21.15.100:0/2870348827 --> [v2:172.21.15.151:6824/547449762,v1:172.21.15.151:6825/547449762] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa38074680 con 0x7ffa38071b70 2024-04-23T14:32:37.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.507+0000 7ffa54c00640 1 -- 172.21.15.100:0/2870348827 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ffa44127050 con 0x7ffa580afb80 2024-04-23T14:32:37.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.507+0000 7fabb3e00640 1 --2- 172.21.15.100:0/1291404021 >> [v2:172.21.15.151:6816/389311850,v1:172.21.15.151:6817/389311850] conn(0x7fab94071bc0 0x7fab94074000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.507+0000 7fabb8e00640 1 --2- 172.21.15.100:0/1291404021 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fab94068450 0x7fab9406a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.507+0000 7ffa5da00640 1 --2- 172.21.15.100:0/2870348827 >> [v2:172.21.15.151:6824/547449762,v1:172.21.15.151:6825/547449762] conn(0x7ffa38071b70 0x7ffa38073fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.507+0000 7ffa5d000640 1 --2- 172.21.15.100:0/2870348827 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ffa38068400 0x7ffa3806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.507+0000 7fbd4be00640 1 --2- 172.21.15.100:0/4218029250 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fbd2c0683d0 0x7fbd2c06a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.507+0000 7fd353400640 1 -- 172.21.15.100:0/249872035 <== osd.2 v2:172.21.15.100:6816/1257826180 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd34801bec0 con 0x7fd33c071bc0 2024-04-23T14:32:37.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.507+0000 7ffa5da00640 1 --2- 172.21.15.100:0/2870348827 >> [v2:172.21.15.151:6824/547449762,v1:172.21.15.151:6825/547449762] conn(0x7ffa38071b70 0x7ffa38073fb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.507+0000 7fbd4b400640 1 --2- 172.21.15.100:0/4218029250 >> [v2:172.21.15.151:6808/4283546311,v1:172.21.15.151:6809/4283546311] conn(0x7fbd2c071b50 0x7fbd2c073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.507+0000 7ffa5d000640 1 --2- 172.21.15.100:0/2870348827 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ffa38068400 0x7ffa3806a8c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffa58163910 tx=0x7ffa50009830 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.507+0000 7fabb8e00640 1 --2- 172.21.15.100:0/1291404021 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fab94068450 0x7fab9406a910 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faba00b2000 tx=0x7faba0004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.507+0000 7fbd4be00640 1 --2- 172.21.15.100:0/4218029250 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fbd2c0683d0 0x7fbd2c06a890 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c164a90 tx=0x7fbd3c006b90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.507+0000 7fec28c00640 1 -- 172.21.15.100:0/2843672516 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fec2401d020 con 0x7fec2c143b20 2024-04-23T14:32:37.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.507+0000 7fec33c00640 1 -- 172.21.15.100:0/2843672516 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec2c147910 con 0x7fec2c143b20 2024-04-23T14:32:37.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.507+0000 7ffa54c00640 1 -- 172.21.15.100:0/2870348827 <== osd.6 v2:172.21.15.151:6824/547449762 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa38074680 con 0x7ffa38071b70 2024-04-23T14:32:37.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.507+0000 7fabb3e00640 1 --2- 172.21.15.100:0/1291404021 >> [v2:172.21.15.151:6816/389311850,v1:172.21.15.151:6817/389311850] conn(0x7fab94071bc0 0x7fab94074000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.507+0000 7fbd4b400640 1 --2- 172.21.15.100:0/4218029250 >> [v2:172.21.15.151:6808/4283546311,v1:172.21.15.151:6809/4283546311] conn(0x7fbd2c071b50 0x7fbd2c073f90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.507+0000 7fec28c00640 1 --2- 172.21.15.100:0/2843672516 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fec0c068450 0x7fec0c06a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.507+0000 7fec28c00640 1 -- 172.21.15.100:0/2843672516 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fec24087260 con 0x7fec2c143b20 2024-04-23T14:32:37.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.507+0000 7fec31200640 1 --2- 172.21.15.100:0/2843672516 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fec0c068450 0x7fec0c06a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7fec28c00640 1 --2- 172.21.15.100:0/2843672516 >> [v2:172.21.15.100:6824/1142032886,v1:172.21.15.100:6825/1142032886] conn(0x7fec0c071bc0 0x7fec0c074000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7fec28c00640 1 -- 172.21.15.100:0/2843672516 --> [v2:172.21.15.100:6824/1142032886,v1:172.21.15.100:6825/1142032886] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec2400e270 con 0x7fec0c071bc0 2024-04-23T14:32:37.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7fec28c00640 1 -- 172.21.15.100:0/2843672516 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fec2408e050 con 0x7fec2c143b20 2024-04-23T14:32:37.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7fbd43e00640 1 -- 172.21.15.100:0/4218029250 <== osd.5 v2:172.21.15.151:6808/4283546311 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbd440b6ea0 con 0x7fbd2c071b50 2024-04-23T14:32:37.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7fec31200640 1 --2- 172.21.15.100:0/2843672516 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fec0c068450 0x7fec0c06a910 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fec200af000 tx=0x7fec20004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7fec2be00640 1 --2- 172.21.15.100:0/2843672516 >> [v2:172.21.15.100:6824/1142032886,v1:172.21.15.100:6825/1142032886] conn(0x7fec0c071bc0 0x7fec0c074000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7f1e3d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e38073c30 0x7f1e3815c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7f1e3d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e380b5c40 0x7f1e380cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7f1e3d400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e380cde20 0x7f1e381ff8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7f1e3d400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3806d090 con 0x7f1e38073c30 2024-04-23T14:32:37.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7f1e3d400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3806cf10 con 0x7f1e380b5c40 2024-04-23T14:32:37.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7f1e3d400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1e3806d210 con 0x7f1e380cde20 2024-04-23T14:32:37.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7f1e35e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e380b5c40 0x7f1e380cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7f1e35e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e380b5c40 0x7f1e380cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:50838/0 (socket says 172.21.15.100:50838) 2024-04-23T14:32:37.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7f1e35e00640 1 -- 172.21.15.100:0/1709271986 learned_addr learned my addr 172.21.15.100:0/1709271986 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7fabb0c00640 1 -- 172.21.15.100:0/1291404021 <== osd.7 v2:172.21.15.151:6816/389311850 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faba8014e70 con 0x7fab94071bc0 2024-04-23T14:32:37.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7f1e36800640 1 --2- 172.21.15.100:0/1709271986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e38073c30 0x7f1e3815c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7f1e35e00640 1 -- 172.21.15.100:0/1709271986 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e380cde20 msgr2=0x7f1e381ff8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-23T14:32:37.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7f1e35e00640 1 --2- 172.21.15.100:0/1709271986 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e380cde20 0x7f1e381ff8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7f1e35e00640 1 -- 172.21.15.100:0/1709271986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e38073c30 msgr2=0x7f1e3815c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7f1e35e00640 1 --2- 172.21.15.100:0/1709271986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e38073c30 0x7f1e3815c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7f1e35e00640 1 -- 172.21.15.100:0/1709271986 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e3815bc00 con 0x7f1e380b5c40 2024-04-23T14:32:37.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7f1e35e00640 1 --2- 172.21.15.100:0/1709271986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e380b5c40 0x7f1e380cbfa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1e2400c3c0 tx=0x7f1e2400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7f1e2ea00640 1 -- 172.21.15.100:0/1709271986 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e24017070 con 0x7f1e380b5c40 2024-04-23T14:32:37.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7f1e2ea00640 1 -- 172.21.15.100:0/1709271986 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e240052d0 con 0x7f1e380b5c40 2024-04-23T14:32:37.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7f1e2ea00640 1 -- 172.21.15.100:0/1709271986 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e24003840 con 0x7f1e380b5c40 2024-04-23T14:32:37.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.511+0000 7fec2be00640 1 --2- 172.21.15.100:0/2843672516 >> [v2:172.21.15.100:6824/1142032886,v1:172.21.15.100:6825/1142032886] conn(0x7fec0c071bc0 0x7fec0c074000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.515+0000 7f1e3d400640 1 -- 172.21.15.100:0/1709271986 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e3815bee0 con 0x7f1e380b5c40 2024-04-23T14:32:37.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.515+0000 7f1e3d400640 1 -- 172.21.15.100:0/1709271986 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e380cc4e0 con 0x7f1e380b5c40 2024-04-23T14:32:37.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.515+0000 7fec28c00640 1 -- 172.21.15.100:0/2843672516 <== osd.3 v2:172.21.15.100:6824/1142032886 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fec2400e270 con 0x7fec0c071bc0 2024-04-23T14:32:37.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.515+0000 7f1e3d400640 1 -- 172.21.15.100:0/1709271986 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1e38068ad0 con 0x7f1e380b5c40 2024-04-23T14:32:37.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.515+0000 7f1e2ea00640 1 -- 172.21.15.100:0/1709271986 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1e2401a070 con 0x7f1e380b5c40 2024-04-23T14:32:37.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.519+0000 7f1e2ea00640 1 --2- 172.21.15.100:0/1709271986 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e18068450 0x7f1e1806a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.519+0000 7f1e2ea00640 1 -- 172.21.15.100:0/1709271986 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1e24085ff0 con 0x7f1e380b5c40 2024-04-23T14:32:37.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.519+0000 7f1e2ea00640 1 --2- 172.21.15.100:0/1709271986 >> [v2:172.21.15.151:6800/4191386446,v1:172.21.15.151:6801/4191386446] conn(0x7f1e18071b40 0x7f1e18073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.519+0000 7f1e2ea00640 1 -- 172.21.15.100:0/1709271986 --> [v2:172.21.15.151:6800/4191386446,v1:172.21.15.151:6801/4191386446] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1e18074650 con 0x7f1e18071b40 2024-04-23T14:32:37.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.519+0000 7f1e2ea00640 1 -- 172.21.15.100:0/1709271986 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1e24051df0 con 0x7f1e380b5c40 2024-04-23T14:32:37.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.519+0000 7f1e37200640 1 --2- 172.21.15.100:0/1709271986 >> [v2:172.21.15.151:6800/4191386446,v1:172.21.15.151:6801/4191386446] conn(0x7f1e18071b40 0x7f1e18073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.519+0000 7f1e36800640 1 --2- 172.21.15.100:0/1709271986 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e18068450 0x7f1e1806a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.519+0000 7feba3200640 1 -- 172.21.15.100:0/13197564 --> [v2:172.21.15.100:6808/3988727915,v1:172.21.15.100:6809/3988727915] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feb9c04f190 con 0x7feb84071b40 2024-04-23T14:32:37.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.523+0000 7feb9a000640 1 -- 172.21.15.100:0/13197564 <== osd.1 v2:172.21.15.100:6808/3988727915 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7feb9c04f190 con 0x7feb84071b40 2024-04-23T14:32:37.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.523+0000 7f1e37200640 1 --2- 172.21.15.100:0/1709271986 >> [v2:172.21.15.151:6800/4191386446,v1:172.21.15.151:6801/4191386446] conn(0x7f1e18071b40 0x7f1e18073f80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.523+0000 7feba3200640 1 -- 172.21.15.100:0/13197564 >> [v2:172.21.15.100:6808/3988727915,v1:172.21.15.100:6809/3988727915] conn(0x7feb84071b40 msgr2=0x7feb84073f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.523+0000 7feba3200640 1 --2- 172.21.15.100:0/13197564 >> [v2:172.21.15.100:6808/3988727915,v1:172.21.15.100:6809/3988727915] conn(0x7feb84071b40 0x7feb84073f80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.523+0000 7feba3200640 1 -- 172.21.15.100:0/13197564 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7feb84068450 msgr2=0x7feb8406a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.523+0000 7feba3200640 1 --2- 172.21.15.100:0/13197564 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7feb84068450 0x7feb8406a910 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feb940ae000 tx=0x7feb940a7000 comp rx=0 tx=0).stop 2024-04-23T14:32:37.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.523+0000 7feba3200640 1 -- 172.21.15.100:0/13197564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb9c0df390 msgr2=0x7feb9c0d5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.523+0000 7feba3200640 1 --2- 172.21.15.100:0/13197564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb9c0df390 0x7feb9c0d5650 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feba406a400 tx=0x7feba40747d0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.523+0000 7feba2800640 1 -- 172.21.15.100:0/13197564 reap_dead start 2024-04-23T14:32:37.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.523+0000 7f1e36800640 1 --2- 172.21.15.100:0/1709271986 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e18068450 0x7f1e1806a910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1e28001570 tx=0x7f1e28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.523+0000 7f1e2ea00640 1 -- 172.21.15.100:0/1709271986 <== osd.4 v2:172.21.15.151:6800/4191386446 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1e18074650 con 0x7f1e18071b40 2024-04-23T14:32:37.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.523+0000 7fd10a000640 1 -- 172.21.15.100:0/3005792584 --> [v2:172.21.15.100:6800/43918908,v1:172.21.15.100:6801/43918908] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd1040b12e0 con 0x7fd0e4071a80 2024-04-23T14:32:37.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.523+0000 7feba3200640 1 -- 172.21.15.100:0/13197564 shutdown_connections 2024-04-23T14:32:37.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.523+0000 7feba3200640 1 -- 172.21.15.100:0/13197564 >> 172.21.15.100:0/13197564 conn(0x7feb9c009fc0 msgr2=0x7feb9c0ee3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.523+0000 7feba3200640 1 -- 172.21.15.100:0/13197564 shutdown_connections 2024-04-23T14:32:37.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.523+0000 7feba3200640 1 -- 172.21.15.100:0/13197564 wait complete. 2024-04-23T14:32:37.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.523+0000 7fd0fb400640 1 -- 172.21.15.100:0/3005792584 <== osd.0 v2:172.21.15.100:6800/43918908 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd1040b12e0 con 0x7fd0e4071a80 2024-04-23T14:32:37.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.523+0000 7fd10a000640 1 -- 172.21.15.100:0/3005792584 >> [v2:172.21.15.100:6800/43918908,v1:172.21.15.100:6801/43918908] conn(0x7fd0e4071a80 msgr2=0x7fd0e4073ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.523+0000 7fd10a000640 1 --2- 172.21.15.100:0/3005792584 >> [v2:172.21.15.100:6800/43918908,v1:172.21.15.100:6801/43918908] conn(0x7fd0e4071a80 0x7fd0e4073ec0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.527+0000 7fd10a000640 1 -- 172.21.15.100:0/3005792584 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd0e4068380 msgr2=0x7fd0e406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.527+0000 7fd10a000640 1 --2- 172.21.15.100:0/3005792584 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd0e4068380 0x7fd0e406a840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd0f401c4a0 tx=0x7fd0f4008040 comp rx=0 tx=0).stop 2024-04-23T14:32:37.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.527+0000 7fd10a000640 1 -- 172.21.15.100:0/3005792584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd104099810 msgr2=0x7fd104204bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.531+0000 7fd10a000640 1 --2- 172.21.15.100:0/3005792584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd104099810 0x7fd104204bd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00eed0 tx=0x7fd0fc00b480 comp rx=0 tx=0).stop 2024-04-23T14:32:37.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.531+0000 7fd103e00640 1 -- 172.21.15.100:0/3005792584 reap_dead start 2024-04-23T14:32:37.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.531+0000 7fd10a000640 1 -- 172.21.15.100:0/3005792584 shutdown_connections 2024-04-23T14:32:37.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.531+0000 7fd10a000640 1 -- 172.21.15.100:0/3005792584 >> 172.21.15.100:0/3005792584 conn(0x7fd104073110 msgr2=0x7fd1040af170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.531+0000 7fd10a000640 1 -- 172.21.15.100:0/3005792584 shutdown_connections 2024-04-23T14:32:37.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.531+0000 7fd10a000640 1 -- 172.21.15.100:0/3005792584 wait complete. 2024-04-23T14:32:37.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.535+0000 7ffa5fa00640 1 -- 172.21.15.100:0/2870348827 --> [v2:172.21.15.151:6824/547449762,v1:172.21.15.151:6825/547449762] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa580b5b20 con 0x7ffa38071b70 2024-04-23T14:32:37.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.535+0000 7fd361200640 1 -- 172.21.15.100:0/249872035 --> [v2:172.21.15.100:6816/1257826180,v1:172.21.15.100:6817/1257826180] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd334002de0 con 0x7fd33c071bc0 2024-04-23T14:32:37.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7ffa54c00640 1 -- 172.21.15.100:0/2870348827 <== osd.6 v2:172.21.15.151:6824/547449762 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffa580b5b20 con 0x7ffa38071b70 2024-04-23T14:32:37.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fd353400640 1 -- 172.21.15.100:0/249872035 <== osd.2 v2:172.21.15.100:6816/1257826180 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd334002de0 con 0x7fd33c071bc0 2024-04-23T14:32:37.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fabba200640 1 -- 172.21.15.100:0/1291404021 --> [v2:172.21.15.151:6816/389311850,v1:172.21.15.151:6817/389311850] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fabac0dc4e0 con 0x7fab94071bc0 2024-04-23T14:32:37.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7ffa5fa00640 1 -- 172.21.15.100:0/2870348827 >> [v2:172.21.15.151:6824/547449762,v1:172.21.15.151:6825/547449762] conn(0x7ffa38071b70 msgr2=0x7ffa38073fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7ffa5fa00640 1 --2- 172.21.15.100:0/2870348827 >> [v2:172.21.15.151:6824/547449762,v1:172.21.15.151:6825/547449762] conn(0x7ffa38071b70 0x7ffa38073fb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fbd41600640 1 -- 172.21.15.100:0/4218029250 --> [v2:172.21.15.151:6808/4283546311,v1:172.21.15.151:6809/4283546311] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbd4c06bab0 con 0x7fbd2c071b50 2024-04-23T14:32:37.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fd361200640 1 -- 172.21.15.100:0/249872035 >> [v2:172.21.15.100:6816/1257826180,v1:172.21.15.100:6817/1257826180] conn(0x7fd33c071bc0 msgr2=0x7fd33c074000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fd361200640 1 --2- 172.21.15.100:0/249872035 >> [v2:172.21.15.100:6816/1257826180,v1:172.21.15.100:6817/1257826180] conn(0x7fd33c071bc0 0x7fd33c074000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7ffa5fa00640 1 -- 172.21.15.100:0/2870348827 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ffa38068400 msgr2=0x7ffa3806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7ffa5fa00640 1 --2- 172.21.15.100:0/2870348827 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ffa38068400 0x7ffa3806a8c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffa58163910 tx=0x7ffa50009830 comp rx=0 tx=0).stop 2024-04-23T14:32:37.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7ffa5fa00640 1 -- 172.21.15.100:0/2870348827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa580afb80 msgr2=0x7ffa5814bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7ffa5fa00640 1 --2- 172.21.15.100:0/2870348827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa580afb80 0x7ffa5814bed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffa44019e90 tx=0x7ffa44019000 comp rx=0 tx=0).stop 2024-04-23T14:32:37.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fd361200640 1 -- 172.21.15.100:0/249872035 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd33c068450 msgr2=0x7fd33c06a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fd361200640 1 --2- 172.21.15.100:0/249872035 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd33c068450 0x7fd33c06a910 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0c48b0 tx=0x7fd354009750 comp rx=0 tx=0).stop 2024-04-23T14:32:37.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fd361200640 1 -- 172.21.15.100:0/249872035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd340004d50 msgr2=0x7fd35c1587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fd361200640 1 --2- 172.21.15.100:0/249872035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd340004d50 0x7fd35c1587f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd348011d30 tx=0x7fd34800f610 comp rx=0 tx=0).stop 2024-04-23T14:32:37.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fbd43e00640 1 -- 172.21.15.100:0/4218029250 <== osd.5 v2:172.21.15.151:6808/4283546311 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbd4c06bab0 con 0x7fbd2c071b50 2024-04-23T14:32:37.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7ffa5da00640 1 -- 172.21.15.100:0/2870348827 reap_dead start 2024-04-23T14:32:37.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7ffa5fa00640 1 -- 172.21.15.100:0/2870348827 shutdown_connections 2024-04-23T14:32:37.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7ffa5fa00640 1 -- 172.21.15.100:0/2870348827 >> 172.21.15.100:0/2870348827 conn(0x7ffa580d07e0 msgr2=0x7ffa5815ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fabb0c00640 1 -- 172.21.15.100:0/1291404021 <== osd.7 v2:172.21.15.151:6816/389311850 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fabac0dc4e0 con 0x7fab94071bc0 2024-04-23T14:32:37.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fd35be00640 1 -- 172.21.15.100:0/249872035 reap_dead start 2024-04-23T14:32:37.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fd361200640 1 -- 172.21.15.100:0/249872035 shutdown_connections 2024-04-23T14:32:37.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fd361200640 1 -- 172.21.15.100:0/249872035 >> 172.21.15.100:0/249872035 conn(0x7fd35c0b1b20 msgr2=0x7fd35c06dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7ffa5fa00640 1 -- 172.21.15.100:0/2870348827 shutdown_connections 2024-04-23T14:32:37.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7ffa5fa00640 1 -- 172.21.15.100:0/2870348827 wait complete. 2024-04-23T14:32:37.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fabba200640 1 -- 172.21.15.100:0/1291404021 >> [v2:172.21.15.151:6816/389311850,v1:172.21.15.151:6817/389311850] conn(0x7fab94071bc0 msgr2=0x7fab94074000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fabba200640 1 --2- 172.21.15.100:0/1291404021 >> [v2:172.21.15.151:6816/389311850,v1:172.21.15.151:6817/389311850] conn(0x7fab94071bc0 0x7fab94074000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fd361200640 1 -- 172.21.15.100:0/249872035 shutdown_connections 2024-04-23T14:32:37.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fd361200640 1 -- 172.21.15.100:0/249872035 wait complete. 2024-04-23T14:32:37.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fabba200640 1 -- 172.21.15.100:0/1291404021 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fab94068450 msgr2=0x7fab9406a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fabba200640 1 --2- 172.21.15.100:0/1291404021 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fab94068450 0x7fab9406a910 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faba00b2000 tx=0x7faba0004740 comp rx=0 tx=0).stop 2024-04-23T14:32:37.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fabba200640 1 -- 172.21.15.100:0/1291404021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabac0ebc00 msgr2=0x7fabac0e9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fabba200640 1 --2- 172.21.15.100:0/1291404021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabac0ebc00 0x7fabac0e9f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faba800b410 tx=0x7faba800b8d0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fabb9800640 1 -- 172.21.15.100:0/1291404021 reap_dead start 2024-04-23T14:32:37.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fabba200640 1 -- 172.21.15.100:0/1291404021 shutdown_connections 2024-04-23T14:32:37.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fabba200640 1 -- 172.21.15.100:0/1291404021 >> 172.21.15.100:0/1291404021 conn(0x7fabac009fc0 msgr2=0x7fabac0e7190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fabba200640 1 -- 172.21.15.100:0/1291404021 shutdown_connections 2024-04-23T14:32:37.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.539+0000 7fabba200640 1 -- 172.21.15.100:0/1291404021 wait complete. 2024-04-23T14:32:37.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.543+0000 7fbd41600640 1 -- 172.21.15.100:0/4218029250 >> [v2:172.21.15.151:6808/4283546311,v1:172.21.15.151:6809/4283546311] conn(0x7fbd2c071b50 msgr2=0x7fbd2c073f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.543+0000 7fbd41600640 1 --2- 172.21.15.100:0/4218029250 >> [v2:172.21.15.151:6808/4283546311,v1:172.21.15.151:6809/4283546311] conn(0x7fbd2c071b50 0x7fbd2c073f90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.543+0000 7fbd41600640 1 -- 172.21.15.100:0/4218029250 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fbd2c0683d0 msgr2=0x7fbd2c06a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.543+0000 7fbd41600640 1 --2- 172.21.15.100:0/4218029250 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fbd2c0683d0 0x7fbd2c06a890 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c164a90 tx=0x7fbd3c006b90 comp rx=0 tx=0).stop 2024-04-23T14:32:37.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.543+0000 7fbd41600640 1 -- 172.21.15.100:0/4218029250 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbd4c15aa80 msgr2=0x7fbd4c1570a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.543+0000 7fbd41600640 1 --2- 172.21.15.100:0/4218029250 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbd4c15aa80 0x7fbd4c1570a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbd440af040 tx=0x7fbd44019730 comp rx=0 tx=0).stop 2024-04-23T14:32:37.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.543+0000 7fbd50e00640 1 -- 172.21.15.100:0/4218029250 reap_dead start 2024-04-23T14:32:37.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.547+0000 7fbd41600640 1 -- 172.21.15.100:0/4218029250 shutdown_connections 2024-04-23T14:32:37.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.547+0000 7fbd41600640 1 -- 172.21.15.100:0/4218029250 >> 172.21.15.100:0/4218029250 conn(0x7fbd4c073110 msgr2=0x7fbd4c0ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.547+0000 7fec33c00640 1 -- 172.21.15.100:0/2843672516 --> [v2:172.21.15.100:6824/1142032886,v1:172.21.15.100:6825/1142032886] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fec2c0b2080 con 0x7fec0c071bc0 2024-04-23T14:32:37.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.547+0000 7fbd41600640 1 -- 172.21.15.100:0/4218029250 shutdown_connections 2024-04-23T14:32:37.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.547+0000 7fbd41600640 1 -- 172.21.15.100:0/4218029250 wait complete. 2024-04-23T14:32:37.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.551+0000 7fec28c00640 1 -- 172.21.15.100:0/2843672516 <== osd.3 v2:172.21.15.100:6824/1142032886 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fec2c0b2080 con 0x7fec0c071bc0 2024-04-23T14:32:37.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.551+0000 7fec33c00640 1 -- 172.21.15.100:0/2843672516 >> [v2:172.21.15.100:6824/1142032886,v1:172.21.15.100:6825/1142032886] conn(0x7fec0c071bc0 msgr2=0x7fec0c074000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.551+0000 7fec33c00640 1 --2- 172.21.15.100:0/2843672516 >> [v2:172.21.15.100:6824/1142032886,v1:172.21.15.100:6825/1142032886] conn(0x7fec0c071bc0 0x7fec0c074000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.551+0000 7fec33c00640 1 -- 172.21.15.100:0/2843672516 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fec0c068450 msgr2=0x7fec0c06a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.551+0000 7fec33c00640 1 --2- 172.21.15.100:0/2843672516 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fec0c068450 0x7fec0c06a910 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fec200af000 tx=0x7fec20004740 comp rx=0 tx=0).stop 2024-04-23T14:32:37.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.551+0000 7fec33c00640 1 -- 172.21.15.100:0/2843672516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec2c143b20 msgr2=0x7fec2c143fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.551+0000 7fec33c00640 1 --2- 172.21.15.100:0/2843672516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec2c143b20 0x7fec2c143fa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0ce540 tx=0x7fec2400ede0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.551+0000 7fec31c00640 1 -- 172.21.15.100:0/2843672516 reap_dead start 2024-04-23T14:32:37.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.555+0000 7fec33c00640 1 -- 172.21.15.100:0/2843672516 shutdown_connections 2024-04-23T14:32:37.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.555+0000 7fec33c00640 1 -- 172.21.15.100:0/2843672516 >> 172.21.15.100:0/2843672516 conn(0x7fec2c0d1310 msgr2=0x7fec2c06c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.555+0000 7f1e3d400640 1 -- 172.21.15.100:0/1709271986 --> [v2:172.21.15.151:6800/4191386446,v1:172.21.15.151:6801/4191386446] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1e3815c780 con 0x7f1e18071b40 2024-04-23T14:32:37.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.555+0000 7fec33c00640 1 -- 172.21.15.100:0/2843672516 shutdown_connections 2024-04-23T14:32:37.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.555+0000 7fec33c00640 1 -- 172.21.15.100:0/2843672516 wait complete. 2024-04-23T14:32:37.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.555+0000 7f1e2ea00640 1 -- 172.21.15.100:0/1709271986 <== osd.4 v2:172.21.15.151:6800/4191386446 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1e3815c780 con 0x7f1e18071b40 2024-04-23T14:32:37.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.555+0000 7f1e3d400640 1 -- 172.21.15.100:0/1709271986 >> [v2:172.21.15.151:6800/4191386446,v1:172.21.15.151:6801/4191386446] conn(0x7f1e18071b40 msgr2=0x7f1e18073f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.555+0000 7f1e3d400640 1 --2- 172.21.15.100:0/1709271986 >> [v2:172.21.15.151:6800/4191386446,v1:172.21.15.151:6801/4191386446] conn(0x7f1e18071b40 0x7f1e18073f80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.555+0000 7f1e3d400640 1 -- 172.21.15.100:0/1709271986 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e18068450 msgr2=0x7f1e1806a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.555+0000 7f1e3d400640 1 --2- 172.21.15.100:0/1709271986 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e18068450 0x7f1e1806a910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1e28001570 tx=0x7f1e28009040 comp rx=0 tx=0).stop 2024-04-23T14:32:37.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.555+0000 7f1e3d400640 1 -- 172.21.15.100:0/1709271986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e380b5c40 msgr2=0x7f1e380cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.555+0000 7f1e3d400640 1 --2- 172.21.15.100:0/1709271986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e380b5c40 0x7f1e380cbfa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1e2400c3c0 tx=0x7f1e2400c880 comp rx=0 tx=0).stop 2024-04-23T14:32:37.569 INFO:teuthology.orchestra.run.smithi100.stdout:55834574860 2024-04-23T14:32:37.569 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-23T14:32:37.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.559+0000 7f1e37200640 1 -- 172.21.15.100:0/1709271986 reap_dead start 2024-04-23T14:32:37.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.559+0000 7f1e3d400640 1 -- 172.21.15.100:0/1709271986 shutdown_connections 2024-04-23T14:32:37.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.559+0000 7f1e3d400640 1 -- 172.21.15.100:0/1709271986 >> 172.21.15.100:0/1709271986 conn(0x7f1e38073110 msgr2=0x7f1e3814ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.559+0000 7f1e3d400640 1 -- 172.21.15.100:0/1709271986 shutdown_connections 2024-04-23T14:32:37.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.559+0000 7f1e3d400640 1 -- 172.21.15.100:0/1709271986 wait complete. 2024-04-23T14:32:37.571 INFO:teuthology.orchestra.run.smithi100.stdout:55834574861 2024-04-23T14:32:37.571 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-23T14:32:37.575 INFO:teuthology.orchestra.run.smithi100.stdout:55834574861 2024-04-23T14:32:37.575 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-23T14:32:37.578 INFO:teuthology.orchestra.run.smithi100.stdout:55834574858 2024-04-23T14:32:37.578 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-23T14:32:37.588 INFO:teuthology.orchestra.run.smithi100.stdout:55834574859 2024-04-23T14:32:37.588 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-23T14:32:37.588 INFO:teuthology.orchestra.run.smithi100.stdout:55834574861 2024-04-23T14:32:37.589 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-23T14:32:37.593 INFO:teuthology.orchestra.run.smithi100.stdout:55834574859 2024-04-23T14:32:37.593 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-23T14:32:37.599 INFO:teuthology.orchestra.run.smithi100.stdout:55834574859 2024-04-23T14:32:37.599 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-23T14:32:37.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.799+0000 7f1164c00640 1 Processor -- start 2024-04-23T14:32:37.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.799+0000 7f1164c00640 1 -- start start 2024-04-23T14:32:37.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.799+0000 7f1164c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f116015a590 0x7f11600cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.799+0000 7f1164c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f11600cde60 0x7f116006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.799+0000 7f1164c00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f116006bff0 0x7f116015e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.803+0000 7f1164c00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f116006c460 con 0x7f116015a590 2024-04-23T14:32:37.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.803+0000 7f115de00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f116015a590 0x7f11600cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.803+0000 7f115de00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f116015a590 0x7f11600cf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57502/0 (socket says 172.21.15.100:57502) 2024-04-23T14:32:37.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.803+0000 7f115de00640 1 -- 172.21.15.100:0/338123888 learned_addr learned my addr 172.21.15.100:0/338123888 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.803+0000 7f115d400640 1 --2- 172.21.15.100:0/338123888 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f11600cde60 0x7f116006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.803+0000 7f115e800640 1 --2- 172.21.15.100:0/338123888 >> v2:172.21.15.151:3300/0 conn(0x7f116006bff0 0x7f116015e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.803+0000 7f1164c00640 1 -- 172.21.15.100:0/338123888 --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f11600ae450 con 0x7f116006bff0 2024-04-23T14:32:37.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.803+0000 7f1164c00640 1 -- 172.21.15.100:0/338123888 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f11600ad880 con 0x7f11600cde60 2024-04-23T14:32:37.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.803+0000 7f115d400640 1 -- 172.21.15.100:0/338123888 >> v2:172.21.15.151:3300/0 conn(0x7f116006bff0 msgr2=0x7f116015e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.803+0000 7f115d400640 1 --2- 172.21.15.100:0/338123888 >> v2:172.21.15.151:3300/0 conn(0x7f116006bff0 0x7f116015e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.803+0000 7f115d400640 1 -- 172.21.15.100:0/338123888 >> v2:172.21.15.100:3300/0 conn(0x7f116015a590 msgr2=0x7f11600cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.803+0000 7f115d400640 1 --2- 172.21.15.100:0/338123888 >> v2:172.21.15.100:3300/0 conn(0x7f116015a590 0x7f11600cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.803+0000 7f115d400640 1 -- 172.21.15.100:0/338123888 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11600adb20 con 0x7f11600cde60 2024-04-23T14:32:37.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.803+0000 7f115e800640 1 --2- 172.21.15.100:0/338123888 >> v2:172.21.15.151:3300/0 conn(0x7f116006bff0 0x7f116015e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:37.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.807+0000 7f115d400640 1 --2- 172.21.15.100:0/338123888 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f11600cde60 0x7f116006bab0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f11600c3b40 tx=0x7f115801c930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1b225ec66ea73dc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.811+0000 7f115ca00640 1 -- 172.21.15.100:0/338123888 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11580b2070 con 0x7f11600cde60 2024-04-23T14:32:37.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.811+0000 7f115ca00640 1 -- 172.21.15.100:0/338123888 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11580a58a0 con 0x7f11600cde60 2024-04-23T14:32:37.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.811+0000 7f115ca00640 1 -- 172.21.15.100:0/338123888 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11580a5c70 con 0x7f11600cde60 2024-04-23T14:32:37.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.811+0000 7f1164c00640 1 -- 172.21.15.100:0/338123888 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f11600cde60 msgr2=0x7f116006bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.811+0000 7f1164c00640 1 --2- 172.21.15.100:0/338123888 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f11600cde60 0x7f116006bab0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f11600c3b40 tx=0x7f115801c930 comp rx=0 tx=0).stop 2024-04-23T14:32:37.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.811+0000 7f1164c00640 1 -- 172.21.15.100:0/338123888 shutdown_connections 2024-04-23T14:32:37.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.811+0000 7f1164c00640 1 --2- 172.21.15.100:0/338123888 >> v2:172.21.15.151:3300/0 conn(0x7f116006bff0 0x7f116015e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.811+0000 7f1164c00640 1 --2- 172.21.15.100:0/338123888 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f11600cde60 0x7f116006bab0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.811+0000 7f1164c00640 1 --2- 172.21.15.100:0/338123888 >> v2:172.21.15.100:3300/0 conn(0x7f116015a590 0x7f11600cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.811+0000 7f1164c00640 1 -- 172.21.15.100:0/338123888 >> 172.21.15.100:0/338123888 conn(0x7f11600d0060 msgr2=0x7f11600d0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.811+0000 7f1164c00640 1 -- 172.21.15.100:0/338123888 shutdown_connections 2024-04-23T14:32:37.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.811+0000 7f1164c00640 1 -- 172.21.15.100:0/338123888 wait complete. 2024-04-23T14:32:37.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.815+0000 7f1164c00640 1 Processor -- start 2024-04-23T14:32:37.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.815+0000 7f1164c00640 1 -- start start 2024-04-23T14:32:37.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.815+0000 7f1164c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f116006bff0 0x7f1160152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.815+0000 7ff8bb200640 1 Processor -- start 2024-04-23T14:32:37.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.815+0000 7ff8bb200640 1 -- start start 2024-04-23T14:32:37.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.815+0000 7ff8bb200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff8bc0d3060 0x7ff8bc0d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.815+0000 7ff8bb200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff8bc0b7e50 0x7ff8bc0cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.815+0000 7ff8bb200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff8bc16eef0 0x7ff8bc16f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7ff8bb200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff8bc06bab0 con 0x7ff8bc16eef0 2024-04-23T14:32:37.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7ff8bb200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7ff8bc06c8f0 con 0x7ff8bc0b7e50 2024-04-23T14:32:37.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7ff8bb200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff8bc0d3980 con 0x7ff8bc0d3060 2024-04-23T14:32:37.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7ff8b9e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff8bc0d3060 0x7ff8bc0d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7ff8b9e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff8bc0d3060 0x7ff8bc0d3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:49742/0 (socket says 172.21.15.100:49742) 2024-04-23T14:32:37.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7ff8b9e00640 1 -- 172.21.15.100:0/2531017210 learned_addr learned my addr 172.21.15.100:0/2531017210 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7f115de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f116006bff0 0x7f1160152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7f115de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f116006bff0 0x7f1160152790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57504/0 (socket says 172.21.15.100:57504) 2024-04-23T14:32:37.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7ff8b9400640 1 --2- 172.21.15.100:0/2531017210 >> v2:172.21.15.151:3300/0 conn(0x7ff8bc0b7e50 0x7ff8bc0cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7ff8ba800640 1 --2- 172.21.15.100:0/2531017210 >> v2:172.21.15.100:3300/0 conn(0x7ff8bc16eef0 0x7ff8bc16f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7ff8b9e00640 1 -- 172.21.15.100:0/2531017210 >> v2:172.21.15.151:3300/0 conn(0x7ff8bc0b7e50 msgr2=0x7ff8bc0cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7ff8b9e00640 1 --2- 172.21.15.100:0/2531017210 >> v2:172.21.15.151:3300/0 conn(0x7ff8bc0b7e50 0x7ff8bc0cf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7ff8b9e00640 1 -- 172.21.15.100:0/2531017210 >> v2:172.21.15.100:3300/0 conn(0x7ff8bc16eef0 msgr2=0x7ff8bc16f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7ff8b9e00640 1 --2- 172.21.15.100:0/2531017210 >> v2:172.21.15.100:3300/0 conn(0x7ff8bc16eef0 0x7ff8bc16f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7ff8b9e00640 1 -- 172.21.15.100:0/2531017210 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8bc06bce0 con 0x7ff8bc0d3060 2024-04-23T14:32:37.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7f1164c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11600cde60 0x7f1160206c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7f115de00640 1 -- 172.21.15.100:0/3245294785 learned_addr learned my addr 172.21.15.100:0/3245294785 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7f1164c00640 1 --2- 172.21.15.100:0/3245294785 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f116015a590 0x7f1160207140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7f1164c00640 1 -- 172.21.15.100:0/3245294785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f116015aaf0 con 0x7f116006bff0 2024-04-23T14:32:37.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7f1164c00640 1 -- 172.21.15.100:0/3245294785 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f116015a970 con 0x7f11600cde60 2024-04-23T14:32:37.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7f1164c00640 1 -- 172.21.15.100:0/3245294785 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f11600ca970 con 0x7f116015a590 2024-04-23T14:32:37.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7f115e800640 1 --2- 172.21.15.100:0/3245294785 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f116015a590 0x7f1160207140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7f115d400640 1 --2- 172.21.15.100:0/3245294785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11600cde60 0x7f1160206c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7f115e800640 1 -- 172.21.15.100:0/3245294785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11600cde60 msgr2=0x7f1160206c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7f115e800640 1 --2- 172.21.15.100:0/3245294785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11600cde60 0x7f1160206c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.819+0000 7f115e800640 1 -- 172.21.15.100:0/3245294785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f116006bff0 msgr2=0x7f1160152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.823+0000 7f115e800640 1 --2- 172.21.15.100:0/3245294785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f116006bff0 0x7f1160152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.823+0000 7f115e800640 1 -- 172.21.15.100:0/3245294785 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1160157be0 con 0x7f116015a590 2024-04-23T14:32:37.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.823+0000 7f115e800640 1 --2- 172.21.15.100:0/3245294785 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f116015a590 0x7f1160207140 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f114c009fd0 tx=0x7f114c00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.823+0000 7f115de00640 1 --2- 172.21.15.100:0/3245294785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f116006bff0 0x7f1160152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:37.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.823+0000 7f1156000640 1 -- 172.21.15.100:0/3245294785 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c017070 con 0x7f116015a590 2024-04-23T14:32:37.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.823+0000 7f1156000640 1 -- 172.21.15.100:0/3245294785 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f114c00c940 con 0x7f116015a590 2024-04-23T14:32:37.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.823+0000 7f1156000640 1 -- 172.21.15.100:0/3245294785 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c00ccf0 con 0x7f116015a590 2024-04-23T14:32:37.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.823+0000 7f1164c00640 1 -- 172.21.15.100:0/3245294785 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1160157e80 con 0x7f116015a590 2024-04-23T14:32:37.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.823+0000 7f1164c00640 1 -- 172.21.15.100:0/3245294785 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1160158440 con 0x7f116015a590 2024-04-23T14:32:37.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.823+0000 7ff8b9e00640 1 --2- 172.21.15.100:0/2531017210 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff8bc0d3060 0x7ff8bc0d3440 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc068300 tx=0x7ff8ac01c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e674f1ec78bee server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.823+0000 7ff8b8a00640 1 -- 172.21.15.100:0/2531017210 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac0aa410 con 0x7ff8bc0d3060 2024-04-23T14:32:37.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.823+0000 7ff8b8a00640 1 -- 172.21.15.100:0/2531017210 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8ac0af070 con 0x7ff8bc0d3060 2024-04-23T14:32:37.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.823+0000 7ff8b8a00640 1 -- 172.21.15.100:0/2531017210 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac0aa7d0 con 0x7ff8bc0d3060 2024-04-23T14:32:37.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7f1164c00640 1 -- 172.21.15.100:0/3245294785 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1160075c20 con 0x7f116015a590 2024-04-23T14:32:37.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7ff8bb200640 1 -- 172.21.15.100:0/2531017210 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff8bc0d3060 msgr2=0x7ff8bc0d3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7ff8bb200640 1 --2- 172.21.15.100:0/2531017210 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff8bc0d3060 0x7ff8bc0d3440 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc068300 tx=0x7ff8ac01c4f0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7ff8bb200640 1 -- 172.21.15.100:0/2531017210 shutdown_connections 2024-04-23T14:32:37.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7ff8bb200640 1 --2- 172.21.15.100:0/2531017210 >> v2:172.21.15.100:3300/0 conn(0x7ff8bc16eef0 0x7ff8bc16f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7ff8bb200640 1 --2- 172.21.15.100:0/2531017210 >> v2:172.21.15.151:3300/0 conn(0x7ff8bc0b7e50 0x7ff8bc0cf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7ff8bb200640 1 --2- 172.21.15.100:0/2531017210 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff8bc0d3060 0x7ff8bc0d3440 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7ff8bb200640 1 -- 172.21.15.100:0/2531017210 >> 172.21.15.100:0/2531017210 conn(0x7ff8bc0b5f70 msgr2=0x7ff8bc0b6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7f2f0ac00640 1 Processor -- start 2024-04-23T14:32:37.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7f2f0ac00640 1 -- start start 2024-04-23T14:32:37.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7ff8bb200640 1 -- 172.21.15.100:0/2531017210 shutdown_connections 2024-04-23T14:32:37.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7ff8bb200640 1 -- 172.21.15.100:0/2531017210 wait complete. 2024-04-23T14:32:37.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7f1156000640 1 -- 172.21.15.100:0/3245294785 <== mon.2 v2:172.21.15.151:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f114c01a070 con 0x7f116015a590 2024-04-23T14:32:37.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7f1156000640 1 --2- 172.21.15.100:0/3245294785 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f11400683d0 0x7f114006a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7f2f0ac00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f2f040aec30 0x7f2f040c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7f2f0ac00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2f040af640 0x7f2f0406c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7f2f0ac00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2f0406c970 0x7f2f04162540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7f2f0ac00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f2f040b5bf0 con 0x7f2f0406c970 2024-04-23T14:32:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7f2f0ac00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f2f040b4fe0 con 0x7f2f040aec30 2024-04-23T14:32:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7f2f0ac00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f2f040b52c0 con 0x7f2f040af640 2024-04-23T14:32:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f2f03e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f2f040aec30 0x7f2f040c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f2f03400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2f040af640 0x7f2f0406c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f2f03400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2f040af640 0x7f2f0406c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:49770/0 (socket says 172.21.15.100:49770) 2024-04-23T14:32:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f2f03400640 1 -- 172.21.15.100:0/4192520908 learned_addr learned my addr 172.21.15.100:0/4192520908 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f2f03e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f2f040aec30 0x7f2f040c6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:50878/0 (socket says 172.21.15.100:50878) 2024-04-23T14:32:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f2f08c00640 1 --2- 172.21.15.100:0/4192520908 >> v2:172.21.15.100:3300/0 conn(0x7f2f0406c970 0x7f2f04162540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7f115de00640 1 --2- 172.21.15.100:0/3245294785 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f11400683d0 0x7f114006a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f1156000640 1 -- 172.21.15.100:0/3245294785 <== mon.2 v2:172.21.15.151:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f114c054720 con 0x7f116015a590 2024-04-23T14:32:37.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f2f03400640 1 -- 172.21.15.100:0/4192520908 >> v2:172.21.15.151:3300/0 conn(0x7f2f040aec30 msgr2=0x7f2f040c6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f2f03400640 1 --2- 172.21.15.100:0/4192520908 >> v2:172.21.15.151:3300/0 conn(0x7f2f040aec30 0x7f2f040c6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f2f03400640 1 -- 172.21.15.100:0/4192520908 >> v2:172.21.15.100:3300/0 conn(0x7f2f0406c970 msgr2=0x7f2f04162540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f2f03400640 1 --2- 172.21.15.100:0/4192520908 >> v2:172.21.15.100:3300/0 conn(0x7f2f0406c970 0x7f2f04162540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f2f03400640 1 -- 172.21.15.100:0/4192520908 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f040b55a0 con 0x7f2f040af640 2024-04-23T14:32:37.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7f84b6400640 1 Processor -- start 2024-04-23T14:32:37.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.827+0000 7f84b6400640 1 -- start start 2024-04-23T14:32:37.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f84b6400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f84b00cd170 0x7f84b006cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7ff8bb200640 1 Processor -- start 2024-04-23T14:32:37.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7ff8bb200640 1 -- start start 2024-04-23T14:32:37.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f115de00640 1 --2- 172.21.15.100:0/3245294785 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f11400683d0 0x7f114006a890 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1150006aa0 tx=0x7f115000c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f84b6400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84b00b5ff0 0x7f84b006e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f84b6400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b006de60 con 0x7f84b0099190 2024-04-23T14:32:37.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f84b6400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f84b006e470 con 0x7f84b00cd170 2024-04-23T14:32:37.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f84b6400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f84b01674c0 con 0x7f84b00b5ff0 2024-04-23T14:32:37.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7ff8bb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8bc0b7e50 0x7ff8bc167570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f84aea00640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f84b0099190 0x7f84b01668f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:58490/0 (socket says 172.21.15.100:58490) 2024-04-23T14:32:37.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f84aea00640 1 -- 172.21.15.100:0/195662072 learned_addr learned my addr 172.21.15.100:0/195662072 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:37.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f84af400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f84b00cd170 0x7f84b006cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7ff8bb200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff8bc0d3060 0x7ff8bc0b9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7ff8bb200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8bc16eef0 0x7ff8bc0b9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7ff8bb200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc0cfa20 con 0x7ff8bc0b7e50 2024-04-23T14:32:37.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7ff8bb200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc16fca0 con 0x7ff8bc16eef0 2024-04-23T14:32:37.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7ff8b9400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff8bc0d3060 0x7ff8bc0b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7ff8b9400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff8bc0d3060 0x7ff8bc0b9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:49790/0 (socket says 172.21.15.100:49790) 2024-04-23T14:32:37.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7ff8b9400640 1 -- 172.21.15.100:0/2001483607 learned_addr learned my addr 172.21.15.100:0/2001483607 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7ff8b9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8bc0b7e50 0x7ff8bc167570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7ff8ba800640 1 --2- 172.21.15.100:0/2001483607 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8bc16eef0 0x7ff8bc0b9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7ff8bb200640 1 -- 172.21.15.100:0/2001483607 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff8bc0b39c0 con 0x7ff8bc0d3060 2024-04-23T14:32:37.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f84afe00640 1 --2- 172.21.15.100:0/195662072 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84b00b5ff0 0x7f84b006e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.831+0000 7f84afe00640 1 -- 172.21.15.100:0/195662072 >> v2:172.21.15.100:3300/0 conn(0x7f84b00cd170 msgr2=0x7f84b006cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7f84afe00640 1 --2- 172.21.15.100:0/195662072 >> v2:172.21.15.100:3300/0 conn(0x7f84b00cd170 0x7f84b006cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7f84afe00640 1 -- 172.21.15.100:0/195662072 >> v1:172.21.15.151:6789/0 conn(0x7f84b0099190 legacy=0x7f84b01668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7f84afe00640 1 -- 172.21.15.100:0/195662072 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b006e100 con 0x7f84b00b5ff0 2024-04-23T14:32:37.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7f84af400640 1 --2- 172.21.15.100:0/195662072 >> v2:172.21.15.100:3300/0 conn(0x7f84b00cd170 0x7f84b006cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:37.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7ff8ba800640 1 -- 172.21.15.100:0/2001483607 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff8bc0d3060 msgr2=0x7ff8bc0b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7ff8ba800640 1 --2- 172.21.15.100:0/2001483607 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff8bc0d3060 0x7ff8bc0b9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7ff8ba800640 1 -- 172.21.15.100:0/2001483607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8bc0b7e50 msgr2=0x7ff8bc167570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7ff8ba800640 1 --2- 172.21.15.100:0/2001483607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8bc0b7e50 0x7ff8bc167570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7ff8ba800640 1 -- 172.21.15.100:0/2001483607 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8bc0cf210 con 0x7ff8bc16eef0 2024-04-23T14:32:37.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7ff8b9400640 1 --2- 172.21.15.100:0/2001483607 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff8bc0d3060 0x7ff8bc0b9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:37.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7f2f03400640 1 --2- 172.21.15.100:0/4192520908 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2f040af640 0x7f2f0406c430 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2f04068480 tx=0x7f2ef001c260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f113057ceb52568d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7ff8ba800640 1 --2- 172.21.15.100:0/2001483607 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8bc16eef0 0x7ff8bc0b9c50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4007d90 tx=0x7ff8b400e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7f2f02a00640 1 -- 172.21.15.100:0/4192520908 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef00b0070 con 0x7f2f040af640 2024-04-23T14:32:37.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7f2f02a00640 1 -- 172.21.15.100:0/4192520908 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef00ab600 con 0x7f2f040af640 2024-04-23T14:32:37.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7f2f02a00640 1 -- 172.21.15.100:0/4192520908 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef00ab9d0 con 0x7f2f040af640 2024-04-23T14:32:37.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7ff8b2000640 1 -- 172.21.15.100:0/2001483607 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b400edd0 con 0x7ff8bc16eef0 2024-04-23T14:32:37.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7ff8b2000640 1 -- 172.21.15.100:0/2001483607 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b40039a0 con 0x7ff8bc16eef0 2024-04-23T14:32:37.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7ff8b2000640 1 -- 172.21.15.100:0/2001483607 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4003d90 con 0x7ff8bc16eef0 2024-04-23T14:32:37.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7ff8bb200640 1 -- 172.21.15.100:0/2001483607 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8bc0c8b30 con 0x7ff8bc16eef0 2024-04-23T14:32:37.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7ff8bb200640 1 -- 172.21.15.100:0/2001483607 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8bc0c90f0 con 0x7ff8bc16eef0 2024-04-23T14:32:37.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7f2f0ac00640 1 -- 172.21.15.100:0/4192520908 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2f040af640 msgr2=0x7f2f0406c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7f2f0ac00640 1 --2- 172.21.15.100:0/4192520908 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2f040af640 0x7f2f0406c430 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2f04068480 tx=0x7f2ef001c260 comp rx=0 tx=0).stop 2024-04-23T14:32:37.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.835+0000 7f1156000640 1 -- 172.21.15.100:0/3245294785 <== mon.2 v2:172.21.15.151:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f114c0093d0 con 0x7f116015a590 2024-04-23T14:32:37.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7f2f0ac00640 1 -- 172.21.15.100:0/4192520908 shutdown_connections 2024-04-23T14:32:37.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7f2f0ac00640 1 --2- 172.21.15.100:0/4192520908 >> v2:172.21.15.100:3300/0 conn(0x7f2f0406c970 0x7f2f04162540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7f2f0ac00640 1 --2- 172.21.15.100:0/4192520908 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2f040af640 0x7f2f0406c430 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7f2f0ac00640 1 --2- 172.21.15.100:0/4192520908 >> v2:172.21.15.151:3300/0 conn(0x7f2f040aec30 0x7f2f040c6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7f2f0ac00640 1 -- 172.21.15.100:0/4192520908 >> 172.21.15.100:0/4192520908 conn(0x7f2f04073110 msgr2=0x7f2f04073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7ff8bb200640 1 -- 172.21.15.100:0/2001483607 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8bc075c20 con 0x7ff8bc16eef0 2024-04-23T14:32:37.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7f2f0ac00640 1 -- 172.21.15.100:0/4192520908 shutdown_connections 2024-04-23T14:32:37.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7f2f0ac00640 1 -- 172.21.15.100:0/4192520908 wait complete. 2024-04-23T14:32:37.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7f84afe00640 1 --2- 172.21.15.100:0/195662072 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84b00b5ff0 0x7f84b006e900 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f84b006e030 tx=0x7f849801c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92e0956c61641911 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7ff8b2000640 1 -- 172.21.15.100:0/2001483607 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff8b401b020 con 0x7ff8bc16eef0 2024-04-23T14:32:37.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7f84ae000640 1 -- 172.21.15.100:0/195662072 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84980b0070 con 0x7f84b00b5ff0 2024-04-23T14:32:37.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7f84ae000640 1 -- 172.21.15.100:0/195662072 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84980ab600 con 0x7f84b00b5ff0 2024-04-23T14:32:37.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7f84ae000640 1 -- 172.21.15.100:0/195662072 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84980ab9d0 con 0x7f84b00b5ff0 2024-04-23T14:32:37.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7ff8a0400640 1 Processor -- start 2024-04-23T14:32:37.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7f84b6400640 1 -- 172.21.15.100:0/195662072 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84b00b5ff0 msgr2=0x7f84b006e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7f84b6400640 1 --2- 172.21.15.100:0/195662072 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84b00b5ff0 0x7f84b006e900 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f84b006e030 tx=0x7f849801c330 comp rx=0 tx=0).stop 2024-04-23T14:32:37.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7ff8a0400640 1 -- start start 2024-04-23T14:32:37.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7f84b6400640 1 -- 172.21.15.100:0/195662072 shutdown_connections 2024-04-23T14:32:37.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7f84b6400640 1 --2- 172.21.15.100:0/195662072 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84b00b5ff0 0x7f84b006e900 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7f84b6400640 1 --2- 172.21.15.100:0/195662072 >> v2:172.21.15.100:3300/0 conn(0x7f84b00cd170 0x7f84b006cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7f84b6400640 1 -- 172.21.15.100:0/195662072 >> 172.21.15.100:0/195662072 conn(0x7f84b0073110 msgr2=0x7f84b0073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7ff8b2000640 1 --2- 172.21.15.100:0/2001483607 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff89c0683b0 0x7ff89c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7ff8b2000640 1 -- 172.21.15.100:0/2001483607 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff8b4086f20 con 0x7ff8bc16eef0 2024-04-23T14:32:37.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7f84b6400640 1 -- 172.21.15.100:0/195662072 shutdown_connections 2024-04-23T14:32:37.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f2f0ac00640 1 Processor -- start 2024-04-23T14:32:37.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f2f0ac00640 1 -- start start 2024-04-23T14:32:37.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7ff8a0400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff8980b5460 0x7ff8980d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7ff8a0400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff89806ee10 con 0x7ff89815fb20 2024-04-23T14:32:37.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7ff8a0400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff89806f0b0 con 0x7ff89815ef80 2024-04-23T14:32:37.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7ff8a0400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff898162300 con 0x7ff8980b5460 2024-04-23T14:32:37.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7ff8b9e00640 1 --2- 172.21.15.100:0/2001483607 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff89c0683b0 0x7ff89c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7ff89e400640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7ff89815ef80 0x7ff898166510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:58496/0 (socket says 172.21.15.100:58496) 2024-04-23T14:32:37.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7ff89e400640 1 -- 172.21.15.100:0/3114690435 learned_addr learned my addr 172.21.15.100:0/3114690435 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:37.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.839+0000 7f84b6400640 1 -- 172.21.15.100:0/195662072 wait complete. 2024-04-23T14:32:37.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f84b6400640 1 Processor -- start 2024-04-23T14:32:37.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f84b6400640 1 -- start start 2024-04-23T14:32:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f2f0ac00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2f0406c970 0x7f2f04204130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f2f0ac00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f040aec30 0x7f2f0415a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f2f0ac00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f040af640 0x7f2f0415a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f2f0ac00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2f040ad2e0 con 0x7f2f040aec30 2024-04-23T14:32:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f2f03e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2f0406c970 0x7f2f04204130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f2f03e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2f0406c970 0x7f2f04204130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:49802/0 (socket says 172.21.15.100:49802) 2024-04-23T14:32:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f2f03e00640 1 -- 172.21.15.100:0/401782842 learned_addr learned my addr 172.21.15.100:0/401782842 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f2f08c00640 1 --2- 172.21.15.100:0/401782842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f040af640 0x7f2f0415a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f84b6400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84b0099190 0x7f84b008c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f84b6400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b00b5ff0 0x7f84b00cf370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7ff89d000640 1 --2- 172.21.15.100:0/3114690435 >> v2:172.21.15.100:3300/0 conn(0x7ff8980b5460 0x7ff8980d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7ff897e00640 1 -- 172.21.15.100:0/3114690435 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 897468990 0 0) 0x7ff89806ee10 con 0x7ff89815fb20 2024-04-23T14:32:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7ff897e00640 1 -- 172.21.15.100:0/3114690435 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff880003530 con 0x7ff89815fb20 2024-04-23T14:32:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f84b6400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84b00cd170 0x7f84b0159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f2f03400640 1 --2- 172.21.15.100:0/401782842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f040aec30 0x7f2f0415a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f2f0ac00640 1 -- 172.21.15.100:0/401782842 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2f040ad160 con 0x7f2f040af640 2024-04-23T14:32:37.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f2f0ac00640 1 -- 172.21.15.100:0/401782842 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f2f040ad460 con 0x7f2f0406c970 2024-04-23T14:32:37.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f84af400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84b0099190 0x7f84b008c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f84af400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84b0099190 0x7f84b008c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:50902/0 (socket says 172.21.15.100:50902) 2024-04-23T14:32:37.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f84b6400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f84b0167000 con 0x7f84b00b5ff0 2024-04-23T14:32:37.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f84b6400640 1 -- 172.21.15.100:0/3004017315 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f84b00cd550 con 0x7f84b0099190 2024-04-23T14:32:37.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f84b6400640 1 -- 172.21.15.100:0/3004017315 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f84b0167180 con 0x7f84b00cd170 2024-04-23T14:32:37.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f84af400640 1 -- 172.21.15.100:0/3004017315 learned_addr learned my addr 172.21.15.100:0/3004017315 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.843+0000 7f84aea00640 1 --2- 172.21.15.100:0/3004017315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b00b5ff0 0x7f84b00cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f84afe00640 1 --2- 172.21.15.100:0/3004017315 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84b00cd170 0x7f84b0159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7ff8b2000640 1 -- 172.21.15.100:0/2001483607 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff8b4057070 con 0x7ff8bc16eef0 2024-04-23T14:32:37.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7ff8b9e00640 1 --2- 172.21.15.100:0/2001483607 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff89c0683b0 0x7ff89c06a870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0ae000 tx=0x7ff8ac0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f2f03e00640 1 -- 172.21.15.100:0/401782842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f040af640 msgr2=0x7f2f0415a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f2f03e00640 1 --2- 172.21.15.100:0/401782842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f040af640 0x7f2f0415a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f2f03e00640 1 -- 172.21.15.100:0/401782842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f040aec30 msgr2=0x7f2f0415a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f2f03e00640 1 --2- 172.21.15.100:0/401782842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f040aec30 0x7f2f0415a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f2f03e00640 1 -- 172.21.15.100:0/401782842 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f04159300 con 0x7f2f0406c970 2024-04-23T14:32:37.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f2f08c00640 1 --2- 172.21.15.100:0/401782842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f040af640 0x7f2f0415a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:37.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f84afe00640 1 -- 172.21.15.100:0/3004017315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84b0099190 msgr2=0x7f84b008c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f84afe00640 1 --2- 172.21.15.100:0/3004017315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84b0099190 0x7f84b008c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f84afe00640 1 -- 172.21.15.100:0/3004017315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b00b5ff0 msgr2=0x7f84b00cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f84afe00640 1 --2- 172.21.15.100:0/3004017315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b00b5ff0 0x7f84b00cf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f84afe00640 1 -- 172.21.15.100:0/3004017315 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b01fc500 con 0x7f84b00cd170 2024-04-23T14:32:37.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f84af400640 1 --2- 172.21.15.100:0/3004017315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84b0099190 0x7f84b008c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:37.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f84aea00640 1 --2- 172.21.15.100:0/3004017315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b00b5ff0 0x7f84b00cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f2f03e00640 1 --2- 172.21.15.100:0/401782842 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2f0406c970 0x7f2f04204130 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400e930 tx=0x7f2ef400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f2efbe00640 1 -- 172.21.15.100:0/401782842 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4017070 con 0x7f2f0406c970 2024-04-23T14:32:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f2efbe00640 1 -- 172.21.15.100:0/401782842 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef400ca00 con 0x7f2f0406c970 2024-04-23T14:32:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7ff89d000640 1 -- 172.21.15.100:0/3114690435 >> v1:172.21.15.151:6789/0 conn(0x7ff89815ef80 legacy=0x7ff898166510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7ff89d000640 1 -- 172.21.15.100:0/3114690435 >> v1:172.21.15.100:6789/0 conn(0x7ff89815fb20 legacy=0x7ff89815c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7ff89d000640 1 -- 172.21.15.100:0/3114690435 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89806f390 con 0x7ff8980b5460 2024-04-23T14:32:37.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f2f0ac00640 1 -- 172.21.15.100:0/401782842 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f041595e0 con 0x7f2f0406c970 2024-04-23T14:32:37.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f317b200640 1 Processor -- start 2024-04-23T14:32:37.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f2efbe00640 1 -- 172.21.15.100:0/401782842 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef400cdb0 con 0x7f2f0406c970 2024-04-23T14:32:37.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f84afe00640 1 --2- 172.21.15.100:0/3004017315 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84b00cd170 0x7f84b0159870 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f84b0070600 tx=0x7f84980190d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f2f0ac00640 1 -- 172.21.15.100:0/401782842 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f04159ba0 con 0x7f2f0406c970 2024-04-23T14:32:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f84a7400640 1 -- 172.21.15.100:0/3004017315 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84980b0040 con 0x7f84b00cd170 2024-04-23T14:32:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f84a7400640 1 -- 172.21.15.100:0/3004017315 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84980ab400 con 0x7f84b00cd170 2024-04-23T14:32:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f84a7400640 1 -- 172.21.15.100:0/3004017315 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84980aa970 con 0x7f84b00cd170 2024-04-23T14:32:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f84b6400640 1 -- 172.21.15.100:0/3004017315 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b01fc7e0 con 0x7f84b00cd170 2024-04-23T14:32:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.851+0000 7f84b6400640 1 -- 172.21.15.100:0/3004017315 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b01fcda0 con 0x7f84b00cd170 2024-04-23T14:32:37.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.851+0000 7f2f0ac00640 1 -- 172.21.15.100:0/401782842 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2edc0052a0 con 0x7f2f0406c970 2024-04-23T14:32:37.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.847+0000 7f317b200640 1 -- start start 2024-04-23T14:32:37.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.851+0000 7f317b200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f317415e4f0 0x7f31740d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.851+0000 7f317b200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f31740d43a0 0x7f31740d47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.851+0000 7f317b200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31740d4ce0 con 0x7f31740b5e90 2024-04-23T14:32:37.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.851+0000 7f317b200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f31740d18a0 con 0x7f31740d43a0 2024-04-23T14:32:37.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.851+0000 7f317b200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f31740d1e10 con 0x7f317415e4f0 2024-04-23T14:32:37.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.851+0000 7f3179e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f317415e4f0 0x7f31740d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.851+0000 7f3179e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f317415e4f0 0x7f31740d1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:50906/0 (socket says 172.21.15.100:50906) 2024-04-23T14:32:37.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.851+0000 7f3179e00640 1 -- 172.21.15.100:0/1220933179 learned_addr learned my addr 172.21.15.100:0/1220933179 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.851+0000 7f3179e00640 1 -- 172.21.15.100:0/1220933179 >> v2:172.21.15.100:3300/0 conn(0x7f31740d43a0 msgr2=0x7f31740d47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.851+0000 7f3179e00640 1 --2- 172.21.15.100:0/1220933179 >> v2:172.21.15.100:3300/0 conn(0x7f31740d43a0 0x7f31740d47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.851+0000 7f3179e00640 1 -- 172.21.15.100:0/1220933179 >> v1:172.21.15.100:6789/0 conn(0x7f31740b5e90 legacy=0x7f31740cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.851+0000 7f3179e00640 1 -- 172.21.15.100:0/1220933179 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3174162180 con 0x7f317415e4f0 2024-04-23T14:32:37.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.851+0000 7f84a7400640 1 -- 172.21.15.100:0/3004017315 <== mon.2 v2:172.21.15.151:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f84980bc020 con 0x7f84b00cd170 2024-04-23T14:32:37.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.851+0000 7f2efbe00640 1 -- 172.21.15.100:0/401782842 <== mon.2 v2:172.21.15.151:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2ef401a070 con 0x7f2f0406c970 2024-04-23T14:32:37.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.851+0000 7ff89d000640 1 --2- 172.21.15.100:0/3114690435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8980b5460 0x7ff8980d09c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff8840097c0 tx=0x7ff88401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3bd2b2961c7ce1e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.851+0000 7f84a7400640 1 --2- 172.21.15.100:0/3004017315 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f848c0683d0 0x7f848c06a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.851+0000 7f84b6400640 1 -- 172.21.15.100:0/3004017315 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84b0068ad0 con 0x7f84b00cd170 2024-04-23T14:32:37.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.855+0000 7ff897e00640 1 -- 172.21.15.100:0/3114690435 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8840b0070 con 0x7ff8980b5460 2024-04-23T14:32:37.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.855+0000 7ff897e00640 1 -- 172.21.15.100:0/3114690435 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8840ab600 con 0x7ff8980b5460 2024-04-23T14:32:37.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.855+0000 7ff897e00640 1 -- 172.21.15.100:0/3114690435 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8840ab9d0 con 0x7ff8980b5460 2024-04-23T14:32:37.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.855+0000 7f2efbe00640 1 --2- 172.21.15.100:0/401782842 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2ee4068480 0x7f2ee406a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.855+0000 7f84af400640 1 --2- 172.21.15.100:0/3004017315 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f848c0683d0 0x7f848c06a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.855+0000 7ff8a0400640 1 -- 172.21.15.100:0/3114690435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8980b5460 msgr2=0x7ff8980d09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.855+0000 7ff8a0400640 1 --2- 172.21.15.100:0/3114690435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8980b5460 0x7ff8980d09c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff8840097c0 tx=0x7ff88401c430 comp rx=0 tx=0).stop 2024-04-23T14:32:37.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.855+0000 7f2efbe00640 1 -- 172.21.15.100:0/401782842 <== mon.2 v2:172.21.15.151:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2ef4086fe0 con 0x7f2f0406c970 2024-04-23T14:32:37.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.855+0000 7f84af400640 1 --2- 172.21.15.100:0/3004017315 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f848c0683d0 0x7f848c06a890 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009600 tx=0x7f84a0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.855+0000 7ff8a0400640 1 -- 172.21.15.100:0/3114690435 shutdown_connections 2024-04-23T14:32:37.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.855+0000 7ff8a0400640 1 --2- 172.21.15.100:0/3114690435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8980b5460 0x7ff8980d09c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.855+0000 7ff8a0400640 1 -- 172.21.15.100:0/3114690435 >> 172.21.15.100:0/3114690435 conn(0x7ff898070670 msgr2=0x7ff898070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.855+0000 7f3179e00640 1 --2- 172.21.15.100:0/1220933179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f317415e4f0 0x7f31740d1360 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f31740b6f30 tx=0x7f316c01c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14c98c1224f00a2d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.855+0000 7f2f03400640 1 --2- 172.21.15.100:0/401782842 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2ee4068480 0x7f2ee406a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.855+0000 7f3178a00640 1 -- 172.21.15.100:0/1220933179 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c0a56d0 con 0x7f317415e4f0 2024-04-23T14:32:37.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.855+0000 7f3178a00640 1 -- 172.21.15.100:0/1220933179 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f316c0a5870 con 0x7f317415e4f0 2024-04-23T14:32:37.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.855+0000 7f3178a00640 1 -- 172.21.15.100:0/1220933179 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c0a56d0 con 0x7f317415e4f0 2024-04-23T14:32:37.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.855+0000 7ff8a0400640 1 -- 172.21.15.100:0/3114690435 shutdown_connections 2024-04-23T14:32:37.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.855+0000 7ff8a0400640 1 -- 172.21.15.100:0/3114690435 wait complete. 2024-04-23T14:32:37.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.855+0000 7f84a7400640 1 -- 172.21.15.100:0/3004017315 <== mon.2 v2:172.21.15.151:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f84980ab5b0 con 0x7f84b00cd170 2024-04-23T14:32:37.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7f317b200640 1 -- 172.21.15.100:0/1220933179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f317415e4f0 msgr2=0x7f31740d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7f317b200640 1 --2- 172.21.15.100:0/1220933179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f317415e4f0 0x7f31740d1360 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f31740b6f30 tx=0x7f316c01c920 comp rx=0 tx=0).stop 2024-04-23T14:32:37.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7f317b200640 1 -- 172.21.15.100:0/1220933179 shutdown_connections 2024-04-23T14:32:37.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7f317b200640 1 --2- 172.21.15.100:0/1220933179 >> v2:172.21.15.100:3300/0 conn(0x7f31740d43a0 0x7f31740d47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7f317b200640 1 --2- 172.21.15.100:0/1220933179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f317415e4f0 0x7f31740d1360 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7f317b200640 1 -- 172.21.15.100:0/1220933179 >> 172.21.15.100:0/1220933179 conn(0x7f317409d790 msgr2=0x7f31740b93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7f317b200640 1 -- 172.21.15.100:0/1220933179 shutdown_connections 2024-04-23T14:32:37.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7ff8a0400640 1 Processor -- start 2024-04-23T14:32:37.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7f2efbe00640 1 -- 172.21.15.100:0/401782842 <== mon.2 v2:172.21.15.151:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2ef4056bf0 con 0x7f2f0406c970 2024-04-23T14:32:37.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7f2f03400640 1 --2- 172.21.15.100:0/401782842 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2ee4068480 0x7f2ee406a940 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00ad000 tx=0x7f2ef00a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7ff8a0400640 1 -- start start 2024-04-23T14:32:37.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7ff8a0400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8980b5460 0x7ff898155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7ff8a0400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff89815ef80 0x7ff89806ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7ff8a0400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff89815fb20 0x7ff89816b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7ff8a0400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff89816c9d0 con 0x7ff8980b5460 2024-04-23T14:32:37.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7ff8a0400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff89816c850 con 0x7ff89815fb20 2024-04-23T14:32:37.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7ff8a0400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff89816cb50 con 0x7ff89815ef80 2024-04-23T14:32:37.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7ff89e400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff89815fb20 0x7ff89816b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7ff89e400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff89815fb20 0x7ff89816b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:50920/0 (socket says 172.21.15.100:50920) 2024-04-23T14:32:37.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7ff89e400640 1 -- 172.21.15.100:0/305654064 learned_addr learned my addr 172.21.15.100:0/305654064 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7ff89d000640 1 --2- 172.21.15.100:0/305654064 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff89815ef80 0x7ff89806ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7ff89da00640 1 --2- 172.21.15.100:0/305654064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8980b5460 0x7ff898155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7f317b200640 1 -- 172.21.15.100:0/1220933179 wait complete. 2024-04-23T14:32:37.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7f317b200640 1 Processor -- start 2024-04-23T14:32:37.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7ff89e400640 1 -- 172.21.15.100:0/305654064 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff89815ef80 msgr2=0x7ff89806ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7ff89e400640 1 --2- 172.21.15.100:0/305654064 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff89815ef80 0x7ff89806ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7ff89e400640 1 -- 172.21.15.100:0/305654064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8980b5460 msgr2=0x7ff898155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7ff89e400640 1 --2- 172.21.15.100:0/305654064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8980b5460 0x7ff898155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.859+0000 7ff89e400640 1 -- 172.21.15.100:0/305654064 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff898156700 con 0x7ff89815fb20 2024-04-23T14:32:37.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.863+0000 7f317b200640 1 -- start start 2024-04-23T14:32:37.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.863+0000 7ff89da00640 1 --2- 172.21.15.100:0/305654064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8980b5460 0x7ff898155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:37.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.863+0000 7ff89e400640 1 --2- 172.21.15.100:0/305654064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff89815fb20 0x7ff89816b500 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff89000cbc0 tx=0x7ff89000a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.863+0000 7ff895600640 1 -- 172.21.15.100:0/305654064 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff890017070 con 0x7ff89815fb20 2024-04-23T14:32:37.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.863+0000 7ff895600640 1 -- 172.21.15.100:0/305654064 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff89000ad40 con 0x7ff89815fb20 2024-04-23T14:32:37.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.863+0000 7ff895600640 1 -- 172.21.15.100:0/305654064 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff890012600 con 0x7ff89815fb20 2024-04-23T14:32:37.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.863+0000 7ff8a0400640 1 -- 172.21.15.100:0/305654064 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8981569e0 con 0x7ff89815fb20 2024-04-23T14:32:37.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.863+0000 7ff8a0400640 1 -- 172.21.15.100:0/305654064 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff898156fa0 con 0x7ff89815fb20 2024-04-23T14:32:37.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.863+0000 7f317b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31740b5e90 0x7f3174207210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.863+0000 7f317b200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31740d43a0 0x7f31741602c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.863+0000 7f317b200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f317415e4f0 0x7f317406cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.863+0000 7f317b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f31740cffd0 con 0x7f31740b5e90 2024-04-23T14:32:37.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.863+0000 7f317b200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f31740b6c50 con 0x7f31740d43a0 2024-04-23T14:32:37.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.863+0000 7f317b200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f317415e8d0 con 0x7f317415e4f0 2024-04-23T14:32:37.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.863+0000 7f3179e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31740b5e90 0x7f3174207210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.863+0000 7ff8a0400640 1 -- 172.21.15.100:0/305654064 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff898068ad0 con 0x7ff89815fb20 2024-04-23T14:32:37.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.863+0000 7f317a800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f317415e4f0 0x7f317406cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.863+0000 7f317a800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f317415e4f0 0x7f317406cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:49828/0 (socket says 172.21.15.100:49828) 2024-04-23T14:32:37.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.863+0000 7f317a800640 1 -- 172.21.15.100:0/3785630019 learned_addr learned my addr 172.21.15.100:0/3785630019 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.863+0000 7f3179400640 1 --2- 172.21.15.100:0/3785630019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31740d43a0 0x7f31741602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.867+0000 7f317a800640 1 -- 172.21.15.100:0/3785630019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31740d43a0 msgr2=0x7f31741602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.867+0000 7f317a800640 1 --2- 172.21.15.100:0/3785630019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31740d43a0 0x7f31741602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.867+0000 7f317a800640 1 -- 172.21.15.100:0/3785630019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31740b5e90 msgr2=0x7f3174207210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.867+0000 7f317a800640 1 --2- 172.21.15.100:0/3785630019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31740b5e90 0x7f3174207210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.867+0000 7f317a800640 1 -- 172.21.15.100:0/3785630019 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3174160cd0 con 0x7f317415e4f0 2024-04-23T14:32:37.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.863+0000 7f84a7400640 1 -- 172.21.15.100:0/3004017315 <== mon.2 v2:172.21.15.151:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8498151e70 con 0x7f84b00cd170 2024-04-23T14:32:37.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.867+0000 7f317a800640 1 --2- 172.21.15.100:0/3785630019 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f317415e4f0 0x7f317406cf80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f317000b430 tx=0x7f317000b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.867+0000 7f316a000640 1 -- 172.21.15.100:0/3785630019 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3170015020 con 0x7f317415e4f0 2024-04-23T14:32:37.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.867+0000 7f316a000640 1 -- 172.21.15.100:0/3785630019 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3170011850 con 0x7f317415e4f0 2024-04-23T14:32:37.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.867+0000 7f316a000640 1 -- 172.21.15.100:0/3785630019 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3170011c40 con 0x7f317415e4f0 2024-04-23T14:32:37.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.867+0000 7f317b200640 1 -- 172.21.15.100:0/3785630019 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3174160fb0 con 0x7f317415e4f0 2024-04-23T14:32:37.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.867+0000 7f317b200640 1 -- 172.21.15.100:0/3785630019 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3174161570 con 0x7f317415e4f0 2024-04-23T14:32:37.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.867+0000 7ff895600640 1 -- 172.21.15.100:0/305654064 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff89001a070 con 0x7ff89815fb20 2024-04-23T14:32:37.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.867+0000 7ff895600640 1 --2- 172.21.15.100:0/305654064 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff878068450 0x7ff87806a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.867+0000 7ff895600640 1 -- 172.21.15.100:0/305654064 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff890086980 con 0x7ff89815fb20 2024-04-23T14:32:37.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.867+0000 7ff89da00640 1 --2- 172.21.15.100:0/305654064 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff878068450 0x7ff87806a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.871+0000 7f317b200640 1 -- 172.21.15.100:0/3785630019 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3174075c20 con 0x7f317415e4f0 2024-04-23T14:32:37.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.871+0000 7ff89da00640 1 --2- 172.21.15.100:0/305654064 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff878068450 0x7ff87806a910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff88c0016e0 tx=0x7ff88c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.871+0000 7f316a000640 1 -- 172.21.15.100:0/3785630019 <== mon.2 v2:172.21.15.151:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3170018020 con 0x7f317415e4f0 2024-04-23T14:32:37.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.871+0000 7ff895600640 1 -- 172.21.15.100:0/305654064 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff890056620 con 0x7ff89815fb20 2024-04-23T14:32:37.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.871+0000 7f316a000640 1 --2- 172.21.15.100:0/3785630019 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3158068480 0x7f315806a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.871+0000 7f316a000640 1 -- 172.21.15.100:0/3785630019 <== mon.2 v2:172.21.15.151:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3170085dd0 con 0x7f317415e4f0 2024-04-23T14:32:37.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.875+0000 7f3179e00640 1 --2- 172.21.15.100:0/3785630019 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3158068480 0x7f315806a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.875+0000 7f3179e00640 1 --2- 172.21.15.100:0/3785630019 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3158068480 0x7f315806a940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f316c0ae000 tx=0x7f316c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.875+0000 7f316a000640 1 -- 172.21.15.100:0/3785630019 <== mon.2 v2:172.21.15.151:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f31700560d0 con 0x7f317415e4f0 2024-04-23T14:32:37.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.891+0000 7f849be00640 1 Processor -- start 2024-04-23T14:32:37.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.891+0000 7f849be00640 1 -- start start 2024-04-23T14:32:37.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.891+0000 7f849be00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f849c15e650 0x7f849c0cb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.891+0000 7f849be00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f849c06d1d0 0x7f849c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.891+0000 7f849be00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f849c0cc2e0 con 0x7f849c0cb3b0 2024-04-23T14:32:37.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.891+0000 7f849be00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f849c0ca680 con 0x7f849c15e650 2024-04-23T14:32:37.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.891+0000 7f849be00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f849c0ca8d0 con 0x7f849c06d1d0 2024-04-23T14:32:37.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.895+0000 7f849b400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f849c06d1d0 0x7f849c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.895+0000 7f849b400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f849c06d1d0 0x7f849c0cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:50930/0 (socket says 172.21.15.100:50930) 2024-04-23T14:32:37.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.895+0000 7f849b400640 1 -- 172.21.15.100:0/877132339 learned_addr learned my addr 172.21.15.100:0/877132339 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.895+0000 7f849aa00640 1 --2- 172.21.15.100:0/877132339 >> v2:172.21.15.100:3300/0 conn(0x7f849c15e650 0x7f849c0cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.895+0000 7f849b400640 1 -- 172.21.15.100:0/877132339 >> v2:172.21.15.100:3300/0 conn(0x7f849c15e650 msgr2=0x7f849c0cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.895+0000 7f849b400640 1 --2- 172.21.15.100:0/877132339 >> v2:172.21.15.100:3300/0 conn(0x7f849c15e650 0x7f849c0cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.895+0000 7f849b400640 1 -- 172.21.15.100:0/877132339 >> v1:172.21.15.151:6789/0 conn(0x7f849c0cb3b0 legacy=0x7f849c06cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.895+0000 7f849b400640 1 -- 172.21.15.100:0/877132339 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849c0cabb0 con 0x7f849c06d1d0 2024-04-23T14:32:37.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.899+0000 7f849aa00640 1 --2- 172.21.15.100:0/877132339 >> v2:172.21.15.100:3300/0 conn(0x7f849c15e650 0x7f849c0cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:37.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.899+0000 7f849b400640 1 --2- 172.21.15.100:0/877132339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f849c06d1d0 0x7f849c0cdc00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f849c1621e0 tx=0x7f848801c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a7869ba44ea62cf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.899+0000 7f8499600640 1 -- 172.21.15.100:0/877132339 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84880b1070 con 0x7f849c06d1d0 2024-04-23T14:32:37.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.899+0000 7f8499600640 1 -- 172.21.15.100:0/877132339 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84880a8050 con 0x7f849c06d1d0 2024-04-23T14:32:37.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.899+0000 7f8499600640 1 -- 172.21.15.100:0/877132339 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84880ac8a0 con 0x7f849c06d1d0 2024-04-23T14:32:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.903+0000 7f849be00640 1 -- 172.21.15.100:0/877132339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f849c06d1d0 msgr2=0x7f849c0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.903+0000 7f849be00640 1 --2- 172.21.15.100:0/877132339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f849c06d1d0 0x7f849c0cdc00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f849c1621e0 tx=0x7f848801c4d0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.903+0000 7f849be00640 1 -- 172.21.15.100:0/877132339 shutdown_connections 2024-04-23T14:32:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.903+0000 7f849be00640 1 --2- 172.21.15.100:0/877132339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f849c06d1d0 0x7f849c0cdc00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.903+0000 7f849be00640 1 --2- 172.21.15.100:0/877132339 >> v2:172.21.15.100:3300/0 conn(0x7f849c15e650 0x7f849c0cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.903+0000 7f849be00640 1 -- 172.21.15.100:0/877132339 >> 172.21.15.100:0/877132339 conn(0x7f849c0b1b20 msgr2=0x7f849c0b1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.903+0000 7f849be00640 1 -- 172.21.15.100:0/877132339 shutdown_connections 2024-04-23T14:32:37.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.903+0000 7f849be00640 1 -- 172.21.15.100:0/877132339 wait complete. 2024-04-23T14:32:37.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.903+0000 7f849be00640 1 Processor -- start 2024-04-23T14:32:37.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.907+0000 7f849be00640 1 -- start start 2024-04-23T14:32:37.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.911+0000 7f849be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c06d1d0 0x7f849c167450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.911+0000 7f849be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f849c0cb3b0 0x7f849c168ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.911+0000 7f849aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c06d1d0 0x7f849c167450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.911+0000 7f849aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c06d1d0 0x7f849c167450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57632/0 (socket says 172.21.15.100:57632) 2024-04-23T14:32:37.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.911+0000 7f849be00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f849c15e650 0x7f849c156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.911+0000 7f849be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f849c16b760 con 0x7f849c06d1d0 2024-04-23T14:32:37.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.911+0000 7f849be00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f849c16b5e0 con 0x7f849c0cb3b0 2024-04-23T14:32:37.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.911+0000 7f849be00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f849c16b8e0 con 0x7f849c15e650 2024-04-23T14:32:37.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.911+0000 7f849aa00640 1 -- 172.21.15.100:0/132553199 learned_addr learned my addr 172.21.15.100:0/132553199 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.911+0000 7f849b400640 1 --2- 172.21.15.100:0/132553199 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f849c15e650 0x7f849c156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.911+0000 7f849a000640 1 --2- 172.21.15.100:0/132553199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f849c0cb3b0 0x7f849c168ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.911+0000 7f849aa00640 1 -- 172.21.15.100:0/132553199 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f849c15e650 msgr2=0x7f849c156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.911+0000 7f758b800640 1 Processor -- start 2024-04-23T14:32:37.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.911+0000 7f849aa00640 1 --2- 172.21.15.100:0/132553199 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f849c15e650 0x7f849c156b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.911+0000 7f758b800640 1 -- start start 2024-04-23T14:32:37.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.911+0000 7f849aa00640 1 -- 172.21.15.100:0/132553199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f849c0cb3b0 msgr2=0x7f849c168ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.911+0000 7f849aa00640 1 --2- 172.21.15.100:0/132553199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f849c0cb3b0 0x7f849c168ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f849aa00640 1 -- 172.21.15.100:0/132553199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849c167a30 con 0x7f849c06d1d0 2024-04-23T14:32:37.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f849a000640 1 --2- 172.21.15.100:0/132553199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f849c0cb3b0 0x7f849c168ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:37.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f849b400640 1 --2- 172.21.15.100:0/132553199 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f849c15e650 0x7f849c156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:37.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f849aa00640 1 --2- 172.21.15.100:0/132553199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c06d1d0 0x7f849c167450 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8494009fc0 tx=0x7f849400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f8492a00640 1 -- 172.21.15.100:0/132553199 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849401a070 con 0x7f849c06d1d0 2024-04-23T14:32:37.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f758b800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f75840b8410 0x7f75840d2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f758b800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75840d1d20 0x7f75840d3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f758b800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75840b6670 con 0x7f7584073190 2024-04-23T14:32:37.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f758b800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f75840b6950 con 0x7f75840b8410 2024-04-23T14:32:37.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f758b800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f75840b6c30 con 0x7f75840d1d20 2024-04-23T14:32:37.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f849be00640 1 -- 172.21.15.100:0/132553199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f849c167d10 con 0x7f849c06d1d0 2024-04-23T14:32:37.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f7583e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f75840b8410 0x7f75840d2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f7583e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f75840b8410 0x7f75840d2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:50954/0 (socket says 172.21.15.100:50954) 2024-04-23T14:32:37.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f7583e00640 1 -- 172.21.15.100:0/1862993073 learned_addr learned my addr 172.21.15.100:0/1862993073 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f849be00640 1 -- 172.21.15.100:0/132553199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f849c1682d0 con 0x7f849c06d1d0 2024-04-23T14:32:37.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f7589800640 1 --2- 172.21.15.100:0/1862993073 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75840d1d20 0x7f75840d3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f8492a00640 1 -- 172.21.15.100:0/132553199 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8494003ba0 con 0x7f849c06d1d0 2024-04-23T14:32:37.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f8492a00640 1 -- 172.21.15.100:0/132553199 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849401a070 con 0x7f849c06d1d0 2024-04-23T14:32:37.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f7583e00640 1 -- 172.21.15.100:0/1862993073 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75840d1d20 msgr2=0x7f75840d3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f7583e00640 1 --2- 172.21.15.100:0/1862993073 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75840d1d20 0x7f75840d3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f7583e00640 1 -- 172.21.15.100:0/1862993073 >> v1:172.21.15.100:6789/0 conn(0x7f7584073190 legacy=0x7f75840d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f7583e00640 1 -- 172.21.15.100:0/1862993073 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75840b6f10 con 0x7f75840b8410 2024-04-23T14:32:37.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.915+0000 7f7589800640 1 --2- 172.21.15.100:0/1862993073 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75840d1d20 0x7f75840d3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:37.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.919+0000 7f8492a00640 1 -- 172.21.15.100:0/132553199 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f849401d070 con 0x7f849c06d1d0 2024-04-23T14:32:37.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.923+0000 7f849be00640 1 -- 172.21.15.100:0/132553199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84740052a0 con 0x7f849c06d1d0 2024-04-23T14:32:37.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.923+0000 7f8492a00640 1 --2- 172.21.15.100:0/132553199 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f847c068780 0x7f847c06ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.923+0000 7f7583e00640 1 --2- 172.21.15.100:0/1862993073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75840b8410 0x7f75840d2bd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f75840b2380 tx=0x7f757401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fa1fe6fca08a78a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.923+0000 7f8492a00640 1 -- 172.21.15.100:0/132553199 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f84940039a0 con 0x7f849c06d1d0 2024-04-23T14:32:37.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.923+0000 7f849a000640 1 --2- 172.21.15.100:0/132553199 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f847c068780 0x7f847c06ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.923+0000 7f7583400640 1 -- 172.21.15.100:0/1862993073 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75740b2070 con 0x7f75840b8410 2024-04-23T14:32:37.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.923+0000 7f7583400640 1 -- 172.21.15.100:0/1862993073 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75740a9070 con 0x7f75840b8410 2024-04-23T14:32:37.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.923+0000 7f758b800640 1 -- 172.21.15.100:0/1862993073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75840b8410 msgr2=0x7f75840d2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.923+0000 7f7583400640 1 -- 172.21.15.100:0/1862993073 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75740b2070 con 0x7f75840b8410 2024-04-23T14:32:37.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.923+0000 7f758b800640 1 --2- 172.21.15.100:0/1862993073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75840b8410 0x7f75840d2bd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f75840b2380 tx=0x7f757401c470 comp rx=0 tx=0).stop 2024-04-23T14:32:37.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.927+0000 7f758b800640 1 -- 172.21.15.100:0/1862993073 shutdown_connections 2024-04-23T14:32:37.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.927+0000 7f758b800640 1 --2- 172.21.15.100:0/1862993073 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75840d1d20 0x7f75840d3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.927+0000 7f758b800640 1 --2- 172.21.15.100:0/1862993073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75840b8410 0x7f75840d2bd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.927+0000 7f758b800640 1 -- 172.21.15.100:0/1862993073 >> 172.21.15.100:0/1862993073 conn(0x7f75840b9bb0 msgr2=0x7f75840b9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.927+0000 7f849a000640 1 --2- 172.21.15.100:0/132553199 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f847c068780 0x7f847c06ac40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f849c09de30 tx=0x7f848c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.927+0000 7f758b800640 1 -- 172.21.15.100:0/1862993073 shutdown_connections 2024-04-23T14:32:37.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.927+0000 7f758b800640 1 -- 172.21.15.100:0/1862993073 wait complete. 2024-04-23T14:32:37.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.931+0000 7f8492a00640 1 -- 172.21.15.100:0/132553199 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8494010070 con 0x7f849c06d1d0 2024-04-23T14:32:37.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.931+0000 7f758b800640 1 Processor -- start 2024-04-23T14:32:37.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.931+0000 7f758b800640 1 -- start start 2024-04-23T14:32:37.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.935+0000 7f758b800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7584073190 0x7f758415f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.935+0000 7f7588e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7584073190 0x7f758415f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.935+0000 7f7588e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7584073190 0x7f758415f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57636/0 (socket says 172.21.15.100:57636) 2024-04-23T14:32:37.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.935+0000 7f7588e00640 1 -- 172.21.15.100:0/3468673230 learned_addr learned my addr 172.21.15.100:0/3468673230 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:37.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.935+0000 7f758b800640 1 --2- 172.21.15.100:0/3468673230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75840b8410 0x7f75840a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.935+0000 7f758b800640 1 --2- 172.21.15.100:0/3468673230 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75840d1d20 0x7f758409ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.935+0000 7f7583e00640 1 --2- 172.21.15.100:0/3468673230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75840b8410 0x7f75840a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.935+0000 7f758b800640 1 -- 172.21.15.100:0/3468673230 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f75840d3650 con 0x7f7584073190 2024-04-23T14:32:37.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.939+0000 7f758b800640 1 -- 172.21.15.100:0/3468673230 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f75840b2810 con 0x7f75840b8410 2024-04-23T14:32:37.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.939+0000 7f758b800640 1 -- 172.21.15.100:0/3468673230 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f75840d37d0 con 0x7f75840d1d20 2024-04-23T14:32:37.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.939+0000 7f7589800640 1 --2- 172.21.15.100:0/3468673230 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75840d1d20 0x7f758409ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.939+0000 7f7588e00640 1 -- 172.21.15.100:0/3468673230 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75840d1d20 msgr2=0x7f758409ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.939+0000 7f7588e00640 1 --2- 172.21.15.100:0/3468673230 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75840d1d20 0x7f758409ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.939+0000 7f7588e00640 1 -- 172.21.15.100:0/3468673230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75840b8410 msgr2=0x7f75840a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.939+0000 7f7588e00640 1 --2- 172.21.15.100:0/3468673230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75840b8410 0x7f75840a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.939+0000 7f7588e00640 1 -- 172.21.15.100:0/3468673230 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75840ca8a0 con 0x7f7584073190 2024-04-23T14:32:37.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.939+0000 7f7588e00640 1 --2- 172.21.15.100:0/3468673230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7584073190 0x7f758415f6b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f757000b3e0 tx=0x7f757000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.939+0000 7f7580c00640 1 -- 172.21.15.100:0/3468673230 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570015020 con 0x7f7584073190 2024-04-23T14:32:37.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.939+0000 7f7580c00640 1 -- 172.21.15.100:0/3468673230 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7570011850 con 0x7f7584073190 2024-04-23T14:32:37.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.939+0000 7f7580c00640 1 -- 172.21.15.100:0/3468673230 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570011c40 con 0x7f7584073190 2024-04-23T14:32:37.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.939+0000 7f758b800640 1 -- 172.21.15.100:0/3468673230 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75840c9d20 con 0x7f7584073190 2024-04-23T14:32:37.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.947+0000 7f758b800640 1 -- 172.21.15.100:0/3468673230 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75840ca2e0 con 0x7f7584073190 2024-04-23T14:32:37.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.947+0000 7f7580c00640 1 -- 172.21.15.100:0/3468673230 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7570018020 con 0x7f7584073190 2024-04-23T14:32:37.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.947+0000 7f7580c00640 1 --2- 172.21.15.100:0/3468673230 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7564068380 0x7f756406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.947+0000 7f7580c00640 1 -- 172.21.15.100:0/3468673230 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f757008a170 con 0x7f7584073190 2024-04-23T14:32:37.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.947+0000 7f7583e00640 1 --2- 172.21.15.100:0/3468673230 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7564068380 0x7f756406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.951+0000 7f758b800640 1 -- 172.21.15.100:0/3468673230 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75840691e0 con 0x7f7584073190 2024-04-23T14:32:37.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.955+0000 7f7583e00640 1 --2- 172.21.15.100:0/3468673230 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7564068380 0x7f756406a840 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f75740af000 tx=0x7f75740a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:37.959+0000 7f7580c00640 1 -- 172.21.15.100:0/3468673230 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7570056080 con 0x7f7584073190 2024-04-23T14:32:38.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.183+0000 7f1164c00640 1 -- 172.21.15.100:0/3245294785 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1160157570 con 0x7f116015a590 2024-04-23T14:32:38.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.191+0000 7f1156000640 1 -- 172.21.15.100:0/3245294785 <== mon.2 v2:172.21.15.151:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f114c056680 con 0x7f116015a590 2024-04-23T14:32:38.190 INFO:teuthology.orchestra.run.smithi100.stdout:55834574859 2024-04-23T14:32:38.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.195+0000 7ff8bb200640 1 -- 172.21.15.100:0/2001483607 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff8bc0c92c0 con 0x7ff8bc16eef0 2024-04-23T14:32:38.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.195+0000 7f1164c00640 1 -- 172.21.15.100:0/3245294785 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f11400683d0 msgr2=0x7f114006a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:38.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.195+0000 7f1164c00640 1 --2- 172.21.15.100:0/3245294785 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f11400683d0 0x7f114006a890 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1150006aa0 tx=0x7f115000c040 comp rx=0 tx=0).stop 2024-04-23T14:32:38.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.195+0000 7f1164c00640 1 -- 172.21.15.100:0/3245294785 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f116015a590 msgr2=0x7f1160207140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:38.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.195+0000 7f1164c00640 1 --2- 172.21.15.100:0/3245294785 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f116015a590 0x7f1160207140 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f114c009fd0 tx=0x7f114c00ecc0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.195+0000 7f1164c00640 1 -- 172.21.15.100:0/3245294785 shutdown_connections 2024-04-23T14:32:38.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.195+0000 7f1164c00640 1 --2- 172.21.15.100:0/3245294785 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f11400683d0 0x7f114006a890 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.195+0000 7f1164c00640 1 --2- 172.21.15.100:0/3245294785 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f116015a590 0x7f1160207140 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.195+0000 7ff8b2000640 1 -- 172.21.15.100:0/2001483607 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff8b4056970 con 0x7ff8bc16eef0 2024-04-23T14:32:38.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.195+0000 7f1164c00640 1 --2- 172.21.15.100:0/3245294785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11600cde60 0x7f1160206c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.195+0000 7f1164c00640 1 --2- 172.21.15.100:0/3245294785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f116006bff0 0x7f1160152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.195+0000 7f1164c00640 1 -- 172.21.15.100:0/3245294785 >> 172.21.15.100:0/3245294785 conn(0x7f11600d0060 msgr2=0x7f116006e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:38.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.195+0000 7f1164c00640 1 -- 172.21.15.100:0/3245294785 shutdown_connections 2024-04-23T14:32:38.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.195+0000 7f1164c00640 1 -- 172.21.15.100:0/3245294785 wait complete. 2024-04-23T14:32:38.198 INFO:teuthology.orchestra.run.smithi100.stdout:55834574858 2024-04-23T14:32:38.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.199+0000 7ff8bb200640 1 -- 172.21.15.100:0/2001483607 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff89c0683b0 msgr2=0x7ff89c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:38.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.199+0000 7ff8bb200640 1 --2- 172.21.15.100:0/2001483607 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff89c0683b0 0x7ff89c06a870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0ae000 tx=0x7ff8ac0a7000 comp rx=0 tx=0).stop 2024-04-23T14:32:38.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.199+0000 7ff8bb200640 1 -- 172.21.15.100:0/2001483607 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8bc16eef0 msgr2=0x7ff8bc0b9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:38.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.199+0000 7ff8bb200640 1 --2- 172.21.15.100:0/2001483607 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8bc16eef0 0x7ff8bc0b9c50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4007d90 tx=0x7ff8b400e4b0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.203+0000 7ff8bb200640 1 -- 172.21.15.100:0/2001483607 shutdown_connections 2024-04-23T14:32:38.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.203+0000 7ff8bb200640 1 --2- 172.21.15.100:0/2001483607 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff89c0683b0 0x7ff89c06a870 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.203+0000 7ff8bb200640 1 --2- 172.21.15.100:0/2001483607 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8bc16eef0 0x7ff8bc0b9c50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.203+0000 7ff8bb200640 1 --2- 172.21.15.100:0/2001483607 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff8bc0d3060 0x7ff8bc0b9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.203+0000 7ff8bb200640 1 --2- 172.21.15.100:0/2001483607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8bc0b7e50 0x7ff8bc167570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.203+0000 7ff8bb200640 1 -- 172.21.15.100:0/2001483607 >> 172.21.15.100:0/2001483607 conn(0x7ff8bc0b5f70 msgr2=0x7ff8bc06d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:38.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.203+0000 7f84b6400640 1 -- 172.21.15.100:0/3004017315 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f84b0073c50 con 0x7f84b00cd170 2024-04-23T14:32:38.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.203+0000 7ff8bb200640 1 -- 172.21.15.100:0/2001483607 shutdown_connections 2024-04-23T14:32:38.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.203+0000 7ff8bb200640 1 -- 172.21.15.100:0/2001483607 wait complete. 2024-04-23T14:32:38.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.203+0000 7f84a7400640 1 -- 172.21.15.100:0/3004017315 <== mon.2 v2:172.21.15.151:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8498015fb0 con 0x7f84b00cd170 2024-04-23T14:32:38.203 INFO:teuthology.orchestra.run.smithi100.stdout:55834574859 2024-04-23T14:32:38.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.207+0000 7f84b6400640 1 -- 172.21.15.100:0/3004017315 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f848c0683d0 msgr2=0x7f848c06a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:38.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.207+0000 7f84b6400640 1 --2- 172.21.15.100:0/3004017315 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f848c0683d0 0x7f848c06a890 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009600 tx=0x7f84a0009190 comp rx=0 tx=0).stop 2024-04-23T14:32:38.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.207+0000 7f84b6400640 1 -- 172.21.15.100:0/3004017315 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84b00cd170 msgr2=0x7f84b0159870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:38.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.207+0000 7f84b6400640 1 --2- 172.21.15.100:0/3004017315 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84b00cd170 0x7f84b0159870 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f84b0070600 tx=0x7f84980190d0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.207+0000 7f84b6400640 1 -- 172.21.15.100:0/3004017315 shutdown_connections 2024-04-23T14:32:38.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.207+0000 7f84b6400640 1 --2- 172.21.15.100:0/3004017315 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f848c0683d0 0x7f848c06a890 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.207+0000 7f84b6400640 1 --2- 172.21.15.100:0/3004017315 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84b00cd170 0x7f84b0159870 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.207+0000 7f84b6400640 1 --2- 172.21.15.100:0/3004017315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b00b5ff0 0x7f84b00cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.207+0000 7f84b6400640 1 --2- 172.21.15.100:0/3004017315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84b0099190 0x7f84b008c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.207+0000 7f84b6400640 1 -- 172.21.15.100:0/3004017315 >> 172.21.15.100:0/3004017315 conn(0x7f84b0073110 msgr2=0x7f84b00cab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:38.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.207+0000 7f2f0ac00640 1 -- 172.21.15.100:0/401782842 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f2edc005590 con 0x7f2f0406c970 2024-04-23T14:32:38.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.207+0000 7f84b6400640 1 -- 172.21.15.100:0/3004017315 shutdown_connections 2024-04-23T14:32:38.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.207+0000 7f84b6400640 1 -- 172.21.15.100:0/3004017315 wait complete. 2024-04-23T14:32:38.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.207+0000 7f2efbe00640 1 -- 172.21.15.100:0/401782842 <== mon.2 v2:172.21.15.151:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2ef405aef0 con 0x7f2f0406c970 2024-04-23T14:32:38.212 INFO:teuthology.orchestra.run.smithi100.stdout:55834574857 2024-04-23T14:32:38.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.211+0000 7f317b200640 1 -- 172.21.15.100:0/3785630019 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f31740cfad0 con 0x7f317415e4f0 2024-04-23T14:32:38.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.211+0000 7f316a000640 1 -- 172.21.15.100:0/3785630019 <== mon.2 v2:172.21.15.151:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f31700559d0 con 0x7f317415e4f0 2024-04-23T14:32:38.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.215+0000 7f2f0ac00640 1 -- 172.21.15.100:0/401782842 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2ee4068480 msgr2=0x7f2ee406a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:38.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.215+0000 7f2f0ac00640 1 --2- 172.21.15.100:0/401782842 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2ee4068480 0x7f2ee406a940 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00ad000 tx=0x7f2ef00a6000 comp rx=0 tx=0).stop 2024-04-23T14:32:38.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.215+0000 7f2f0ac00640 1 -- 172.21.15.100:0/401782842 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2f0406c970 msgr2=0x7f2f04204130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:38.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.215+0000 7f2f0ac00640 1 --2- 172.21.15.100:0/401782842 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2f0406c970 0x7f2f04204130 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400e930 tx=0x7f2ef400edf0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.215+0000 7f2f0ac00640 1 -- 172.21.15.100:0/401782842 shutdown_connections 2024-04-23T14:32:38.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.215+0000 7f2f0ac00640 1 --2- 172.21.15.100:0/401782842 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2ee4068480 0x7f2ee406a940 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.215+0000 7f2f0ac00640 1 --2- 172.21.15.100:0/401782842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f040af640 0x7f2f0415a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.215+0000 7f2f0ac00640 1 --2- 172.21.15.100:0/401782842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f040aec30 0x7f2f0415a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.215+0000 7f2f0ac00640 1 --2- 172.21.15.100:0/401782842 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2f0406c970 0x7f2f04204130 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.215+0000 7f2f0ac00640 1 -- 172.21.15.100:0/401782842 >> 172.21.15.100:0/401782842 conn(0x7f2f04073110 msgr2=0x7f2f0406f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:38.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.215+0000 7f2f0ac00640 1 -- 172.21.15.100:0/401782842 shutdown_connections 2024-04-23T14:32:38.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.215+0000 7f2f0ac00640 1 -- 172.21.15.100:0/401782842 wait complete. 2024-04-23T14:32:38.216 INFO:teuthology.orchestra.run.smithi100.stdout:55834574856 2024-04-23T14:32:38.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.215+0000 7ff8a0400640 1 -- 172.21.15.100:0/305654064 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ff8980cde10 con 0x7ff89815fb20 2024-04-23T14:32:38.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.219+0000 7ff895600640 1 -- 172.21.15.100:0/305654064 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff89005a920 con 0x7ff89815fb20 2024-04-23T14:32:38.218 INFO:teuthology.orchestra.run.smithi100.stdout:55834574859 2024-04-23T14:32:38.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.219+0000 7f317b200640 1 -- 172.21.15.100:0/3785630019 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3158068480 msgr2=0x7f315806a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:38.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.219+0000 7f317b200640 1 --2- 172.21.15.100:0/3785630019 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3158068480 0x7f315806a940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f316c0ae000 tx=0x7f316c0a7070 comp rx=0 tx=0).stop 2024-04-23T14:32:38.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.219+0000 7f317b200640 1 -- 172.21.15.100:0/3785630019 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f317415e4f0 msgr2=0x7f317406cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:38.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.219+0000 7f317b200640 1 --2- 172.21.15.100:0/3785630019 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f317415e4f0 0x7f317406cf80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f317000b430 tx=0x7f317000b8f0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.223+0000 7f317b200640 1 -- 172.21.15.100:0/3785630019 shutdown_connections 2024-04-23T14:32:38.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.223+0000 7f317b200640 1 --2- 172.21.15.100:0/3785630019 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3158068480 0x7f315806a940 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f316c0ae000 tx=0x7f316c0a7070 comp rx=0 tx=0).stop 2024-04-23T14:32:38.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.223+0000 7f317b200640 1 --2- 172.21.15.100:0/3785630019 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f317415e4f0 0x7f317406cf80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.223+0000 7f317b200640 1 --2- 172.21.15.100:0/3785630019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31740d43a0 0x7f31741602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.223+0000 7f317b200640 1 --2- 172.21.15.100:0/3785630019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31740b5e90 0x7f3174207210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.223+0000 7f317b200640 1 -- 172.21.15.100:0/3785630019 >> 172.21.15.100:0/3785630019 conn(0x7f317409d790 msgr2=0x7f31740d0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:38.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.223+0000 7ff8a0400640 1 -- 172.21.15.100:0/305654064 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff878068450 msgr2=0x7ff87806a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:38.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.223+0000 7ff8a0400640 1 --2- 172.21.15.100:0/305654064 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff878068450 0x7ff87806a910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff88c0016e0 tx=0x7ff88c009040 comp rx=0 tx=0).stop 2024-04-23T14:32:38.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.223+0000 7ff8a0400640 1 -- 172.21.15.100:0/305654064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff89815fb20 msgr2=0x7ff89816b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:38.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.223+0000 7ff8a0400640 1 --2- 172.21.15.100:0/305654064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff89815fb20 0x7ff89816b500 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff89000cbc0 tx=0x7ff89000a430 comp rx=0 tx=0).stop 2024-04-23T14:32:38.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.223+0000 7ff8a0400640 1 -- 172.21.15.100:0/305654064 shutdown_connections 2024-04-23T14:32:38.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.223+0000 7ff8a0400640 1 --2- 172.21.15.100:0/305654064 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff878068450 0x7ff87806a910 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.223+0000 7ff8a0400640 1 --2- 172.21.15.100:0/305654064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff89815fb20 0x7ff89816b500 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.223+0000 7ff8a0400640 1 --2- 172.21.15.100:0/305654064 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff89815ef80 0x7ff89806ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.223+0000 7ff8a0400640 1 --2- 172.21.15.100:0/305654064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8980b5460 0x7ff898155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.223+0000 7ff8a0400640 1 -- 172.21.15.100:0/305654064 >> 172.21.15.100:0/305654064 conn(0x7ff898070670 msgr2=0x7ff8980b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:38.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.223+0000 7f317b200640 1 -- 172.21.15.100:0/3785630019 shutdown_connections 2024-04-23T14:32:38.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.223+0000 7f317b200640 1 -- 172.21.15.100:0/3785630019 wait complete. 2024-04-23T14:32:38.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.223+0000 7ff8a0400640 1 -- 172.21.15.100:0/305654064 shutdown_connections 2024-04-23T14:32:38.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.223+0000 7ff8a0400640 1 -- 172.21.15.100:0/305654064 wait complete. 2024-04-23T14:32:38.232 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574861 got 55834574859 for osd.0 2024-04-23T14:32:38.247 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574858 for osd.1 2024-04-23T14:32:38.248 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.5 2024-04-23T14:32:38.250 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574861 got 55834574859 for osd.2 2024-04-23T14:32:38.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.251+0000 7f849be00640 1 -- 172.21.15.100:0/132553199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f8474005590 con 0x7f849c06d1d0 2024-04-23T14:32:38.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.255+0000 7f8492a00640 1 -- 172.21.15.100:0/132553199 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8494056570 con 0x7f849c06d1d0 2024-04-23T14:32:38.254 INFO:teuthology.orchestra.run.smithi100.stdout:55834574857 2024-04-23T14:32:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.255+0000 7f849be00640 1 -- 172.21.15.100:0/132553199 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f847c068780 msgr2=0x7f847c06ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.255+0000 7f849be00640 1 --2- 172.21.15.100:0/132553199 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f847c068780 0x7f847c06ac40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f849c09de30 tx=0x7f848c009040 comp rx=0 tx=0).stop 2024-04-23T14:32:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.255+0000 7f849be00640 1 -- 172.21.15.100:0/132553199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c06d1d0 msgr2=0x7f849c167450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.255+0000 7f849be00640 1 --2- 172.21.15.100:0/132553199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c06d1d0 0x7f849c167450 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8494009fc0 tx=0x7f849400d560 comp rx=0 tx=0).stop 2024-04-23T14:32:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.255+0000 7f849be00640 1 -- 172.21.15.100:0/132553199 shutdown_connections 2024-04-23T14:32:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.255+0000 7f849be00640 1 --2- 172.21.15.100:0/132553199 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f847c068780 0x7f847c06ac40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.255+0000 7f849be00640 1 --2- 172.21.15.100:0/132553199 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f849c15e650 0x7f849c156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.255+0000 7f849be00640 1 --2- 172.21.15.100:0/132553199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f849c0cb3b0 0x7f849c168ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.255+0000 7f849be00640 1 --2- 172.21.15.100:0/132553199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f849c06d1d0 0x7f849c167450 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.255+0000 7f849be00640 1 -- 172.21.15.100:0/132553199 >> 172.21.15.100:0/132553199 conn(0x7f849c0b1b20 msgr2=0x7f849c06e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:38.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.255+0000 7f849be00640 1 -- 172.21.15.100:0/132553199 shutdown_connections 2024-04-23T14:32:38.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.255+0000 7f849be00640 1 -- 172.21.15.100:0/132553199 wait complete. 2024-04-23T14:32:38.258 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574861 got 55834574859 for osd.7 2024-04-23T14:32:38.260 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.6 2024-04-23T14:32:38.277 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.4 2024-04-23T14:32:38.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.279+0000 7f758b800640 1 -- 172.21.15.100:0/3468673230 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f75840cb320 con 0x7f7584073190 2024-04-23T14:32:38.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.279+0000 7f7580c00640 1 -- 172.21.15.100:0/3468673230 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7570055980 con 0x7f7584073190 2024-04-23T14:32:38.281 INFO:teuthology.orchestra.run.smithi100.stdout:55834574857 2024-04-23T14:32:38.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.283+0000 7f758b800640 1 -- 172.21.15.100:0/3468673230 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7564068380 msgr2=0x7f756406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:38.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.283+0000 7f758b800640 1 --2- 172.21.15.100:0/3468673230 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7564068380 0x7f756406a840 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f75740af000 tx=0x7f75740a7000 comp rx=0 tx=0).stop 2024-04-23T14:32:38.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.283+0000 7f758b800640 1 -- 172.21.15.100:0/3468673230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7584073190 msgr2=0x7f758415f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:38.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.283+0000 7f758b800640 1 --2- 172.21.15.100:0/3468673230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7584073190 0x7f758415f6b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f757000b3e0 tx=0x7f757000b8a0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.283+0000 7f758b800640 1 -- 172.21.15.100:0/3468673230 shutdown_connections 2024-04-23T14:32:38.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.283+0000 7f758b800640 1 --2- 172.21.15.100:0/3468673230 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7564068380 0x7f756406a840 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.283+0000 7f758b800640 1 --2- 172.21.15.100:0/3468673230 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75840d1d20 0x7f758409ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.283+0000 7f758b800640 1 --2- 172.21.15.100:0/3468673230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75840b8410 0x7f75840a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.283+0000 7f758b800640 1 --2- 172.21.15.100:0/3468673230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7584073190 0x7f758415f6b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.283+0000 7f758b800640 1 -- 172.21.15.100:0/3468673230 >> 172.21.15.100:0/3468673230 conn(0x7f75840b9bb0 msgr2=0x7f75840b9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:38.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.283+0000 7f758b800640 1 -- 172.21.15.100:0/3468673230 shutdown_connections 2024-04-23T14:32:38.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:38.283+0000 7f758b800640 1 -- 172.21.15.100:0/3468673230 wait complete. 2024-04-23T14:32:38.302 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.3 2024-04-23T14:32:39.233 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-23T14:32:39.248 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-23T14:32:39.249 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-23T14:32:39.250 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-23T14:32:39.259 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-23T14:32:39.261 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-23T14:32:39.278 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-23T14:32:39.303 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-23T14:32:39.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.435+0000 7f182bc00640 1 Processor -- start 2024-04-23T14:32:39.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.435+0000 7f182bc00640 1 -- start start 2024-04-23T14:32:39.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.435+0000 7f182bc00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f182415a590 0x7f18240cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.435+0000 7f182bc00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f18240cde60 0x7f182406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.435+0000 7f182bc00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f182406bff0 0x7f182415e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.439+0000 7f1829200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f182415a590 0x7f18240cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.439+0000 7f1823e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f18240cde60 0x7f182406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.439+0000 7f1823e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f18240cde60 0x7f182406bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:44528/0 (socket says 172.21.15.100:44528) 2024-04-23T14:32:39.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.439+0000 7f1829200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f182415a590 0x7f18240cf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:55164/0 (socket says 172.21.15.100:55164) 2024-04-23T14:32:39.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.439+0000 7f1823e00640 1 -- 172.21.15.100:0/3484601864 learned_addr learned my addr 172.21.15.100:0/3484601864 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:39.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.439+0000 7f182bc00640 1 -- 172.21.15.100:0/3484601864 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f182406c460 con 0x7f182406bff0 2024-04-23T14:32:39.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.439+0000 7f182bc00640 1 -- 172.21.15.100:0/3484601864 --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f18240ae450 con 0x7f18240cde60 2024-04-23T14:32:39.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.439+0000 7f182bc00640 1 -- 172.21.15.100:0/3484601864 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f18240ad880 con 0x7f182415a590 2024-04-23T14:32:39.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.439+0000 7f1829200640 1 -- 172.21.15.100:0/3484601864 >> v2:172.21.15.151:3300/0 conn(0x7f18240cde60 msgr2=0x7f182406bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:39.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.439+0000 7f1829200640 1 --2- 172.21.15.100:0/3484601864 >> v2:172.21.15.151:3300/0 conn(0x7f18240cde60 0x7f182406bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.439+0000 7f1829200640 1 -- 172.21.15.100:0/3484601864 >> v2:172.21.15.100:3300/0 conn(0x7f182406bff0 msgr2=0x7f182415e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:39.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.439+0000 7f1829200640 1 --2- 172.21.15.100:0/3484601864 >> v2:172.21.15.100:3300/0 conn(0x7f182406bff0 0x7f182415e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.443+0000 7f1829200640 1 -- 172.21.15.100:0/3484601864 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18240adb20 con 0x7f182415a590 2024-04-23T14:32:39.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.443+0000 7f1823e00640 1 --2- 172.21.15.100:0/3484601864 >> v2:172.21.15.151:3300/0 conn(0x7f18240cde60 0x7f182406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:39.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.447+0000 7f1829200640 1 --2- 172.21.15.100:0/3484601864 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f182415a590 0x7f18240cf670 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f18240b2bb0 tx=0x7f181401d6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75da986aed309110 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.447+0000 7f1823400640 1 -- 172.21.15.100:0/3484601864 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18140ad070 con 0x7f182415a590 2024-04-23T14:32:39.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.447+0000 7f1823400640 1 -- 172.21.15.100:0/3484601864 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18140a7620 con 0x7f182415a590 2024-04-23T14:32:39.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.447+0000 7f1823400640 1 -- 172.21.15.100:0/3484601864 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18140a79f0 con 0x7f182415a590 2024-04-23T14:32:39.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.447+0000 7f182bc00640 1 -- 172.21.15.100:0/3484601864 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f182415a590 msgr2=0x7f18240cf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.447+0000 7f182bc00640 1 --2- 172.21.15.100:0/3484601864 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f182415a590 0x7f18240cf670 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f18240b2bb0 tx=0x7f181401d6d0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.447+0000 7f182bc00640 1 -- 172.21.15.100:0/3484601864 shutdown_connections 2024-04-23T14:32:39.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.447+0000 7f182bc00640 1 --2- 172.21.15.100:0/3484601864 >> v2:172.21.15.100:3300/0 conn(0x7f182406bff0 0x7f182415e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.447+0000 7f182bc00640 1 --2- 172.21.15.100:0/3484601864 >> v2:172.21.15.151:3300/0 conn(0x7f18240cde60 0x7f182406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.447+0000 7f182bc00640 1 --2- 172.21.15.100:0/3484601864 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f182415a590 0x7f18240cf670 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.447+0000 7f182bc00640 1 -- 172.21.15.100:0/3484601864 >> 172.21.15.100:0/3484601864 conn(0x7f18240d0060 msgr2=0x7f18240d0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:39.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.447+0000 7f182bc00640 1 -- 172.21.15.100:0/3484601864 shutdown_connections 2024-04-23T14:32:39.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.451+0000 7f182bc00640 1 -- 172.21.15.100:0/3484601864 wait complete. 2024-04-23T14:32:39.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.451+0000 7f182bc00640 1 Processor -- start 2024-04-23T14:32:39.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.451+0000 7f182bc00640 1 -- start start 2024-04-23T14:32:39.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.455+0000 7f182bc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f182406bff0 0x7f182415bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.455+0000 7f182bc00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18240cde60 0x7f1824152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.455+0000 7f182bc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f182415a590 0x7f182414e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.455+0000 7f1823e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18240cde60 0x7f1824152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.455+0000 7f1823e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18240cde60 0x7f1824152980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:55174/0 (socket says 172.21.15.100:55174) 2024-04-23T14:32:39.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.455+0000 7f1829200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f182406bff0 0x7f182415bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.455+0000 7f1829200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f182406bff0 0x7f182415bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:44540/0 (socket says 172.21.15.100:44540) 2024-04-23T14:32:39.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.455+0000 7f1823e00640 1 -- 172.21.15.100:0/170613723 learned_addr learned my addr 172.21.15.100:0/170613723 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:39.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.455+0000 7f182bc00640 1 -- 172.21.15.100:0/170613723 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f182415aaf0 con 0x7f182415a590 2024-04-23T14:32:39.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.455+0000 7f182bc00640 1 -- 172.21.15.100:0/170613723 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f182415a970 con 0x7f182406bff0 2024-04-23T14:32:39.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.455+0000 7f182bc00640 1 -- 172.21.15.100:0/170613723 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f18240ca990 con 0x7f18240cde60 2024-04-23T14:32:39.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.455+0000 7f1829c00640 1 --2- 172.21.15.100:0/170613723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f182415a590 0x7f182414e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.459+0000 7f1823e00640 1 -- 172.21.15.100:0/170613723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f182406bff0 msgr2=0x7f182415bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.459+0000 7f1823e00640 1 --2- 172.21.15.100:0/170613723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f182406bff0 0x7f182415bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.459+0000 7f1823e00640 1 -- 172.21.15.100:0/170613723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f182415a590 msgr2=0x7f182414e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.459+0000 7f1823e00640 1 --2- 172.21.15.100:0/170613723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f182415a590 0x7f182414e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.459+0000 7f1823e00640 1 -- 172.21.15.100:0/170613723 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18241f9d20 con 0x7f18240cde60 2024-04-23T14:32:39.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.459+0000 7f1829c00640 1 --2- 172.21.15.100:0/170613723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f182415a590 0x7f182414e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:39.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.459+0000 7f1823e00640 1 --2- 172.21.15.100:0/170613723 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18240cde60 0x7f1824152980 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f181c009f00 tx=0x7f181c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.459+0000 7f1820c00640 1 -- 172.21.15.100:0/170613723 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181c014480 con 0x7f18240cde60 2024-04-23T14:32:39.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.459+0000 7f1820c00640 1 -- 172.21.15.100:0/170613723 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f181c019070 con 0x7f18240cde60 2024-04-23T14:32:39.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.459+0000 7f182bc00640 1 -- 172.21.15.100:0/170613723 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18241540a0 con 0x7f18240cde60 2024-04-23T14:32:39.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.459+0000 7f182bc00640 1 -- 172.21.15.100:0/170613723 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f182414db00 con 0x7f18240cde60 2024-04-23T14:32:39.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.459+0000 7f1820c00640 1 -- 172.21.15.100:0/170613723 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181c014860 con 0x7f18240cde60 2024-04-23T14:32:39.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.463+0000 7f1820c00640 1 -- 172.21.15.100:0/170613723 <== mon.2 v2:172.21.15.151:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f181c012050 con 0x7f18240cde60 2024-04-23T14:32:39.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.463+0000 7f181a000640 1 -- 172.21.15.100:0/170613723 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f80052a0 con 0x7f18240cde60 2024-04-23T14:32:39.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.463+0000 7f1820c00640 1 --2- 172.21.15.100:0/170613723 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1804068380 0x7f180406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.467+0000 7f1820c00640 1 -- 172.21.15.100:0/170613723 <== mon.2 v2:172.21.15.151:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f181c087270 con 0x7f18240cde60 2024-04-23T14:32:39.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.467+0000 7f1829200640 1 --2- 172.21.15.100:0/170613723 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1804068380 0x7f180406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.467+0000 7f1829200640 1 --2- 172.21.15.100:0/170613723 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1804068380 0x7f180406a840 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f18140af000 tx=0x7f18140a8000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.475+0000 7f1820c00640 1 -- 172.21.15.100:0/170613723 <== mon.2 v2:172.21.15.151:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f181c010040 con 0x7f18240cde60 2024-04-23T14:32:39.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.523+0000 7ff2b8000640 1 Processor -- start 2024-04-23T14:32:39.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.523+0000 7ff2b8000640 1 -- start start 2024-04-23T14:32:39.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7ff2b8000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff2b009dd50 0x7ff2b00c3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7ff2b8000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2b00ceba0 con 0x7ff2b00c3680 2024-04-23T14:32:39.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7ff2b8000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2b00cee80 con 0x7ff2b0162a40 2024-04-23T14:32:39.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7ff2b8000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff2b00cfc40 con 0x7ff2b009dd50 2024-04-23T14:32:39.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7ff2b5600640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7ff2b0162a40 0x7ff2b00ad8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:54938/0 (socket says 172.21.15.100:54938) 2024-04-23T14:32:39.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7ff2b5600640 1 -- 172.21.15.100:0/3459419257 learned_addr learned my addr 172.21.15.100:0/3459419257 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:39.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7ff2b4c00640 1 --2- 172.21.15.100:0/3459419257 >> v2:172.21.15.100:3300/0 conn(0x7ff2b009dd50 0x7ff2b00c3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7ff2afe00640 1 -- 172.21.15.100:0/3459419257 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1822446521 0 0) 0x7ff2b00cee80 con 0x7ff2b0162a40 2024-04-23T14:32:39.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7ff2afe00640 1 -- 172.21.15.100:0/3459419257 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff298003530 con 0x7ff2b0162a40 2024-04-23T14:32:39.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.523+0000 7fc543200640 1 Processor -- start 2024-04-23T14:32:39.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7f799c800640 1 Processor -- start 2024-04-23T14:32:39.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.523+0000 7fc543200640 1 -- start start 2024-04-23T14:32:39.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7fc543200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc53c04fa80 0x7fc53c04fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7fc543200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc53c04f5c0 0x7fc53c038020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7fc542800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc53c04f5c0 0x7fc53c038020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7fc542800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc53c04f5c0 0x7fc53c038020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:44554/0 (socket says 172.21.15.100:44554) 2024-04-23T14:32:39.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7fc542800640 1 -- 172.21.15.100:0/1929916298 learned_addr learned my addr 172.21.15.100:0/1929916298 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:39.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7fc543200640 1 -- 172.21.15.100:0/1929916298 --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc53c054390 con 0x7fc53c0df390 2024-04-23T14:32:39.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7fc543200640 1 -- 172.21.15.100:0/1929916298 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc53c005ea0 con 0x7fc53c04fa80 2024-04-23T14:32:39.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7fc543200640 1 -- 172.21.15.100:0/1929916298 --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fc53c006d00 con 0x7fc53c04f5c0 2024-04-23T14:32:39.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7fc541e00640 1 --2- 172.21.15.100:0/1929916298 >> v2:172.21.15.100:3300/0 conn(0x7fc53c04fa80 0x7fc53c04fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7fc540a00640 1 -- 172.21.15.100:0/1929916298 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3086447433 0 0) 0x7fc53c054390 con 0x7fc53c0df390 2024-04-23T14:32:39.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7fc540a00640 1 -- 172.21.15.100:0/1929916298 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc528003530 con 0x7fc53c0df390 2024-04-23T14:32:39.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7fc540a00640 1 -- 172.21.15.100:0/1929916298 <== mon.1 v1:172.21.15.151:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1742451751 0 0) 0x7fc528003530 con 0x7fc53c0df390 2024-04-23T14:32:39.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7fc540a00640 1 -- 172.21.15.100:0/1929916298 >> v2:172.21.15.151:3300/0 conn(0x7fc53c04f5c0 msgr2=0x7fc53c038020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:39.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7fc540a00640 1 --2- 172.21.15.100:0/1929916298 >> v2:172.21.15.151:3300/0 conn(0x7fc53c04f5c0 0x7fc53c038020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7fc540a00640 1 -- 172.21.15.100:0/1929916298 >> v2:172.21.15.100:3300/0 conn(0x7fc53c04fa80 msgr2=0x7fc53c04fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:39.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7fc540a00640 1 --2- 172.21.15.100:0/1929916298 >> v2:172.21.15.100:3300/0 conn(0x7fc53c04fa80 0x7fc53c04fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7fc540a00640 1 -- 172.21.15.100:0/1929916298 --> v1:172.21.15.151:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53c0503a0 con 0x7fc53c0df390 2024-04-23T14:32:39.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7fc540a00640 1 -- 172.21.15.100:0/1929916298 <== mon.1 v1:172.21.15.151:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7fc534001680 con 0x7fc53c0df390 2024-04-23T14:32:39.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7f799c800640 1 -- start start 2024-04-23T14:32:39.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7ff2afe00640 1 -- 172.21.15.100:0/3459419257 <== mon.1 v1:172.21.15.151:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 523205857 0 0) 0x7ff298003530 con 0x7ff2b0162a40 2024-04-23T14:32:39.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7ff2afe00640 1 -- 172.21.15.100:0/3459419257 >> v2:172.21.15.100:3300/0 conn(0x7ff2b009dd50 msgr2=0x7ff2b00c3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:39.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7ff2afe00640 1 --2- 172.21.15.100:0/3459419257 >> v2:172.21.15.100:3300/0 conn(0x7ff2b009dd50 0x7ff2b00c3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7ff2afe00640 1 -- 172.21.15.100:0/3459419257 >> v1:172.21.15.100:6789/0 conn(0x7ff2b00c3680 legacy=0x7ff2b006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:39.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7ff2afe00640 1 -- 172.21.15.100:0/3459419257 --> v1:172.21.15.151:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b00cf160 con 0x7ff2b0162a40 2024-04-23T14:32:39.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7ff2afe00640 1 -- 172.21.15.100:0/3459419257 <== mon.1 v1:172.21.15.151:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7ff29c001680 con 0x7ff2b0162a40 2024-04-23T14:32:39.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7ff2afe00640 1 -- 172.21.15.100:0/3459419257 >> v1:172.21.15.151:6789/0 conn(0x7ff2b0162a40 legacy=0x7ff2b00ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7ff2afe00640 1 --2- 172.21.15.100:0/3459419257 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2980039c0 0x7ff298003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7ff2afe00640 1 --2- 172.21.15.100:0/3459419257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff298004810 0x7ff298004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7ff2afe00640 1 --2- 172.21.15.100:0/3459419257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2980051d0 0x7ff29800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7ff2afe00640 1 -- 172.21.15.100:0/3459419257 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b00cfc40 con 0x7ff2980051d0 2024-04-23T14:32:39.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7ff2afe00640 1 -- 172.21.15.100:0/3459419257 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff29800c100 con 0x7ff298004810 2024-04-23T14:32:39.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7ff2afe00640 1 -- 172.21.15.100:0/3459419257 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff29800c300 con 0x7ff2980039c0 2024-04-23T14:32:39.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7ff2b5600640 1 --2- 172.21.15.100:0/3459419257 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2980039c0 0x7ff298003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7fc540a00640 1 -- 172.21.15.100:0/1929916298 >> v1:172.21.15.151:6789/0 conn(0x7fc53c0df390 legacy=0x7fc53c007190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.527+0000 7fc540a00640 1 --2- 172.21.15.100:0/1929916298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc528003a10 0x7fc528003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7fc540a00640 1 --2- 172.21.15.100:0/1929916298 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc528004810 0x7fc528004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7fc540a00640 1 --2- 172.21.15.100:0/1929916298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc528005220 0x7fc52800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7fc540a00640 1 -- 172.21.15.100:0/1929916298 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c005ea0 con 0x7fc528005220 2024-04-23T14:32:39.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7fc540a00640 1 -- 172.21.15.100:0/1929916298 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c006d00 con 0x7fc528003a10 2024-04-23T14:32:39.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7fc540a00640 1 -- 172.21.15.100:0/1929916298 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc528005a50 con 0x7fc528004810 2024-04-23T14:32:39.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7fc541400640 1 --2- 172.21.15.100:0/1929916298 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc528004810 0x7fc528004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7fc541400640 1 -- 172.21.15.100:0/1929916298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc528003a10 msgr2=0x7fc528003e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:32:39.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7fc541400640 1 --2- 172.21.15.100:0/1929916298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc528003a10 0x7fc528003e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7fc541400640 1 -- 172.21.15.100:0/1929916298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc528005220 msgr2=0x7fc52800dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:39.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7fc541400640 1 --2- 172.21.15.100:0/1929916298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc528005220 0x7fc52800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7fc541400640 1 -- 172.21.15.100:0/1929916298 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc528003530 con 0x7fc528004810 2024-04-23T14:32:39.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7ff2b5600640 1 -- 172.21.15.100:0/3459419257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff298004810 msgr2=0x7ff298004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:32:39.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7ff2b5600640 1 --2- 172.21.15.100:0/3459419257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff298004810 0x7ff298004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7ff2b5600640 1 -- 172.21.15.100:0/3459419257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2980051d0 msgr2=0x7ff29800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:39.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7ff2b5600640 1 --2- 172.21.15.100:0/3459419257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2980051d0 0x7ff29800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7ff2b5600640 1 -- 172.21.15.100:0/3459419257 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b00cf160 con 0x7ff2980039c0 2024-04-23T14:32:39.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7ff2b6000640 1 -- 172.21.15.100:0/3459419257 reap_dead start 2024-04-23T14:32:39.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7fc542800640 1 -- 172.21.15.100:0/1929916298 reap_dead start 2024-04-23T14:32:39.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7f799c800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79900df2b0 0x7f799004faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7f799c800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7990007550 0x7f799004ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7f799c800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7990054120 con 0x7f7990033d00 2024-04-23T14:32:39.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7f799c800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f79900df690 con 0x7f7990007550 2024-04-23T14:32:39.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7f799c800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7990054400 con 0x7f79900df2b0 2024-04-23T14:32:39.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7f7999e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79900df2b0 0x7f799004faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7f7999e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79900df2b0 0x7f799004faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:55200/0 (socket says 172.21.15.100:55200) 2024-04-23T14:32:39.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7f7999e00640 1 -- 172.21.15.100:0/3557856387 learned_addr learned my addr 172.21.15.100:0/3557856387 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:39.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7f799a800640 1 --2- 172.21.15.100:0/3557856387 >> v2:172.21.15.100:3300/0 conn(0x7f7990007550 0x7f799004ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7f7999e00640 1 -- 172.21.15.100:0/3557856387 >> v2:172.21.15.100:3300/0 conn(0x7f7990007550 msgr2=0x7f799004ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:39.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7f7999e00640 1 --2- 172.21.15.100:0/3557856387 >> v2:172.21.15.100:3300/0 conn(0x7f7990007550 0x7f799004ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7f7999e00640 1 -- 172.21.15.100:0/3557856387 >> v1:172.21.15.151:6789/0 conn(0x7f7990033d00 legacy=0x7f79900e2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7fdb87a00640 1 Processor -- start 2024-04-23T14:32:39.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7fdb87a00640 1 -- start start 2024-04-23T14:32:39.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7f7999e00640 1 -- 172.21.15.100:0/3557856387 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79900546e0 con 0x7f79900df2b0 2024-04-23T14:32:39.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.531+0000 7f799a800640 1 --2- 172.21.15.100:0/3557856387 >> v2:172.21.15.100:3300/0 conn(0x7f7990007550 0x7f799004ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:39.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fd146e00640 1 Processor -- start 2024-04-23T14:32:39.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fd146e00640 1 -- start start 2024-04-23T14:32:39.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7faf3c600640 1 Processor -- start 2024-04-23T14:32:39.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fdb87a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdb80068ad0 0x7fdb8006cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fdb87a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb8015a880 0x7fdb8006f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fdb87a00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb8006bf40 con 0x7fdb8006bab0 2024-04-23T14:32:39.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fdb87a00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fdb8006cb50 con 0x7fdb80068ad0 2024-04-23T14:32:39.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fdb87a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fdb8006c220 con 0x7fdb8015a880 2024-04-23T14:32:39.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7faf3c600640 1 -- start start 2024-04-23T14:32:39.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7ff2b5600640 1 --2- 172.21.15.100:0/3459419257 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2980039c0 0x7ff298003dc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff29c01d340 tx=0x7ff29c02dd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d43d82a353a4d7f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fdb85a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb8015a880 0x7fdb8006f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fdb85a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb8015a880 0x7fdb8006f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:55212/0 (socket says 172.21.15.100:55212) 2024-04-23T14:32:39.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fdb85a00640 1 -- 172.21.15.100:0/2786839592 learned_addr learned my addr 172.21.15.100:0/2786839592 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:39.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fdb7fe00640 1 --2- 172.21.15.100:0/2786839592 >> v2:172.21.15.100:3300/0 conn(0x7fdb80068ad0 0x7fdb8006cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fdb85000640 1 --1- 172.21.15.100:0/2786839592 >> v1:172.21.15.151:6789/0 conn(0x7fdb8006bab0 0x7fdb800cc7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:54970/0 (socket says 172.21.15.100:54970) 2024-04-23T14:32:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7ff2afe00640 1 -- 172.21.15.100:0/3459419257 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c0c3070 con 0x7ff2980039c0 2024-04-23T14:32:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7ff2afe00640 1 -- 172.21.15.100:0/3459419257 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff29c0b6c00 con 0x7ff2980039c0 2024-04-23T14:32:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7ff2afe00640 1 -- 172.21.15.100:0/3459419257 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c018d90 con 0x7ff2980039c0 2024-04-23T14:32:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fdb7fe00640 1 -- 172.21.15.100:0/2786839592 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb8015a880 msgr2=0x7fdb8006f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fdb7fe00640 1 --2- 172.21.15.100:0/2786839592 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb8015a880 0x7fdb8006f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fdb7fe00640 1 -- 172.21.15.100:0/2786839592 >> v1:172.21.15.151:6789/0 conn(0x7fdb8006bab0 legacy=0x7fdb800cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fdb7fe00640 1 -- 172.21.15.100:0/2786839592 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8006c500 con 0x7fdb80068ad0 2024-04-23T14:32:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fd146e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd1400cd370 0x7fd1400ad160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fd146e00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1400c70e0 con 0x7fd14006c490 2024-04-23T14:32:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fd146e00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1400c6b20 con 0x7fd14016b780 2024-04-23T14:32:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fd146e00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fd1400c6e00 con 0x7fd1400cd370 2024-04-23T14:32:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fd13f400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd1400cd370 0x7fd1400ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fd13f400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd1400cd370 0x7fd1400ad160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52888/0 (socket says 172.21.15.100:52888) 2024-04-23T14:32:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fd13f400640 1 -- 172.21.15.100:0/4246692954 learned_addr learned my addr 172.21.15.100:0/4246692954 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.535+0000 7fc541400640 1 --2- 172.21.15.100:0/1929916298 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc528004810 0x7fc528004c90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc534012400 tx=0x7fc53402db50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bce62e9e64676445 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7fc540a00640 1 -- 172.21.15.100:0/1929916298 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5340c3070 con 0x7fc528004810 2024-04-23T14:32:39.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7ff2b8000640 1 -- 172.21.15.100:0/3459419257 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2980039c0 msgr2=0x7ff298003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7ff2b8000640 1 --2- 172.21.15.100:0/3459419257 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2980039c0 0x7ff298003dc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff29c01d340 tx=0x7ff29c02dd00 comp rx=0 tx=0).stop 2024-04-23T14:32:39.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7faf3c600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faf34162b00 0x7faf340b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7fc540a00640 1 -- 172.21.15.100:0/1929916298 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5340b6a50 con 0x7fc528004810 2024-04-23T14:32:39.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7fc540a00640 1 -- 172.21.15.100:0/1929916298 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534012440 con 0x7fc528004810 2024-04-23T14:32:39.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7f7999e00640 1 --2- 172.21.15.100:0/3557856387 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79900df2b0 0x7f799004faa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f798800b9f0 tx=0x7f798801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0309c4aae09fe8a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7fd13f400640 1 -- 172.21.15.100:0/4246692954 >> v1:172.21.15.151:6789/0 conn(0x7fd14016b780 legacy=0x7fd1400ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:39.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7ff2b8000640 1 -- 172.21.15.100:0/3459419257 shutdown_connections 2024-04-23T14:32:39.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7fd13f400640 1 -- 172.21.15.100:0/4246692954 >> v1:172.21.15.100:6789/0 conn(0x7fd14006c490 legacy=0x7fd1400b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7ff2b8000640 1 --2- 172.21.15.100:0/3459419257 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2980039c0 0x7ff298003dc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7ff2b8000640 1 -- 172.21.15.100:0/3459419257 >> 172.21.15.100:0/3459419257 conn(0x7ff2b00d0170 msgr2=0x7ff2b00d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:39.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7f7998a00640 1 -- 172.21.15.100:0/3557856387 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79880b0070 con 0x7f79900df2b0 2024-04-23T14:32:39.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7f7998a00640 1 -- 172.21.15.100:0/3557856387 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79880ab600 con 0x7f79900df2b0 2024-04-23T14:32:39.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7ff2b8000640 1 -- 172.21.15.100:0/3459419257 shutdown_connections 2024-04-23T14:32:39.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7faf3c600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faf34162ee0 0x7faf340cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7faf3c600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf3406cc20 con 0x7faf3406f620 2024-04-23T14:32:39.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7faf3c600640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7faf3406fda0 con 0x7faf34162b00 2024-04-23T14:32:39.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7faf39200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faf34162ee0 0x7faf340cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7ff2b8000640 1 -- 172.21.15.100:0/3459419257 wait complete. 2024-04-23T14:32:39.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7faf39200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faf34162ee0 0x7faf340cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:55216/0 (socket says 172.21.15.100:55216) 2024-04-23T14:32:39.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7faf39200640 1 -- 172.21.15.100:0/3997803185 learned_addr learned my addr 172.21.15.100:0/3997803185 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:39.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7f7998a00640 1 -- 172.21.15.100:0/3557856387 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79880b0070 con 0x7f79900df2b0 2024-04-23T14:32:39.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7faf3c600640 1 -- 172.21.15.100:0/3997803185 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7faf340b54d0 con 0x7faf34162ee0 2024-04-23T14:32:39.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7fc543200640 1 -- 172.21.15.100:0/1929916298 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc528004810 msgr2=0x7fc528004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7fc543200640 1 --2- 172.21.15.100:0/1929916298 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc528004810 0x7fc528004c90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc534012400 tx=0x7fc53402db50 comp rx=0 tx=0).stop 2024-04-23T14:32:39.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7f799c800640 1 -- 172.21.15.100:0/3557856387 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79900df2b0 msgr2=0x7f799004faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7f799c800640 1 --2- 172.21.15.100:0/3557856387 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79900df2b0 0x7f799004faa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f798800b9f0 tx=0x7f798801c8e0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7faf39c00640 1 --2- 172.21.15.100:0/3997803185 >> v2:172.21.15.100:3300/0 conn(0x7faf34162b00 0x7faf340b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7faf39200640 1 -- 172.21.15.100:0/3997803185 >> v2:172.21.15.100:3300/0 conn(0x7faf34162b00 msgr2=0x7faf340b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:39.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7fd13f400640 1 -- 172.21.15.100:0/4246692954 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1400b5080 con 0x7fd1400cd370 2024-04-23T14:32:39.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7faf39200640 1 --2- 172.21.15.100:0/3997803185 >> v2:172.21.15.100:3300/0 conn(0x7faf34162b00 0x7faf340b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7faf39200640 1 -- 172.21.15.100:0/3997803185 >> v1:172.21.15.100:6789/0 conn(0x7faf3406f620 legacy=0x7faf341668a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:39.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7faf39200640 1 -- 172.21.15.100:0/3997803185 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf340b57b0 con 0x7faf34162ee0 2024-04-23T14:32:39.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7faf39c00640 1 --2- 172.21.15.100:0/3997803185 >> v2:172.21.15.100:3300/0 conn(0x7faf34162b00 0x7faf340b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:39.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7f799c800640 1 -- 172.21.15.100:0/3557856387 shutdown_connections 2024-04-23T14:32:39.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7f799c800640 1 --2- 172.21.15.100:0/3557856387 >> v2:172.21.15.100:3300/0 conn(0x7f7990007550 0x7f799004ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7f799c800640 1 --2- 172.21.15.100:0/3557856387 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79900df2b0 0x7f799004faa0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7f799c800640 1 -- 172.21.15.100:0/3557856387 >> 172.21.15.100:0/3557856387 conn(0x7f7990009fc0 msgr2=0x7f799000a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:39.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7f799c800640 1 -- 172.21.15.100:0/3557856387 shutdown_connections 2024-04-23T14:32:39.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7f799c800640 1 -- 172.21.15.100:0/3557856387 wait complete. 2024-04-23T14:32:39.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7fdb7fe00640 1 --2- 172.21.15.100:0/2786839592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb80068ad0 0x7fdb8006cfe0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0097c0 tx=0x7fdb6c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa3a74b63ed7436 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7ff2b8000640 1 Processor -- start 2024-04-23T14:32:39.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7ff2b8000640 1 -- start start 2024-04-23T14:32:39.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7ff2b8000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2980039c0 0x7ff2b0157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7fc543200640 1 -- 172.21.15.100:0/1929916298 shutdown_connections 2024-04-23T14:32:39.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7fc543200640 1 --2- 172.21.15.100:0/1929916298 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc528004810 0x7fc528004c90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.539+0000 7fc543200640 1 -- 172.21.15.100:0/1929916298 >> 172.21.15.100:0/1929916298 conn(0x7fc53c009fc0 msgr2=0x7fc53c00a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:39.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7fc543200640 1 -- 172.21.15.100:0/1929916298 shutdown_connections 2024-04-23T14:32:39.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7fc543200640 1 -- 172.21.15.100:0/1929916298 wait complete. 2024-04-23T14:32:39.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7f799c800640 1 Processor -- start 2024-04-23T14:32:39.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7fdb7f400640 1 -- 172.21.15.100:0/2786839592 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c0b0070 con 0x7fdb80068ad0 2024-04-23T14:32:39.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7fdb7f400640 1 -- 172.21.15.100:0/2786839592 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb6c0ab600 con 0x7fdb80068ad0 2024-04-23T14:32:39.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7fdb7f400640 1 -- 172.21.15.100:0/2786839592 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c0ab9d0 con 0x7fdb80068ad0 2024-04-23T14:32:39.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7fdb87a00640 1 -- 172.21.15.100:0/2786839592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb80068ad0 msgr2=0x7fdb8006cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7fdb87a00640 1 --2- 172.21.15.100:0/2786839592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb80068ad0 0x7fdb8006cfe0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0097c0 tx=0x7fdb6c01c4a0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7ff2b8000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2b01f0490 0x7ff2b015fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7fc543200640 1 Processor -- start 2024-04-23T14:32:39.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7f799c800640 1 -- start start 2024-04-23T14:32:39.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7fc543200640 1 -- start start 2024-04-23T14:32:39.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7fdb87a00640 1 -- 172.21.15.100:0/2786839592 shutdown_connections 2024-04-23T14:32:39.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7fdb87a00640 1 --2- 172.21.15.100:0/2786839592 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb8015a880 0x7fdb8006f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7ff2b8000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2b015ce70 0x7ff2b00b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7fdb87a00640 1 --2- 172.21.15.100:0/2786839592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb80068ad0 0x7fdb8006cfe0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7fdb87a00640 1 -- 172.21.15.100:0/2786839592 >> 172.21.15.100:0/2786839592 conn(0x7fdb800d07e0 msgr2=0x7fdb800d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:39.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7ff2b5600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2980039c0 0x7ff2b0157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7ff2b5600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2980039c0 0x7ff2b0157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:55226/0 (socket says 172.21.15.100:55226) 2024-04-23T14:32:39.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7fdb87a00640 1 -- 172.21.15.100:0/2786839592 shutdown_connections 2024-04-23T14:32:39.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7ff2b8000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b00ce850 con 0x7ff2b01f0490 2024-04-23T14:32:39.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7ff2b8000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b00c3ad0 con 0x7ff2b015ce70 2024-04-23T14:32:39.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7ff2b8000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff2b0162e20 con 0x7ff2980039c0 2024-04-23T14:32:39.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7ff2b5600640 1 -- 172.21.15.100:0/2059667939 learned_addr learned my addr 172.21.15.100:0/2059667939 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:39.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7faf39200640 1 --2- 172.21.15.100:0/3997803185 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faf34162ee0 0x7faf340cfb70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faf34068460 tx=0x7faf2801c9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63c71b0c3a66a40f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7fd13f400640 1 --2- 172.21.15.100:0/4246692954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1400cd370 0x7fd1400ad160 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd12c013150 tx=0x7fd12c024970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbc3c935f6a5586 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.543+0000 7fdb87a00640 1 -- 172.21.15.100:0/2786839592 wait complete. 2024-04-23T14:32:39.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7ff2b6000640 1 --2- 172.21.15.100:0/2059667939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2b015ce70 0x7ff2b00b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7ff2b4c00640 1 --2- 172.21.15.100:0/2059667939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2b01f0490 0x7ff2b015fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7f799c800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7990007550 0x7f79900357e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7f799c800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7990033d00 0x7f799004bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7faf33e00640 1 -- 172.21.15.100:0/3997803185 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf280a5830 con 0x7faf34162ee0 2024-04-23T14:32:39.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7faf33e00640 1 -- 172.21.15.100:0/3997803185 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf280a59d0 con 0x7faf34162ee0 2024-04-23T14:32:39.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7faf33e00640 1 -- 172.21.15.100:0/3997803185 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf280a5da0 con 0x7faf34162ee0 2024-04-23T14:32:39.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fd13ea00640 1 -- 172.21.15.100:0/4246692954 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c0110d0 con 0x7fd1400cd370 2024-04-23T14:32:39.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fd13ea00640 1 -- 172.21.15.100:0/4246692954 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd12c011270 con 0x7fd1400cd370 2024-04-23T14:32:39.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fd13ea00640 1 -- 172.21.15.100:0/4246692954 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c011640 con 0x7fd1400cd370 2024-04-23T14:32:39.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7f799c800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79900df2b0 0x7f799004c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7f799c800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f79900ecbf0 con 0x7f79900df2b0 2024-04-23T14:32:39.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7f799c800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f79900eca70 con 0x7f7990033d00 2024-04-23T14:32:39.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7f799c800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f79900ecd70 con 0x7f7990007550 2024-04-23T14:32:39.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7f7999400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7990033d00 0x7f799004bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7f799a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79900df2b0 0x7f799004c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7f7999400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7990033d00 0x7f799004bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:44568/0 (socket says 172.21.15.100:44568) 2024-04-23T14:32:39.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7f7999400640 1 -- 172.21.15.100:0/901348395 learned_addr learned my addr 172.21.15.100:0/901348395 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:39.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7f7999e00640 1 --2- 172.21.15.100:0/901348395 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7990007550 0x7f79900357e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7ff2b5600640 1 -- 172.21.15.100:0/2059667939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2b015ce70 msgr2=0x7ff2b00b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7ff2b5600640 1 --2- 172.21.15.100:0/2059667939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2b015ce70 0x7ff2b00b0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7ff2b5600640 1 -- 172.21.15.100:0/2059667939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2b01f0490 msgr2=0x7ff2b015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7ff2b5600640 1 --2- 172.21.15.100:0/2059667939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2b01f0490 0x7ff2b015fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7ff2b5600640 1 -- 172.21.15.100:0/2059667939 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b01603d0 con 0x7ff2980039c0 2024-04-23T14:32:39.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fc543200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc528004810 0x7fc53c175ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fc543200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc53c1722e0 0x7fc53c04df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fc543200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc53c04e450 0x7fc53c0dafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fc543200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c0066b0 con 0x7fc53c1722e0 2024-04-23T14:32:39.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fc543200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c0063d0 con 0x7fc53c04e450 2024-04-23T14:32:39.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fc543200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc53c006990 con 0x7fc528004810 2024-04-23T14:32:39.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7f7999400640 1 -- 172.21.15.100:0/901348395 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7990007550 msgr2=0x7f79900357e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7f7999400640 1 --2- 172.21.15.100:0/901348395 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7990007550 0x7f79900357e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7f7999400640 1 -- 172.21.15.100:0/901348395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79900df2b0 msgr2=0x7f799004c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7f7999400640 1 --2- 172.21.15.100:0/901348395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79900df2b0 0x7f799004c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7f7999400640 1 -- 172.21.15.100:0/901348395 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79900d8cb0 con 0x7f7990033d00 2024-04-23T14:32:39.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7faf3c600640 1 -- 172.21.15.100:0/3997803185 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faf34162ee0 msgr2=0x7faf340cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7faf3c600640 1 --2- 172.21.15.100:0/3997803185 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faf34162ee0 0x7faf340cfb70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faf34068460 tx=0x7faf2801c9b0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fd146e00640 1 -- 172.21.15.100:0/4246692954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1400cd370 msgr2=0x7fd1400ad160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fd146e00640 1 --2- 172.21.15.100:0/4246692954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1400cd370 0x7fd1400ad160 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd12c013150 tx=0x7fd12c024970 comp rx=0 tx=0).stop 2024-04-23T14:32:39.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7faf3c600640 1 -- 172.21.15.100:0/3997803185 shutdown_connections 2024-04-23T14:32:39.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7faf3c600640 1 --2- 172.21.15.100:0/3997803185 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faf34162ee0 0x7faf340cfb70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7faf3c600640 1 --2- 172.21.15.100:0/3997803185 >> v2:172.21.15.100:3300/0 conn(0x7faf34162b00 0x7faf340b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7faf3c600640 1 -- 172.21.15.100:0/3997803185 >> 172.21.15.100:0/3997803185 conn(0x7faf34070650 msgr2=0x7faf34070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:39.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fd146e00640 1 -- 172.21.15.100:0/4246692954 shutdown_connections 2024-04-23T14:32:39.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fd146e00640 1 --2- 172.21.15.100:0/4246692954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1400cd370 0x7fd1400ad160 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fc541e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc528004810 0x7fc53c175ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fdb87a00640 1 Processor -- start 2024-04-23T14:32:39.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fd146e00640 1 -- 172.21.15.100:0/4246692954 >> 172.21.15.100:0/4246692954 conn(0x7fd140073110 msgr2=0x7fd140073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:39.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7faf3c600640 1 -- 172.21.15.100:0/3997803185 shutdown_connections 2024-04-23T14:32:39.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7f7999400640 1 --2- 172.21.15.100:0/901348395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7990033d00 0x7f799004bba0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f798000c780 tx=0x7f798000cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7faf3c600640 1 -- 172.21.15.100:0/3997803185 wait complete. 2024-04-23T14:32:39.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fc541e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc528004810 0x7fc53c175ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:55242/0 (socket says 172.21.15.100:55242) 2024-04-23T14:32:39.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fc541e00640 1 -- 172.21.15.100:0/1344547731 learned_addr learned my addr 172.21.15.100:0/1344547731 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:39.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fc542800640 1 --2- 172.21.15.100:0/1344547731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc53c04e450 0x7fc53c0dafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fc541400640 1 --2- 172.21.15.100:0/1344547731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc53c1722e0 0x7fc53c04df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7ff2b5600640 1 --2- 172.21.15.100:0/2059667939 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2980039c0 0x7ff2b0157ad0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff29c000c10 tx=0x7ff298005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7f798e000640 1 -- 172.21.15.100:0/901348395 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980017070 con 0x7f7990033d00 2024-04-23T14:32:39.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fdb87a00640 1 -- start start 2024-04-23T14:32:39.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fc542800640 1 -- 172.21.15.100:0/1344547731 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc528004810 msgr2=0x7fc53c175ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fc542800640 1 --2- 172.21.15.100:0/1344547731 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc528004810 0x7fc53c175ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fc542800640 1 -- 172.21.15.100:0/1344547731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc53c1722e0 msgr2=0x7fc53c04df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7f799c800640 1 -- 172.21.15.100:0/901348395 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7990033950 con 0x7f7990033d00 2024-04-23T14:32:39.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7f799c800640 1 -- 172.21.15.100:0/901348395 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79900d5af0 con 0x7f7990033d00 2024-04-23T14:32:39.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fc542800640 1 --2- 172.21.15.100:0/1344547731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc53c1722e0 0x7fc53c04df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fc542800640 1 -- 172.21.15.100:0/1344547731 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53c0dbd20 con 0x7fc53c04e450 2024-04-23T14:32:39.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fc541e00640 1 --2- 172.21.15.100:0/1344547731 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc528004810 0x7fc53c175ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:39.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fc541400640 1 --2- 172.21.15.100:0/1344547731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc53c1722e0 0x7fc53c04df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:39.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7f798e000640 1 -- 172.21.15.100:0/901348395 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7980005270 con 0x7f7990033d00 2024-04-23T14:32:39.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7f798e000640 1 -- 172.21.15.100:0/901348395 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980017070 con 0x7f7990033d00 2024-04-23T14:32:39.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fd146e00640 1 -- 172.21.15.100:0/4246692954 shutdown_connections 2024-04-23T14:32:39.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fd146e00640 1 -- 172.21.15.100:0/4246692954 wait complete. 2024-04-23T14:32:39.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7ff2ad600640 1 -- 172.21.15.100:0/2059667939 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c0cc070 con 0x7ff2980039c0 2024-04-23T14:32:39.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fc542800640 1 --2- 172.21.15.100:0/1344547731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc53c04e450 0x7fc53c0dafb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc54406a250 tx=0x7fc5440747d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7ff2b8000640 1 -- 172.21.15.100:0/2059667939 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b0156d10 con 0x7ff2980039c0 2024-04-23T14:32:39.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7ff2b8000640 1 -- 172.21.15.100:0/2059667939 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b01572d0 con 0x7ff2980039c0 2024-04-23T14:32:39.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7ff2ad600640 1 -- 172.21.15.100:0/2059667939 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff29c0b6b40 con 0x7ff2980039c0 2024-04-23T14:32:39.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7fc53a000640 1 -- 172.21.15.100:0/1344547731 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544099070 con 0x7fc53c04e450 2024-04-23T14:32:39.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7fc53a000640 1 -- 172.21.15.100:0/1344547731 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc544074900 con 0x7fc53c04e450 2024-04-23T14:32:39.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7fc53a000640 1 -- 172.21.15.100:0/1344547731 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544075590 con 0x7fc53c04e450 2024-04-23T14:32:39.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7fc543200640 1 -- 172.21.15.100:0/1344547731 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53c04bba0 con 0x7fc53c04e450 2024-04-23T14:32:39.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fdb87a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb80068ad0 0x7fdb801465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fdb87a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb8015a880 0x7fdb80145060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fdb87a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb800b12b0 0x7fdb8015afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fdb87a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdb800ce050 con 0x7fdb80068ad0 2024-04-23T14:32:39.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fdb87a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdb800cded0 con 0x7fdb800b12b0 2024-04-23T14:32:39.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.547+0000 7fdb87a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fdb800ce1d0 con 0x7fdb8015a880 2024-04-23T14:32:39.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7ff2ad600640 1 -- 172.21.15.100:0/2059667939 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c0cc070 con 0x7ff2980039c0 2024-04-23T14:32:39.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7ff2b8000640 1 -- 172.21.15.100:0/2059667939 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff28c0052a0 con 0x7ff2980039c0 2024-04-23T14:32:39.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7fc543200640 1 -- 172.21.15.100:0/1344547731 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc53c04c160 con 0x7fc53c04e450 2024-04-23T14:32:39.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7faf3c600640 1 Processor -- start 2024-04-23T14:32:39.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7fd146e00640 1 Processor -- start 2024-04-23T14:32:39.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7faf3c600640 1 -- start start 2024-04-23T14:32:39.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7fc543200640 1 -- 172.21.15.100:0/1344547731 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc53c04f190 con 0x7fc53c04e450 2024-04-23T14:32:39.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7f799c800640 1 -- 172.21.15.100:0/901348395 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79900089e0 con 0x7f7990033d00 2024-04-23T14:32:39.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7fdb85000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb80068ad0 0x7fdb801465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7fdb85000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb80068ad0 0x7fdb801465d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52936/0 (socket says 172.21.15.100:52936) 2024-04-23T14:32:39.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7fdb85000640 1 -- 172.21.15.100:0/2760381951 learned_addr learned my addr 172.21.15.100:0/2760381951 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:39.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7fdb85000640 1 -- 172.21.15.100:0/2760381951 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb8015a880 msgr2=0x7fdb80145060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7fdb85000640 1 --2- 172.21.15.100:0/2760381951 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb8015a880 0x7fdb80145060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7fdb85000640 1 -- 172.21.15.100:0/2760381951 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb800b12b0 msgr2=0x7fdb8015afd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-23T14:32:39.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7fdb85000640 1 --2- 172.21.15.100:0/2760381951 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb800b12b0 0x7fdb8015afd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7fdb85000640 1 -- 172.21.15.100:0/2760381951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb80146e50 con 0x7fdb80068ad0 2024-04-23T14:32:39.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7fdb85000640 1 --2- 172.21.15.100:0/2760381951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb80068ad0 0x7fdb801465d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000c420 tx=0x7fdb7000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7fd146e00640 1 -- start start 2024-04-23T14:32:39.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7fdb7cc00640 1 -- 172.21.15.100:0/2760381951 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb700074e0 con 0x7fdb80068ad0 2024-04-23T14:32:39.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7faf3c600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf3406f620 0x7faf3414e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7faf3c600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faf34162b00 0x7faf3414f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7faf3c600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf34162ee0 0x7faf34158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fdb7cc00640 1 -- 172.21.15.100:0/2760381951 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb70007680 con 0x7fdb80068ad0 2024-04-23T14:32:39.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fdb7cc00640 1 -- 172.21.15.100:0/2760381951 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb700049f0 con 0x7fdb80068ad0 2024-04-23T14:32:39.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7faf3c600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faf3416fc20 con 0x7faf3406f620 2024-04-23T14:32:39.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7faf3c600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faf340d04c0 con 0x7faf34162ee0 2024-04-23T14:32:39.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7faf3c600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7faf3416fda0 con 0x7faf34162b00 2024-04-23T14:32:39.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7fd146e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd14006c490 0x7fd1400d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7fd146e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd1400cd370 0x7fd140092c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.551+0000 7fd146e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14016b780 0x7fd1400b1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fdb87a00640 1 -- 172.21.15.100:0/2760381951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb80147130 con 0x7fdb80068ad0 2024-04-23T14:32:39.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fd146e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd140099d70 con 0x7fd14016b780 2024-04-23T14:32:39.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fd146e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd140099bf0 con 0x7fd14006c490 2024-04-23T14:32:39.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fd146e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd1400b5cb0 con 0x7fd1400cd370 2024-04-23T14:32:39.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fd13fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd14006c490 0x7fd1400d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fd13f400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd1400cd370 0x7fd140092c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fd13fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd14006c490 0x7fd1400d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:44614/0 (socket says 172.21.15.100:44614) 2024-04-23T14:32:39.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fd13fe00640 1 -- 172.21.15.100:0/3132940009 learned_addr learned my addr 172.21.15.100:0/3132940009 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:39.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fd144e00640 1 --2- 172.21.15.100:0/3132940009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14016b780 0x7fd1400b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fd13f400640 1 -- 172.21.15.100:0/3132940009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd14006c490 msgr2=0x7fd1400d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fd13f400640 1 --2- 172.21.15.100:0/3132940009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd14006c490 0x7fd1400d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fd13f400640 1 -- 172.21.15.100:0/3132940009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14016b780 msgr2=0x7fd1400b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fd13f400640 1 --2- 172.21.15.100:0/3132940009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14016b780 0x7fd1400b1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fd13f400640 1 -- 172.21.15.100:0/3132940009 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd140095850 con 0x7fd1400cd370 2024-04-23T14:32:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fd13f400640 1 --2- 172.21.15.100:0/3132940009 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd1400cd370 0x7fd140092c60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0b3000 tx=0x7fd12c021730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fc53a000640 1 -- 172.21.15.100:0/1344547731 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc54409c070 con 0x7fc53c04e450 2024-04-23T14:32:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fc53a000640 1 --2- 172.21.15.100:0/1344547731 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc524068480 0x7fc52406a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fc53a000640 1 -- 172.21.15.100:0/1344547731 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc5441019e0 con 0x7fc53c04e450 2024-04-23T14:32:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fdb87a00640 1 -- 172.21.15.100:0/2760381951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb800b1760 con 0x7fdb80068ad0 2024-04-23T14:32:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7faf3a600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf34162ee0 0x7faf34158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7faf3a600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf34162ee0 0x7faf34158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:44606/0 (socket says 172.21.15.100:44606) 2024-04-23T14:32:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7faf3a600640 1 -- 172.21.15.100:0/3499136600 learned_addr learned my addr 172.21.15.100:0/3499136600 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7faf39200640 1 --2- 172.21.15.100:0/3499136600 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faf34162b00 0x7faf3414f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7faf3a600640 1 -- 172.21.15.100:0/3499136600 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faf34162b00 msgr2=0x7faf3414f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7faf3a600640 1 --2- 172.21.15.100:0/3499136600 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faf34162b00 0x7faf3414f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7faf3a600640 1 -- 172.21.15.100:0/3499136600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf3406f620 msgr2=0x7faf3414e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7faf3a600640 1 --2- 172.21.15.100:0/3499136600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf3406f620 0x7faf3414e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7faf3a600640 1 -- 172.21.15.100:0/3499136600 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3414faf0 con 0x7faf34162ee0 2024-04-23T14:32:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7faf3a600640 1 --2- 172.21.15.100:0/3499136600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf34162ee0 0x7faf34158870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faf2000c3c0 tx=0x7faf2000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fc541e00640 1 --2- 172.21.15.100:0/1344547731 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc524068480 0x7fc52406a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7f798e000640 1 -- 172.21.15.100:0/901348395 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f798001a070 con 0x7f7990033d00 2024-04-23T14:32:39.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7ff2ad600640 1 -- 172.21.15.100:0/2059667939 <== mon.2 v2:172.21.15.151:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff29c0049f0 con 0x7ff2980039c0 2024-04-23T14:32:39.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.555+0000 7fc541e00640 1 --2- 172.21.15.100:0/1344547731 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc524068480 0x7fc52406a940 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc53c04d870 tx=0x7fc53000c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.559+0000 7f798e000640 1 --2- 172.21.15.100:0/901348395 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7974068580 0x7f797406aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.559+0000 7f798e000640 1 -- 172.21.15.100:0/901348395 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7980085cc0 con 0x7f7990033d00 2024-04-23T14:32:39.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.559+0000 7ff2ad600640 1 --2- 172.21.15.100:0/2059667939 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff2900687b0 0x7ff29006ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.559+0000 7fdb87a00640 1 -- 172.21.15.100:0/2760381951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb800b0b80 con 0x7fdb80068ad0 2024-04-23T14:32:39.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.559+0000 7ff2ad600640 1 -- 172.21.15.100:0/2059667939 <== mon.2 v2:172.21.15.151:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff29c0047f0 con 0x7ff2980039c0 2024-04-23T14:32:39.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.559+0000 7f7999e00640 1 --2- 172.21.15.100:0/901348395 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7974068580 0x7f797406aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.559+0000 7fdb7cc00640 1 -- 172.21.15.100:0/2760381951 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdb7001a020 con 0x7fdb80068ad0 2024-04-23T14:32:39.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.559+0000 7fd137e00640 1 -- 172.21.15.100:0/3132940009 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c0c5020 con 0x7fd1400cd370 2024-04-23T14:32:39.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.559+0000 7fd137e00640 1 -- 172.21.15.100:0/3132940009 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd12c00fca0 con 0x7fd1400cd370 2024-04-23T14:32:39.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.559+0000 7fd137e00640 1 -- 172.21.15.100:0/3132940009 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c0c06b0 con 0x7fd1400cd370 2024-04-23T14:32:39.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.559+0000 7f7999e00640 1 --2- 172.21.15.100:0/901348395 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7974068580 0x7f797406aa40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f79880ad000 tx=0x7f7988009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.559+0000 7fd146e00640 1 -- 172.21.15.100:0/3132940009 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1400ce2c0 con 0x7fd1400cd370 2024-04-23T14:32:39.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.559+0000 7fd146e00640 1 -- 172.21.15.100:0/3132940009 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1400b0a10 con 0x7fd1400cd370 2024-04-23T14:32:39.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.559+0000 7ff2ad600640 1 -- 172.21.15.100:0/2059667939 <== mon.2 v2:172.21.15.151:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff29c027a10 con 0x7ff2980039c0 2024-04-23T14:32:39.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.559+0000 7ff2b4c00640 1 --2- 172.21.15.100:0/2059667939 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff2900687b0 0x7ff29006ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.559+0000 7fdb7cc00640 1 --2- 172.21.15.100:0/2760381951 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb60068380 0x7fdb6006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.559+0000 7fd146e00640 1 -- 172.21.15.100:0/3132940009 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd140068ad0 con 0x7fd1400cd370 2024-04-23T14:32:39.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.559+0000 7fdb7fe00640 1 --2- 172.21.15.100:0/2760381951 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb60068380 0x7fdb6006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.559+0000 7ff2b4c00640 1 --2- 172.21.15.100:0/2059667939 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff2900687b0 0x7ff29006ac70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8000b70 tx=0x7ff2a800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.559+0000 7fc53a000640 1 -- 172.21.15.100:0/1344547731 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc544091070 con 0x7fc53c04e450 2024-04-23T14:32:39.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.559+0000 7fdb7cc00640 1 -- 172.21.15.100:0/2760381951 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdb70086510 con 0x7fdb80068ad0 2024-04-23T14:32:39.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.563+0000 7faf31600640 1 -- 172.21.15.100:0/3499136600 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20011790 con 0x7faf34162ee0 2024-04-23T14:32:39.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.563+0000 7faf3c600640 1 -- 172.21.15.100:0/3499136600 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf3414fdd0 con 0x7faf34162ee0 2024-04-23T14:32:39.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.563+0000 7fdb7fe00640 1 --2- 172.21.15.100:0/2760381951 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb60068380 0x7fdb6006a840 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0ad000 tx=0x7fdb6c004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.563+0000 7faf3c600640 1 -- 172.21.15.100:0/3499136600 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf34150390 con 0x7faf34162ee0 2024-04-23T14:32:39.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.563+0000 7faf3c600640 1 -- 172.21.15.100:0/3499136600 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf340691e0 con 0x7faf34162ee0 2024-04-23T14:32:39.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.563+0000 7fd137e00640 1 -- 172.21.15.100:0/3132940009 <== mon.2 v2:172.21.15.151:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd12c0c8070 con 0x7fd1400cd370 2024-04-23T14:32:39.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.563+0000 7fd137e00640 1 --2- 172.21.15.100:0/3132940009 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd1200683d0 0x7fd12006a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.563+0000 7fd137e00640 1 -- 172.21.15.100:0/3132940009 <== mon.2 v2:172.21.15.151:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd12c129630 con 0x7fd1400cd370 2024-04-23T14:32:39.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.563+0000 7faf31600640 1 -- 172.21.15.100:0/3499136600 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf20017070 con 0x7faf34162ee0 2024-04-23T14:32:39.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.563+0000 7faf31600640 1 -- 172.21.15.100:0/3499136600 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20011790 con 0x7faf34162ee0 2024-04-23T14:32:39.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.563+0000 7f798e000640 1 -- 172.21.15.100:0/901348395 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f79800558e0 con 0x7f7990033d00 2024-04-23T14:32:39.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.563+0000 7fd13fe00640 1 --2- 172.21.15.100:0/3132940009 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd1200683d0 0x7fd12006a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.567+0000 7fd13fe00640 1 --2- 172.21.15.100:0/3132940009 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd1200683d0 0x7fd12006a890 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd1300099b0 tx=0x7fd130009110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.567+0000 7faf31600640 1 -- 172.21.15.100:0/3499136600 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7faf20011930 con 0x7faf34162ee0 2024-04-23T14:32:39.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.567+0000 7faf31600640 1 --2- 172.21.15.100:0/3499136600 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faf14068580 0x7faf1406aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.567+0000 7faf31600640 1 -- 172.21.15.100:0/3499136600 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7faf20086a40 con 0x7faf34162ee0 2024-04-23T14:32:39.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.567+0000 7fd137e00640 1 -- 172.21.15.100:0/3132940009 <== mon.2 v2:172.21.15.151:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd1400ce2c0 con 0x7fd1400cd370 2024-04-23T14:32:39.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.571+0000 7faf39c00640 1 --2- 172.21.15.100:0/3499136600 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faf14068580 0x7faf1406aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.571+0000 7faf39c00640 1 --2- 172.21.15.100:0/3499136600 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faf14068580 0x7faf1406aa40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7faf2c006f10 tx=0x7faf2c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.571+0000 7faf31600640 1 -- 172.21.15.100:0/3499136600 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faf20020020 con 0x7faf34162ee0 2024-04-23T14:32:39.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.571+0000 7fdb7cc00640 1 -- 172.21.15.100:0/2760381951 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdb70056780 con 0x7fdb80068ad0 2024-04-23T14:32:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.587+0000 7fa077e00640 1 Processor -- start 2024-04-23T14:32:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.587+0000 7fa077e00640 1 -- start start 2024-04-23T14:32:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.591+0000 7fa077e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa07806d450 0x7fa07806e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.591+0000 7fa077e00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0780d1000 con 0x7fa07806f510 2024-04-23T14:32:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.591+0000 7fa077e00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa07806ea00 con 0x7fa07815b240 2024-04-23T14:32:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.591+0000 7fa077e00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fa0780b41e0 con 0x7fa07806d450 2024-04-23T14:32:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.591+0000 7fa076000640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fa07806f510 0x7fa0780cf9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:44956/0 (socket says 172.21.15.100:44956) 2024-04-23T14:32:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.591+0000 7fa076000640 1 -- 172.21.15.100:0/3208842862 learned_addr learned my addr 172.21.15.100:0/3208842862 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.591+0000 7fa077400640 1 --2- 172.21.15.100:0/3208842862 >> v2:172.21.15.100:3300/0 conn(0x7fa07806d450 0x7fa07806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.591+0000 7fa077400640 1 -- 172.21.15.100:0/3208842862 >> v1:172.21.15.151:6789/0 conn(0x7fa07815b240 legacy=0x7fa07806ee00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T14:32:39.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.591+0000 7fa077400640 1 -- 172.21.15.100:0/3208842862 >> v1:172.21.15.100:6789/0 conn(0x7fa07806f510 legacy=0x7fa0780cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.595+0000 7fa077400640 1 -- 172.21.15.100:0/3208842862 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0780cef00 con 0x7fa07806d450 2024-04-23T14:32:39.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.599+0000 7fa077400640 1 --2- 172.21.15.100:0/3208842862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa07806d450 0x7fa07806e340 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa070009d90 tx=0x7fa07001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4245f0bcf26a1b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.599+0000 7fa075600640 1 -- 172.21.15.100:0/3208842862 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0700b2070 con 0x7fa07806d450 2024-04-23T14:32:39.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.599+0000 7fa075600640 1 -- 172.21.15.100:0/3208842862 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0700a59f0 con 0x7fa07806d450 2024-04-23T14:32:39.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.599+0000 7fa077e00640 1 -- 172.21.15.100:0/3208842862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa07806d450 msgr2=0x7fa07806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.599+0000 7fa077e00640 1 --2- 172.21.15.100:0/3208842862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa07806d450 0x7fa07806e340 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa070009d90 tx=0x7fa07001c930 comp rx=0 tx=0).stop 2024-04-23T14:32:39.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.599+0000 7fa077e00640 1 -- 172.21.15.100:0/3208842862 shutdown_connections 2024-04-23T14:32:39.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.599+0000 7fa077e00640 1 --2- 172.21.15.100:0/3208842862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa07806d450 0x7fa07806e340 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.599+0000 7fa077e00640 1 -- 172.21.15.100:0/3208842862 >> 172.21.15.100:0/3208842862 conn(0x7fa0780d11d0 msgr2=0x7fa0780d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:39.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.603+0000 7fa077e00640 1 -- 172.21.15.100:0/3208842862 shutdown_connections 2024-04-23T14:32:39.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.607+0000 7fa077e00640 1 -- 172.21.15.100:0/3208842862 wait complete. 2024-04-23T14:32:39.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.607+0000 7fa077e00640 1 Processor -- start 2024-04-23T14:32:39.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.607+0000 7fa077e00640 1 -- start start 2024-04-23T14:32:39.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.611+0000 7fa077e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa07806d450 0x7fa07813d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.611+0000 7fa077e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa07806f510 0x7fa0781526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.611+0000 7fa077e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa07815b240 0x7fa0780b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.611+0000 7fa077e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa0780d0900 con 0x7fa07806f510 2024-04-23T14:32:39.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.611+0000 7fa077e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa07815b620 con 0x7fa07806d450 2024-04-23T14:32:39.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.611+0000 7fa077e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa0780d0a80 con 0x7fa07815b240 2024-04-23T14:32:39.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.611+0000 7fa076a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa07806d450 0x7fa07813d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.611+0000 7fa076a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa07806d450 0x7fa07813d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:44622/0 (socket says 172.21.15.100:44622) 2024-04-23T14:32:39.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.611+0000 7fa076a00640 1 -- 172.21.15.100:0/3371334509 learned_addr learned my addr 172.21.15.100:0/3371334509 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.611+0000 7fa077400640 1 --2- 172.21.15.100:0/3371334509 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa07815b240 0x7fa0780b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.611+0000 7fa076a00640 1 -- 172.21.15.100:0/3371334509 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa07815b240 msgr2=0x7fa0780b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.611+0000 7fa076a00640 1 --2- 172.21.15.100:0/3371334509 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa07815b240 0x7fa0780b4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.611+0000 7fa076a00640 1 -- 172.21.15.100:0/3371334509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa07806f510 msgr2=0x7fa0781526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.611+0000 7fa076a00640 1 --2- 172.21.15.100:0/3371334509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa07806f510 0x7fa0781526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.611+0000 7fa076a00640 1 -- 172.21.15.100:0/3371334509 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0781531a0 con 0x7fa07806d450 2024-04-23T14:32:39.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.611+0000 7fa076a00640 1 --2- 172.21.15.100:0/3371334509 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa07806d450 0x7fa07813d9d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa06000b3e0 tx=0x7fa06000b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.611+0000 7fa06ea00640 1 -- 172.21.15.100:0/3371334509 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa060015020 con 0x7fa07806d450 2024-04-23T14:32:39.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.611+0000 7fa06ea00640 1 -- 172.21.15.100:0/3371334509 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa060011850 con 0x7fa07806d450 2024-04-23T14:32:39.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.611+0000 7fa06ea00640 1 -- 172.21.15.100:0/3371334509 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa060011c40 con 0x7fa07806d450 2024-04-23T14:32:39.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.615+0000 7fa077e00640 1 -- 172.21.15.100:0/3371334509 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa078153480 con 0x7fa07806d450 2024-04-23T14:32:39.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.615+0000 7fa077e00640 1 -- 172.21.15.100:0/3371334509 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa07814ec50 con 0x7fa07806d450 2024-04-23T14:32:39.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.619+0000 7fa077e00640 1 -- 172.21.15.100:0/3371334509 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa078075c20 con 0x7fa07806d450 2024-04-23T14:32:39.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.619+0000 7fa06ea00640 1 -- 172.21.15.100:0/3371334509 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa060018020 con 0x7fa07806d450 2024-04-23T14:32:39.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.619+0000 7fa06ea00640 1 --2- 172.21.15.100:0/3371334509 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa054068380 0x7fa05406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.619+0000 7fa06ea00640 1 -- 172.21.15.100:0/3371334509 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa060086cc0 con 0x7fa07806d450 2024-04-23T14:32:39.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.623+0000 7fa076000640 1 --2- 172.21.15.100:0/3371334509 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa054068380 0x7fa05406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.623+0000 7fa076000640 1 --2- 172.21.15.100:0/3371334509 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa054068380 0x7fa05406a840 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa0780b5a20 tx=0x7fa068007a50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.627+0000 7fa06ea00640 1 -- 172.21.15.100:0/3371334509 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa06001f020 con 0x7fa07806d450 2024-04-23T14:32:39.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.815+0000 7f181a000640 1 -- 172.21.15.100:0/170613723 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f17f8005590 con 0x7f18240cde60 2024-04-23T14:32:39.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.823+0000 7f1820c00640 1 -- 172.21.15.100:0/170613723 <== mon.2 v2:172.21.15.151:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f181c057450 con 0x7f18240cde60 2024-04-23T14:32:39.822 INFO:teuthology.orchestra.run.smithi100.stdout:55834574861 2024-04-23T14:32:39.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.823+0000 7f181a000640 1 -- 172.21.15.100:0/170613723 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1804068380 msgr2=0x7f180406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.823+0000 7f181a000640 1 --2- 172.21.15.100:0/170613723 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1804068380 0x7f180406a840 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f18140af000 tx=0x7f18140a8000 comp rx=0 tx=0).stop 2024-04-23T14:32:39.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.823+0000 7f181a000640 1 -- 172.21.15.100:0/170613723 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18240cde60 msgr2=0x7f1824152980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.823+0000 7f181a000640 1 --2- 172.21.15.100:0/170613723 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18240cde60 0x7f1824152980 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f181c009f00 tx=0x7f181c00c490 comp rx=0 tx=0).stop 2024-04-23T14:32:39.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.823+0000 7f181a000640 1 -- 172.21.15.100:0/170613723 shutdown_connections 2024-04-23T14:32:39.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.827+0000 7f181a000640 1 --2- 172.21.15.100:0/170613723 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1804068380 0x7f180406a840 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.827+0000 7f181a000640 1 --2- 172.21.15.100:0/170613723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f182415a590 0x7f182414e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.827+0000 7f181a000640 1 --2- 172.21.15.100:0/170613723 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18240cde60 0x7f1824152980 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.827+0000 7f181a000640 1 --2- 172.21.15.100:0/170613723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f182406bff0 0x7f182415bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.827+0000 7f181a000640 1 -- 172.21.15.100:0/170613723 >> 172.21.15.100:0/170613723 conn(0x7f18240d0060 msgr2=0x7f1824067810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:39.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.827+0000 7f181a000640 1 -- 172.21.15.100:0/170613723 shutdown_connections 2024-04-23T14:32:39.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.827+0000 7f181a000640 1 -- 172.21.15.100:0/170613723 wait complete. 2024-04-23T14:32:39.858 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574861 got 55834574861 for osd.0 2024-04-23T14:32:39.858 DEBUG:teuthology.parallel:result is None 2024-04-23T14:32:39.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.883+0000 7fdb87a00640 1 -- 172.21.15.100:0/2760381951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fdb800691e0 con 0x7fdb80068ad0 2024-04-23T14:32:39.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.883+0000 7fdb7cc00640 1 -- 172.21.15.100:0/2760381951 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdb70056080 con 0x7fdb80068ad0 2024-04-23T14:32:39.884 INFO:teuthology.orchestra.run.smithi100.stdout:55834574861 2024-04-23T14:32:39.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.883+0000 7fdb87a00640 1 -- 172.21.15.100:0/2760381951 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb60068380 msgr2=0x7fdb6006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.883+0000 7fdb87a00640 1 --2- 172.21.15.100:0/2760381951 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb60068380 0x7fdb6006a840 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0ad000 tx=0x7fdb6c004740 comp rx=0 tx=0).stop 2024-04-23T14:32:39.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.883+0000 7fdb87a00640 1 -- 172.21.15.100:0/2760381951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb80068ad0 msgr2=0x7fdb801465d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.883+0000 7fdb87a00640 1 --2- 172.21.15.100:0/2760381951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb80068ad0 0x7fdb801465d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000c420 tx=0x7fdb7000c8e0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.887+0000 7fdb87a00640 1 -- 172.21.15.100:0/2760381951 shutdown_connections 2024-04-23T14:32:39.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.887+0000 7fdb87a00640 1 --2- 172.21.15.100:0/2760381951 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb60068380 0x7fdb6006a840 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.887+0000 7fdb87a00640 1 --2- 172.21.15.100:0/2760381951 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb800b12b0 0x7fdb8015afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.887+0000 7fdb87a00640 1 --2- 172.21.15.100:0/2760381951 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb8015a880 0x7fdb80145060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.887+0000 7fdb87a00640 1 --2- 172.21.15.100:0/2760381951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb80068ad0 0x7fdb801465d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.887+0000 7fdb87a00640 1 -- 172.21.15.100:0/2760381951 >> 172.21.15.100:0/2760381951 conn(0x7fdb800d07e0 msgr2=0x7fdb8014fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:39.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.887+0000 7fdb87a00640 1 -- 172.21.15.100:0/2760381951 shutdown_connections 2024-04-23T14:32:39.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.887+0000 7fdb87a00640 1 -- 172.21.15.100:0/2760381951 wait complete. 2024-04-23T14:32:39.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.895+0000 7ff2b8000640 1 -- 172.21.15.100:0/2059667939 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff28c005860 con 0x7ff2980039c0 2024-04-23T14:32:39.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.895+0000 7ff2ad600640 1 -- 172.21.15.100:0/2059667939 <== mon.2 v2:172.21.15.151:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff29c100c00 con 0x7ff2980039c0 2024-04-23T14:32:39.897 INFO:teuthology.orchestra.run.smithi100.stdout:55834574860 2024-04-23T14:32:39.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.899+0000 7fc543200640 1 -- 172.21.15.100:0/1344547731 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc53c053d50 con 0x7fc53c04e450 2024-04-23T14:32:39.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.899+0000 7ff2b8000640 1 -- 172.21.15.100:0/2059667939 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff2900687b0 msgr2=0x7ff29006ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.899+0000 7ff2b8000640 1 --2- 172.21.15.100:0/2059667939 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff2900687b0 0x7ff29006ac70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8000b70 tx=0x7ff2a800b040 comp rx=0 tx=0).stop 2024-04-23T14:32:39.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.899+0000 7ff2b8000640 1 -- 172.21.15.100:0/2059667939 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2980039c0 msgr2=0x7ff2b0157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.899+0000 7ff2b8000640 1 --2- 172.21.15.100:0/2059667939 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2980039c0 0x7ff2b0157ad0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff29c000c10 tx=0x7ff298005840 comp rx=0 tx=0).stop 2024-04-23T14:32:39.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.899+0000 7fc53a000640 1 -- 172.21.15.100:0/1344547731 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc5440d15f0 con 0x7fc53c04e450 2024-04-23T14:32:39.901 INFO:teuthology.orchestra.run.smithi100.stdout:55834574859 2024-04-23T14:32:39.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.903+0000 7ff2b8000640 1 -- 172.21.15.100:0/2059667939 shutdown_connections 2024-04-23T14:32:39.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.903+0000 7ff2b8000640 1 --2- 172.21.15.100:0/2059667939 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff2900687b0 0x7ff29006ac70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.903+0000 7ff2b8000640 1 --2- 172.21.15.100:0/2059667939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2b015ce70 0x7ff2b00b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.903+0000 7ff2b8000640 1 --2- 172.21.15.100:0/2059667939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2b01f0490 0x7ff2b015fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.903+0000 7ff2b8000640 1 --2- 172.21.15.100:0/2059667939 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2980039c0 0x7ff2b0157ad0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.903+0000 7ff2b8000640 1 -- 172.21.15.100:0/2059667939 >> 172.21.15.100:0/2059667939 conn(0x7ff2b00d0170 msgr2=0x7ff2b00cb760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:39.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.903+0000 7ff2b8000640 1 -- 172.21.15.100:0/2059667939 shutdown_connections 2024-04-23T14:32:39.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.903+0000 7ff2b8000640 1 -- 172.21.15.100:0/2059667939 wait complete. 2024-04-23T14:32:39.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.903+0000 7fd146e00640 1 -- 172.21.15.100:0/3132940009 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd140073c30 con 0x7fd1400cd370 2024-04-23T14:32:39.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.903+0000 7fc543200640 1 -- 172.21.15.100:0/1344547731 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc524068480 msgr2=0x7fc52406a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.903+0000 7fc543200640 1 --2- 172.21.15.100:0/1344547731 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc524068480 0x7fc52406a940 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc53c04d870 tx=0x7fc53000c040 comp rx=0 tx=0).stop 2024-04-23T14:32:39.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.903+0000 7fc543200640 1 -- 172.21.15.100:0/1344547731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc53c04e450 msgr2=0x7fc53c0dafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.903+0000 7fc543200640 1 --2- 172.21.15.100:0/1344547731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc53c04e450 0x7fc53c0dafb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc54406a250 tx=0x7fc5440747d0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.903+0000 7fc543200640 1 -- 172.21.15.100:0/1344547731 shutdown_connections 2024-04-23T14:32:39.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.903+0000 7fc543200640 1 --2- 172.21.15.100:0/1344547731 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc524068480 0x7fc52406a940 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.903+0000 7fc543200640 1 --2- 172.21.15.100:0/1344547731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc53c04e450 0x7fc53c0dafb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.903+0000 7fc543200640 1 --2- 172.21.15.100:0/1344547731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc53c1722e0 0x7fc53c04df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.903+0000 7fc543200640 1 --2- 172.21.15.100:0/1344547731 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc528004810 0x7fc53c175ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.903+0000 7fc543200640 1 -- 172.21.15.100:0/1344547731 >> 172.21.15.100:0/1344547731 conn(0x7fc53c009fc0 msgr2=0x7fc53c03a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:39.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.903+0000 7fc543200640 1 -- 172.21.15.100:0/1344547731 shutdown_connections 2024-04-23T14:32:39.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.903+0000 7fc543200640 1 -- 172.21.15.100:0/1344547731 wait complete. 2024-04-23T14:32:39.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.903+0000 7fd137e00640 1 -- 172.21.15.100:0/3132940009 <== mon.2 v2:172.21.15.151:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd12c01e840 con 0x7fd1400cd370 2024-04-23T14:32:39.906 INFO:teuthology.orchestra.run.smithi100.stdout:55834574859 2024-04-23T14:32:39.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.907+0000 7fd146e00640 1 -- 172.21.15.100:0/3132940009 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd1200683d0 msgr2=0x7fd12006a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.907+0000 7fd146e00640 1 --2- 172.21.15.100:0/3132940009 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd1200683d0 0x7fd12006a890 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd1300099b0 tx=0x7fd130009110 comp rx=0 tx=0).stop 2024-04-23T14:32:39.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.907+0000 7fd146e00640 1 -- 172.21.15.100:0/3132940009 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd1400cd370 msgr2=0x7fd140092c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.907+0000 7fd146e00640 1 --2- 172.21.15.100:0/3132940009 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd1400cd370 0x7fd140092c60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0b3000 tx=0x7fd12c021730 comp rx=0 tx=0).stop 2024-04-23T14:32:39.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.907+0000 7fd146e00640 1 -- 172.21.15.100:0/3132940009 shutdown_connections 2024-04-23T14:32:39.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.907+0000 7fd146e00640 1 --2- 172.21.15.100:0/3132940009 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd1200683d0 0x7fd12006a890 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.907+0000 7fd146e00640 1 --2- 172.21.15.100:0/3132940009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14016b780 0x7fd1400b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.907+0000 7fd146e00640 1 --2- 172.21.15.100:0/3132940009 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd1400cd370 0x7fd140092c60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.907+0000 7fd146e00640 1 --2- 172.21.15.100:0/3132940009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd14006c490 0x7fd1400d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.907+0000 7fd146e00640 1 -- 172.21.15.100:0/3132940009 >> 172.21.15.100:0/3132940009 conn(0x7fd140073110 msgr2=0x7fd1400734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:39.911 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574861 got 55834574861 for osd.7 2024-04-23T14:32:39.911 DEBUG:teuthology.parallel:result is None 2024-04-23T14:32:39.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.911+0000 7fd146e00640 1 -- 172.21.15.100:0/3132940009 shutdown_connections 2024-04-23T14:32:39.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.911+0000 7fd146e00640 1 -- 172.21.15.100:0/3132940009 wait complete. 2024-04-23T14:32:39.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.915+0000 7faf3c600640 1 -- 172.21.15.100:0/3499136600 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7faf340b1540 con 0x7faf34162ee0 2024-04-23T14:32:39.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.915+0000 7faf31600640 1 -- 172.21.15.100:0/3499136600 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faf20056d30 con 0x7faf34162ee0 2024-04-23T14:32:39.918 INFO:teuthology.orchestra.run.smithi100.stdout:55834574858 2024-04-23T14:32:39.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.919+0000 7f799c800640 1 -- 172.21.15.100:0/901348395 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f799003a4e0 con 0x7f7990033d00 2024-04-23T14:32:39.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.919+0000 7f798e000640 1 -- 172.21.15.100:0/901348395 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7980059be0 con 0x7f7990033d00 2024-04-23T14:32:39.921 INFO:teuthology.orchestra.run.smithi100.stdout:55834574861 2024-04-23T14:32:39.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.919+0000 7faf3c600640 1 -- 172.21.15.100:0/3499136600 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faf14068580 msgr2=0x7faf1406aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.919+0000 7faf3c600640 1 --2- 172.21.15.100:0/3499136600 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faf14068580 0x7faf1406aa40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7faf2c006f10 tx=0x7faf2c008040 comp rx=0 tx=0).stop 2024-04-23T14:32:39.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.919+0000 7faf3c600640 1 -- 172.21.15.100:0/3499136600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf34162ee0 msgr2=0x7faf34158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.919+0000 7faf3c600640 1 --2- 172.21.15.100:0/3499136600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf34162ee0 0x7faf34158870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faf2000c3c0 tx=0x7faf2000c880 comp rx=0 tx=0).stop 2024-04-23T14:32:39.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.919+0000 7faf3c600640 1 -- 172.21.15.100:0/3499136600 shutdown_connections 2024-04-23T14:32:39.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.919+0000 7faf3c600640 1 --2- 172.21.15.100:0/3499136600 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faf14068580 0x7faf1406aa40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.919+0000 7faf3c600640 1 --2- 172.21.15.100:0/3499136600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf34162ee0 0x7faf34158870 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.919+0000 7faf3c600640 1 --2- 172.21.15.100:0/3499136600 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faf34162b00 0x7faf3414f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.919+0000 7faf3c600640 1 --2- 172.21.15.100:0/3499136600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf3406f620 0x7faf3414e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.919+0000 7faf3c600640 1 -- 172.21.15.100:0/3499136600 >> 172.21.15.100:0/3499136600 conn(0x7faf34070650 msgr2=0x7faf340ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:39.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.923+0000 7faf3c600640 1 -- 172.21.15.100:0/3499136600 shutdown_connections 2024-04-23T14:32:39.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.923+0000 7faf3c600640 1 -- 172.21.15.100:0/3499136600 wait complete. 2024-04-23T14:32:39.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.923+0000 7f799c800640 1 -- 172.21.15.100:0/901348395 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7974068580 msgr2=0x7f797406aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.923+0000 7f799c800640 1 --2- 172.21.15.100:0/901348395 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7974068580 0x7f797406aa40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f79880ad000 tx=0x7f7988009040 comp rx=0 tx=0).stop 2024-04-23T14:32:39.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.923+0000 7f799c800640 1 -- 172.21.15.100:0/901348395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7990033d00 msgr2=0x7f799004bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.923+0000 7f799c800640 1 --2- 172.21.15.100:0/901348395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7990033d00 0x7f799004bba0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f798000c780 tx=0x7f798000cc40 comp rx=0 tx=0).stop 2024-04-23T14:32:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.927+0000 7f799c800640 1 -- 172.21.15.100:0/901348395 shutdown_connections 2024-04-23T14:32:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.927+0000 7f799c800640 1 --2- 172.21.15.100:0/901348395 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7974068580 0x7f797406aa40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.927+0000 7f799c800640 1 --2- 172.21.15.100:0/901348395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79900df2b0 0x7f799004c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.927+0000 7f799c800640 1 --2- 172.21.15.100:0/901348395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7990033d00 0x7f799004bba0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.927+0000 7f799c800640 1 --2- 172.21.15.100:0/901348395 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7990007550 0x7f79900357e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.927+0000 7f799c800640 1 -- 172.21.15.100:0/901348395 >> 172.21.15.100:0/901348395 conn(0x7f7990009fc0 msgr2=0x7f79900386f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.927+0000 7f799c800640 1 -- 172.21.15.100:0/901348395 shutdown_connections 2024-04-23T14:32:39.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.927+0000 7f799c800640 1 -- 172.21.15.100:0/901348395 wait complete. 2024-04-23T14:32:39.940 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.1 2024-04-23T14:32:39.940 DEBUG:teuthology.parallel:result is None 2024-04-23T14:32:39.942 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.4 2024-04-23T14:32:39.942 DEBUG:teuthology.parallel:result is None 2024-04-23T14:32:39.944 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.5 2024-04-23T14:32:39.944 DEBUG:teuthology.parallel:result is None 2024-04-23T14:32:39.953 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574861 got 55834574861 for osd.2 2024-04-23T14:32:39.953 DEBUG:teuthology.parallel:result is None 2024-04-23T14:32:39.953 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.6 2024-04-23T14:32:39.954 DEBUG:teuthology.parallel:result is None 2024-04-23T14:32:39.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.955+0000 7fa077e00640 1 -- 172.21.15.100:0/3371334509 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa07806daa0 con 0x7fa07806d450 2024-04-23T14:32:39.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.955+0000 7fa06ea00640 1 -- 172.21.15.100:0/3371334509 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa06006c090 con 0x7fa07806d450 2024-04-23T14:32:39.955 INFO:teuthology.orchestra.run.smithi100.stdout:55834574859 2024-04-23T14:32:39.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.955+0000 7fa077e00640 1 -- 172.21.15.100:0/3371334509 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa054068380 msgr2=0x7fa05406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.955+0000 7fa077e00640 1 --2- 172.21.15.100:0/3371334509 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa054068380 0x7fa05406a840 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa0780b5a20 tx=0x7fa068007a50 comp rx=0 tx=0).stop 2024-04-23T14:32:39.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.955+0000 7fa077e00640 1 -- 172.21.15.100:0/3371334509 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa07806d450 msgr2=0x7fa07813d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.955+0000 7fa077e00640 1 --2- 172.21.15.100:0/3371334509 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa07806d450 0x7fa07813d9d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa06000b3e0 tx=0x7fa06000b8a0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.955+0000 7fa077e00640 1 -- 172.21.15.100:0/3371334509 shutdown_connections 2024-04-23T14:32:39.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.955+0000 7fa077e00640 1 --2- 172.21.15.100:0/3371334509 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa054068380 0x7fa05406a840 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.955+0000 7fa077e00640 1 --2- 172.21.15.100:0/3371334509 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa07815b240 0x7fa0780b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.955+0000 7fa077e00640 1 --2- 172.21.15.100:0/3371334509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa07806f510 0x7fa0781526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.955+0000 7fa077e00640 1 --2- 172.21.15.100:0/3371334509 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa07806d450 0x7fa07813d9d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.955+0000 7fa077e00640 1 -- 172.21.15.100:0/3371334509 >> 172.21.15.100:0/3371334509 conn(0x7fa0780d11d0 msgr2=0x7fa0780b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:39.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.955+0000 7fa077e00640 1 -- 172.21.15.100:0/3371334509 shutdown_connections 2024-04-23T14:32:39.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:39.955+0000 7fa077e00640 1 -- 172.21.15.100:0/3371334509 wait complete. 2024-04-23T14:32:39.978 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2024-04-23T14:32:39.978 DEBUG:teuthology.parallel:result is None 2024-04-23T14:32:39.978 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-23T14:32:39.978 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:32:40.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.159+0000 7f3556a00640 1 Processor -- start 2024-04-23T14:32:40.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.159+0000 7f3556a00640 1 -- start start 2024-04-23T14:32:40.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.159+0000 7f3556a00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f355015c7e0 0x7f35500d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.159+0000 7f3556a00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35500d04a0 con 0x7f355015d600 2024-04-23T14:32:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.159+0000 7f3556a00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35500d0780 con 0x7f35500cca50 2024-04-23T14:32:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.159+0000 7f3556a00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f355015db00 con 0x7f355015c7e0 2024-04-23T14:32:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.159+0000 7f3554a00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f355015d600 0x7f35500d17a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:44972/0 (socket says 172.21.15.100:44972) 2024-04-23T14:32:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.159+0000 7f3554a00640 1 -- 172.21.15.100:0/1910518454 learned_addr learned my addr 172.21.15.100:0/1910518454 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.159+0000 7f354f400640 1 --2- 172.21.15.100:0/1910518454 >> v2:172.21.15.151:3300/0 conn(0x7f355015c7e0 0x7f35500d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.159+0000 7f354ea00640 1 -- 172.21.15.100:0/1910518454 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 607867680 0 0) 0x7f35500d04a0 con 0x7f355015d600 2024-04-23T14:32:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.159+0000 7f354ea00640 1 -- 172.21.15.100:0/1910518454 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3534003530 con 0x7f355015d600 2024-04-23T14:32:40.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.159+0000 7f354f400640 1 -- 172.21.15.100:0/1910518454 >> v1:172.21.15.151:6789/0 conn(0x7f35500cca50 legacy=0x7f35500cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.159+0000 7f354f400640 1 -- 172.21.15.100:0/1910518454 >> v1:172.21.15.100:6789/0 conn(0x7f355015d600 legacy=0x7f35500d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.159+0000 7f354f400640 1 -- 172.21.15.100:0/1910518454 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35500d0a60 con 0x7f355015c7e0 2024-04-23T14:32:40.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.163+0000 7f354f400640 1 --2- 172.21.15.100:0/1910518454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f355015c7e0 0x7f35500d1260 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f353c0097c0 tx=0x7f353c01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80061dc12300aba7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:40.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.163+0000 7f354ea00640 1 -- 172.21.15.100:0/1910518454 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c0ab600 con 0x7f355015c7e0 2024-04-23T14:32:40.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.163+0000 7f354ea00640 1 -- 172.21.15.100:0/1910518454 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f353c0b0070 con 0x7f355015c7e0 2024-04-23T14:32:40.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.163+0000 7f354ea00640 1 -- 172.21.15.100:0/1910518454 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c0ab9c0 con 0x7f355015c7e0 2024-04-23T14:32:40.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.163+0000 7f3556a00640 1 -- 172.21.15.100:0/1910518454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f355015c7e0 msgr2=0x7f35500d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.163+0000 7f3556a00640 1 --2- 172.21.15.100:0/1910518454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f355015c7e0 0x7f35500d1260 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f353c0097c0 tx=0x7f353c01c3e0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.163+0000 7f3556a00640 1 -- 172.21.15.100:0/1910518454 shutdown_connections 2024-04-23T14:32:40.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.163+0000 7f3556a00640 1 --2- 172.21.15.100:0/1910518454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f355015c7e0 0x7f35500d1260 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.163+0000 7f3556a00640 1 -- 172.21.15.100:0/1910518454 >> 172.21.15.100:0/1910518454 conn(0x7f35500cee80 msgr2=0x7f35500cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:40.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.163+0000 7f3556a00640 1 -- 172.21.15.100:0/1910518454 shutdown_connections 2024-04-23T14:32:40.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.163+0000 7f3556a00640 1 -- 172.21.15.100:0/1910518454 wait complete. 2024-04-23T14:32:40.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f3556a00640 1 Processor -- start 2024-04-23T14:32:40.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f3556a00640 1 -- start start 2024-04-23T14:32:40.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f3556a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35500cca50 0x7f35500b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:40.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f3556a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f355015c7e0 0x7f355015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:40.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f3556a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f355015d600 0x7f3550146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:40.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f3556a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f355016ac60 con 0x7f35500cca50 2024-04-23T14:32:40.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f3556a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f355016aae0 con 0x7f355015d600 2024-04-23T14:32:40.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f3556a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f355016ade0 con 0x7f355015c7e0 2024-04-23T14:32:40.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f354fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35500cca50 0x7f35500b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:40.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f354fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35500cca50 0x7f35500b58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52972/0 (socket says 172.21.15.100:52972) 2024-04-23T14:32:40.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f354fe00640 1 -- 172.21.15.100:0/3661437108 learned_addr learned my addr 172.21.15.100:0/3661437108 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:40.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f3554a00640 1 --2- 172.21.15.100:0/3661437108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f355015d600 0x7f3550146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:40.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f354f400640 1 --2- 172.21.15.100:0/3661437108 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f355015c7e0 0x7f355015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:40.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f354fe00640 1 -- 172.21.15.100:0/3661437108 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f355015c7e0 msgr2=0x7f355015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f354fe00640 1 --2- 172.21.15.100:0/3661437108 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f355015c7e0 0x7f355015a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f354fe00640 1 -- 172.21.15.100:0/3661437108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f355015d600 msgr2=0x7f3550146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f354fe00640 1 --2- 172.21.15.100:0/3661437108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f355015d600 0x7f3550146860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f354fe00640 1 -- 172.21.15.100:0/3661437108 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35500c5840 con 0x7f35500cca50 2024-04-23T14:32:40.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f354fe00640 1 --2- 172.21.15.100:0/3661437108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35500cca50 0x7f35500b58e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f354000c7c0 tx=0x7f354000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:40.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f3547e00640 1 -- 172.21.15.100:0/3661437108 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3540016020 con 0x7f35500cca50 2024-04-23T14:32:40.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f3547e00640 1 -- 172.21.15.100:0/3661437108 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3540006d60 con 0x7f35500cca50 2024-04-23T14:32:40.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f3556a00640 1 -- 172.21.15.100:0/3661437108 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35500c1800 con 0x7f35500cca50 2024-04-23T14:32:40.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f3547e00640 1 -- 172.21.15.100:0/3661437108 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3540003bc0 con 0x7f35500cca50 2024-04-23T14:32:40.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.167+0000 7f3556a00640 1 -- 172.21.15.100:0/3661437108 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f355015bb60 con 0x7f35500cca50 2024-04-23T14:32:40.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.171+0000 7f3556a00640 1 -- 172.21.15.100:0/3661437108 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3550068ad0 con 0x7f35500cca50 2024-04-23T14:32:40.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.171+0000 7f3547e00640 1 -- 172.21.15.100:0/3661437108 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f354000a850 con 0x7f35500cca50 2024-04-23T14:32:40.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.171+0000 7f3547e00640 1 --2- 172.21.15.100:0/3661437108 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3530068380 0x7f353006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:40.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.171+0000 7f3547e00640 1 -- 172.21.15.100:0/3661437108 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3540086a50 con 0x7f35500cca50 2024-04-23T14:32:40.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.171+0000 7f354f400640 1 --2- 172.21.15.100:0/3661437108 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3530068380 0x7f353006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:40.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.171+0000 7f354f400640 1 --2- 172.21.15.100:0/3661437108 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3530068380 0x7f353006a840 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f353c0af000 tx=0x7f353c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:40.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.175+0000 7f3547e00640 1 -- 172.21.15.100:0/3661437108 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3540020020 con 0x7f35500cca50 2024-04-23T14:32:40.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.327+0000 7f3556a00640 1 -- 172.21.15.100:0/3661437108 --> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f355006fab0 con 0x7f3530068380 2024-04-23T14:32:40.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.335+0000 7f3547e00640 1 -- 172.21.15.100:0/3661437108 <== mgr.4101 v2:172.21.15.100:6832/2859853957 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332537 (secure 0 0 0) 0x7f355006fab0 con 0x7f3530068380 2024-04-23T14:32:40.340 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:40.342 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-04-23T14:32:40.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.339+0000 7f3556a00640 1 -- 172.21.15.100:0/3661437108 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3530068380 msgr2=0x7f353006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.339+0000 7f3556a00640 1 --2- 172.21.15.100:0/3661437108 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3530068380 0x7f353006a840 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f353c0af000 tx=0x7f353c0a7000 comp rx=0 tx=0).stop 2024-04-23T14:32:40.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.339+0000 7f3556a00640 1 -- 172.21.15.100:0/3661437108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35500cca50 msgr2=0x7f35500b58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.339+0000 7f3556a00640 1 --2- 172.21.15.100:0/3661437108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35500cca50 0x7f35500b58e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f354000c7c0 tx=0x7f354000cc80 comp rx=0 tx=0).stop 2024-04-23T14:32:40.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.339+0000 7f3556a00640 1 -- 172.21.15.100:0/3661437108 shutdown_connections 2024-04-23T14:32:40.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.339+0000 7f3556a00640 1 --2- 172.21.15.100:0/3661437108 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3530068380 0x7f353006a840 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.339+0000 7f3556a00640 1 --2- 172.21.15.100:0/3661437108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f355015d600 0x7f3550146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.339+0000 7f3556a00640 1 --2- 172.21.15.100:0/3661437108 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f355015c7e0 0x7f355015a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.339+0000 7f3556a00640 1 --2- 172.21.15.100:0/3661437108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35500cca50 0x7f35500b58e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.339+0000 7f3556a00640 1 -- 172.21.15.100:0/3661437108 >> 172.21.15.100:0/3661437108 conn(0x7f35500cee80 msgr2=0x7f35500b6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:40.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.339+0000 7f3556a00640 1 -- 172.21.15.100:0/3661437108 shutdown_connections 2024-04-23T14:32:40.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.339+0000 7f3556a00640 1 -- 172.21.15.100:0/3661437108 wait complete. 2024-04-23T14:32:40.359 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-23T14:32:38.741603+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219376,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755344,"statfs":{"total":773094113280,"available":772869472256,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002866"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640233+0000","last_change":"2024-04-23T14:32:08.611525+0000","last_active":"2024-04-23T14:32:14.640233+0000","last_peered":"2024-04-23T14:32:14.640233+0000","last_clean":"2024-04-23T14:32:14.640233+0000","last_became_active":"2024-04-23T14:32:08.611163+0000","last_became_peered":"2024-04-23T14:32:08.611163+0000","last_unstale":"2024-04-23T14:32:14.640233+0000","last_undegraded":"2024-04-23T14:32:14.640233+0000","last_fullsized":"2024-04-23T14:32:14.640233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:08:50.925677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181653+0000","last_change":"2024-04-23T14:32:09.625243+0000","last_active":"2024-04-23T14:32:15.181653+0000","last_peered":"2024-04-23T14:32:15.181653+0000","last_clean":"2024-04-23T14:32:15.181653+0000","last_became_active":"2024-04-23T14:32:09.624765+0000","last_became_peered":"2024-04-23T14:32:09.624765+0000","last_unstale":"2024-04-23T14:32:15.181653+0000","last_undegraded":"2024-04-23T14:32:15.181653+0000","last_fullsized":"2024-04-23T14:32:15.181653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:50.876901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181363+0000","last_change":"2024-04-23T14:32:13.779583+0000","last_active":"2024-04-23T14:32:15.181363+0000","last_peered":"2024-04-23T14:32:15.181363+0000","last_clean":"2024-04-23T14:32:15.181363+0000","last_became_active":"2024-04-23T14:32:13.779224+0000","last_became_peered":"2024-04-23T14:32:13.779224+0000","last_unstale":"2024-04-23T14:32:15.181363+0000","last_undegraded":"2024-04-23T14:32:15.181363+0000","last_fullsized":"2024-04-23T14:32:15.181363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:55.041747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181862+0000","last_change":"2024-04-23T14:32:08.587781+0000","last_active":"2024-04-23T14:32:15.181862+0000","last_peered":"2024-04-23T14:32:15.181862+0000","last_clean":"2024-04-23T14:32:15.181862+0000","last_became_active":"2024-04-23T14:32:08.587435+0000","last_became_peered":"2024-04-23T14:32:08.587435+0000","last_unstale":"2024-04-23T14:32:15.181862+0000","last_undegraded":"2024-04-23T14:32:15.181862+0000","last_fullsized":"2024-04-23T14:32:15.181862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:51.650822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640102+0000","last_change":"2024-04-23T14:32:09.621038+0000","last_active":"2024-04-23T14:32:14.640102+0000","last_peered":"2024-04-23T14:32:14.640102+0000","last_clean":"2024-04-23T14:32:14.640102+0000","last_became_active":"2024-04-23T14:32:09.620776+0000","last_became_peered":"2024-04-23T14:32:09.620776+0000","last_unstale":"2024-04-23T14:32:14.640102+0000","last_undegraded":"2024-04-23T14:32:14.640102+0000","last_fullsized":"2024-04-23T14:32:14.640102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:12.933947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151642+0000","last_change":"2024-04-23T14:32:13.671138+0000","last_active":"2024-04-23T14:32:15.151642+0000","last_peered":"2024-04-23T14:32:15.151642+0000","last_clean":"2024-04-23T14:32:15.151642+0000","last_became_active":"2024-04-23T14:32:13.670514+0000","last_became_peered":"2024-04-23T14:32:13.670514+0000","last_unstale":"2024-04-23T14:32:15.151642+0000","last_undegraded":"2024-04-23T14:32:15.151642+0000","last_fullsized":"2024-04-23T14:32:15.151642+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:26.271219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179964+0000","last_change":"2024-04-23T14:32:08.616316+0000","last_active":"2024-04-23T14:32:15.179964+0000","last_peered":"2024-04-23T14:32:15.179964+0000","last_clean":"2024-04-23T14:32:15.179964+0000","last_became_active":"2024-04-23T14:32:08.616020+0000","last_became_peered":"2024-04-23T14:32:08.616020+0000","last_unstale":"2024-04-23T14:32:15.179964+0000","last_undegraded":"2024-04-23T14:32:15.179964+0000","last_fullsized":"2024-04-23T14:32:15.179964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:53:40.004436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640291+0000","last_change":"2024-04-23T14:32:09.635504+0000","last_active":"2024-04-23T14:32:14.640291+0000","last_peered":"2024-04-23T14:32:14.640291+0000","last_clean":"2024-04-23T14:32:14.640291+0000","last_became_active":"2024-04-23T14:32:09.635179+0000","last_became_peered":"2024-04-23T14:32:09.635179+0000","last_unstale":"2024-04-23T14:32:14.640291+0000","last_undegraded":"2024-04-23T14:32:14.640291+0000","last_fullsized":"2024-04-23T14:32:14.640291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:56.096479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151795+0000","last_change":"2024-04-23T14:32:13.667280+0000","last_active":"2024-04-23T14:32:15.151795+0000","last_peered":"2024-04-23T14:32:15.151795+0000","last_clean":"2024-04-23T14:32:15.151795+0000","last_became_active":"2024-04-23T14:32:13.666861+0000","last_became_peered":"2024-04-23T14:32:13.666861+0000","last_unstale":"2024-04-23T14:32:15.151795+0000","last_undegraded":"2024-04-23T14:32:15.151795+0000","last_fullsized":"2024-04-23T14:32:15.151795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:56.509994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179707+0000","last_change":"2024-04-23T14:32:08.615668+0000","last_active":"2024-04-23T14:32:15.179707+0000","last_peered":"2024-04-23T14:32:15.179707+0000","last_clean":"2024-04-23T14:32:15.179707+0000","last_became_active":"2024-04-23T14:32:08.615384+0000","last_became_peered":"2024-04-23T14:32:08.615384+0000","last_unstale":"2024-04-23T14:32:15.179707+0000","last_undegraded":"2024-04-23T14:32:15.179707+0000","last_fullsized":"2024-04-23T14:32:15.179707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:11.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640191+0000","last_change":"2024-04-23T14:32:09.634014+0000","last_active":"2024-04-23T14:32:14.640191+0000","last_peered":"2024-04-23T14:32:14.640191+0000","last_clean":"2024-04-23T14:32:14.640191+0000","last_became_active":"2024-04-23T14:32:09.633633+0000","last_became_peered":"2024-04-23T14:32:09.633633+0000","last_unstale":"2024-04-23T14:32:14.640191+0000","last_undegraded":"2024-04-23T14:32:14.640191+0000","last_fullsized":"2024-04-23T14:32:14.640191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:47:31.805399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151732+0000","last_change":"2024-04-23T14:32:13.661944+0000","last_active":"2024-04-23T14:32:15.151732+0000","last_peered":"2024-04-23T14:32:15.151732+0000","last_clean":"2024-04-23T14:32:15.151732+0000","last_became_active":"2024-04-23T14:32:13.661584+0000","last_became_peered":"2024-04-23T14:32:13.661584+0000","last_unstale":"2024-04-23T14:32:15.151732+0000","last_undegraded":"2024-04-23T14:32:15.151732+0000","last_fullsized":"2024-04-23T14:32:15.151732+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:54.927834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641604+0000","last_change":"2024-04-23T14:32:09.645728+0000","last_active":"2024-04-23T14:32:14.641604+0000","last_peered":"2024-04-23T14:32:14.641604+0000","last_clean":"2024-04-23T14:32:14.641604+0000","last_became_active":"2024-04-23T14:32:09.641089+0000","last_became_peered":"2024-04-23T14:32:09.641089+0000","last_unstale":"2024-04-23T14:32:14.641604+0000","last_undegraded":"2024-04-23T14:32:14.641604+0000","last_fullsized":"2024-04-23T14:32:14.641604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:53.674486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179794+0000","last_change":"2024-04-23T14:32:08.616537+0000","last_active":"2024-04-23T14:32:15.179794+0000","last_peered":"2024-04-23T14:32:15.179794+0000","last_clean":"2024-04-23T14:32:15.179794+0000","last_became_active":"2024-04-23T14:32:08.616145+0000","last_became_peered":"2024-04-23T14:32:08.616145+0000","last_unstale":"2024-04-23T14:32:15.179794+0000","last_undegraded":"2024-04-23T14:32:15.179794+0000","last_fullsized":"2024-04-23T14:32:15.179794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:00.056567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152246+0000","last_change":"2024-04-23T14:32:13.667224+0000","last_active":"2024-04-23T14:32:15.152246+0000","last_peered":"2024-04-23T14:32:15.152246+0000","last_clean":"2024-04-23T14:32:15.152246+0000","last_became_active":"2024-04-23T14:32:13.666816+0000","last_became_peered":"2024-04-23T14:32:13.666816+0000","last_unstale":"2024-04-23T14:32:15.152246+0000","last_undegraded":"2024-04-23T14:32:15.152246+0000","last_fullsized":"2024-04-23T14:32:15.152246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:22.418010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181037+0000","last_change":"2024-04-23T14:32:09.622489+0000","last_active":"2024-04-23T14:32:15.181037+0000","last_peered":"2024-04-23T14:32:15.181037+0000","last_clean":"2024-04-23T14:32:15.181037+0000","last_became_active":"2024-04-23T14:32:09.617206+0000","last_became_peered":"2024-04-23T14:32:09.617206+0000","last_unstale":"2024-04-23T14:32:15.181037+0000","last_undegraded":"2024-04-23T14:32:15.181037+0000","last_fullsized":"2024-04-23T14:32:15.181037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:03.128923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182693+0000","last_change":"2024-04-23T14:32:08.600628+0000","last_active":"2024-04-23T14:32:15.182693+0000","last_peered":"2024-04-23T14:32:15.182693+0000","last_clean":"2024-04-23T14:32:15.182693+0000","last_became_active":"2024-04-23T14:32:08.600318+0000","last_became_peered":"2024-04-23T14:32:08.600318+0000","last_unstale":"2024-04-23T14:32:15.182693+0000","last_undegraded":"2024-04-23T14:32:15.182693+0000","last_fullsized":"2024-04-23T14:32:15.182693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:54.495261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152192+0000","last_change":"2024-04-23T14:32:13.661586+0000","last_active":"2024-04-23T14:32:15.152192+0000","last_peered":"2024-04-23T14:32:15.152192+0000","last_clean":"2024-04-23T14:32:15.152192+0000","last_became_active":"2024-04-23T14:32:13.660406+0000","last_became_peered":"2024-04-23T14:32:13.660406+0000","last_unstale":"2024-04-23T14:32:15.152192+0000","last_undegraded":"2024-04-23T14:32:15.152192+0000","last_fullsized":"2024-04-23T14:32:15.152192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:47:35.856332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181164+0000","last_change":"2024-04-23T14:32:09.627921+0000","last_active":"2024-04-23T14:32:15.181164+0000","last_peered":"2024-04-23T14:32:15.181164+0000","last_clean":"2024-04-23T14:32:15.181164+0000","last_became_active":"2024-04-23T14:32:09.622420+0000","last_became_peered":"2024-04-23T14:32:09.622420+0000","last_unstale":"2024-04-23T14:32:15.181164+0000","last_undegraded":"2024-04-23T14:32:15.181164+0000","last_fullsized":"2024-04-23T14:32:15.181164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:53.674486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180797+0000","last_change":"2024-04-23T14:32:08.612306+0000","last_active":"2024-04-23T14:32:15.180797+0000","last_peered":"2024-04-23T14:32:15.180797+0000","last_clean":"2024-04-23T14:32:15.180797+0000","last_became_active":"2024-04-23T14:32:08.612018+0000","last_became_peered":"2024-04-23T14:32:08.612018+0000","last_unstale":"2024-04-23T14:32:15.180797+0000","last_undegraded":"2024-04-23T14:32:15.180797+0000","last_fullsized":"2024-04-23T14:32:15.180797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:59:03.639740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.183304+0000","last_change":"2024-04-23T14:32:13.780668+0000","last_active":"2024-04-23T14:32:15.183304+0000","last_peered":"2024-04-23T14:32:15.183304+0000","last_clean":"2024-04-23T14:32:15.183304+0000","last_became_active":"2024-04-23T14:32:13.780323+0000","last_became_peered":"2024-04-23T14:32:13.780323+0000","last_unstale":"2024-04-23T14:32:15.183304+0000","last_undegraded":"2024-04-23T14:32:15.183304+0000","last_fullsized":"2024-04-23T14:32:15.183304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:12:23.488042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151992+0000","last_change":"2024-04-23T14:32:09.627663+0000","last_active":"2024-04-23T14:32:15.151992+0000","last_peered":"2024-04-23T14:32:15.151992+0000","last_clean":"2024-04-23T14:32:15.151992+0000","last_became_active":"2024-04-23T14:32:09.620464+0000","last_became_peered":"2024-04-23T14:32:09.620464+0000","last_unstale":"2024-04-23T14:32:15.151992+0000","last_undegraded":"2024-04-23T14:32:15.151992+0000","last_fullsized":"2024-04-23T14:32:15.151992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:52.656649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179300+0000","last_change":"2024-04-23T14:32:08.612870+0000","last_active":"2024-04-23T14:32:15.179300+0000","last_peered":"2024-04-23T14:32:15.179300+0000","last_clean":"2024-04-23T14:32:15.179300+0000","last_became_active":"2024-04-23T14:32:08.610830+0000","last_became_peered":"2024-04-23T14:32:08.610830+0000","last_unstale":"2024-04-23T14:32:15.179300+0000","last_undegraded":"2024-04-23T14:32:15.179300+0000","last_fullsized":"2024-04-23T14:32:15.179300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:51.650822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152268+0000","last_change":"2024-04-23T14:32:13.660771+0000","last_active":"2024-04-23T14:32:15.152268+0000","last_peered":"2024-04-23T14:32:15.152268+0000","last_clean":"2024-04-23T14:32:15.152268+0000","last_became_active":"2024-04-23T14:32:13.660291+0000","last_became_peered":"2024-04-23T14:32:13.660291+0000","last_unstale":"2024-04-23T14:32:15.152268+0000","last_undegraded":"2024-04-23T14:32:15.152268+0000","last_fullsized":"2024-04-23T14:32:15.152268+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:28.688885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640262+0000","last_change":"2024-04-23T14:32:09.612595+0000","last_active":"2024-04-23T14:32:14.640262+0000","last_peered":"2024-04-23T14:32:14.640262+0000","last_clean":"2024-04-23T14:32:14.640262+0000","last_became_active":"2024-04-23T14:32:09.612168+0000","last_became_peered":"2024-04-23T14:32:09.612168+0000","last_unstale":"2024-04-23T14:32:14.640262+0000","last_undegraded":"2024-04-23T14:32:14.640262+0000","last_fullsized":"2024-04-23T14:32:14.640262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:03.128923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.642236+0000","last_change":"2024-04-23T14:32:08.615099+0000","last_active":"2024-04-23T14:32:14.642236+0000","last_peered":"2024-04-23T14:32:14.642236+0000","last_clean":"2024-04-23T14:32:14.642236+0000","last_became_active":"2024-04-23T14:32:08.614350+0000","last_became_peered":"2024-04-23T14:32:08.614350+0000","last_unstale":"2024-04-23T14:32:14.642236+0000","last_undegraded":"2024-04-23T14:32:14.642236+0000","last_fullsized":"2024-04-23T14:32:14.642236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:59:03.639740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151716+0000","last_change":"2024-04-23T14:32:13.775674+0000","last_active":"2024-04-23T14:32:15.151716+0000","last_peered":"2024-04-23T14:32:15.151716+0000","last_clean":"2024-04-23T14:32:15.151716+0000","last_became_active":"2024-04-23T14:32:13.774695+0000","last_became_peered":"2024-04-23T14:32:13.774695+0000","last_unstale":"2024-04-23T14:32:15.151716+0000","last_undegraded":"2024-04-23T14:32:15.151716+0000","last_fullsized":"2024-04-23T14:32:15.151716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:13.117688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182184+0000","last_change":"2024-04-23T14:32:09.630421+0000","last_active":"2024-04-23T14:32:15.182184+0000","last_peered":"2024-04-23T14:32:15.182184+0000","last_clean":"2024-04-23T14:32:15.182184+0000","last_became_active":"2024-04-23T14:32:09.630104+0000","last_became_peered":"2024-04-23T14:32:09.630104+0000","last_unstale":"2024-04-23T14:32:15.182184+0000","last_undegraded":"2024-04-23T14:32:15.182184+0000","last_fullsized":"2024-04-23T14:32:15.182184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:56.096479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.183107+0000","last_change":"2024-04-23T14:32:08.607576+0000","last_active":"2024-04-23T14:32:15.183107+0000","last_peered":"2024-04-23T14:32:15.183107+0000","last_clean":"2024-04-23T14:32:15.183107+0000","last_became_active":"2024-04-23T14:32:08.606611+0000","last_became_peered":"2024-04-23T14:32:08.606611+0000","last_unstale":"2024-04-23T14:32:15.183107+0000","last_undegraded":"2024-04-23T14:32:15.183107+0000","last_fullsized":"2024-04-23T14:32:15.183107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:00.056567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180827+0000","last_change":"2024-04-23T14:32:13.778406+0000","last_active":"2024-04-23T14:32:15.180827+0000","last_peered":"2024-04-23T14:32:15.180827+0000","last_clean":"2024-04-23T14:32:15.180827+0000","last_became_active":"2024-04-23T14:32:13.778034+0000","last_became_peered":"2024-04-23T14:32:13.778034+0000","last_unstale":"2024-04-23T14:32:15.180827+0000","last_undegraded":"2024-04-23T14:32:15.180827+0000","last_fullsized":"2024-04-23T14:32:15.180827+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:21:36.288246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181961+0000","last_change":"2024-04-23T14:32:09.613694+0000","last_active":"2024-04-23T14:32:15.181961+0000","last_peered":"2024-04-23T14:32:15.181961+0000","last_clean":"2024-04-23T14:32:15.181961+0000","last_became_active":"2024-04-23T14:32:09.613413+0000","last_became_peered":"2024-04-23T14:32:09.613413+0000","last_unstale":"2024-04-23T14:32:15.181961+0000","last_undegraded":"2024-04-23T14:32:15.181961+0000","last_fullsized":"2024-04-23T14:32:15.181961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:12.933947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180126+0000","last_change":"2024-04-23T14:32:08.612923+0000","last_active":"2024-04-23T14:32:15.180126+0000","last_peered":"2024-04-23T14:32:15.180126+0000","last_clean":"2024-04-23T14:32:15.180126+0000","last_became_active":"2024-04-23T14:32:08.610847+0000","last_became_peered":"2024-04-23T14:32:08.610847+0000","last_unstale":"2024-04-23T14:32:15.180126+0000","last_undegraded":"2024-04-23T14:32:15.180126+0000","last_fullsized":"2024-04-23T14:32:15.180126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:40:25.063520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640453+0000","last_change":"2024-04-23T14:32:13.775013+0000","last_active":"2024-04-23T14:32:14.640453+0000","last_peered":"2024-04-23T14:32:14.640453+0000","last_clean":"2024-04-23T14:32:14.640453+0000","last_became_active":"2024-04-23T14:32:13.774565+0000","last_became_peered":"2024-04-23T14:32:13.774565+0000","last_unstale":"2024-04-23T14:32:14.640453+0000","last_undegraded":"2024-04-23T14:32:14.640453+0000","last_fullsized":"2024-04-23T14:32:14.640453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:26:08.003427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640083+0000","last_change":"2024-04-23T14:32:09.633886+0000","last_active":"2024-04-23T14:32:14.640083+0000","last_peered":"2024-04-23T14:32:14.640083+0000","last_clean":"2024-04-23T14:32:14.640083+0000","last_became_active":"2024-04-23T14:32:09.633258+0000","last_became_peered":"2024-04-23T14:32:09.633258+0000","last_unstale":"2024-04-23T14:32:14.640083+0000","last_undegraded":"2024-04-23T14:32:14.640083+0000","last_fullsized":"2024-04-23T14:32:14.640083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:24.637952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182012+0000","last_change":"2024-04-23T14:32:08.602248+0000","last_active":"2024-04-23T14:32:15.182012+0000","last_peered":"2024-04-23T14:32:15.182012+0000","last_clean":"2024-04-23T14:32:15.182012+0000","last_became_active":"2024-04-23T14:32:08.601913+0000","last_became_peered":"2024-04-23T14:32:08.601913+0000","last_unstale":"2024-04-23T14:32:15.182012+0000","last_undegraded":"2024-04-23T14:32:15.182012+0000","last_fullsized":"2024-04-23T14:32:15.182012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:11.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180987+0000","last_change":"2024-04-23T14:32:13.666952+0000","last_active":"2024-04-23T14:32:15.180987+0000","last_peered":"2024-04-23T14:32:15.180987+0000","last_clean":"2024-04-23T14:32:15.180987+0000","last_became_active":"2024-04-23T14:32:13.663335+0000","last_became_peered":"2024-04-23T14:32:13.663335+0000","last_unstale":"2024-04-23T14:32:15.180987+0000","last_undegraded":"2024-04-23T14:32:15.180987+0000","last_fullsized":"2024-04-23T14:32:15.180987+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:22.418010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152000+0000","last_change":"2024-04-23T14:32:09.627852+0000","last_active":"2024-04-23T14:32:15.152000+0000","last_peered":"2024-04-23T14:32:15.152000+0000","last_clean":"2024-04-23T14:32:15.152000+0000","last_became_active":"2024-04-23T14:32:09.627575+0000","last_became_peered":"2024-04-23T14:32:09.627575+0000","last_unstale":"2024-04-23T14:32:15.152000+0000","last_undegraded":"2024-04-23T14:32:15.152000+0000","last_fullsized":"2024-04-23T14:32:15.152000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:55.501089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640731+0000","last_change":"2024-04-23T14:32:08.603753+0000","last_active":"2024-04-23T14:32:14.640731+0000","last_peered":"2024-04-23T14:32:14.640731+0000","last_clean":"2024-04-23T14:32:14.640731+0000","last_became_active":"2024-04-23T14:32:08.602944+0000","last_became_peered":"2024-04-23T14:32:08.602944+0000","last_unstale":"2024-04-23T14:32:14.640731+0000","last_undegraded":"2024-04-23T14:32:14.640731+0000","last_fullsized":"2024-04-23T14:32:14.640731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:54.495261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181918+0000","last_change":"2024-04-23T14:32:13.667268+0000","last_active":"2024-04-23T14:32:15.181918+0000","last_peered":"2024-04-23T14:32:15.181918+0000","last_clean":"2024-04-23T14:32:15.181918+0000","last_became_active":"2024-04-23T14:32:13.666836+0000","last_became_peered":"2024-04-23T14:32:13.666836+0000","last_unstale":"2024-04-23T14:32:15.181918+0000","last_undegraded":"2024-04-23T14:32:15.181918+0000","last_fullsized":"2024-04-23T14:32:15.181918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:14:16.782670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180774+0000","last_change":"2024-04-23T14:32:09.613896+0000","last_active":"2024-04-23T14:32:15.180774+0000","last_peered":"2024-04-23T14:32:15.180774+0000","last_clean":"2024-04-23T14:32:15.180774+0000","last_became_active":"2024-04-23T14:32:09.613535+0000","last_became_peered":"2024-04-23T14:32:09.613535+0000","last_unstale":"2024-04-23T14:32:15.180774+0000","last_undegraded":"2024-04-23T14:32:15.180774+0000","last_fullsized":"2024-04-23T14:32:15.180774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:50:24.043980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181650+0000","last_change":"2024-04-23T14:32:08.606268+0000","last_active":"2024-04-23T14:32:15.181650+0000","last_peered":"2024-04-23T14:32:15.181650+0000","last_clean":"2024-04-23T14:32:15.181650+0000","last_became_active":"2024-04-23T14:32:08.605958+0000","last_became_peered":"2024-04-23T14:32:08.605958+0000","last_unstale":"2024-04-23T14:32:15.181650+0000","last_undegraded":"2024-04-23T14:32:15.181650+0000","last_fullsized":"2024-04-23T14:32:15.181650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:45:45.243948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.183441+0000","last_change":"2024-04-23T14:32:13.664183+0000","last_active":"2024-04-23T14:32:15.183441+0000","last_peered":"2024-04-23T14:32:15.183441+0000","last_clean":"2024-04-23T14:32:15.183441+0000","last_became_active":"2024-04-23T14:32:13.663836+0000","last_became_peered":"2024-04-23T14:32:13.663836+0000","last_unstale":"2024-04-23T14:32:15.183441+0000","last_undegraded":"2024-04-23T14:32:15.183441+0000","last_fullsized":"2024-04-23T14:32:15.183441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:09:12.045283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639679+0000","last_change":"2024-04-23T14:32:09.633686+0000","last_active":"2024-04-23T14:32:14.639679+0000","last_peered":"2024-04-23T14:32:14.639679+0000","last_clean":"2024-04-23T14:32:14.639679+0000","last_became_active":"2024-04-23T14:32:09.631825+0000","last_became_peered":"2024-04-23T14:32:09.631825+0000","last_unstale":"2024-04-23T14:32:14.639679+0000","last_undegraded":"2024-04-23T14:32:14.639679+0000","last_fullsized":"2024-04-23T14:32:14.639679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:23:57.869625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179303+0000","last_change":"2024-04-23T14:32:08.613375+0000","last_active":"2024-04-23T14:32:15.179303+0000","last_peered":"2024-04-23T14:32:15.179303+0000","last_clean":"2024-04-23T14:32:15.179303+0000","last_became_active":"2024-04-23T14:32:08.613129+0000","last_became_peered":"2024-04-23T14:32:08.613129+0000","last_unstale":"2024-04-23T14:32:15.179303+0000","last_undegraded":"2024-04-23T14:32:15.179303+0000","last_fullsized":"2024-04-23T14:32:15.179303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:54.495261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640202+0000","last_change":"2024-04-23T14:32:13.779707+0000","last_active":"2024-04-23T14:32:14.640202+0000","last_peered":"2024-04-23T14:32:14.640202+0000","last_clean":"2024-04-23T14:32:14.640202+0000","last_became_active":"2024-04-23T14:32:13.779357+0000","last_became_peered":"2024-04-23T14:32:13.779357+0000","last_unstale":"2024-04-23T14:32:14.640202+0000","last_undegraded":"2024-04-23T14:32:14.640202+0000","last_fullsized":"2024-04-23T14:32:14.640202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:12:23.488042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181182+0000","last_change":"2024-04-23T14:32:09.621797+0000","last_active":"2024-04-23T14:32:15.181182+0000","last_peered":"2024-04-23T14:32:15.181182+0000","last_clean":"2024-04-23T14:32:15.181182+0000","last_became_active":"2024-04-23T14:32:09.618825+0000","last_became_peered":"2024-04-23T14:32:09.618825+0000","last_unstale":"2024-04-23T14:32:15.181182+0000","last_undegraded":"2024-04-23T14:32:15.181182+0000","last_fullsized":"2024-04-23T14:32:15.181182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:22.220286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639965+0000","last_change":"2024-04-23T14:32:08.603338+0000","last_active":"2024-04-23T14:32:14.639965+0000","last_peered":"2024-04-23T14:32:14.639965+0000","last_clean":"2024-04-23T14:32:14.639965+0000","last_became_active":"2024-04-23T14:32:08.602563+0000","last_became_peered":"2024-04-23T14:32:08.602563+0000","last_unstale":"2024-04-23T14:32:14.639965+0000","last_undegraded":"2024-04-23T14:32:14.639965+0000","last_fullsized":"2024-04-23T14:32:14.639965+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:45:45.243948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179439+0000","last_change":"2024-04-23T14:31:58.314235+0000","last_active":"2024-04-23T14:32:15.179439+0000","last_peered":"2024-04-23T14:32:15.179439+0000","last_clean":"2024-04-23T14:32:15.179439+0000","last_became_active":"2024-04-23T14:31:58.313165+0000","last_became_peered":"2024-04-23T14:31:58.313165+0000","last_unstale":"2024-04-23T14:32:15.179439+0000","last_undegraded":"2024-04-23T14:32:15.179439+0000","last_fullsized":"2024-04-23T14:32:15.179439+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_clean_scrub_stamp":"2024-04-23T14:31:57.140712+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:44.170436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641636+0000","last_change":"2024-04-23T14:32:13.777292+0000","last_active":"2024-04-23T14:32:14.641636+0000","last_peered":"2024-04-23T14:32:14.641636+0000","last_clean":"2024-04-23T14:32:14.641636+0000","last_became_active":"2024-04-23T14:32:13.776933+0000","last_became_peered":"2024-04-23T14:32:13.776933+0000","last_unstale":"2024-04-23T14:32:14.641636+0000","last_undegraded":"2024-04-23T14:32:14.641636+0000","last_fullsized":"2024-04-23T14:32:14.641636+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:47:35.856332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179793+0000","last_change":"2024-04-23T14:32:09.639015+0000","last_active":"2024-04-23T14:32:15.179793+0000","last_peered":"2024-04-23T14:32:15.179793+0000","last_clean":"2024-04-23T14:32:15.179793+0000","last_became_active":"2024-04-23T14:32:09.638622+0000","last_became_peered":"2024-04-23T14:32:09.638622+0000","last_unstale":"2024-04-23T14:32:15.179793+0000","last_undegraded":"2024-04-23T14:32:15.179793+0000","last_fullsized":"2024-04-23T14:32:15.179793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:18.367077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181794+0000","last_change":"2024-04-23T14:32:08.607926+0000","last_active":"2024-04-23T14:32:15.181794+0000","last_peered":"2024-04-23T14:32:15.181794+0000","last_clean":"2024-04-23T14:32:15.181794+0000","last_became_active":"2024-04-23T14:32:08.607649+0000","last_became_peered":"2024-04-23T14:32:08.607649+0000","last_unstale":"2024-04-23T14:32:15.181794+0000","last_undegraded":"2024-04-23T14:32:15.181794+0000","last_fullsized":"2024-04-23T14:32:15.181794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:59:03.639740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641732+0000","last_change":"2024-04-23T14:32:13.774449+0000","last_active":"2024-04-23T14:32:14.641732+0000","last_peered":"2024-04-23T14:32:14.641732+0000","last_clean":"2024-04-23T14:32:14.641732+0000","last_became_active":"2024-04-23T14:32:13.773937+0000","last_became_peered":"2024-04-23T14:32:13.773937+0000","last_unstale":"2024-04-23T14:32:14.641732+0000","last_undegraded":"2024-04-23T14:32:14.641732+0000","last_fullsized":"2024-04-23T14:32:14.641732+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:22.418010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640399+0000","last_change":"2024-04-23T14:32:09.626405+0000","last_active":"2024-04-23T14:32:14.640399+0000","last_peered":"2024-04-23T14:32:14.640399+0000","last_clean":"2024-04-23T14:32:14.640399+0000","last_became_active":"2024-04-23T14:32:09.626056+0000","last_became_peered":"2024-04-23T14:32:09.626056+0000","last_unstale":"2024-04-23T14:32:14.640399+0000","last_undegraded":"2024-04-23T14:32:14.640399+0000","last_fullsized":"2024-04-23T14:32:14.640399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:18.367077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182898+0000","last_change":"2024-04-23T14:32:08.608148+0000","last_active":"2024-04-23T14:32:15.182898+0000","last_peered":"2024-04-23T14:32:15.182898+0000","last_clean":"2024-04-23T14:32:15.182898+0000","last_became_active":"2024-04-23T14:32:08.607623+0000","last_became_peered":"2024-04-23T14:32:08.607623+0000","last_unstale":"2024-04-23T14:32:15.182898+0000","last_undegraded":"2024-04-23T14:32:15.182898+0000","last_fullsized":"2024-04-23T14:32:15.182898+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:51.650822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640246+0000","last_change":"2024-04-23T14:32:13.663295+0000","last_active":"2024-04-23T14:32:14.640246+0000","last_peered":"2024-04-23T14:32:14.640246+0000","last_clean":"2024-04-23T14:32:14.640246+0000","last_became_active":"2024-04-23T14:32:13.662900+0000","last_became_peered":"2024-04-23T14:32:13.662900+0000","last_unstale":"2024-04-23T14:32:14.640246+0000","last_undegraded":"2024-04-23T14:32:14.640246+0000","last_fullsized":"2024-04-23T14:32:14.640246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:09:12.045283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180115+0000","last_change":"2024-04-23T14:32:09.641393+0000","last_active":"2024-04-23T14:32:15.180115+0000","last_peered":"2024-04-23T14:32:15.180115+0000","last_clean":"2024-04-23T14:32:15.180115+0000","last_became_active":"2024-04-23T14:32:09.641075+0000","last_became_peered":"2024-04-23T14:32:09.641075+0000","last_unstale":"2024-04-23T14:32:15.180115+0000","last_undegraded":"2024-04-23T14:32:15.180115+0000","last_fullsized":"2024-04-23T14:32:15.180115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:26:03.952494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182813+0000","last_change":"2024-04-23T14:32:08.607178+0000","last_active":"2024-04-23T14:32:15.182813+0000","last_peered":"2024-04-23T14:32:15.182813+0000","last_clean":"2024-04-23T14:32:15.182813+0000","last_became_active":"2024-04-23T14:32:08.606863+0000","last_became_peered":"2024-04-23T14:32:08.606863+0000","last_unstale":"2024-04-23T14:32:15.182813+0000","last_undegraded":"2024-04-23T14:32:15.182813+0000","last_fullsized":"2024-04-23T14:32:15.182813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:35.854078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639398+0000","last_change":"2024-04-23T14:32:13.773533+0000","last_active":"2024-04-23T14:32:14.639398+0000","last_peered":"2024-04-23T14:32:14.639398+0000","last_clean":"2024-04-23T14:32:14.639398+0000","last_became_active":"2024-04-23T14:32:13.773057+0000","last_became_peered":"2024-04-23T14:32:13.773057+0000","last_unstale":"2024-04-23T14:32:14.639398+0000","last_undegraded":"2024-04-23T14:32:14.639398+0000","last_fullsized":"2024-04-23T14:32:14.639398+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:08.115745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180012+0000","last_change":"2024-04-23T14:32:09.641317+0000","last_active":"2024-04-23T14:32:15.180012+0000","last_peered":"2024-04-23T14:32:15.180012+0000","last_clean":"2024-04-23T14:32:15.180012+0000","last_became_active":"2024-04-23T14:32:09.640930+0000","last_became_peered":"2024-04-23T14:32:09.640930+0000","last_unstale":"2024-04-23T14:32:15.180012+0000","last_undegraded":"2024-04-23T14:32:15.180012+0000","last_fullsized":"2024-04-23T14:32:15.180012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:24.637952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152438+0000","last_change":"2024-04-23T14:32:08.601936+0000","last_active":"2024-04-23T14:32:15.152438+0000","last_peered":"2024-04-23T14:32:15.152438+0000","last_clean":"2024-04-23T14:32:15.152438+0000","last_became_active":"2024-04-23T14:32:08.601653+0000","last_became_peered":"2024-04-23T14:32:08.601653+0000","last_unstale":"2024-04-23T14:32:15.152438+0000","last_undegraded":"2024-04-23T14:32:15.152438+0000","last_fullsized":"2024-04-23T14:32:15.152438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:07:08.639311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151588+0000","last_change":"2024-04-23T14:32:13.664132+0000","last_active":"2024-04-23T14:32:15.151588+0000","last_peered":"2024-04-23T14:32:15.151588+0000","last_clean":"2024-04-23T14:32:15.151588+0000","last_became_active":"2024-04-23T14:32:13.663741+0000","last_became_peered":"2024-04-23T14:32:13.663741+0000","last_unstale":"2024-04-23T14:32:15.151588+0000","last_undegraded":"2024-04-23T14:32:15.151588+0000","last_fullsized":"2024-04-23T14:32:15.151588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:07.179856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179517+0000","last_change":"2024-04-23T14:32:09.641942+0000","last_active":"2024-04-23T14:32:15.179517+0000","last_peered":"2024-04-23T14:32:15.179517+0000","last_clean":"2024-04-23T14:32:15.179517+0000","last_became_active":"2024-04-23T14:32:09.641680+0000","last_became_peered":"2024-04-23T14:32:09.641680+0000","last_unstale":"2024-04-23T14:32:15.179517+0000","last_undegraded":"2024-04-23T14:32:15.179517+0000","last_fullsized":"2024-04-23T14:32:15.179517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:50.876901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181283+0000","last_change":"2024-04-23T14:32:08.603405+0000","last_active":"2024-04-23T14:32:15.181283+0000","last_peered":"2024-04-23T14:32:15.181283+0000","last_clean":"2024-04-23T14:32:15.181283+0000","last_became_active":"2024-04-23T14:32:08.603116+0000","last_became_peered":"2024-04-23T14:32:08.603116+0000","last_unstale":"2024-04-23T14:32:15.181283+0000","last_undegraded":"2024-04-23T14:32:15.181283+0000","last_fullsized":"2024-04-23T14:32:15.181283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:43:58.790056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640138+0000","last_change":"2024-04-23T14:32:13.660242+0000","last_active":"2024-04-23T14:32:14.640138+0000","last_peered":"2024-04-23T14:32:14.640138+0000","last_clean":"2024-04-23T14:32:14.640138+0000","last_became_active":"2024-04-23T14:32:13.659712+0000","last_became_peered":"2024-04-23T14:32:13.659712+0000","last_unstale":"2024-04-23T14:32:14.640138+0000","last_undegraded":"2024-04-23T14:32:14.640138+0000","last_fullsized":"2024-04-23T14:32:14.640138+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:14:16.782670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641152+0000","last_change":"2024-04-23T14:32:09.619773+0000","last_active":"2024-04-23T14:32:14.641152+0000","last_peered":"2024-04-23T14:32:14.641152+0000","last_clean":"2024-04-23T14:32:14.641152+0000","last_became_active":"2024-04-23T14:32:09.619451+0000","last_became_peered":"2024-04-23T14:32:09.619451+0000","last_unstale":"2024-04-23T14:32:14.641152+0000","last_undegraded":"2024-04-23T14:32:14.641152+0000","last_fullsized":"2024-04-23T14:32:14.641152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:12.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641439+0000","last_change":"2024-04-23T14:32:08.613751+0000","last_active":"2024-04-23T14:32:14.641439+0000","last_peered":"2024-04-23T14:32:14.641439+0000","last_clean":"2024-04-23T14:32:14.641439+0000","last_became_active":"2024-04-23T14:32:08.607415+0000","last_became_peered":"2024-04-23T14:32:08.607415+0000","last_unstale":"2024-04-23T14:32:14.641439+0000","last_undegraded":"2024-04-23T14:32:14.641439+0000","last_fullsized":"2024-04-23T14:32:14.641439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:35.854078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182048+0000","last_change":"2024-04-23T14:32:13.780614+0000","last_active":"2024-04-23T14:32:15.182048+0000","last_peered":"2024-04-23T14:32:15.182048+0000","last_clean":"2024-04-23T14:32:15.182048+0000","last_became_active":"2024-04-23T14:32:13.780135+0000","last_became_peered":"2024-04-23T14:32:13.780135+0000","last_unstale":"2024-04-23T14:32:15.182048+0000","last_undegraded":"2024-04-23T14:32:15.182048+0000","last_fullsized":"2024-04-23T14:32:15.182048+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:16:27.476884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179624+0000","last_change":"2024-04-23T14:32:09.642908+0000","last_active":"2024-04-23T14:32:15.179624+0000","last_peered":"2024-04-23T14:32:15.179624+0000","last_clean":"2024-04-23T14:32:15.179624+0000","last_became_active":"2024-04-23T14:32:09.642670+0000","last_became_peered":"2024-04-23T14:32:09.642670+0000","last_unstale":"2024-04-23T14:32:15.179624+0000","last_undegraded":"2024-04-23T14:32:15.179624+0000","last_fullsized":"2024-04-23T14:32:15.179624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:56.096479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640627+0000","last_change":"2024-04-23T14:32:08.598260+0000","last_active":"2024-04-23T14:32:14.640627+0000","last_peered":"2024-04-23T14:32:14.640627+0000","last_clean":"2024-04-23T14:32:14.640627+0000","last_became_active":"2024-04-23T14:32:08.597931+0000","last_became_peered":"2024-04-23T14:32:08.597931+0000","last_unstale":"2024-04-23T14:32:14.640627+0000","last_undegraded":"2024-04-23T14:32:14.640627+0000","last_fullsized":"2024-04-23T14:32:14.640627+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:51.650822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182382+0000","last_change":"2024-04-23T14:32:13.775555+0000","last_active":"2024-04-23T14:32:15.182382+0000","last_peered":"2024-04-23T14:32:15.182382+0000","last_clean":"2024-04-23T14:32:15.182382+0000","last_became_active":"2024-04-23T14:32:13.774654+0000","last_became_peered":"2024-04-23T14:32:13.774654+0000","last_unstale":"2024-04-23T14:32:15.182382+0000","last_undegraded":"2024-04-23T14:32:15.182382+0000","last_fullsized":"2024-04-23T14:32:15.182382+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:57.743802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639957+0000","last_change":"2024-04-23T14:32:09.626794+0000","last_active":"2024-04-23T14:32:14.639957+0000","last_peered":"2024-04-23T14:32:14.639957+0000","last_clean":"2024-04-23T14:32:14.639957+0000","last_became_active":"2024-04-23T14:32:09.626382+0000","last_became_peered":"2024-04-23T14:32:09.626382+0000","last_unstale":"2024-04-23T14:32:14.639957+0000","last_undegraded":"2024-04-23T14:32:14.639957+0000","last_fullsized":"2024-04-23T14:32:14.639957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:54:08.664672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640520+0000","last_change":"2024-04-23T14:32:08.600425+0000","last_active":"2024-04-23T14:32:14.640520+0000","last_peered":"2024-04-23T14:32:14.640520+0000","last_clean":"2024-04-23T14:32:14.640520+0000","last_became_active":"2024-04-23T14:32:08.600113+0000","last_became_peered":"2024-04-23T14:32:08.600113+0000","last_unstale":"2024-04-23T14:32:14.640520+0000","last_undegraded":"2024-04-23T14:32:14.640520+0000","last_fullsized":"2024-04-23T14:32:14.640520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:07:08.639311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639973+0000","last_change":"2024-04-23T14:32:13.777092+0000","last_active":"2024-04-23T14:32:14.639973+0000","last_peered":"2024-04-23T14:32:14.639973+0000","last_clean":"2024-04-23T14:32:14.639973+0000","last_became_active":"2024-04-23T14:32:13.776681+0000","last_became_peered":"2024-04-23T14:32:13.776681+0000","last_unstale":"2024-04-23T14:32:14.639973+0000","last_undegraded":"2024-04-23T14:32:14.639973+0000","last_fullsized":"2024-04-23T14:32:14.639973+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:21:36.288246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179674+0000","last_change":"2024-04-23T14:32:09.629485+0000","last_active":"2024-04-23T14:32:15.179674+0000","last_peered":"2024-04-23T14:32:15.179674+0000","last_clean":"2024-04-23T14:32:15.179674+0000","last_became_active":"2024-04-23T14:32:09.629157+0000","last_became_peered":"2024-04-23T14:32:09.629157+0000","last_unstale":"2024-04-23T14:32:15.179674+0000","last_undegraded":"2024-04-23T14:32:15.179674+0000","last_fullsized":"2024-04-23T14:32:15.179674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:47:31.805399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:25.916019+0000","last_change":"2024-04-23T14:32:08.604477+0000","last_active":"2024-04-23T14:32:25.916019+0000","last_peered":"2024-04-23T14:32:25.916019+0000","last_clean":"2024-04-23T14:32:25.916019+0000","last_became_active":"2024-04-23T14:32:08.604148+0000","last_became_peered":"2024-04-23T14:32:08.604148+0000","last_unstale":"2024-04-23T14:32:25.916019+0000","last_undegraded":"2024-04-23T14:32:25.916019+0000","last_fullsized":"2024-04-23T14:32:25.916019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:51.650822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181124+0000","last_change":"2024-04-23T14:32:13.660428+0000","last_active":"2024-04-23T14:32:15.181124+0000","last_peered":"2024-04-23T14:32:15.181124+0000","last_clean":"2024-04-23T14:32:15.181124+0000","last_became_active":"2024-04-23T14:32:13.659779+0000","last_became_peered":"2024-04-23T14:32:13.659779+0000","last_unstale":"2024-04-23T14:32:15.181124+0000","last_undegraded":"2024-04-23T14:32:15.181124+0000","last_fullsized":"2024-04-23T14:32:15.181124+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:28.688885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152295+0000","last_change":"2024-04-23T14:32:09.628609+0000","last_active":"2024-04-23T14:32:15.152295+0000","last_peered":"2024-04-23T14:32:15.152295+0000","last_clean":"2024-04-23T14:32:15.152295+0000","last_became_active":"2024-04-23T14:32:09.628357+0000","last_became_peered":"2024-04-23T14:32:09.628357+0000","last_unstale":"2024-04-23T14:32:15.152295+0000","last_undegraded":"2024-04-23T14:32:15.152295+0000","last_fullsized":"2024-04-23T14:32:15.152295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:01.062394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182825+0000","last_change":"2024-04-23T14:32:08.607931+0000","last_active":"2024-04-23T14:32:15.182825+0000","last_peered":"2024-04-23T14:32:15.182825+0000","last_clean":"2024-04-23T14:32:15.182825+0000","last_became_active":"2024-04-23T14:32:08.607268+0000","last_became_peered":"2024-04-23T14:32:08.607268+0000","last_unstale":"2024-04-23T14:32:15.182825+0000","last_undegraded":"2024-04-23T14:32:15.182825+0000","last_fullsized":"2024-04-23T14:32:15.182825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:35.854078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152109+0000","last_change":"2024-04-23T14:32:13.661080+0000","last_active":"2024-04-23T14:32:15.152109+0000","last_peered":"2024-04-23T14:32:15.152109+0000","last_clean":"2024-04-23T14:32:15.152109+0000","last_became_active":"2024-04-23T14:32:13.660376+0000","last_became_peered":"2024-04-23T14:32:13.660376+0000","last_unstale":"2024-04-23T14:32:15.152109+0000","last_undegraded":"2024-04-23T14:32:15.152109+0000","last_fullsized":"2024-04-23T14:32:15.152109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:40:00.147412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640230+0000","last_change":"2024-04-23T14:32:09.621197+0000","last_active":"2024-04-23T14:32:14.640230+0000","last_peered":"2024-04-23T14:32:14.640230+0000","last_clean":"2024-04-23T14:32:14.640230+0000","last_became_active":"2024-04-23T14:32:09.620883+0000","last_became_peered":"2024-04-23T14:32:09.620883+0000","last_unstale":"2024-04-23T14:32:14.640230+0000","last_undegraded":"2024-04-23T14:32:14.640230+0000","last_fullsized":"2024-04-23T14:32:14.640230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:12.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640178+0000","last_change":"2024-04-23T14:32:08.606025+0000","last_active":"2024-04-23T14:32:14.640178+0000","last_peered":"2024-04-23T14:32:14.640178+0000","last_clean":"2024-04-23T14:32:14.640178+0000","last_became_active":"2024-04-23T14:32:08.605727+0000","last_became_peered":"2024-04-23T14:32:08.605727+0000","last_unstale":"2024-04-23T14:32:14.640178+0000","last_undegraded":"2024-04-23T14:32:14.640178+0000","last_fullsized":"2024-04-23T14:32:14.640178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:59:03.639740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181124+0000","last_change":"2024-04-23T14:32:13.669518+0000","last_active":"2024-04-23T14:32:15.181124+0000","last_peered":"2024-04-23T14:32:15.181124+0000","last_clean":"2024-04-23T14:32:15.181124+0000","last_became_active":"2024-04-23T14:32:13.669004+0000","last_became_peered":"2024-04-23T14:32:13.669004+0000","last_unstale":"2024-04-23T14:32:15.181124+0000","last_undegraded":"2024-04-23T14:32:15.181124+0000","last_fullsized":"2024-04-23T14:32:15.181124+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:31:45.244234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640043+0000","last_change":"2024-04-23T14:32:09.625280+0000","last_active":"2024-04-23T14:32:14.640043+0000","last_peered":"2024-04-23T14:32:14.640043+0000","last_clean":"2024-04-23T14:32:14.640043+0000","last_became_active":"2024-04-23T14:32:09.625033+0000","last_became_peered":"2024-04-23T14:32:09.625033+0000","last_unstale":"2024-04-23T14:32:14.640043+0000","last_undegraded":"2024-04-23T14:32:14.640043+0000","last_fullsized":"2024-04-23T14:32:14.640043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:14:12.731737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.642133+0000","last_change":"2024-04-23T14:32:08.605039+0000","last_active":"2024-04-23T14:32:14.642133+0000","last_peered":"2024-04-23T14:32:14.642133+0000","last_clean":"2024-04-23T14:32:14.642133+0000","last_became_active":"2024-04-23T14:32:08.604450+0000","last_became_peered":"2024-04-23T14:32:08.604450+0000","last_unstale":"2024-04-23T14:32:14.642133+0000","last_undegraded":"2024-04-23T14:32:14.642133+0000","last_fullsized":"2024-04-23T14:32:14.642133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:45:45.243948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181369+0000","last_change":"2024-04-23T14:32:13.779606+0000","last_active":"2024-04-23T14:32:15.181369+0000","last_peered":"2024-04-23T14:32:15.181369+0000","last_clean":"2024-04-23T14:32:15.181369+0000","last_became_active":"2024-04-23T14:32:13.779243+0000","last_became_peered":"2024-04-23T14:32:13.779243+0000","last_unstale":"2024-04-23T14:32:15.181369+0000","last_undegraded":"2024-04-23T14:32:15.181369+0000","last_fullsized":"2024-04-23T14:32:15.181369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:58:40.249561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640030+0000","last_change":"2024-04-23T14:32:13.779659+0000","last_active":"2024-04-23T14:32:14.640030+0000","last_peered":"2024-04-23T14:32:14.640030+0000","last_clean":"2024-04-23T14:32:14.640030+0000","last_became_active":"2024-04-23T14:32:13.779312+0000","last_became_peered":"2024-04-23T14:32:13.779312+0000","last_unstale":"2024-04-23T14:32:14.640030+0000","last_undegraded":"2024-04-23T14:32:14.640030+0000","last_fullsized":"2024-04-23T14:32:14.640030+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:28.688885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180700+0000","last_change":"2024-04-23T14:32:09.632498+0000","last_active":"2024-04-23T14:32:15.180700+0000","last_peered":"2024-04-23T14:32:15.180700+0000","last_clean":"2024-04-23T14:32:15.180700+0000","last_became_active":"2024-04-23T14:32:09.632191+0000","last_became_peered":"2024-04-23T14:32:09.632191+0000","last_unstale":"2024-04-23T14:32:15.180700+0000","last_undegraded":"2024-04-23T14:32:15.180700+0000","last_fullsized":"2024-04-23T14:32:15.180700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:53.674486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639631+0000","last_change":"2024-04-23T14:32:08.607308+0000","last_active":"2024-04-23T14:32:14.639631+0000","last_peered":"2024-04-23T14:32:14.639631+0000","last_clean":"2024-04-23T14:32:14.639631+0000","last_became_active":"2024-04-23T14:32:08.606840+0000","last_became_peered":"2024-04-23T14:32:08.606840+0000","last_unstale":"2024-04-23T14:32:14.639631+0000","last_undegraded":"2024-04-23T14:32:14.639631+0000","last_fullsized":"2024-04-23T14:32:14.639631+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:35.854078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.642125+0000","last_change":"2024-04-23T14:32:13.654800+0000","last_active":"2024-04-23T14:32:14.642125+0000","last_peered":"2024-04-23T14:32:14.642125+0000","last_clean":"2024-04-23T14:32:14.642125+0000","last_became_active":"2024-04-23T14:32:13.654191+0000","last_became_peered":"2024-04-23T14:32:13.654191+0000","last_unstale":"2024-04-23T14:32:14.642125+0000","last_undegraded":"2024-04-23T14:32:14.642125+0000","last_fullsized":"2024-04-23T14:32:14.642125+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:28.688885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181062+0000","last_change":"2024-04-23T14:32:09.623316+0000","last_active":"2024-04-23T14:32:15.181062+0000","last_peered":"2024-04-23T14:32:15.181062+0000","last_clean":"2024-04-23T14:32:15.181062+0000","last_became_active":"2024-04-23T14:32:09.621723+0000","last_became_peered":"2024-04-23T14:32:09.621723+0000","last_unstale":"2024-04-23T14:32:15.181062+0000","last_undegraded":"2024-04-23T14:32:15.181062+0000","last_fullsized":"2024-04-23T14:32:15.181062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:12.933947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181611+0000","last_change":"2024-04-23T14:32:08.609985+0000","last_active":"2024-04-23T14:32:15.181611+0000","last_peered":"2024-04-23T14:32:15.181611+0000","last_clean":"2024-04-23T14:32:15.181611+0000","last_became_active":"2024-04-23T14:32:08.609621+0000","last_became_peered":"2024-04-23T14:32:08.609621+0000","last_unstale":"2024-04-23T14:32:15.181611+0000","last_undegraded":"2024-04-23T14:32:15.181611+0000","last_fullsized":"2024-04-23T14:32:15.181611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:11.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641831+0000","last_change":"2024-04-23T14:32:08.609884+0000","last_active":"2024-04-23T14:32:14.641831+0000","last_peered":"2024-04-23T14:32:14.641831+0000","last_clean":"2024-04-23T14:32:14.641831+0000","last_became_active":"2024-04-23T14:32:08.607367+0000","last_became_peered":"2024-04-23T14:32:08.607367+0000","last_unstale":"2024-04-23T14:32:14.641831+0000","last_undegraded":"2024-04-23T14:32:14.641831+0000","last_fullsized":"2024-04-23T14:32:14.641831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:54.495261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640439+0000","last_change":"2024-04-23T14:32:09.632614+0000","last_active":"2024-04-23T14:32:14.640439+0000","last_peered":"2024-04-23T14:32:14.640439+0000","last_clean":"2024-04-23T14:32:14.640439+0000","last_became_active":"2024-04-23T14:32:09.632230+0000","last_became_peered":"2024-04-23T14:32:09.632230+0000","last_unstale":"2024-04-23T14:32:14.640439+0000","last_undegraded":"2024-04-23T14:32:14.640439+0000","last_fullsized":"2024-04-23T14:32:14.640439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:53.674486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181628+0000","last_change":"2024-04-23T14:32:13.671189+0000","last_active":"2024-04-23T14:32:15.181628+0000","last_peered":"2024-04-23T14:32:15.181628+0000","last_clean":"2024-04-23T14:32:15.181628+0000","last_became_active":"2024-04-23T14:32:13.670033+0000","last_became_peered":"2024-04-23T14:32:13.670033+0000","last_unstale":"2024-04-23T14:32:15.181628+0000","last_undegraded":"2024-04-23T14:32:15.181628+0000","last_fullsized":"2024-04-23T14:32:15.181628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:08:35.206423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181132+0000","last_change":"2024-04-23T14:32:08.612716+0000","last_active":"2024-04-23T14:32:15.181132+0000","last_peered":"2024-04-23T14:32:15.181132+0000","last_clean":"2024-04-23T14:32:15.181132+0000","last_became_active":"2024-04-23T14:32:08.612378+0000","last_became_peered":"2024-04-23T14:32:08.612378+0000","last_unstale":"2024-04-23T14:32:15.181132+0000","last_undegraded":"2024-04-23T14:32:15.181132+0000","last_fullsized":"2024-04-23T14:32:15.181132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:43:58.790056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151820+0000","last_change":"2024-04-23T14:32:09.620811+0000","last_active":"2024-04-23T14:32:15.151820+0000","last_peered":"2024-04-23T14:32:15.151820+0000","last_clean":"2024-04-23T14:32:15.151820+0000","last_became_active":"2024-04-23T14:32:09.620393+0000","last_became_peered":"2024-04-23T14:32:09.620393+0000","last_unstale":"2024-04-23T14:32:15.151820+0000","last_undegraded":"2024-04-23T14:32:15.151820+0000","last_fullsized":"2024-04-23T14:32:15.151820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:36.859905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181533+0000","last_change":"2024-04-23T14:32:13.670681+0000","last_active":"2024-04-23T14:32:15.181533+0000","last_peered":"2024-04-23T14:32:15.181533+0000","last_clean":"2024-04-23T14:32:15.181533+0000","last_became_active":"2024-04-23T14:32:13.670304+0000","last_became_peered":"2024-04-23T14:32:13.670304+0000","last_unstale":"2024-04-23T14:32:15.181533+0000","last_undegraded":"2024-04-23T14:32:15.181533+0000","last_fullsized":"2024-04-23T14:32:15.181533+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:35:37.731501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641029+0000","last_change":"2024-04-23T14:32:09.617499+0000","last_active":"2024-04-23T14:32:14.641029+0000","last_peered":"2024-04-23T14:32:14.641029+0000","last_clean":"2024-04-23T14:32:14.641029+0000","last_became_active":"2024-04-23T14:32:09.617154+0000","last_became_peered":"2024-04-23T14:32:09.617154+0000","last_unstale":"2024-04-23T14:32:14.641029+0000","last_undegraded":"2024-04-23T14:32:14.641029+0000","last_fullsized":"2024-04-23T14:32:14.641029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:43:49.773005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181553+0000","last_change":"2024-04-23T14:32:09.619214+0000","last_active":"2024-04-23T14:32:15.181553+0000","last_peered":"2024-04-23T14:32:15.181553+0000","last_clean":"2024-04-23T14:32:15.181553+0000","last_became_active":"2024-04-23T14:32:09.618943+0000","last_became_peered":"2024-04-23T14:32:09.618943+0000","last_unstale":"2024-04-23T14:32:15.181553+0000","last_undegraded":"2024-04-23T14:32:15.181553+0000","last_fullsized":"2024-04-23T14:32:15.181553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:12.536442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640531+0000","last_change":"2024-04-23T14:32:09.626499+0000","last_active":"2024-04-23T14:32:14.640531+0000","last_peered":"2024-04-23T14:32:14.640531+0000","last_clean":"2024-04-23T14:32:14.640531+0000","last_became_active":"2024-04-23T14:32:09.626070+0000","last_became_peered":"2024-04-23T14:32:09.626070+0000","last_unstale":"2024-04-23T14:32:14.640531+0000","last_undegraded":"2024-04-23T14:32:14.640531+0000","last_fullsized":"2024-04-23T14:32:14.640531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:50.876901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640295+0000","last_change":"2024-04-23T14:32:09.625391+0000","last_active":"2024-04-23T14:32:14.640295+0000","last_peered":"2024-04-23T14:32:14.640295+0000","last_clean":"2024-04-23T14:32:14.640295+0000","last_became_active":"2024-04-23T14:32:09.625049+0000","last_became_peered":"2024-04-23T14:32:09.625049+0000","last_unstale":"2024-04-23T14:32:14.640295+0000","last_undegraded":"2024-04-23T14:32:14.640295+0000","last_fullsized":"2024-04-23T14:32:14.640295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:52:42.867377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182436+0000","last_change":"2024-04-23T14:32:09.627850+0000","last_active":"2024-04-23T14:32:15.182436+0000","last_peered":"2024-04-23T14:32:15.182436+0000","last_clean":"2024-04-23T14:32:15.182436+0000","last_became_active":"2024-04-23T14:32:09.627603+0000","last_became_peered":"2024-04-23T14:32:09.627603+0000","last_unstale":"2024-04-23T14:32:15.182436+0000","last_undegraded":"2024-04-23T14:32:15.182436+0000","last_fullsized":"2024-04-23T14:32:15.182436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:09.066755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182767+0000","last_change":"2024-04-23T14:32:09.630409+0000","last_active":"2024-04-23T14:32:15.182767+0000","last_peered":"2024-04-23T14:32:15.182767+0000","last_clean":"2024-04-23T14:32:15.182767+0000","last_became_active":"2024-04-23T14:32:09.630100+0000","last_became_peered":"2024-04-23T14:32:09.630100+0000","last_unstale":"2024-04-23T14:32:15.182767+0000","last_undegraded":"2024-04-23T14:32:15.182767+0000","last_fullsized":"2024-04-23T14:32:15.182767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:03.128923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641996+0000","last_change":"2024-04-23T14:32:09.631521+0000","last_active":"2024-04-23T14:32:14.641996+0000","last_peered":"2024-04-23T14:32:14.641996+0000","last_clean":"2024-04-23T14:32:14.641996+0000","last_became_active":"2024-04-23T14:32:09.631291+0000","last_became_peered":"2024-04-23T14:32:09.631291+0000","last_unstale":"2024-04-23T14:32:14.641996+0000","last_undegraded":"2024-04-23T14:32:14.641996+0000","last_fullsized":"2024-04-23T14:32:14.641996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:12.933947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182097+0000","last_change":"2024-04-23T14:32:09.642262+0000","last_active":"2024-04-23T14:32:15.182097+0000","last_peered":"2024-04-23T14:32:15.182097+0000","last_clean":"2024-04-23T14:32:15.182097+0000","last_became_active":"2024-04-23T14:32:09.641982+0000","last_became_peered":"2024-04-23T14:32:09.641982+0000","last_unstale":"2024-04-23T14:32:15.182097+0000","last_undegraded":"2024-04-23T14:32:15.182097+0000","last_fullsized":"2024-04-23T14:32:15.182097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:52.459061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639502+0000","last_change":"2024-04-23T14:32:09.613189+0000","last_active":"2024-04-23T14:32:14.639502+0000","last_peered":"2024-04-23T14:32:14.639502+0000","last_clean":"2024-04-23T14:32:14.639502+0000","last_became_active":"2024-04-23T14:32:09.612846+0000","last_became_peered":"2024-04-23T14:32:09.612846+0000","last_unstale":"2024-04-23T14:32:14.639502+0000","last_undegraded":"2024-04-23T14:32:14.639502+0000","last_fullsized":"2024-04-23T14:32:14.639502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:03.128923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181523+0000","last_change":"2024-04-23T14:32:09.642268+0000","last_active":"2024-04-23T14:32:15.181523+0000","last_peered":"2024-04-23T14:32:15.181523+0000","last_clean":"2024-04-23T14:32:15.181523+0000","last_became_active":"2024-04-23T14:32:09.642007+0000","last_became_peered":"2024-04-23T14:32:09.642007+0000","last_unstale":"2024-04-23T14:32:15.181523+0000","last_undegraded":"2024-04-23T14:32:15.181523+0000","last_fullsized":"2024-04-23T14:32:15.181523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:50:24.043980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640425+0000","last_change":"2024-04-23T14:32:09.626670+0000","last_active":"2024-04-23T14:32:14.640425+0000","last_peered":"2024-04-23T14:32:14.640425+0000","last_clean":"2024-04-23T14:32:14.640425+0000","last_became_active":"2024-04-23T14:32:09.625568+0000","last_became_peered":"2024-04-23T14:32:09.625568+0000","last_unstale":"2024-04-23T14:32:14.640425+0000","last_undegraded":"2024-04-23T14:32:14.640425+0000","last_fullsized":"2024-04-23T14:32:14.640425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:09.066755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639884+0000","last_change":"2024-04-23T14:32:09.627089+0000","last_active":"2024-04-23T14:32:14.639884+0000","last_peered":"2024-04-23T14:32:14.639884+0000","last_clean":"2024-04-23T14:32:14.639884+0000","last_became_active":"2024-04-23T14:32:09.626429+0000","last_became_peered":"2024-04-23T14:32:09.626429+0000","last_unstale":"2024-04-23T14:32:14.639884+0000","last_undegraded":"2024-04-23T14:32:14.639884+0000","last_fullsized":"2024-04-23T14:32:14.639884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:43:49.773005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641487+0000","last_change":"2024-04-23T14:32:09.643824+0000","last_active":"2024-04-23T14:32:14.641487+0000","last_peered":"2024-04-23T14:32:14.641487+0000","last_clean":"2024-04-23T14:32:14.641487+0000","last_became_active":"2024-04-23T14:32:09.643454+0000","last_became_peered":"2024-04-23T14:32:09.643454+0000","last_unstale":"2024-04-23T14:32:14.641487+0000","last_undegraded":"2024-04-23T14:32:14.641487+0000","last_fullsized":"2024-04-23T14:32:14.641487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:03.128923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179997+0000","last_change":"2024-04-23T14:32:09.627274+0000","last_active":"2024-04-23T14:32:15.179997+0000","last_peered":"2024-04-23T14:32:15.179997+0000","last_clean":"2024-04-23T14:32:15.179997+0000","last_became_active":"2024-04-23T14:32:09.626999+0000","last_became_peered":"2024-04-23T14:32:09.626999+0000","last_unstale":"2024-04-23T14:32:15.179997+0000","last_undegraded":"2024-04-23T14:32:15.179997+0000","last_fullsized":"2024-04-23T14:32:15.179997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:12:19.437109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182768+0000","last_change":"2024-04-23T14:32:09.615554+0000","last_active":"2024-04-23T14:32:15.182768+0000","last_peered":"2024-04-23T14:32:15.182768+0000","last_clean":"2024-04-23T14:32:15.182768+0000","last_became_active":"2024-04-23T14:32:09.614793+0000","last_became_peered":"2024-04-23T14:32:09.614793+0000","last_unstale":"2024-04-23T14:32:15.182768+0000","last_undegraded":"2024-04-23T14:32:15.182768+0000","last_fullsized":"2024-04-23T14:32:15.182768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:01.062394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640359+0000","last_change":"2024-04-23T14:32:09.625443+0000","last_active":"2024-04-23T14:32:14.640359+0000","last_peered":"2024-04-23T14:32:14.640359+0000","last_clean":"2024-04-23T14:32:14.640359+0000","last_became_active":"2024-04-23T14:32:09.625140+0000","last_became_peered":"2024-04-23T14:32:09.625140+0000","last_unstale":"2024-04-23T14:32:14.640359+0000","last_undegraded":"2024-04-23T14:32:14.640359+0000","last_fullsized":"2024-04-23T14:32:14.640359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:12:19.437109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640042+0000","last_change":"2024-04-23T14:32:09.614187+0000","last_active":"2024-04-23T14:32:14.640042+0000","last_peered":"2024-04-23T14:32:14.640042+0000","last_clean":"2024-04-23T14:32:14.640042+0000","last_became_active":"2024-04-23T14:32:09.613813+0000","last_became_peered":"2024-04-23T14:32:09.613813+0000","last_unstale":"2024-04-23T14:32:14.640042+0000","last_undegraded":"2024-04-23T14:32:14.640042+0000","last_fullsized":"2024-04-23T14:32:14.640042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:12.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180552+0000","last_change":"2024-04-23T14:32:09.630660+0000","last_active":"2024-04-23T14:32:15.180552+0000","last_peered":"2024-04-23T14:32:15.180552+0000","last_clean":"2024-04-23T14:32:15.180552+0000","last_became_active":"2024-04-23T14:32:09.630381+0000","last_became_peered":"2024-04-23T14:32:09.630381+0000","last_unstale":"2024-04-23T14:32:15.180552+0000","last_undegraded":"2024-04-23T14:32:15.180552+0000","last_fullsized":"2024-04-23T14:32:15.180552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:12.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639936+0000","last_change":"2024-04-23T14:32:09.613974+0000","last_active":"2024-04-23T14:32:14.639936+0000","last_peered":"2024-04-23T14:32:14.639936+0000","last_clean":"2024-04-23T14:32:14.639936+0000","last_became_active":"2024-04-23T14:32:09.613699+0000","last_became_peered":"2024-04-23T14:32:09.613699+0000","last_unstale":"2024-04-23T14:32:14.639936+0000","last_undegraded":"2024-04-23T14:32:14.639936+0000","last_fullsized":"2024-04-23T14:32:14.639936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:08:31.155490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180930+0000","last_change":"2024-04-23T14:32:09.622174+0000","last_active":"2024-04-23T14:32:15.180930+0000","last_peered":"2024-04-23T14:32:15.180930+0000","last_clean":"2024-04-23T14:32:15.180930+0000","last_became_active":"2024-04-23T14:32:09.621360+0000","last_became_peered":"2024-04-23T14:32:09.621360+0000","last_unstale":"2024-04-23T14:32:15.180930+0000","last_undegraded":"2024-04-23T14:32:15.180930+0000","last_fullsized":"2024-04-23T14:32:15.180930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:12.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152292+0000","last_change":"2024-04-23T14:32:09.625529+0000","last_active":"2024-04-23T14:32:15.152292+0000","last_peered":"2024-04-23T14:32:15.152292+0000","last_clean":"2024-04-23T14:32:15.152292+0000","last_became_active":"2024-04-23T14:32:09.625259+0000","last_became_peered":"2024-04-23T14:32:09.625259+0000","last_unstale":"2024-04-23T14:32:15.152292+0000","last_undegraded":"2024-04-23T14:32:15.152292+0000","last_fullsized":"2024-04-23T14:32:15.152292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:12.536442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641776+0000","last_change":"2024-04-23T14:32:09.634616+0000","last_active":"2024-04-23T14:32:14.641776+0000","last_peered":"2024-04-23T14:32:14.641776+0000","last_clean":"2024-04-23T14:32:14.641776+0000","last_became_active":"2024-04-23T14:32:09.634328+0000","last_became_peered":"2024-04-23T14:32:09.634328+0000","last_unstale":"2024-04-23T14:32:14.641776+0000","last_undegraded":"2024-04-23T14:32:14.641776+0000","last_fullsized":"2024-04-23T14:32:14.641776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:52:42.867377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641980+0000","last_change":"2024-04-23T14:32:09.618704+0000","last_active":"2024-04-23T14:32:14.641980+0000","last_peered":"2024-04-23T14:32:14.641980+0000","last_clean":"2024-04-23T14:32:14.641980+0000","last_became_active":"2024-04-23T14:32:09.618328+0000","last_became_peered":"2024-04-23T14:32:09.618328+0000","last_unstale":"2024-04-23T14:32:14.641980+0000","last_undegraded":"2024-04-23T14:32:14.641980+0000","last_fullsized":"2024-04-23T14:32:14.641980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:50:24.043980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180068+0000","last_change":"2024-04-23T14:32:09.637988+0000","last_active":"2024-04-23T14:32:15.180068+0000","last_peered":"2024-04-23T14:32:15.180068+0000","last_clean":"2024-04-23T14:32:15.180068+0000","last_became_active":"2024-04-23T14:32:09.637766+0000","last_became_peered":"2024-04-23T14:32:09.637766+0000","last_unstale":"2024-04-23T14:32:15.180068+0000","last_undegraded":"2024-04-23T14:32:15.180068+0000","last_fullsized":"2024-04-23T14:32:15.180068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:09:07.994350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180933+0000","last_change":"2024-04-23T14:32:09.623580+0000","last_active":"2024-04-23T14:32:15.180933+0000","last_peered":"2024-04-23T14:32:15.180933+0000","last_clean":"2024-04-23T14:32:15.180933+0000","last_became_active":"2024-04-23T14:32:09.622942+0000","last_became_peered":"2024-04-23T14:32:09.622942+0000","last_unstale":"2024-04-23T14:32:15.180933+0000","last_undegraded":"2024-04-23T14:32:15.180933+0000","last_fullsized":"2024-04-23T14:32:15.180933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:52:42.867377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179379+0000","last_change":"2024-04-23T14:32:09.624523+0000","last_active":"2024-04-23T14:32:15.179379+0000","last_peered":"2024-04-23T14:32:15.179379+0000","last_clean":"2024-04-23T14:32:15.179379+0000","last_became_active":"2024-04-23T14:32:09.624229+0000","last_became_peered":"2024-04-23T14:32:09.624229+0000","last_unstale":"2024-04-23T14:32:15.179379+0000","last_undegraded":"2024-04-23T14:32:15.179379+0000","last_fullsized":"2024-04-23T14:32:15.179379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:09.066755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640175+0000","last_change":"2024-04-23T14:32:09.614929+0000","last_active":"2024-04-23T14:32:14.640175+0000","last_peered":"2024-04-23T14:32:14.640175+0000","last_clean":"2024-04-23T14:32:14.640175+0000","last_became_active":"2024-04-23T14:32:09.614584+0000","last_became_peered":"2024-04-23T14:32:09.614584+0000","last_unstale":"2024-04-23T14:32:14.640175+0000","last_undegraded":"2024-04-23T14:32:14.640175+0000","last_fullsized":"2024-04-23T14:32:14.640175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:22.220286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179498+0000","last_change":"2024-04-23T14:32:09.623580+0000","last_active":"2024-04-23T14:32:15.179498+0000","last_peered":"2024-04-23T14:32:15.179498+0000","last_clean":"2024-04-23T14:32:15.179498+0000","last_became_active":"2024-04-23T14:32:09.623197+0000","last_became_peered":"2024-04-23T14:32:09.623197+0000","last_unstale":"2024-04-23T14:32:15.179498+0000","last_undegraded":"2024-04-23T14:32:15.179498+0000","last_fullsized":"2024-04-23T14:32:15.179498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:52.459061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640154+0000","last_change":"2024-04-23T14:32:09.622573+0000","last_active":"2024-04-23T14:32:14.640154+0000","last_peered":"2024-04-23T14:32:14.640154+0000","last_clean":"2024-04-23T14:32:14.640154+0000","last_became_active":"2024-04-23T14:32:09.622232+0000","last_became_peered":"2024-04-23T14:32:09.622232+0000","last_unstale":"2024-04-23T14:32:14.640154+0000","last_undegraded":"2024-04-23T14:32:14.640154+0000","last_fullsized":"2024-04-23T14:32:14.640154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:52.459061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639881+0000","last_change":"2024-04-23T14:32:09.604425+0000","last_active":"2024-04-23T14:32:14.639881+0000","last_peered":"2024-04-23T14:32:14.639881+0000","last_clean":"2024-04-23T14:32:14.639881+0000","last_became_active":"2024-04-23T14:32:09.604159+0000","last_became_peered":"2024-04-23T14:32:09.604159+0000","last_unstale":"2024-04-23T14:32:14.639881+0000","last_undegraded":"2024-04-23T14:32:14.639881+0000","last_fullsized":"2024-04-23T14:32:14.639881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:50:24.043980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.183079+0000","last_change":"2024-04-23T14:32:09.614842+0000","last_active":"2024-04-23T14:32:15.183079+0000","last_peered":"2024-04-23T14:32:15.183079+0000","last_clean":"2024-04-23T14:32:15.183079+0000","last_became_active":"2024-04-23T14:32:09.614385+0000","last_became_peered":"2024-04-23T14:32:09.614385+0000","last_unstale":"2024-04-23T14:32:15.183079+0000","last_undegraded":"2024-04-23T14:32:15.183079+0000","last_fullsized":"2024-04-23T14:32:15.183079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:22.220286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179991+0000","last_change":"2024-04-23T14:32:09.628001+0000","last_active":"2024-04-23T14:32:15.179991+0000","last_peered":"2024-04-23T14:32:15.179991+0000","last_clean":"2024-04-23T14:32:15.179991+0000","last_became_active":"2024-04-23T14:32:09.627722+0000","last_became_peered":"2024-04-23T14:32:09.627722+0000","last_unstale":"2024-04-23T14:32:15.179991+0000","last_undegraded":"2024-04-23T14:32:15.179991+0000","last_fullsized":"2024-04-23T14:32:15.179991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:14:12.731737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:32:40.364 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:32:40.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.539+0000 7f0f24e00640 1 Processor -- start 2024-04-23T14:32:40.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.539+0000 7f0f24e00640 1 -- start start 2024-04-23T14:32:40.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.539+0000 7f0f24e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0f200cea60 0x7f0f200d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:40.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.539+0000 7f0f24e00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f200b1f90 con 0x7f0f200cd460 2024-04-23T14:32:40.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.539+0000 7f0f24e00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f200b2270 con 0x7f0f2006d450 2024-04-23T14:32:40.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.539+0000 7f0f24e00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f0f200b2550 con 0x7f0f200cea60 2024-04-23T14:32:40.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.539+0000 7f0f15400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0f200cea60 0x7f0f200d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:40.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.539+0000 7f0f1e800640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f0f200cd460 0x7f0f200b2d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:44974/0 (socket says 172.21.15.100:44974) 2024-04-23T14:32:40.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.539+0000 7f0f1e800640 1 -- 172.21.15.100:0/2061517634 learned_addr learned my addr 172.21.15.100:0/2061517634 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:40.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.539+0000 7f0f15400640 1 -- 172.21.15.100:0/2061517634 >> v1:172.21.15.151:6789/0 conn(0x7f0f2006d450 legacy=0x7f0f2006d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.539+0000 7f0f15400640 1 -- 172.21.15.100:0/2061517634 >> v1:172.21.15.100:6789/0 conn(0x7f0f200cd460 legacy=0x7f0f200b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.539+0000 7f0f15400640 1 -- 172.21.15.100:0/2061517634 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f200b2830 con 0x7f0f200cea60 2024-04-23T14:32:40.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.543+0000 7f0f15400640 1 --2- 172.21.15.100:0/2061517634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f200cea60 0x7f0f200d0300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0f080097c0 tx=0x7f0f0801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d0fee674dd620d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:40.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.543+0000 7f0f1d400640 1 -- 172.21.15.100:0/2061517634 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f080b0070 con 0x7f0f200cea60 2024-04-23T14:32:40.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.543+0000 7f0f1d400640 1 -- 172.21.15.100:0/2061517634 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f080ab600 con 0x7f0f200cea60 2024-04-23T14:32:40.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.543+0000 7f0f1d400640 1 -- 172.21.15.100:0/2061517634 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f080ab9d0 con 0x7f0f200cea60 2024-04-23T14:32:40.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.543+0000 7f0f24e00640 1 -- 172.21.15.100:0/2061517634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f200cea60 msgr2=0x7f0f200d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.543+0000 7f0f24e00640 1 --2- 172.21.15.100:0/2061517634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f200cea60 0x7f0f200d0300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0f080097c0 tx=0x7f0f0801c410 comp rx=0 tx=0).stop 2024-04-23T14:32:40.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.543+0000 7f0f24e00640 1 -- 172.21.15.100:0/2061517634 shutdown_connections 2024-04-23T14:32:40.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.543+0000 7f0f24e00640 1 --2- 172.21.15.100:0/2061517634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f200cea60 0x7f0f200d0300 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.543+0000 7f0f24e00640 1 -- 172.21.15.100:0/2061517634 >> 172.21.15.100:0/2061517634 conn(0x7f0f200cee80 msgr2=0x7f0f200cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:40.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.543+0000 7f0f24e00640 1 -- 172.21.15.100:0/2061517634 shutdown_connections 2024-04-23T14:32:40.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.543+0000 7f0f24e00640 1 -- 172.21.15.100:0/2061517634 wait complete. 2024-04-23T14:32:40.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.543+0000 7f0f24e00640 1 Processor -- start 2024-04-23T14:32:40.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f24e00640 1 -- start start 2024-04-23T14:32:40.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f24e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0f2006d450 0x7f0f201528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:40.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f24e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f200cd460 0x7f0f20159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:40.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f24e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f200cea60 0x7f0f2015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:40.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f24e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2016cd50 con 0x7f0f200cd460 2024-04-23T14:32:40.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f24e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2016cbd0 con 0x7f0f200cea60 2024-04-23T14:32:40.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f24e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0f2016ced0 con 0x7f0f2006d450 2024-04-23T14:32:40.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f15400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f200cd460 0x7f0f20159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:40.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f15400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f200cd460 0x7f0f20159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52988/0 (socket says 172.21.15.100:52988) 2024-04-23T14:32:40.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f15400640 1 -- 172.21.15.100:0/1680398707 learned_addr learned my addr 172.21.15.100:0/1680398707 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:40.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f1e800640 1 --2- 172.21.15.100:0/1680398707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f200cea60 0x7f0f2015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:40.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f1de00640 1 --2- 172.21.15.100:0/1680398707 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0f2006d450 0x7f0f201528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:40.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f15400640 1 -- 172.21.15.100:0/1680398707 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0f2006d450 msgr2=0x7f0f201528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f15400640 1 --2- 172.21.15.100:0/1680398707 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0f2006d450 0x7f0f201528e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f15400640 1 -- 172.21.15.100:0/1680398707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f200cea60 msgr2=0x7f0f2015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f15400640 1 --2- 172.21.15.100:0/1680398707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f200cea60 0x7f0f2015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f15400640 1 -- 172.21.15.100:0/1680398707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f20152e20 con 0x7f0f200cd460 2024-04-23T14:32:40.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f15400640 1 --2- 172.21.15.100:0/1680398707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f200cd460 0x7f0f20159b30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0f08019020 tx=0x7f0f080191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:40.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f16a00640 1 -- 172.21.15.100:0/1680398707 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f080b8030 con 0x7f0f200cd460 2024-04-23T14:32:40.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f16a00640 1 -- 172.21.15.100:0/1680398707 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f080b4410 con 0x7f0f200cd460 2024-04-23T14:32:40.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f24e00640 1 -- 172.21.15.100:0/1680398707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2015a510 con 0x7f0f200cd460 2024-04-23T14:32:40.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f16a00640 1 -- 172.21.15.100:0/1680398707 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f080b0040 con 0x7f0f200cd460 2024-04-23T14:32:40.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f24e00640 1 -- 172.21.15.100:0/1680398707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f2015aad0 con 0x7f0f200cd460 2024-04-23T14:32:40.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.547+0000 7f0f24e00640 1 -- 172.21.15.100:0/1680398707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f20068ad0 con 0x7f0f200cd460 2024-04-23T14:32:40.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.551+0000 7f0f16a00640 1 -- 172.21.15.100:0/1680398707 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0f080bb070 con 0x7f0f200cd460 2024-04-23T14:32:40.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.551+0000 7f0f16a00640 1 --2- 172.21.15.100:0/1680398707 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0efc068380 0x7f0efc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:40.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.551+0000 7f0f16a00640 1 -- 172.21.15.100:0/1680398707 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0f08120b00 con 0x7f0f200cd460 2024-04-23T14:32:40.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.551+0000 7f0f1de00640 1 --2- 172.21.15.100:0/1680398707 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0efc068380 0x7f0efc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:40.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.551+0000 7f0f1de00640 1 --2- 172.21.15.100:0/1680398707 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0efc068380 0x7f0efc06a840 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0f10001650 tx=0x7f0f10009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:40.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.555+0000 7f0f16a00640 1 -- 172.21.15.100:0/1680398707 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0f080c3020 con 0x7f0f200cd460 2024-04-23T14:32:40.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.711+0000 7f0f24e00640 1 -- 172.21.15.100:0/1680398707 --> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0f200cc7b0 con 0x7f0efc068380 2024-04-23T14:32:40.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.719+0000 7f0f16a00640 1 -- 172.21.15.100:0/1680398707 <== mgr.4101 v2:172.21.15.100:6832/2859853957 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332537 (secure 0 0 0) 0x7f0f200cc7b0 con 0x7f0efc068380 2024-04-23T14:32:40.722 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:40.725 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-04-23T14:32:40.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.723+0000 7f0f24e00640 1 -- 172.21.15.100:0/1680398707 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0efc068380 msgr2=0x7f0efc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.723+0000 7f0f24e00640 1 --2- 172.21.15.100:0/1680398707 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0efc068380 0x7f0efc06a840 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0f10001650 tx=0x7f0f10009040 comp rx=0 tx=0).stop 2024-04-23T14:32:40.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.723+0000 7f0f24e00640 1 -- 172.21.15.100:0/1680398707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f200cd460 msgr2=0x7f0f20159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.723+0000 7f0f24e00640 1 --2- 172.21.15.100:0/1680398707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f200cd460 0x7f0f20159b30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0f08019020 tx=0x7f0f080191b0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.723+0000 7f0f24e00640 1 -- 172.21.15.100:0/1680398707 shutdown_connections 2024-04-23T14:32:40.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.723+0000 7f0f24e00640 1 --2- 172.21.15.100:0/1680398707 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0efc068380 0x7f0efc06a840 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.723+0000 7f0f24e00640 1 --2- 172.21.15.100:0/1680398707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f200cea60 0x7f0f2015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.723+0000 7f0f24e00640 1 --2- 172.21.15.100:0/1680398707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f200cd460 0x7f0f20159b30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.723+0000 7f0f24e00640 1 --2- 172.21.15.100:0/1680398707 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0f2006d450 0x7f0f201528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.723+0000 7f0f24e00640 1 -- 172.21.15.100:0/1680398707 >> 172.21.15.100:0/1680398707 conn(0x7f0f200cee80 msgr2=0x7f0f2006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:40.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.723+0000 7f0f24e00640 1 -- 172.21.15.100:0/1680398707 shutdown_connections 2024-04-23T14:32:40.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.723+0000 7f0f24e00640 1 -- 172.21.15.100:0/1680398707 wait complete. 2024-04-23T14:32:40.742 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-23T14:32:38.741603+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219376,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755344,"statfs":{"total":773094113280,"available":772869472256,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002866"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640233+0000","last_change":"2024-04-23T14:32:08.611525+0000","last_active":"2024-04-23T14:32:14.640233+0000","last_peered":"2024-04-23T14:32:14.640233+0000","last_clean":"2024-04-23T14:32:14.640233+0000","last_became_active":"2024-04-23T14:32:08.611163+0000","last_became_peered":"2024-04-23T14:32:08.611163+0000","last_unstale":"2024-04-23T14:32:14.640233+0000","last_undegraded":"2024-04-23T14:32:14.640233+0000","last_fullsized":"2024-04-23T14:32:14.640233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:08:50.925677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181653+0000","last_change":"2024-04-23T14:32:09.625243+0000","last_active":"2024-04-23T14:32:15.181653+0000","last_peered":"2024-04-23T14:32:15.181653+0000","last_clean":"2024-04-23T14:32:15.181653+0000","last_became_active":"2024-04-23T14:32:09.624765+0000","last_became_peered":"2024-04-23T14:32:09.624765+0000","last_unstale":"2024-04-23T14:32:15.181653+0000","last_undegraded":"2024-04-23T14:32:15.181653+0000","last_fullsized":"2024-04-23T14:32:15.181653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:50.876901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181363+0000","last_change":"2024-04-23T14:32:13.779583+0000","last_active":"2024-04-23T14:32:15.181363+0000","last_peered":"2024-04-23T14:32:15.181363+0000","last_clean":"2024-04-23T14:32:15.181363+0000","last_became_active":"2024-04-23T14:32:13.779224+0000","last_became_peered":"2024-04-23T14:32:13.779224+0000","last_unstale":"2024-04-23T14:32:15.181363+0000","last_undegraded":"2024-04-23T14:32:15.181363+0000","last_fullsized":"2024-04-23T14:32:15.181363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:55.041747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181862+0000","last_change":"2024-04-23T14:32:08.587781+0000","last_active":"2024-04-23T14:32:15.181862+0000","last_peered":"2024-04-23T14:32:15.181862+0000","last_clean":"2024-04-23T14:32:15.181862+0000","last_became_active":"2024-04-23T14:32:08.587435+0000","last_became_peered":"2024-04-23T14:32:08.587435+0000","last_unstale":"2024-04-23T14:32:15.181862+0000","last_undegraded":"2024-04-23T14:32:15.181862+0000","last_fullsized":"2024-04-23T14:32:15.181862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:51.650822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640102+0000","last_change":"2024-04-23T14:32:09.621038+0000","last_active":"2024-04-23T14:32:14.640102+0000","last_peered":"2024-04-23T14:32:14.640102+0000","last_clean":"2024-04-23T14:32:14.640102+0000","last_became_active":"2024-04-23T14:32:09.620776+0000","last_became_peered":"2024-04-23T14:32:09.620776+0000","last_unstale":"2024-04-23T14:32:14.640102+0000","last_undegraded":"2024-04-23T14:32:14.640102+0000","last_fullsized":"2024-04-23T14:32:14.640102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:12.933947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151642+0000","last_change":"2024-04-23T14:32:13.671138+0000","last_active":"2024-04-23T14:32:15.151642+0000","last_peered":"2024-04-23T14:32:15.151642+0000","last_clean":"2024-04-23T14:32:15.151642+0000","last_became_active":"2024-04-23T14:32:13.670514+0000","last_became_peered":"2024-04-23T14:32:13.670514+0000","last_unstale":"2024-04-23T14:32:15.151642+0000","last_undegraded":"2024-04-23T14:32:15.151642+0000","last_fullsized":"2024-04-23T14:32:15.151642+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:26.271219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179964+0000","last_change":"2024-04-23T14:32:08.616316+0000","last_active":"2024-04-23T14:32:15.179964+0000","last_peered":"2024-04-23T14:32:15.179964+0000","last_clean":"2024-04-23T14:32:15.179964+0000","last_became_active":"2024-04-23T14:32:08.616020+0000","last_became_peered":"2024-04-23T14:32:08.616020+0000","last_unstale":"2024-04-23T14:32:15.179964+0000","last_undegraded":"2024-04-23T14:32:15.179964+0000","last_fullsized":"2024-04-23T14:32:15.179964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:53:40.004436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640291+0000","last_change":"2024-04-23T14:32:09.635504+0000","last_active":"2024-04-23T14:32:14.640291+0000","last_peered":"2024-04-23T14:32:14.640291+0000","last_clean":"2024-04-23T14:32:14.640291+0000","last_became_active":"2024-04-23T14:32:09.635179+0000","last_became_peered":"2024-04-23T14:32:09.635179+0000","last_unstale":"2024-04-23T14:32:14.640291+0000","last_undegraded":"2024-04-23T14:32:14.640291+0000","last_fullsized":"2024-04-23T14:32:14.640291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:56.096479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151795+0000","last_change":"2024-04-23T14:32:13.667280+0000","last_active":"2024-04-23T14:32:15.151795+0000","last_peered":"2024-04-23T14:32:15.151795+0000","last_clean":"2024-04-23T14:32:15.151795+0000","last_became_active":"2024-04-23T14:32:13.666861+0000","last_became_peered":"2024-04-23T14:32:13.666861+0000","last_unstale":"2024-04-23T14:32:15.151795+0000","last_undegraded":"2024-04-23T14:32:15.151795+0000","last_fullsized":"2024-04-23T14:32:15.151795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:56.509994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179707+0000","last_change":"2024-04-23T14:32:08.615668+0000","last_active":"2024-04-23T14:32:15.179707+0000","last_peered":"2024-04-23T14:32:15.179707+0000","last_clean":"2024-04-23T14:32:15.179707+0000","last_became_active":"2024-04-23T14:32:08.615384+0000","last_became_peered":"2024-04-23T14:32:08.615384+0000","last_unstale":"2024-04-23T14:32:15.179707+0000","last_undegraded":"2024-04-23T14:32:15.179707+0000","last_fullsized":"2024-04-23T14:32:15.179707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:11.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640191+0000","last_change":"2024-04-23T14:32:09.634014+0000","last_active":"2024-04-23T14:32:14.640191+0000","last_peered":"2024-04-23T14:32:14.640191+0000","last_clean":"2024-04-23T14:32:14.640191+0000","last_became_active":"2024-04-23T14:32:09.633633+0000","last_became_peered":"2024-04-23T14:32:09.633633+0000","last_unstale":"2024-04-23T14:32:14.640191+0000","last_undegraded":"2024-04-23T14:32:14.640191+0000","last_fullsized":"2024-04-23T14:32:14.640191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:47:31.805399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151732+0000","last_change":"2024-04-23T14:32:13.661944+0000","last_active":"2024-04-23T14:32:15.151732+0000","last_peered":"2024-04-23T14:32:15.151732+0000","last_clean":"2024-04-23T14:32:15.151732+0000","last_became_active":"2024-04-23T14:32:13.661584+0000","last_became_peered":"2024-04-23T14:32:13.661584+0000","last_unstale":"2024-04-23T14:32:15.151732+0000","last_undegraded":"2024-04-23T14:32:15.151732+0000","last_fullsized":"2024-04-23T14:32:15.151732+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:54.927834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641604+0000","last_change":"2024-04-23T14:32:09.645728+0000","last_active":"2024-04-23T14:32:14.641604+0000","last_peered":"2024-04-23T14:32:14.641604+0000","last_clean":"2024-04-23T14:32:14.641604+0000","last_became_active":"2024-04-23T14:32:09.641089+0000","last_became_peered":"2024-04-23T14:32:09.641089+0000","last_unstale":"2024-04-23T14:32:14.641604+0000","last_undegraded":"2024-04-23T14:32:14.641604+0000","last_fullsized":"2024-04-23T14:32:14.641604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:53.674486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179794+0000","last_change":"2024-04-23T14:32:08.616537+0000","last_active":"2024-04-23T14:32:15.179794+0000","last_peered":"2024-04-23T14:32:15.179794+0000","last_clean":"2024-04-23T14:32:15.179794+0000","last_became_active":"2024-04-23T14:32:08.616145+0000","last_became_peered":"2024-04-23T14:32:08.616145+0000","last_unstale":"2024-04-23T14:32:15.179794+0000","last_undegraded":"2024-04-23T14:32:15.179794+0000","last_fullsized":"2024-04-23T14:32:15.179794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:00.056567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152246+0000","last_change":"2024-04-23T14:32:13.667224+0000","last_active":"2024-04-23T14:32:15.152246+0000","last_peered":"2024-04-23T14:32:15.152246+0000","last_clean":"2024-04-23T14:32:15.152246+0000","last_became_active":"2024-04-23T14:32:13.666816+0000","last_became_peered":"2024-04-23T14:32:13.666816+0000","last_unstale":"2024-04-23T14:32:15.152246+0000","last_undegraded":"2024-04-23T14:32:15.152246+0000","last_fullsized":"2024-04-23T14:32:15.152246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:22.418010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181037+0000","last_change":"2024-04-23T14:32:09.622489+0000","last_active":"2024-04-23T14:32:15.181037+0000","last_peered":"2024-04-23T14:32:15.181037+0000","last_clean":"2024-04-23T14:32:15.181037+0000","last_became_active":"2024-04-23T14:32:09.617206+0000","last_became_peered":"2024-04-23T14:32:09.617206+0000","last_unstale":"2024-04-23T14:32:15.181037+0000","last_undegraded":"2024-04-23T14:32:15.181037+0000","last_fullsized":"2024-04-23T14:32:15.181037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:03.128923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182693+0000","last_change":"2024-04-23T14:32:08.600628+0000","last_active":"2024-04-23T14:32:15.182693+0000","last_peered":"2024-04-23T14:32:15.182693+0000","last_clean":"2024-04-23T14:32:15.182693+0000","last_became_active":"2024-04-23T14:32:08.600318+0000","last_became_peered":"2024-04-23T14:32:08.600318+0000","last_unstale":"2024-04-23T14:32:15.182693+0000","last_undegraded":"2024-04-23T14:32:15.182693+0000","last_fullsized":"2024-04-23T14:32:15.182693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:54.495261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152192+0000","last_change":"2024-04-23T14:32:13.661586+0000","last_active":"2024-04-23T14:32:15.152192+0000","last_peered":"2024-04-23T14:32:15.152192+0000","last_clean":"2024-04-23T14:32:15.152192+0000","last_became_active":"2024-04-23T14:32:13.660406+0000","last_became_peered":"2024-04-23T14:32:13.660406+0000","last_unstale":"2024-04-23T14:32:15.152192+0000","last_undegraded":"2024-04-23T14:32:15.152192+0000","last_fullsized":"2024-04-23T14:32:15.152192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:47:35.856332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181164+0000","last_change":"2024-04-23T14:32:09.627921+0000","last_active":"2024-04-23T14:32:15.181164+0000","last_peered":"2024-04-23T14:32:15.181164+0000","last_clean":"2024-04-23T14:32:15.181164+0000","last_became_active":"2024-04-23T14:32:09.622420+0000","last_became_peered":"2024-04-23T14:32:09.622420+0000","last_unstale":"2024-04-23T14:32:15.181164+0000","last_undegraded":"2024-04-23T14:32:15.181164+0000","last_fullsized":"2024-04-23T14:32:15.181164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:53.674486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180797+0000","last_change":"2024-04-23T14:32:08.612306+0000","last_active":"2024-04-23T14:32:15.180797+0000","last_peered":"2024-04-23T14:32:15.180797+0000","last_clean":"2024-04-23T14:32:15.180797+0000","last_became_active":"2024-04-23T14:32:08.612018+0000","last_became_peered":"2024-04-23T14:32:08.612018+0000","last_unstale":"2024-04-23T14:32:15.180797+0000","last_undegraded":"2024-04-23T14:32:15.180797+0000","last_fullsized":"2024-04-23T14:32:15.180797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:59:03.639740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.183304+0000","last_change":"2024-04-23T14:32:13.780668+0000","last_active":"2024-04-23T14:32:15.183304+0000","last_peered":"2024-04-23T14:32:15.183304+0000","last_clean":"2024-04-23T14:32:15.183304+0000","last_became_active":"2024-04-23T14:32:13.780323+0000","last_became_peered":"2024-04-23T14:32:13.780323+0000","last_unstale":"2024-04-23T14:32:15.183304+0000","last_undegraded":"2024-04-23T14:32:15.183304+0000","last_fullsized":"2024-04-23T14:32:15.183304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:12:23.488042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151992+0000","last_change":"2024-04-23T14:32:09.627663+0000","last_active":"2024-04-23T14:32:15.151992+0000","last_peered":"2024-04-23T14:32:15.151992+0000","last_clean":"2024-04-23T14:32:15.151992+0000","last_became_active":"2024-04-23T14:32:09.620464+0000","last_became_peered":"2024-04-23T14:32:09.620464+0000","last_unstale":"2024-04-23T14:32:15.151992+0000","last_undegraded":"2024-04-23T14:32:15.151992+0000","last_fullsized":"2024-04-23T14:32:15.151992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:52.656649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179300+0000","last_change":"2024-04-23T14:32:08.612870+0000","last_active":"2024-04-23T14:32:15.179300+0000","last_peered":"2024-04-23T14:32:15.179300+0000","last_clean":"2024-04-23T14:32:15.179300+0000","last_became_active":"2024-04-23T14:32:08.610830+0000","last_became_peered":"2024-04-23T14:32:08.610830+0000","last_unstale":"2024-04-23T14:32:15.179300+0000","last_undegraded":"2024-04-23T14:32:15.179300+0000","last_fullsized":"2024-04-23T14:32:15.179300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:51.650822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152268+0000","last_change":"2024-04-23T14:32:13.660771+0000","last_active":"2024-04-23T14:32:15.152268+0000","last_peered":"2024-04-23T14:32:15.152268+0000","last_clean":"2024-04-23T14:32:15.152268+0000","last_became_active":"2024-04-23T14:32:13.660291+0000","last_became_peered":"2024-04-23T14:32:13.660291+0000","last_unstale":"2024-04-23T14:32:15.152268+0000","last_undegraded":"2024-04-23T14:32:15.152268+0000","last_fullsized":"2024-04-23T14:32:15.152268+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:28.688885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640262+0000","last_change":"2024-04-23T14:32:09.612595+0000","last_active":"2024-04-23T14:32:14.640262+0000","last_peered":"2024-04-23T14:32:14.640262+0000","last_clean":"2024-04-23T14:32:14.640262+0000","last_became_active":"2024-04-23T14:32:09.612168+0000","last_became_peered":"2024-04-23T14:32:09.612168+0000","last_unstale":"2024-04-23T14:32:14.640262+0000","last_undegraded":"2024-04-23T14:32:14.640262+0000","last_fullsized":"2024-04-23T14:32:14.640262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:03.128923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.642236+0000","last_change":"2024-04-23T14:32:08.615099+0000","last_active":"2024-04-23T14:32:14.642236+0000","last_peered":"2024-04-23T14:32:14.642236+0000","last_clean":"2024-04-23T14:32:14.642236+0000","last_became_active":"2024-04-23T14:32:08.614350+0000","last_became_peered":"2024-04-23T14:32:08.614350+0000","last_unstale":"2024-04-23T14:32:14.642236+0000","last_undegraded":"2024-04-23T14:32:14.642236+0000","last_fullsized":"2024-04-23T14:32:14.642236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:59:03.639740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151716+0000","last_change":"2024-04-23T14:32:13.775674+0000","last_active":"2024-04-23T14:32:15.151716+0000","last_peered":"2024-04-23T14:32:15.151716+0000","last_clean":"2024-04-23T14:32:15.151716+0000","last_became_active":"2024-04-23T14:32:13.774695+0000","last_became_peered":"2024-04-23T14:32:13.774695+0000","last_unstale":"2024-04-23T14:32:15.151716+0000","last_undegraded":"2024-04-23T14:32:15.151716+0000","last_fullsized":"2024-04-23T14:32:15.151716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:13.117688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182184+0000","last_change":"2024-04-23T14:32:09.630421+0000","last_active":"2024-04-23T14:32:15.182184+0000","last_peered":"2024-04-23T14:32:15.182184+0000","last_clean":"2024-04-23T14:32:15.182184+0000","last_became_active":"2024-04-23T14:32:09.630104+0000","last_became_peered":"2024-04-23T14:32:09.630104+0000","last_unstale":"2024-04-23T14:32:15.182184+0000","last_undegraded":"2024-04-23T14:32:15.182184+0000","last_fullsized":"2024-04-23T14:32:15.182184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:56.096479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.183107+0000","last_change":"2024-04-23T14:32:08.607576+0000","last_active":"2024-04-23T14:32:15.183107+0000","last_peered":"2024-04-23T14:32:15.183107+0000","last_clean":"2024-04-23T14:32:15.183107+0000","last_became_active":"2024-04-23T14:32:08.606611+0000","last_became_peered":"2024-04-23T14:32:08.606611+0000","last_unstale":"2024-04-23T14:32:15.183107+0000","last_undegraded":"2024-04-23T14:32:15.183107+0000","last_fullsized":"2024-04-23T14:32:15.183107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:00.056567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180827+0000","last_change":"2024-04-23T14:32:13.778406+0000","last_active":"2024-04-23T14:32:15.180827+0000","last_peered":"2024-04-23T14:32:15.180827+0000","last_clean":"2024-04-23T14:32:15.180827+0000","last_became_active":"2024-04-23T14:32:13.778034+0000","last_became_peered":"2024-04-23T14:32:13.778034+0000","last_unstale":"2024-04-23T14:32:15.180827+0000","last_undegraded":"2024-04-23T14:32:15.180827+0000","last_fullsized":"2024-04-23T14:32:15.180827+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:21:36.288246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181961+0000","last_change":"2024-04-23T14:32:09.613694+0000","last_active":"2024-04-23T14:32:15.181961+0000","last_peered":"2024-04-23T14:32:15.181961+0000","last_clean":"2024-04-23T14:32:15.181961+0000","last_became_active":"2024-04-23T14:32:09.613413+0000","last_became_peered":"2024-04-23T14:32:09.613413+0000","last_unstale":"2024-04-23T14:32:15.181961+0000","last_undegraded":"2024-04-23T14:32:15.181961+0000","last_fullsized":"2024-04-23T14:32:15.181961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:12.933947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180126+0000","last_change":"2024-04-23T14:32:08.612923+0000","last_active":"2024-04-23T14:32:15.180126+0000","last_peered":"2024-04-23T14:32:15.180126+0000","last_clean":"2024-04-23T14:32:15.180126+0000","last_became_active":"2024-04-23T14:32:08.610847+0000","last_became_peered":"2024-04-23T14:32:08.610847+0000","last_unstale":"2024-04-23T14:32:15.180126+0000","last_undegraded":"2024-04-23T14:32:15.180126+0000","last_fullsized":"2024-04-23T14:32:15.180126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:40:25.063520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640453+0000","last_change":"2024-04-23T14:32:13.775013+0000","last_active":"2024-04-23T14:32:14.640453+0000","last_peered":"2024-04-23T14:32:14.640453+0000","last_clean":"2024-04-23T14:32:14.640453+0000","last_became_active":"2024-04-23T14:32:13.774565+0000","last_became_peered":"2024-04-23T14:32:13.774565+0000","last_unstale":"2024-04-23T14:32:14.640453+0000","last_undegraded":"2024-04-23T14:32:14.640453+0000","last_fullsized":"2024-04-23T14:32:14.640453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:26:08.003427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640083+0000","last_change":"2024-04-23T14:32:09.633886+0000","last_active":"2024-04-23T14:32:14.640083+0000","last_peered":"2024-04-23T14:32:14.640083+0000","last_clean":"2024-04-23T14:32:14.640083+0000","last_became_active":"2024-04-23T14:32:09.633258+0000","last_became_peered":"2024-04-23T14:32:09.633258+0000","last_unstale":"2024-04-23T14:32:14.640083+0000","last_undegraded":"2024-04-23T14:32:14.640083+0000","last_fullsized":"2024-04-23T14:32:14.640083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:24.637952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182012+0000","last_change":"2024-04-23T14:32:08.602248+0000","last_active":"2024-04-23T14:32:15.182012+0000","last_peered":"2024-04-23T14:32:15.182012+0000","last_clean":"2024-04-23T14:32:15.182012+0000","last_became_active":"2024-04-23T14:32:08.601913+0000","last_became_peered":"2024-04-23T14:32:08.601913+0000","last_unstale":"2024-04-23T14:32:15.182012+0000","last_undegraded":"2024-04-23T14:32:15.182012+0000","last_fullsized":"2024-04-23T14:32:15.182012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:11.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180987+0000","last_change":"2024-04-23T14:32:13.666952+0000","last_active":"2024-04-23T14:32:15.180987+0000","last_peered":"2024-04-23T14:32:15.180987+0000","last_clean":"2024-04-23T14:32:15.180987+0000","last_became_active":"2024-04-23T14:32:13.663335+0000","last_became_peered":"2024-04-23T14:32:13.663335+0000","last_unstale":"2024-04-23T14:32:15.180987+0000","last_undegraded":"2024-04-23T14:32:15.180987+0000","last_fullsized":"2024-04-23T14:32:15.180987+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:22.418010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152000+0000","last_change":"2024-04-23T14:32:09.627852+0000","last_active":"2024-04-23T14:32:15.152000+0000","last_peered":"2024-04-23T14:32:15.152000+0000","last_clean":"2024-04-23T14:32:15.152000+0000","last_became_active":"2024-04-23T14:32:09.627575+0000","last_became_peered":"2024-04-23T14:32:09.627575+0000","last_unstale":"2024-04-23T14:32:15.152000+0000","last_undegraded":"2024-04-23T14:32:15.152000+0000","last_fullsized":"2024-04-23T14:32:15.152000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:55.501089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640731+0000","last_change":"2024-04-23T14:32:08.603753+0000","last_active":"2024-04-23T14:32:14.640731+0000","last_peered":"2024-04-23T14:32:14.640731+0000","last_clean":"2024-04-23T14:32:14.640731+0000","last_became_active":"2024-04-23T14:32:08.602944+0000","last_became_peered":"2024-04-23T14:32:08.602944+0000","last_unstale":"2024-04-23T14:32:14.640731+0000","last_undegraded":"2024-04-23T14:32:14.640731+0000","last_fullsized":"2024-04-23T14:32:14.640731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:54.495261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181918+0000","last_change":"2024-04-23T14:32:13.667268+0000","last_active":"2024-04-23T14:32:15.181918+0000","last_peered":"2024-04-23T14:32:15.181918+0000","last_clean":"2024-04-23T14:32:15.181918+0000","last_became_active":"2024-04-23T14:32:13.666836+0000","last_became_peered":"2024-04-23T14:32:13.666836+0000","last_unstale":"2024-04-23T14:32:15.181918+0000","last_undegraded":"2024-04-23T14:32:15.181918+0000","last_fullsized":"2024-04-23T14:32:15.181918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:14:16.782670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180774+0000","last_change":"2024-04-23T14:32:09.613896+0000","last_active":"2024-04-23T14:32:15.180774+0000","last_peered":"2024-04-23T14:32:15.180774+0000","last_clean":"2024-04-23T14:32:15.180774+0000","last_became_active":"2024-04-23T14:32:09.613535+0000","last_became_peered":"2024-04-23T14:32:09.613535+0000","last_unstale":"2024-04-23T14:32:15.180774+0000","last_undegraded":"2024-04-23T14:32:15.180774+0000","last_fullsized":"2024-04-23T14:32:15.180774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:50:24.043980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181650+0000","last_change":"2024-04-23T14:32:08.606268+0000","last_active":"2024-04-23T14:32:15.181650+0000","last_peered":"2024-04-23T14:32:15.181650+0000","last_clean":"2024-04-23T14:32:15.181650+0000","last_became_active":"2024-04-23T14:32:08.605958+0000","last_became_peered":"2024-04-23T14:32:08.605958+0000","last_unstale":"2024-04-23T14:32:15.181650+0000","last_undegraded":"2024-04-23T14:32:15.181650+0000","last_fullsized":"2024-04-23T14:32:15.181650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:45:45.243948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.183441+0000","last_change":"2024-04-23T14:32:13.664183+0000","last_active":"2024-04-23T14:32:15.183441+0000","last_peered":"2024-04-23T14:32:15.183441+0000","last_clean":"2024-04-23T14:32:15.183441+0000","last_became_active":"2024-04-23T14:32:13.663836+0000","last_became_peered":"2024-04-23T14:32:13.663836+0000","last_unstale":"2024-04-23T14:32:15.183441+0000","last_undegraded":"2024-04-23T14:32:15.183441+0000","last_fullsized":"2024-04-23T14:32:15.183441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:09:12.045283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639679+0000","last_change":"2024-04-23T14:32:09.633686+0000","last_active":"2024-04-23T14:32:14.639679+0000","last_peered":"2024-04-23T14:32:14.639679+0000","last_clean":"2024-04-23T14:32:14.639679+0000","last_became_active":"2024-04-23T14:32:09.631825+0000","last_became_peered":"2024-04-23T14:32:09.631825+0000","last_unstale":"2024-04-23T14:32:14.639679+0000","last_undegraded":"2024-04-23T14:32:14.639679+0000","last_fullsized":"2024-04-23T14:32:14.639679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:23:57.869625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179303+0000","last_change":"2024-04-23T14:32:08.613375+0000","last_active":"2024-04-23T14:32:15.179303+0000","last_peered":"2024-04-23T14:32:15.179303+0000","last_clean":"2024-04-23T14:32:15.179303+0000","last_became_active":"2024-04-23T14:32:08.613129+0000","last_became_peered":"2024-04-23T14:32:08.613129+0000","last_unstale":"2024-04-23T14:32:15.179303+0000","last_undegraded":"2024-04-23T14:32:15.179303+0000","last_fullsized":"2024-04-23T14:32:15.179303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:54.495261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640202+0000","last_change":"2024-04-23T14:32:13.779707+0000","last_active":"2024-04-23T14:32:14.640202+0000","last_peered":"2024-04-23T14:32:14.640202+0000","last_clean":"2024-04-23T14:32:14.640202+0000","last_became_active":"2024-04-23T14:32:13.779357+0000","last_became_peered":"2024-04-23T14:32:13.779357+0000","last_unstale":"2024-04-23T14:32:14.640202+0000","last_undegraded":"2024-04-23T14:32:14.640202+0000","last_fullsized":"2024-04-23T14:32:14.640202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:12:23.488042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181182+0000","last_change":"2024-04-23T14:32:09.621797+0000","last_active":"2024-04-23T14:32:15.181182+0000","last_peered":"2024-04-23T14:32:15.181182+0000","last_clean":"2024-04-23T14:32:15.181182+0000","last_became_active":"2024-04-23T14:32:09.618825+0000","last_became_peered":"2024-04-23T14:32:09.618825+0000","last_unstale":"2024-04-23T14:32:15.181182+0000","last_undegraded":"2024-04-23T14:32:15.181182+0000","last_fullsized":"2024-04-23T14:32:15.181182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:22.220286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639965+0000","last_change":"2024-04-23T14:32:08.603338+0000","last_active":"2024-04-23T14:32:14.639965+0000","last_peered":"2024-04-23T14:32:14.639965+0000","last_clean":"2024-04-23T14:32:14.639965+0000","last_became_active":"2024-04-23T14:32:08.602563+0000","last_became_peered":"2024-04-23T14:32:08.602563+0000","last_unstale":"2024-04-23T14:32:14.639965+0000","last_undegraded":"2024-04-23T14:32:14.639965+0000","last_fullsized":"2024-04-23T14:32:14.639965+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:45:45.243948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179439+0000","last_change":"2024-04-23T14:31:58.314235+0000","last_active":"2024-04-23T14:32:15.179439+0000","last_peered":"2024-04-23T14:32:15.179439+0000","last_clean":"2024-04-23T14:32:15.179439+0000","last_became_active":"2024-04-23T14:31:58.313165+0000","last_became_peered":"2024-04-23T14:31:58.313165+0000","last_unstale":"2024-04-23T14:32:15.179439+0000","last_undegraded":"2024-04-23T14:32:15.179439+0000","last_fullsized":"2024-04-23T14:32:15.179439+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_clean_scrub_stamp":"2024-04-23T14:31:57.140712+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:44.170436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641636+0000","last_change":"2024-04-23T14:32:13.777292+0000","last_active":"2024-04-23T14:32:14.641636+0000","last_peered":"2024-04-23T14:32:14.641636+0000","last_clean":"2024-04-23T14:32:14.641636+0000","last_became_active":"2024-04-23T14:32:13.776933+0000","last_became_peered":"2024-04-23T14:32:13.776933+0000","last_unstale":"2024-04-23T14:32:14.641636+0000","last_undegraded":"2024-04-23T14:32:14.641636+0000","last_fullsized":"2024-04-23T14:32:14.641636+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:47:35.856332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179793+0000","last_change":"2024-04-23T14:32:09.639015+0000","last_active":"2024-04-23T14:32:15.179793+0000","last_peered":"2024-04-23T14:32:15.179793+0000","last_clean":"2024-04-23T14:32:15.179793+0000","last_became_active":"2024-04-23T14:32:09.638622+0000","last_became_peered":"2024-04-23T14:32:09.638622+0000","last_unstale":"2024-04-23T14:32:15.179793+0000","last_undegraded":"2024-04-23T14:32:15.179793+0000","last_fullsized":"2024-04-23T14:32:15.179793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:18.367077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181794+0000","last_change":"2024-04-23T14:32:08.607926+0000","last_active":"2024-04-23T14:32:15.181794+0000","last_peered":"2024-04-23T14:32:15.181794+0000","last_clean":"2024-04-23T14:32:15.181794+0000","last_became_active":"2024-04-23T14:32:08.607649+0000","last_became_peered":"2024-04-23T14:32:08.607649+0000","last_unstale":"2024-04-23T14:32:15.181794+0000","last_undegraded":"2024-04-23T14:32:15.181794+0000","last_fullsized":"2024-04-23T14:32:15.181794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:59:03.639740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641732+0000","last_change":"2024-04-23T14:32:13.774449+0000","last_active":"2024-04-23T14:32:14.641732+0000","last_peered":"2024-04-23T14:32:14.641732+0000","last_clean":"2024-04-23T14:32:14.641732+0000","last_became_active":"2024-04-23T14:32:13.773937+0000","last_became_peered":"2024-04-23T14:32:13.773937+0000","last_unstale":"2024-04-23T14:32:14.641732+0000","last_undegraded":"2024-04-23T14:32:14.641732+0000","last_fullsized":"2024-04-23T14:32:14.641732+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:22.418010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640399+0000","last_change":"2024-04-23T14:32:09.626405+0000","last_active":"2024-04-23T14:32:14.640399+0000","last_peered":"2024-04-23T14:32:14.640399+0000","last_clean":"2024-04-23T14:32:14.640399+0000","last_became_active":"2024-04-23T14:32:09.626056+0000","last_became_peered":"2024-04-23T14:32:09.626056+0000","last_unstale":"2024-04-23T14:32:14.640399+0000","last_undegraded":"2024-04-23T14:32:14.640399+0000","last_fullsized":"2024-04-23T14:32:14.640399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:18.367077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182898+0000","last_change":"2024-04-23T14:32:08.608148+0000","last_active":"2024-04-23T14:32:15.182898+0000","last_peered":"2024-04-23T14:32:15.182898+0000","last_clean":"2024-04-23T14:32:15.182898+0000","last_became_active":"2024-04-23T14:32:08.607623+0000","last_became_peered":"2024-04-23T14:32:08.607623+0000","last_unstale":"2024-04-23T14:32:15.182898+0000","last_undegraded":"2024-04-23T14:32:15.182898+0000","last_fullsized":"2024-04-23T14:32:15.182898+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:51.650822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640246+0000","last_change":"2024-04-23T14:32:13.663295+0000","last_active":"2024-04-23T14:32:14.640246+0000","last_peered":"2024-04-23T14:32:14.640246+0000","last_clean":"2024-04-23T14:32:14.640246+0000","last_became_active":"2024-04-23T14:32:13.662900+0000","last_became_peered":"2024-04-23T14:32:13.662900+0000","last_unstale":"2024-04-23T14:32:14.640246+0000","last_undegraded":"2024-04-23T14:32:14.640246+0000","last_fullsized":"2024-04-23T14:32:14.640246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:09:12.045283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180115+0000","last_change":"2024-04-23T14:32:09.641393+0000","last_active":"2024-04-23T14:32:15.180115+0000","last_peered":"2024-04-23T14:32:15.180115+0000","last_clean":"2024-04-23T14:32:15.180115+0000","last_became_active":"2024-04-23T14:32:09.641075+0000","last_became_peered":"2024-04-23T14:32:09.641075+0000","last_unstale":"2024-04-23T14:32:15.180115+0000","last_undegraded":"2024-04-23T14:32:15.180115+0000","last_fullsized":"2024-04-23T14:32:15.180115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:26:03.952494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182813+0000","last_change":"2024-04-23T14:32:08.607178+0000","last_active":"2024-04-23T14:32:15.182813+0000","last_peered":"2024-04-23T14:32:15.182813+0000","last_clean":"2024-04-23T14:32:15.182813+0000","last_became_active":"2024-04-23T14:32:08.606863+0000","last_became_peered":"2024-04-23T14:32:08.606863+0000","last_unstale":"2024-04-23T14:32:15.182813+0000","last_undegraded":"2024-04-23T14:32:15.182813+0000","last_fullsized":"2024-04-23T14:32:15.182813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:35.854078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639398+0000","last_change":"2024-04-23T14:32:13.773533+0000","last_active":"2024-04-23T14:32:14.639398+0000","last_peered":"2024-04-23T14:32:14.639398+0000","last_clean":"2024-04-23T14:32:14.639398+0000","last_became_active":"2024-04-23T14:32:13.773057+0000","last_became_peered":"2024-04-23T14:32:13.773057+0000","last_unstale":"2024-04-23T14:32:14.639398+0000","last_undegraded":"2024-04-23T14:32:14.639398+0000","last_fullsized":"2024-04-23T14:32:14.639398+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:08.115745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180012+0000","last_change":"2024-04-23T14:32:09.641317+0000","last_active":"2024-04-23T14:32:15.180012+0000","last_peered":"2024-04-23T14:32:15.180012+0000","last_clean":"2024-04-23T14:32:15.180012+0000","last_became_active":"2024-04-23T14:32:09.640930+0000","last_became_peered":"2024-04-23T14:32:09.640930+0000","last_unstale":"2024-04-23T14:32:15.180012+0000","last_undegraded":"2024-04-23T14:32:15.180012+0000","last_fullsized":"2024-04-23T14:32:15.180012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:24.637952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152438+0000","last_change":"2024-04-23T14:32:08.601936+0000","last_active":"2024-04-23T14:32:15.152438+0000","last_peered":"2024-04-23T14:32:15.152438+0000","last_clean":"2024-04-23T14:32:15.152438+0000","last_became_active":"2024-04-23T14:32:08.601653+0000","last_became_peered":"2024-04-23T14:32:08.601653+0000","last_unstale":"2024-04-23T14:32:15.152438+0000","last_undegraded":"2024-04-23T14:32:15.152438+0000","last_fullsized":"2024-04-23T14:32:15.152438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:07:08.639311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151588+0000","last_change":"2024-04-23T14:32:13.664132+0000","last_active":"2024-04-23T14:32:15.151588+0000","last_peered":"2024-04-23T14:32:15.151588+0000","last_clean":"2024-04-23T14:32:15.151588+0000","last_became_active":"2024-04-23T14:32:13.663741+0000","last_became_peered":"2024-04-23T14:32:13.663741+0000","last_unstale":"2024-04-23T14:32:15.151588+0000","last_undegraded":"2024-04-23T14:32:15.151588+0000","last_fullsized":"2024-04-23T14:32:15.151588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:07.179856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179517+0000","last_change":"2024-04-23T14:32:09.641942+0000","last_active":"2024-04-23T14:32:15.179517+0000","last_peered":"2024-04-23T14:32:15.179517+0000","last_clean":"2024-04-23T14:32:15.179517+0000","last_became_active":"2024-04-23T14:32:09.641680+0000","last_became_peered":"2024-04-23T14:32:09.641680+0000","last_unstale":"2024-04-23T14:32:15.179517+0000","last_undegraded":"2024-04-23T14:32:15.179517+0000","last_fullsized":"2024-04-23T14:32:15.179517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:50.876901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181283+0000","last_change":"2024-04-23T14:32:08.603405+0000","last_active":"2024-04-23T14:32:15.181283+0000","last_peered":"2024-04-23T14:32:15.181283+0000","last_clean":"2024-04-23T14:32:15.181283+0000","last_became_active":"2024-04-23T14:32:08.603116+0000","last_became_peered":"2024-04-23T14:32:08.603116+0000","last_unstale":"2024-04-23T14:32:15.181283+0000","last_undegraded":"2024-04-23T14:32:15.181283+0000","last_fullsized":"2024-04-23T14:32:15.181283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:43:58.790056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640138+0000","last_change":"2024-04-23T14:32:13.660242+0000","last_active":"2024-04-23T14:32:14.640138+0000","last_peered":"2024-04-23T14:32:14.640138+0000","last_clean":"2024-04-23T14:32:14.640138+0000","last_became_active":"2024-04-23T14:32:13.659712+0000","last_became_peered":"2024-04-23T14:32:13.659712+0000","last_unstale":"2024-04-23T14:32:14.640138+0000","last_undegraded":"2024-04-23T14:32:14.640138+0000","last_fullsized":"2024-04-23T14:32:14.640138+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:14:16.782670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641152+0000","last_change":"2024-04-23T14:32:09.619773+0000","last_active":"2024-04-23T14:32:14.641152+0000","last_peered":"2024-04-23T14:32:14.641152+0000","last_clean":"2024-04-23T14:32:14.641152+0000","last_became_active":"2024-04-23T14:32:09.619451+0000","last_became_peered":"2024-04-23T14:32:09.619451+0000","last_unstale":"2024-04-23T14:32:14.641152+0000","last_undegraded":"2024-04-23T14:32:14.641152+0000","last_fullsized":"2024-04-23T14:32:14.641152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:12.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641439+0000","last_change":"2024-04-23T14:32:08.613751+0000","last_active":"2024-04-23T14:32:14.641439+0000","last_peered":"2024-04-23T14:32:14.641439+0000","last_clean":"2024-04-23T14:32:14.641439+0000","last_became_active":"2024-04-23T14:32:08.607415+0000","last_became_peered":"2024-04-23T14:32:08.607415+0000","last_unstale":"2024-04-23T14:32:14.641439+0000","last_undegraded":"2024-04-23T14:32:14.641439+0000","last_fullsized":"2024-04-23T14:32:14.641439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:35.854078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182048+0000","last_change":"2024-04-23T14:32:13.780614+0000","last_active":"2024-04-23T14:32:15.182048+0000","last_peered":"2024-04-23T14:32:15.182048+0000","last_clean":"2024-04-23T14:32:15.182048+0000","last_became_active":"2024-04-23T14:32:13.780135+0000","last_became_peered":"2024-04-23T14:32:13.780135+0000","last_unstale":"2024-04-23T14:32:15.182048+0000","last_undegraded":"2024-04-23T14:32:15.182048+0000","last_fullsized":"2024-04-23T14:32:15.182048+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:16:27.476884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179624+0000","last_change":"2024-04-23T14:32:09.642908+0000","last_active":"2024-04-23T14:32:15.179624+0000","last_peered":"2024-04-23T14:32:15.179624+0000","last_clean":"2024-04-23T14:32:15.179624+0000","last_became_active":"2024-04-23T14:32:09.642670+0000","last_became_peered":"2024-04-23T14:32:09.642670+0000","last_unstale":"2024-04-23T14:32:15.179624+0000","last_undegraded":"2024-04-23T14:32:15.179624+0000","last_fullsized":"2024-04-23T14:32:15.179624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:56.096479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640627+0000","last_change":"2024-04-23T14:32:08.598260+0000","last_active":"2024-04-23T14:32:14.640627+0000","last_peered":"2024-04-23T14:32:14.640627+0000","last_clean":"2024-04-23T14:32:14.640627+0000","last_became_active":"2024-04-23T14:32:08.597931+0000","last_became_peered":"2024-04-23T14:32:08.597931+0000","last_unstale":"2024-04-23T14:32:14.640627+0000","last_undegraded":"2024-04-23T14:32:14.640627+0000","last_fullsized":"2024-04-23T14:32:14.640627+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:51.650822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182382+0000","last_change":"2024-04-23T14:32:13.775555+0000","last_active":"2024-04-23T14:32:15.182382+0000","last_peered":"2024-04-23T14:32:15.182382+0000","last_clean":"2024-04-23T14:32:15.182382+0000","last_became_active":"2024-04-23T14:32:13.774654+0000","last_became_peered":"2024-04-23T14:32:13.774654+0000","last_unstale":"2024-04-23T14:32:15.182382+0000","last_undegraded":"2024-04-23T14:32:15.182382+0000","last_fullsized":"2024-04-23T14:32:15.182382+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:57.743802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639957+0000","last_change":"2024-04-23T14:32:09.626794+0000","last_active":"2024-04-23T14:32:14.639957+0000","last_peered":"2024-04-23T14:32:14.639957+0000","last_clean":"2024-04-23T14:32:14.639957+0000","last_became_active":"2024-04-23T14:32:09.626382+0000","last_became_peered":"2024-04-23T14:32:09.626382+0000","last_unstale":"2024-04-23T14:32:14.639957+0000","last_undegraded":"2024-04-23T14:32:14.639957+0000","last_fullsized":"2024-04-23T14:32:14.639957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:54:08.664672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640520+0000","last_change":"2024-04-23T14:32:08.600425+0000","last_active":"2024-04-23T14:32:14.640520+0000","last_peered":"2024-04-23T14:32:14.640520+0000","last_clean":"2024-04-23T14:32:14.640520+0000","last_became_active":"2024-04-23T14:32:08.600113+0000","last_became_peered":"2024-04-23T14:32:08.600113+0000","last_unstale":"2024-04-23T14:32:14.640520+0000","last_undegraded":"2024-04-23T14:32:14.640520+0000","last_fullsized":"2024-04-23T14:32:14.640520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:07:08.639311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639973+0000","last_change":"2024-04-23T14:32:13.777092+0000","last_active":"2024-04-23T14:32:14.639973+0000","last_peered":"2024-04-23T14:32:14.639973+0000","last_clean":"2024-04-23T14:32:14.639973+0000","last_became_active":"2024-04-23T14:32:13.776681+0000","last_became_peered":"2024-04-23T14:32:13.776681+0000","last_unstale":"2024-04-23T14:32:14.639973+0000","last_undegraded":"2024-04-23T14:32:14.639973+0000","last_fullsized":"2024-04-23T14:32:14.639973+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:21:36.288246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179674+0000","last_change":"2024-04-23T14:32:09.629485+0000","last_active":"2024-04-23T14:32:15.179674+0000","last_peered":"2024-04-23T14:32:15.179674+0000","last_clean":"2024-04-23T14:32:15.179674+0000","last_became_active":"2024-04-23T14:32:09.629157+0000","last_became_peered":"2024-04-23T14:32:09.629157+0000","last_unstale":"2024-04-23T14:32:15.179674+0000","last_undegraded":"2024-04-23T14:32:15.179674+0000","last_fullsized":"2024-04-23T14:32:15.179674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:47:31.805399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:25.916019+0000","last_change":"2024-04-23T14:32:08.604477+0000","last_active":"2024-04-23T14:32:25.916019+0000","last_peered":"2024-04-23T14:32:25.916019+0000","last_clean":"2024-04-23T14:32:25.916019+0000","last_became_active":"2024-04-23T14:32:08.604148+0000","last_became_peered":"2024-04-23T14:32:08.604148+0000","last_unstale":"2024-04-23T14:32:25.916019+0000","last_undegraded":"2024-04-23T14:32:25.916019+0000","last_fullsized":"2024-04-23T14:32:25.916019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:51.650822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181124+0000","last_change":"2024-04-23T14:32:13.660428+0000","last_active":"2024-04-23T14:32:15.181124+0000","last_peered":"2024-04-23T14:32:15.181124+0000","last_clean":"2024-04-23T14:32:15.181124+0000","last_became_active":"2024-04-23T14:32:13.659779+0000","last_became_peered":"2024-04-23T14:32:13.659779+0000","last_unstale":"2024-04-23T14:32:15.181124+0000","last_undegraded":"2024-04-23T14:32:15.181124+0000","last_fullsized":"2024-04-23T14:32:15.181124+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:28.688885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152295+0000","last_change":"2024-04-23T14:32:09.628609+0000","last_active":"2024-04-23T14:32:15.152295+0000","last_peered":"2024-04-23T14:32:15.152295+0000","last_clean":"2024-04-23T14:32:15.152295+0000","last_became_active":"2024-04-23T14:32:09.628357+0000","last_became_peered":"2024-04-23T14:32:09.628357+0000","last_unstale":"2024-04-23T14:32:15.152295+0000","last_undegraded":"2024-04-23T14:32:15.152295+0000","last_fullsized":"2024-04-23T14:32:15.152295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:01.062394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182825+0000","last_change":"2024-04-23T14:32:08.607931+0000","last_active":"2024-04-23T14:32:15.182825+0000","last_peered":"2024-04-23T14:32:15.182825+0000","last_clean":"2024-04-23T14:32:15.182825+0000","last_became_active":"2024-04-23T14:32:08.607268+0000","last_became_peered":"2024-04-23T14:32:08.607268+0000","last_unstale":"2024-04-23T14:32:15.182825+0000","last_undegraded":"2024-04-23T14:32:15.182825+0000","last_fullsized":"2024-04-23T14:32:15.182825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:35.854078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152109+0000","last_change":"2024-04-23T14:32:13.661080+0000","last_active":"2024-04-23T14:32:15.152109+0000","last_peered":"2024-04-23T14:32:15.152109+0000","last_clean":"2024-04-23T14:32:15.152109+0000","last_became_active":"2024-04-23T14:32:13.660376+0000","last_became_peered":"2024-04-23T14:32:13.660376+0000","last_unstale":"2024-04-23T14:32:15.152109+0000","last_undegraded":"2024-04-23T14:32:15.152109+0000","last_fullsized":"2024-04-23T14:32:15.152109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:40:00.147412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640230+0000","last_change":"2024-04-23T14:32:09.621197+0000","last_active":"2024-04-23T14:32:14.640230+0000","last_peered":"2024-04-23T14:32:14.640230+0000","last_clean":"2024-04-23T14:32:14.640230+0000","last_became_active":"2024-04-23T14:32:09.620883+0000","last_became_peered":"2024-04-23T14:32:09.620883+0000","last_unstale":"2024-04-23T14:32:14.640230+0000","last_undegraded":"2024-04-23T14:32:14.640230+0000","last_fullsized":"2024-04-23T14:32:14.640230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:12.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640178+0000","last_change":"2024-04-23T14:32:08.606025+0000","last_active":"2024-04-23T14:32:14.640178+0000","last_peered":"2024-04-23T14:32:14.640178+0000","last_clean":"2024-04-23T14:32:14.640178+0000","last_became_active":"2024-04-23T14:32:08.605727+0000","last_became_peered":"2024-04-23T14:32:08.605727+0000","last_unstale":"2024-04-23T14:32:14.640178+0000","last_undegraded":"2024-04-23T14:32:14.640178+0000","last_fullsized":"2024-04-23T14:32:14.640178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:59:03.639740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181124+0000","last_change":"2024-04-23T14:32:13.669518+0000","last_active":"2024-04-23T14:32:15.181124+0000","last_peered":"2024-04-23T14:32:15.181124+0000","last_clean":"2024-04-23T14:32:15.181124+0000","last_became_active":"2024-04-23T14:32:13.669004+0000","last_became_peered":"2024-04-23T14:32:13.669004+0000","last_unstale":"2024-04-23T14:32:15.181124+0000","last_undegraded":"2024-04-23T14:32:15.181124+0000","last_fullsized":"2024-04-23T14:32:15.181124+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:31:45.244234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640043+0000","last_change":"2024-04-23T14:32:09.625280+0000","last_active":"2024-04-23T14:32:14.640043+0000","last_peered":"2024-04-23T14:32:14.640043+0000","last_clean":"2024-04-23T14:32:14.640043+0000","last_became_active":"2024-04-23T14:32:09.625033+0000","last_became_peered":"2024-04-23T14:32:09.625033+0000","last_unstale":"2024-04-23T14:32:14.640043+0000","last_undegraded":"2024-04-23T14:32:14.640043+0000","last_fullsized":"2024-04-23T14:32:14.640043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:14:12.731737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.642133+0000","last_change":"2024-04-23T14:32:08.605039+0000","last_active":"2024-04-23T14:32:14.642133+0000","last_peered":"2024-04-23T14:32:14.642133+0000","last_clean":"2024-04-23T14:32:14.642133+0000","last_became_active":"2024-04-23T14:32:08.604450+0000","last_became_peered":"2024-04-23T14:32:08.604450+0000","last_unstale":"2024-04-23T14:32:14.642133+0000","last_undegraded":"2024-04-23T14:32:14.642133+0000","last_fullsized":"2024-04-23T14:32:14.642133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:45:45.243948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181369+0000","last_change":"2024-04-23T14:32:13.779606+0000","last_active":"2024-04-23T14:32:15.181369+0000","last_peered":"2024-04-23T14:32:15.181369+0000","last_clean":"2024-04-23T14:32:15.181369+0000","last_became_active":"2024-04-23T14:32:13.779243+0000","last_became_peered":"2024-04-23T14:32:13.779243+0000","last_unstale":"2024-04-23T14:32:15.181369+0000","last_undegraded":"2024-04-23T14:32:15.181369+0000","last_fullsized":"2024-04-23T14:32:15.181369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:58:40.249561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640030+0000","last_change":"2024-04-23T14:32:13.779659+0000","last_active":"2024-04-23T14:32:14.640030+0000","last_peered":"2024-04-23T14:32:14.640030+0000","last_clean":"2024-04-23T14:32:14.640030+0000","last_became_active":"2024-04-23T14:32:13.779312+0000","last_became_peered":"2024-04-23T14:32:13.779312+0000","last_unstale":"2024-04-23T14:32:14.640030+0000","last_undegraded":"2024-04-23T14:32:14.640030+0000","last_fullsized":"2024-04-23T14:32:14.640030+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:28.688885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180700+0000","last_change":"2024-04-23T14:32:09.632498+0000","last_active":"2024-04-23T14:32:15.180700+0000","last_peered":"2024-04-23T14:32:15.180700+0000","last_clean":"2024-04-23T14:32:15.180700+0000","last_became_active":"2024-04-23T14:32:09.632191+0000","last_became_peered":"2024-04-23T14:32:09.632191+0000","last_unstale":"2024-04-23T14:32:15.180700+0000","last_undegraded":"2024-04-23T14:32:15.180700+0000","last_fullsized":"2024-04-23T14:32:15.180700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:53.674486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639631+0000","last_change":"2024-04-23T14:32:08.607308+0000","last_active":"2024-04-23T14:32:14.639631+0000","last_peered":"2024-04-23T14:32:14.639631+0000","last_clean":"2024-04-23T14:32:14.639631+0000","last_became_active":"2024-04-23T14:32:08.606840+0000","last_became_peered":"2024-04-23T14:32:08.606840+0000","last_unstale":"2024-04-23T14:32:14.639631+0000","last_undegraded":"2024-04-23T14:32:14.639631+0000","last_fullsized":"2024-04-23T14:32:14.639631+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:35.854078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.642125+0000","last_change":"2024-04-23T14:32:13.654800+0000","last_active":"2024-04-23T14:32:14.642125+0000","last_peered":"2024-04-23T14:32:14.642125+0000","last_clean":"2024-04-23T14:32:14.642125+0000","last_became_active":"2024-04-23T14:32:13.654191+0000","last_became_peered":"2024-04-23T14:32:13.654191+0000","last_unstale":"2024-04-23T14:32:14.642125+0000","last_undegraded":"2024-04-23T14:32:14.642125+0000","last_fullsized":"2024-04-23T14:32:14.642125+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:28.688885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181062+0000","last_change":"2024-04-23T14:32:09.623316+0000","last_active":"2024-04-23T14:32:15.181062+0000","last_peered":"2024-04-23T14:32:15.181062+0000","last_clean":"2024-04-23T14:32:15.181062+0000","last_became_active":"2024-04-23T14:32:09.621723+0000","last_became_peered":"2024-04-23T14:32:09.621723+0000","last_unstale":"2024-04-23T14:32:15.181062+0000","last_undegraded":"2024-04-23T14:32:15.181062+0000","last_fullsized":"2024-04-23T14:32:15.181062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:12.933947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181611+0000","last_change":"2024-04-23T14:32:08.609985+0000","last_active":"2024-04-23T14:32:15.181611+0000","last_peered":"2024-04-23T14:32:15.181611+0000","last_clean":"2024-04-23T14:32:15.181611+0000","last_became_active":"2024-04-23T14:32:08.609621+0000","last_became_peered":"2024-04-23T14:32:08.609621+0000","last_unstale":"2024-04-23T14:32:15.181611+0000","last_undegraded":"2024-04-23T14:32:15.181611+0000","last_fullsized":"2024-04-23T14:32:15.181611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:11.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641831+0000","last_change":"2024-04-23T14:32:08.609884+0000","last_active":"2024-04-23T14:32:14.641831+0000","last_peered":"2024-04-23T14:32:14.641831+0000","last_clean":"2024-04-23T14:32:14.641831+0000","last_became_active":"2024-04-23T14:32:08.607367+0000","last_became_peered":"2024-04-23T14:32:08.607367+0000","last_unstale":"2024-04-23T14:32:14.641831+0000","last_undegraded":"2024-04-23T14:32:14.641831+0000","last_fullsized":"2024-04-23T14:32:14.641831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:54.495261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640439+0000","last_change":"2024-04-23T14:32:09.632614+0000","last_active":"2024-04-23T14:32:14.640439+0000","last_peered":"2024-04-23T14:32:14.640439+0000","last_clean":"2024-04-23T14:32:14.640439+0000","last_became_active":"2024-04-23T14:32:09.632230+0000","last_became_peered":"2024-04-23T14:32:09.632230+0000","last_unstale":"2024-04-23T14:32:14.640439+0000","last_undegraded":"2024-04-23T14:32:14.640439+0000","last_fullsized":"2024-04-23T14:32:14.640439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:53.674486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181628+0000","last_change":"2024-04-23T14:32:13.671189+0000","last_active":"2024-04-23T14:32:15.181628+0000","last_peered":"2024-04-23T14:32:15.181628+0000","last_clean":"2024-04-23T14:32:15.181628+0000","last_became_active":"2024-04-23T14:32:13.670033+0000","last_became_peered":"2024-04-23T14:32:13.670033+0000","last_unstale":"2024-04-23T14:32:15.181628+0000","last_undegraded":"2024-04-23T14:32:15.181628+0000","last_fullsized":"2024-04-23T14:32:15.181628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:08:35.206423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181132+0000","last_change":"2024-04-23T14:32:08.612716+0000","last_active":"2024-04-23T14:32:15.181132+0000","last_peered":"2024-04-23T14:32:15.181132+0000","last_clean":"2024-04-23T14:32:15.181132+0000","last_became_active":"2024-04-23T14:32:08.612378+0000","last_became_peered":"2024-04-23T14:32:08.612378+0000","last_unstale":"2024-04-23T14:32:15.181132+0000","last_undegraded":"2024-04-23T14:32:15.181132+0000","last_fullsized":"2024-04-23T14:32:15.181132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:07.569279+0000","last_clean_scrub_stamp":"2024-04-23T14:32:07.569279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:43:58.790056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.151820+0000","last_change":"2024-04-23T14:32:09.620811+0000","last_active":"2024-04-23T14:32:15.151820+0000","last_peered":"2024-04-23T14:32:15.151820+0000","last_clean":"2024-04-23T14:32:15.151820+0000","last_became_active":"2024-04-23T14:32:09.620393+0000","last_became_peered":"2024-04-23T14:32:09.620393+0000","last_unstale":"2024-04-23T14:32:15.151820+0000","last_undegraded":"2024-04-23T14:32:15.151820+0000","last_fullsized":"2024-04-23T14:32:15.151820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:36.859905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181533+0000","last_change":"2024-04-23T14:32:13.670681+0000","last_active":"2024-04-23T14:32:15.181533+0000","last_peered":"2024-04-23T14:32:15.181533+0000","last_clean":"2024-04-23T14:32:15.181533+0000","last_became_active":"2024-04-23T14:32:13.670304+0000","last_became_peered":"2024-04-23T14:32:13.670304+0000","last_unstale":"2024-04-23T14:32:15.181533+0000","last_undegraded":"2024-04-23T14:32:15.181533+0000","last_fullsized":"2024-04-23T14:32:15.181533+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:12.626039+0000","last_clean_scrub_stamp":"2024-04-23T14:32:12.626039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:35:37.731501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641029+0000","last_change":"2024-04-23T14:32:09.617499+0000","last_active":"2024-04-23T14:32:14.641029+0000","last_peered":"2024-04-23T14:32:14.641029+0000","last_clean":"2024-04-23T14:32:14.641029+0000","last_became_active":"2024-04-23T14:32:09.617154+0000","last_became_peered":"2024-04-23T14:32:09.617154+0000","last_unstale":"2024-04-23T14:32:14.641029+0000","last_undegraded":"2024-04-23T14:32:14.641029+0000","last_fullsized":"2024-04-23T14:32:14.641029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:43:49.773005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181553+0000","last_change":"2024-04-23T14:32:09.619214+0000","last_active":"2024-04-23T14:32:15.181553+0000","last_peered":"2024-04-23T14:32:15.181553+0000","last_clean":"2024-04-23T14:32:15.181553+0000","last_became_active":"2024-04-23T14:32:09.618943+0000","last_became_peered":"2024-04-23T14:32:09.618943+0000","last_unstale":"2024-04-23T14:32:15.181553+0000","last_undegraded":"2024-04-23T14:32:15.181553+0000","last_fullsized":"2024-04-23T14:32:15.181553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:12.536442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640531+0000","last_change":"2024-04-23T14:32:09.626499+0000","last_active":"2024-04-23T14:32:14.640531+0000","last_peered":"2024-04-23T14:32:14.640531+0000","last_clean":"2024-04-23T14:32:14.640531+0000","last_became_active":"2024-04-23T14:32:09.626070+0000","last_became_peered":"2024-04-23T14:32:09.626070+0000","last_unstale":"2024-04-23T14:32:14.640531+0000","last_undegraded":"2024-04-23T14:32:14.640531+0000","last_fullsized":"2024-04-23T14:32:14.640531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:50.876901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640295+0000","last_change":"2024-04-23T14:32:09.625391+0000","last_active":"2024-04-23T14:32:14.640295+0000","last_peered":"2024-04-23T14:32:14.640295+0000","last_clean":"2024-04-23T14:32:14.640295+0000","last_became_active":"2024-04-23T14:32:09.625049+0000","last_became_peered":"2024-04-23T14:32:09.625049+0000","last_unstale":"2024-04-23T14:32:14.640295+0000","last_undegraded":"2024-04-23T14:32:14.640295+0000","last_fullsized":"2024-04-23T14:32:14.640295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:52:42.867377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182436+0000","last_change":"2024-04-23T14:32:09.627850+0000","last_active":"2024-04-23T14:32:15.182436+0000","last_peered":"2024-04-23T14:32:15.182436+0000","last_clean":"2024-04-23T14:32:15.182436+0000","last_became_active":"2024-04-23T14:32:09.627603+0000","last_became_peered":"2024-04-23T14:32:09.627603+0000","last_unstale":"2024-04-23T14:32:15.182436+0000","last_undegraded":"2024-04-23T14:32:15.182436+0000","last_fullsized":"2024-04-23T14:32:15.182436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:09.066755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182767+0000","last_change":"2024-04-23T14:32:09.630409+0000","last_active":"2024-04-23T14:32:15.182767+0000","last_peered":"2024-04-23T14:32:15.182767+0000","last_clean":"2024-04-23T14:32:15.182767+0000","last_became_active":"2024-04-23T14:32:09.630100+0000","last_became_peered":"2024-04-23T14:32:09.630100+0000","last_unstale":"2024-04-23T14:32:15.182767+0000","last_undegraded":"2024-04-23T14:32:15.182767+0000","last_fullsized":"2024-04-23T14:32:15.182767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:03.128923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641996+0000","last_change":"2024-04-23T14:32:09.631521+0000","last_active":"2024-04-23T14:32:14.641996+0000","last_peered":"2024-04-23T14:32:14.641996+0000","last_clean":"2024-04-23T14:32:14.641996+0000","last_became_active":"2024-04-23T14:32:09.631291+0000","last_became_peered":"2024-04-23T14:32:09.631291+0000","last_unstale":"2024-04-23T14:32:14.641996+0000","last_undegraded":"2024-04-23T14:32:14.641996+0000","last_fullsized":"2024-04-23T14:32:14.641996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:12.933947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182097+0000","last_change":"2024-04-23T14:32:09.642262+0000","last_active":"2024-04-23T14:32:15.182097+0000","last_peered":"2024-04-23T14:32:15.182097+0000","last_clean":"2024-04-23T14:32:15.182097+0000","last_became_active":"2024-04-23T14:32:09.641982+0000","last_became_peered":"2024-04-23T14:32:09.641982+0000","last_unstale":"2024-04-23T14:32:15.182097+0000","last_undegraded":"2024-04-23T14:32:15.182097+0000","last_fullsized":"2024-04-23T14:32:15.182097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:52.459061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639502+0000","last_change":"2024-04-23T14:32:09.613189+0000","last_active":"2024-04-23T14:32:14.639502+0000","last_peered":"2024-04-23T14:32:14.639502+0000","last_clean":"2024-04-23T14:32:14.639502+0000","last_became_active":"2024-04-23T14:32:09.612846+0000","last_became_peered":"2024-04-23T14:32:09.612846+0000","last_unstale":"2024-04-23T14:32:14.639502+0000","last_undegraded":"2024-04-23T14:32:14.639502+0000","last_fullsized":"2024-04-23T14:32:14.639502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:03.128923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.181523+0000","last_change":"2024-04-23T14:32:09.642268+0000","last_active":"2024-04-23T14:32:15.181523+0000","last_peered":"2024-04-23T14:32:15.181523+0000","last_clean":"2024-04-23T14:32:15.181523+0000","last_became_active":"2024-04-23T14:32:09.642007+0000","last_became_peered":"2024-04-23T14:32:09.642007+0000","last_unstale":"2024-04-23T14:32:15.181523+0000","last_undegraded":"2024-04-23T14:32:15.181523+0000","last_fullsized":"2024-04-23T14:32:15.181523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:50:24.043980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640425+0000","last_change":"2024-04-23T14:32:09.626670+0000","last_active":"2024-04-23T14:32:14.640425+0000","last_peered":"2024-04-23T14:32:14.640425+0000","last_clean":"2024-04-23T14:32:14.640425+0000","last_became_active":"2024-04-23T14:32:09.625568+0000","last_became_peered":"2024-04-23T14:32:09.625568+0000","last_unstale":"2024-04-23T14:32:14.640425+0000","last_undegraded":"2024-04-23T14:32:14.640425+0000","last_fullsized":"2024-04-23T14:32:14.640425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:09.066755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639884+0000","last_change":"2024-04-23T14:32:09.627089+0000","last_active":"2024-04-23T14:32:14.639884+0000","last_peered":"2024-04-23T14:32:14.639884+0000","last_clean":"2024-04-23T14:32:14.639884+0000","last_became_active":"2024-04-23T14:32:09.626429+0000","last_became_peered":"2024-04-23T14:32:09.626429+0000","last_unstale":"2024-04-23T14:32:14.639884+0000","last_undegraded":"2024-04-23T14:32:14.639884+0000","last_fullsized":"2024-04-23T14:32:14.639884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:43:49.773005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641487+0000","last_change":"2024-04-23T14:32:09.643824+0000","last_active":"2024-04-23T14:32:14.641487+0000","last_peered":"2024-04-23T14:32:14.641487+0000","last_clean":"2024-04-23T14:32:14.641487+0000","last_became_active":"2024-04-23T14:32:09.643454+0000","last_became_peered":"2024-04-23T14:32:09.643454+0000","last_unstale":"2024-04-23T14:32:14.641487+0000","last_undegraded":"2024-04-23T14:32:14.641487+0000","last_fullsized":"2024-04-23T14:32:14.641487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:03.128923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179997+0000","last_change":"2024-04-23T14:32:09.627274+0000","last_active":"2024-04-23T14:32:15.179997+0000","last_peered":"2024-04-23T14:32:15.179997+0000","last_clean":"2024-04-23T14:32:15.179997+0000","last_became_active":"2024-04-23T14:32:09.626999+0000","last_became_peered":"2024-04-23T14:32:09.626999+0000","last_unstale":"2024-04-23T14:32:15.179997+0000","last_undegraded":"2024-04-23T14:32:15.179997+0000","last_fullsized":"2024-04-23T14:32:15.179997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:12:19.437109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.182768+0000","last_change":"2024-04-23T14:32:09.615554+0000","last_active":"2024-04-23T14:32:15.182768+0000","last_peered":"2024-04-23T14:32:15.182768+0000","last_clean":"2024-04-23T14:32:15.182768+0000","last_became_active":"2024-04-23T14:32:09.614793+0000","last_became_peered":"2024-04-23T14:32:09.614793+0000","last_unstale":"2024-04-23T14:32:15.182768+0000","last_undegraded":"2024-04-23T14:32:15.182768+0000","last_fullsized":"2024-04-23T14:32:15.182768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:01.062394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640359+0000","last_change":"2024-04-23T14:32:09.625443+0000","last_active":"2024-04-23T14:32:14.640359+0000","last_peered":"2024-04-23T14:32:14.640359+0000","last_clean":"2024-04-23T14:32:14.640359+0000","last_became_active":"2024-04-23T14:32:09.625140+0000","last_became_peered":"2024-04-23T14:32:09.625140+0000","last_unstale":"2024-04-23T14:32:14.640359+0000","last_undegraded":"2024-04-23T14:32:14.640359+0000","last_fullsized":"2024-04-23T14:32:14.640359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:12:19.437109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640042+0000","last_change":"2024-04-23T14:32:09.614187+0000","last_active":"2024-04-23T14:32:14.640042+0000","last_peered":"2024-04-23T14:32:14.640042+0000","last_clean":"2024-04-23T14:32:14.640042+0000","last_became_active":"2024-04-23T14:32:09.613813+0000","last_became_peered":"2024-04-23T14:32:09.613813+0000","last_unstale":"2024-04-23T14:32:14.640042+0000","last_undegraded":"2024-04-23T14:32:14.640042+0000","last_fullsized":"2024-04-23T14:32:14.640042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:12.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180552+0000","last_change":"2024-04-23T14:32:09.630660+0000","last_active":"2024-04-23T14:32:15.180552+0000","last_peered":"2024-04-23T14:32:15.180552+0000","last_clean":"2024-04-23T14:32:15.180552+0000","last_became_active":"2024-04-23T14:32:09.630381+0000","last_became_peered":"2024-04-23T14:32:09.630381+0000","last_unstale":"2024-04-23T14:32:15.180552+0000","last_undegraded":"2024-04-23T14:32:15.180552+0000","last_fullsized":"2024-04-23T14:32:15.180552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:12.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639936+0000","last_change":"2024-04-23T14:32:09.613974+0000","last_active":"2024-04-23T14:32:14.639936+0000","last_peered":"2024-04-23T14:32:14.639936+0000","last_clean":"2024-04-23T14:32:14.639936+0000","last_became_active":"2024-04-23T14:32:09.613699+0000","last_became_peered":"2024-04-23T14:32:09.613699+0000","last_unstale":"2024-04-23T14:32:14.639936+0000","last_undegraded":"2024-04-23T14:32:14.639936+0000","last_fullsized":"2024-04-23T14:32:14.639936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:08:31.155490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180930+0000","last_change":"2024-04-23T14:32:09.622174+0000","last_active":"2024-04-23T14:32:15.180930+0000","last_peered":"2024-04-23T14:32:15.180930+0000","last_clean":"2024-04-23T14:32:15.180930+0000","last_became_active":"2024-04-23T14:32:09.621360+0000","last_became_peered":"2024-04-23T14:32:09.621360+0000","last_unstale":"2024-04-23T14:32:15.180930+0000","last_undegraded":"2024-04-23T14:32:15.180930+0000","last_fullsized":"2024-04-23T14:32:15.180930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:12.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.152292+0000","last_change":"2024-04-23T14:32:09.625529+0000","last_active":"2024-04-23T14:32:15.152292+0000","last_peered":"2024-04-23T14:32:15.152292+0000","last_clean":"2024-04-23T14:32:15.152292+0000","last_became_active":"2024-04-23T14:32:09.625259+0000","last_became_peered":"2024-04-23T14:32:09.625259+0000","last_unstale":"2024-04-23T14:32:15.152292+0000","last_undegraded":"2024-04-23T14:32:15.152292+0000","last_fullsized":"2024-04-23T14:32:15.152292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:12.536442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641776+0000","last_change":"2024-04-23T14:32:09.634616+0000","last_active":"2024-04-23T14:32:14.641776+0000","last_peered":"2024-04-23T14:32:14.641776+0000","last_clean":"2024-04-23T14:32:14.641776+0000","last_became_active":"2024-04-23T14:32:09.634328+0000","last_became_peered":"2024-04-23T14:32:09.634328+0000","last_unstale":"2024-04-23T14:32:14.641776+0000","last_undegraded":"2024-04-23T14:32:14.641776+0000","last_fullsized":"2024-04-23T14:32:14.641776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:52:42.867377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.641980+0000","last_change":"2024-04-23T14:32:09.618704+0000","last_active":"2024-04-23T14:32:14.641980+0000","last_peered":"2024-04-23T14:32:14.641980+0000","last_clean":"2024-04-23T14:32:14.641980+0000","last_became_active":"2024-04-23T14:32:09.618328+0000","last_became_peered":"2024-04-23T14:32:09.618328+0000","last_unstale":"2024-04-23T14:32:14.641980+0000","last_undegraded":"2024-04-23T14:32:14.641980+0000","last_fullsized":"2024-04-23T14:32:14.641980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:50:24.043980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180068+0000","last_change":"2024-04-23T14:32:09.637988+0000","last_active":"2024-04-23T14:32:15.180068+0000","last_peered":"2024-04-23T14:32:15.180068+0000","last_clean":"2024-04-23T14:32:15.180068+0000","last_became_active":"2024-04-23T14:32:09.637766+0000","last_became_peered":"2024-04-23T14:32:09.637766+0000","last_unstale":"2024-04-23T14:32:15.180068+0000","last_undegraded":"2024-04-23T14:32:15.180068+0000","last_fullsized":"2024-04-23T14:32:15.180068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:09:07.994350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.180933+0000","last_change":"2024-04-23T14:32:09.623580+0000","last_active":"2024-04-23T14:32:15.180933+0000","last_peered":"2024-04-23T14:32:15.180933+0000","last_clean":"2024-04-23T14:32:15.180933+0000","last_became_active":"2024-04-23T14:32:09.622942+0000","last_became_peered":"2024-04-23T14:32:09.622942+0000","last_unstale":"2024-04-23T14:32:15.180933+0000","last_undegraded":"2024-04-23T14:32:15.180933+0000","last_fullsized":"2024-04-23T14:32:15.180933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:52:42.867377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179379+0000","last_change":"2024-04-23T14:32:09.624523+0000","last_active":"2024-04-23T14:32:15.179379+0000","last_peered":"2024-04-23T14:32:15.179379+0000","last_clean":"2024-04-23T14:32:15.179379+0000","last_became_active":"2024-04-23T14:32:09.624229+0000","last_became_peered":"2024-04-23T14:32:09.624229+0000","last_unstale":"2024-04-23T14:32:15.179379+0000","last_undegraded":"2024-04-23T14:32:15.179379+0000","last_fullsized":"2024-04-23T14:32:15.179379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:09.066755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640175+0000","last_change":"2024-04-23T14:32:09.614929+0000","last_active":"2024-04-23T14:32:14.640175+0000","last_peered":"2024-04-23T14:32:14.640175+0000","last_clean":"2024-04-23T14:32:14.640175+0000","last_became_active":"2024-04-23T14:32:09.614584+0000","last_became_peered":"2024-04-23T14:32:09.614584+0000","last_unstale":"2024-04-23T14:32:14.640175+0000","last_undegraded":"2024-04-23T14:32:14.640175+0000","last_fullsized":"2024-04-23T14:32:14.640175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:22.220286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179498+0000","last_change":"2024-04-23T14:32:09.623580+0000","last_active":"2024-04-23T14:32:15.179498+0000","last_peered":"2024-04-23T14:32:15.179498+0000","last_clean":"2024-04-23T14:32:15.179498+0000","last_became_active":"2024-04-23T14:32:09.623197+0000","last_became_peered":"2024-04-23T14:32:09.623197+0000","last_unstale":"2024-04-23T14:32:15.179498+0000","last_undegraded":"2024-04-23T14:32:15.179498+0000","last_fullsized":"2024-04-23T14:32:15.179498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:52.459061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.640154+0000","last_change":"2024-04-23T14:32:09.622573+0000","last_active":"2024-04-23T14:32:14.640154+0000","last_peered":"2024-04-23T14:32:14.640154+0000","last_clean":"2024-04-23T14:32:14.640154+0000","last_became_active":"2024-04-23T14:32:09.622232+0000","last_became_peered":"2024-04-23T14:32:09.622232+0000","last_unstale":"2024-04-23T14:32:14.640154+0000","last_undegraded":"2024-04-23T14:32:14.640154+0000","last_fullsized":"2024-04-23T14:32:14.640154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:52.459061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:14.639881+0000","last_change":"2024-04-23T14:32:09.604425+0000","last_active":"2024-04-23T14:32:14.639881+0000","last_peered":"2024-04-23T14:32:14.639881+0000","last_clean":"2024-04-23T14:32:14.639881+0000","last_became_active":"2024-04-23T14:32:09.604159+0000","last_became_peered":"2024-04-23T14:32:09.604159+0000","last_unstale":"2024-04-23T14:32:14.639881+0000","last_undegraded":"2024-04-23T14:32:14.639881+0000","last_fullsized":"2024-04-23T14:32:14.639881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:50:24.043980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.183079+0000","last_change":"2024-04-23T14:32:09.614842+0000","last_active":"2024-04-23T14:32:15.183079+0000","last_peered":"2024-04-23T14:32:15.183079+0000","last_clean":"2024-04-23T14:32:15.183079+0000","last_became_active":"2024-04-23T14:32:09.614385+0000","last_became_peered":"2024-04-23T14:32:09.614385+0000","last_unstale":"2024-04-23T14:32:15.183079+0000","last_undegraded":"2024-04-23T14:32:15.183079+0000","last_fullsized":"2024-04-23T14:32:15.183079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:22.220286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-23T14:32:15.179991+0000","last_change":"2024-04-23T14:32:09.628001+0000","last_active":"2024-04-23T14:32:15.179991+0000","last_peered":"2024-04-23T14:32:15.179991+0000","last_clean":"2024-04-23T14:32:15.179991+0000","last_became_active":"2024-04-23T14:32:09.627722+0000","last_became_peered":"2024-04-23T14:32:09.627722+0000","last_unstale":"2024-04-23T14:32:15.179991+0000","last_undegraded":"2024-04-23T14:32:15.179991+0000","last_fullsized":"2024-04-23T14:32:15.179991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:08.575106+0000","last_clean_scrub_stamp":"2024-04-23T14:32:08.575106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:14:12.731737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:32:40.746 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-23T14:32:40.747 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-23T14:32:40.747 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-23T14:32:40.747 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-23T14:32:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.919+0000 7f72fae00640 1 Processor -- start 2024-04-23T14:32:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.919+0000 7f72fae00640 1 -- start start 2024-04-23T14:32:40.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.919+0000 7f72fae00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72f415c7e0 0x7f72f40d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:40.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.919+0000 7f72fae00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f72f415d600 0x7f72f40d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:40.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.919+0000 7f72fae00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72f40d04a0 con 0x7f72f40cca50 2024-04-23T14:32:40.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.919+0000 7f72fae00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f72f415db00 con 0x7f72f415d600 2024-04-23T14:32:40.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.919+0000 7f72fae00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f72f40d0780 con 0x7f72f415c7e0 2024-04-23T14:32:40.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.919+0000 7f72f8e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f72f415d600 0x7f72f40d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:40.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.919+0000 7f72f3400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72f415c7e0 0x7f72f40d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:40.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.919+0000 7f72f3400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72f415c7e0 0x7f72f40d1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:55312/0 (socket says 172.21.15.100:55312) 2024-04-23T14:32:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.919+0000 7f72f8e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f72f415d600 0x7f72f40d17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:44656/0 (socket says 172.21.15.100:44656) 2024-04-23T14:32:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.919+0000 7f72f3400640 1 -- 172.21.15.100:0/3207035496 learned_addr learned my addr 172.21.15.100:0/3207035496 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.919+0000 7f72f3e00640 1 --1- 172.21.15.100:0/3207035496 >> v1:172.21.15.151:6789/0 conn(0x7f72f40cca50 0x7f72f40cce30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:55010/0 (socket says 172.21.15.100:55010) 2024-04-23T14:32:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.919+0000 7f72f3400640 1 -- 172.21.15.100:0/3207035496 >> v2:172.21.15.151:3300/0 conn(0x7f72f415d600 msgr2=0x7f72f40d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.919+0000 7f72f3400640 1 --2- 172.21.15.100:0/3207035496 >> v2:172.21.15.151:3300/0 conn(0x7f72f415d600 0x7f72f40d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.919+0000 7f72f3400640 1 -- 172.21.15.100:0/3207035496 >> v1:172.21.15.151:6789/0 conn(0x7f72f40cca50 legacy=0x7f72f40cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.919+0000 7f72f3400640 1 -- 172.21.15.100:0/3207035496 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f40d0a60 con 0x7f72f415c7e0 2024-04-23T14:32:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.919+0000 7f72f8e00640 1 --2- 172.21.15.100:0/3207035496 >> v2:172.21.15.151:3300/0 conn(0x7f72f415d600 0x7f72f40d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:40.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.923+0000 7f72f3400640 1 --2- 172.21.15.100:0/3207035496 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72f415c7e0 0x7f72f40d1260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009870 tx=0x7f72e001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a39d3a10c0672d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:40.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.923+0000 7f72f2a00640 1 -- 172.21.15.100:0/3207035496 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e00aa620 con 0x7f72f415c7e0 2024-04-23T14:32:40.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.923+0000 7f72f2a00640 1 -- 172.21.15.100:0/3207035496 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72e00af070 con 0x7f72f415c7e0 2024-04-23T14:32:40.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.923+0000 7f72f2a00640 1 -- 172.21.15.100:0/3207035496 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e00aa9e0 con 0x7f72f415c7e0 2024-04-23T14:32:40.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.923+0000 7f72fae00640 1 -- 172.21.15.100:0/3207035496 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72f415c7e0 msgr2=0x7f72f40d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.923+0000 7f72fae00640 1 --2- 172.21.15.100:0/3207035496 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72f415c7e0 0x7f72f40d1260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009870 tx=0x7f72e001c4a0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.923+0000 7f72fae00640 1 -- 172.21.15.100:0/3207035496 shutdown_connections 2024-04-23T14:32:40.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.923+0000 7f72fae00640 1 --2- 172.21.15.100:0/3207035496 >> v2:172.21.15.151:3300/0 conn(0x7f72f415d600 0x7f72f40d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.923+0000 7f72fae00640 1 --2- 172.21.15.100:0/3207035496 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72f415c7e0 0x7f72f40d1260 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.923+0000 7f72fae00640 1 -- 172.21.15.100:0/3207035496 >> 172.21.15.100:0/3207035496 conn(0x7f72f40cee80 msgr2=0x7f72f40cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:40.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.923+0000 7f72fae00640 1 -- 172.21.15.100:0/3207035496 shutdown_connections 2024-04-23T14:32:40.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.923+0000 7f72fae00640 1 -- 172.21.15.100:0/3207035496 wait complete. 2024-04-23T14:32:40.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72fae00640 1 Processor -- start 2024-04-23T14:32:40.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72fae00640 1 -- start start 2024-04-23T14:32:40.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72fae00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72f40cca50 0x7f72f4149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:40.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72fae00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72f415c7e0 0x7f72f4157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:40.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72fae00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72f415d600 0x7f72f40c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:40.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72fae00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f72f40b3d70 con 0x7f72f415d600 2024-04-23T14:32:40.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72fae00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f72f40b3bf0 con 0x7f72f415c7e0 2024-04-23T14:32:40.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72fae00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f72f4169ad0 con 0x7f72f40cca50 2024-04-23T14:32:40.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72f8e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72f415d600 0x7f72f40c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:40.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72f8e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72f415d600 0x7f72f40c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52990/0 (socket says 172.21.15.100:52990) 2024-04-23T14:32:40.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72f8e00640 1 -- 172.21.15.100:0/4105791952 learned_addr learned my addr 172.21.15.100:0/4105791952 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:40.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72f3e00640 1 --2- 172.21.15.100:0/4105791952 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72f40cca50 0x7f72f4149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:40.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72f3400640 1 --2- 172.21.15.100:0/4105791952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72f415c7e0 0x7f72f4157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:40.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72f8e00640 1 -- 172.21.15.100:0/4105791952 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72f40cca50 msgr2=0x7f72f4149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72f8e00640 1 --2- 172.21.15.100:0/4105791952 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72f40cca50 0x7f72f4149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72f8e00640 1 -- 172.21.15.100:0/4105791952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72f415c7e0 msgr2=0x7f72f4157040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72f8e00640 1 --2- 172.21.15.100:0/4105791952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72f415c7e0 0x7f72f4157040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72f8e00640 1 -- 172.21.15.100:0/4105791952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f4157b00 con 0x7f72f415d600 2024-04-23T14:32:40.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72f8e00640 1 --2- 172.21.15.100:0/4105791952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72f415d600 0x7f72f40c82a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00efc0 tx=0x7f72ec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:40.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72ebe00640 1 -- 172.21.15.100:0/4105791952 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec018020 con 0x7f72f415d600 2024-04-23T14:32:40.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72ebe00640 1 -- 172.21.15.100:0/4105791952 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72ec014480 con 0x7f72f415d600 2024-04-23T14:32:40.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72fae00640 1 -- 172.21.15.100:0/4105791952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72f4157de0 con 0x7f72f415d600 2024-04-23T14:32:40.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72ebe00640 1 -- 172.21.15.100:0/4105791952 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec014870 con 0x7f72f415d600 2024-04-23T14:32:40.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.927+0000 7f72fae00640 1 -- 172.21.15.100:0/4105791952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72f40b55e0 con 0x7f72f415d600 2024-04-23T14:32:40.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.931+0000 7f72fae00640 1 -- 172.21.15.100:0/4105791952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72f4068ad0 con 0x7f72f415d600 2024-04-23T14:32:40.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.931+0000 7f72ebe00640 1 -- 172.21.15.100:0/4105791952 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f72ec01b020 con 0x7f72f415d600 2024-04-23T14:32:40.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.931+0000 7f72ebe00640 1 --2- 172.21.15.100:0/4105791952 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f72d40683b0 0x7f72d406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:40.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.931+0000 7f72ebe00640 1 -- 172.21.15.100:0/4105791952 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f72ec087920 con 0x7f72f415d600 2024-04-23T14:32:40.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.931+0000 7f72f3e00640 1 --2- 172.21.15.100:0/4105791952 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f72d40683b0 0x7f72d406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:40.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.931+0000 7f72f3e00640 1 --2- 172.21.15.100:0/4105791952 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f72d40683b0 0x7f72d406a870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f72e4000f70 tx=0x7f72e4007e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:40.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:40.935+0000 7f72ebe00640 1 -- 172.21.15.100:0/4105791952 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f72ec022020 con 0x7f72f415d600 2024-04-23T14:32:41.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.127+0000 7f72fae00640 1 -- 172.21.15.100:0/4105791952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f72f40b37a0 con 0x7f72f415d600 2024-04-23T14:32:41.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.127+0000 7f72ebe00640 1 -- 172.21.15.100:0/4105791952 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f72ec010040 con 0x7f72f415d600 2024-04-23T14:32:41.128 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:41.128 INFO:teuthology.orchestra.run.smithi100.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-23T14:32:41.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.127+0000 7f72fae00640 1 -- 172.21.15.100:0/4105791952 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f72d40683b0 msgr2=0x7f72d406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.127+0000 7f72fae00640 1 --2- 172.21.15.100:0/4105791952 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f72d40683b0 0x7f72d406a870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f72e4000f70 tx=0x7f72e4007e90 comp rx=0 tx=0).stop 2024-04-23T14:32:41.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.127+0000 7f72fae00640 1 -- 172.21.15.100:0/4105791952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72f415d600 msgr2=0x7f72f40c82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.127+0000 7f72fae00640 1 --2- 172.21.15.100:0/4105791952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72f415d600 0x7f72f40c82a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00efc0 tx=0x7f72ec00c490 comp rx=0 tx=0).stop 2024-04-23T14:32:41.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.127+0000 7f72fae00640 1 -- 172.21.15.100:0/4105791952 shutdown_connections 2024-04-23T14:32:41.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.127+0000 7f72fae00640 1 --2- 172.21.15.100:0/4105791952 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f72d40683b0 0x7f72d406a870 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.127+0000 7f72fae00640 1 --2- 172.21.15.100:0/4105791952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72f415d600 0x7f72f40c82a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.127+0000 7f72fae00640 1 --2- 172.21.15.100:0/4105791952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72f415c7e0 0x7f72f4157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.127+0000 7f72fae00640 1 --2- 172.21.15.100:0/4105791952 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72f40cca50 0x7f72f4149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.127+0000 7f72fae00640 1 -- 172.21.15.100:0/4105791952 >> 172.21.15.100:0/4105791952 conn(0x7f72f40cee80 msgr2=0x7f72f4156860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:41.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.127+0000 7f72fae00640 1 -- 172.21.15.100:0/4105791952 shutdown_connections 2024-04-23T14:32:41.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.127+0000 7f72fae00640 1 -- 172.21.15.100:0/4105791952 wait complete. 2024-04-23T14:32:41.148 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-23T14:32:41.148 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:41.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.283+0000 7f6213000640 1 Processor -- start 2024-04-23T14:32:41.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.283+0000 7f6213000640 1 -- start start 2024-04-23T14:32:41.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.283+0000 7f6213000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f620c06ec40 0x7f620c072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.283+0000 7f6213000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f620c0733c0 0x7f620c0cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.283+0000 7f6213000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f620c06f220 con 0x7f620c15c550 2024-04-23T14:32:41.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.283+0000 7f6213000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f620c073830 con 0x7f620c0733c0 2024-04-23T14:32:41.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.283+0000 7f6213000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f620c06f500 con 0x7f620c06ec40 2024-04-23T14:32:41.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.283+0000 7f6211000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f620c0733c0 0x7f620c0cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.283+0000 7f620be00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f620c15c550 0x7f620c0ce730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:44980/0 (socket says 172.21.15.100:44980) 2024-04-23T14:32:41.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.283+0000 7f6211000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f620c0733c0 0x7f620c0cfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53002/0 (socket says 172.21.15.100:53002) 2024-04-23T14:32:41.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.283+0000 7f620be00640 1 -- 172.21.15.100:0/406629193 learned_addr learned my addr 172.21.15.100:0/406629193 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:41.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.283+0000 7f620b400640 1 --2- 172.21.15.100:0/406629193 >> v2:172.21.15.151:3300/0 conn(0x7f620c06ec40 0x7f620c072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.283+0000 7f6211000640 1 -- 172.21.15.100:0/406629193 >> v2:172.21.15.151:3300/0 conn(0x7f620c06ec40 msgr2=0x7f620c072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:41.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.283+0000 7f6211000640 1 --2- 172.21.15.100:0/406629193 >> v2:172.21.15.151:3300/0 conn(0x7f620c06ec40 0x7f620c072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.283+0000 7f6211000640 1 -- 172.21.15.100:0/406629193 >> v1:172.21.15.100:6789/0 conn(0x7f620c15c550 legacy=0x7f620c0ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.283+0000 7f6211000640 1 -- 172.21.15.100:0/406629193 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620c06f7e0 con 0x7f620c0733c0 2024-04-23T14:32:41.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.283+0000 7f620aa00640 1 -- 172.21.15.100:0/406629193 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2364512269 0 0) 0x7f620c06f220 con 0x7f620c15c550 2024-04-23T14:32:41.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.283+0000 7f620b400640 1 --2- 172.21.15.100:0/406629193 >> v2:172.21.15.151:3300/0 conn(0x7f620c06ec40 0x7f620c072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:41.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.287+0000 7f6211000640 1 --2- 172.21.15.100:0/406629193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f620c0733c0 0x7f620c0cfb90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f620c0cdae0 tx=0x7f620401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b5d55d66c1f348 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:41.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.287+0000 7f620aa00640 1 -- 172.21.15.100:0/406629193 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62040b0070 con 0x7f620c0733c0 2024-04-23T14:32:41.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.287+0000 7f620aa00640 1 -- 172.21.15.100:0/406629193 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62040ab600 con 0x7f620c0733c0 2024-04-23T14:32:41.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.287+0000 7f620aa00640 1 -- 172.21.15.100:0/406629193 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62040ab9d0 con 0x7f620c0733c0 2024-04-23T14:32:41.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.287+0000 7f6213000640 1 -- 172.21.15.100:0/406629193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f620c0733c0 msgr2=0x7f620c0cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.287+0000 7f6213000640 1 --2- 172.21.15.100:0/406629193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f620c0733c0 0x7f620c0cfb90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f620c0cdae0 tx=0x7f620401c3f0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.287+0000 7f6213000640 1 -- 172.21.15.100:0/406629193 shutdown_connections 2024-04-23T14:32:41.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.287+0000 7f6213000640 1 --2- 172.21.15.100:0/406629193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f620c0733c0 0x7f620c0cfb90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.287+0000 7f6213000640 1 --2- 172.21.15.100:0/406629193 >> v2:172.21.15.151:3300/0 conn(0x7f620c06ec40 0x7f620c072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.287+0000 7f6213000640 1 -- 172.21.15.100:0/406629193 >> 172.21.15.100:0/406629193 conn(0x7f620c0b3050 msgr2=0x7f620c0b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:41.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.287+0000 7f6213000640 1 -- 172.21.15.100:0/406629193 shutdown_connections 2024-04-23T14:32:41.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.287+0000 7f6213000640 1 -- 172.21.15.100:0/406629193 wait complete. 2024-04-23T14:32:41.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f6213000640 1 Processor -- start 2024-04-23T14:32:41.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f6213000640 1 -- start start 2024-04-23T14:32:41.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f6213000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f620c06ec40 0x7f620c1534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f6213000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f620c0733c0 0x7f620c1637a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f6213000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f620c15c550 0x7f620c163ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f6213000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f620c0d0a60 con 0x7f620c0733c0 2024-04-23T14:32:41.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f6213000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f620c0d08e0 con 0x7f620c15c550 2024-04-23T14:32:41.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f6213000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f620c0d0be0 con 0x7f620c06ec40 2024-04-23T14:32:41.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f620b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f620c0733c0 0x7f620c1637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f620b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f620c0733c0 0x7f620c1637a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53016/0 (socket says 172.21.15.100:53016) 2024-04-23T14:32:41.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f620b400640 1 -- 172.21.15.100:0/169678722 learned_addr learned my addr 172.21.15.100:0/169678722 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:41.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f6211000640 1 --2- 172.21.15.100:0/169678722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f620c15c550 0x7f620c163ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f620be00640 1 --2- 172.21.15.100:0/169678722 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f620c06ec40 0x7f620c1534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f620b400640 1 -- 172.21.15.100:0/169678722 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f620c06ec40 msgr2=0x7f620c1534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f620b400640 1 --2- 172.21.15.100:0/169678722 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f620c06ec40 0x7f620c1534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f620b400640 1 -- 172.21.15.100:0/169678722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f620c15c550 msgr2=0x7f620c163ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f620b400640 1 --2- 172.21.15.100:0/169678722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f620c15c550 0x7f620c163ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f620b400640 1 -- 172.21.15.100:0/169678722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620c162580 con 0x7f620c0733c0 2024-04-23T14:32:41.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f620b400640 1 --2- 172.21.15.100:0/169678722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f620c0733c0 0x7f620c1637a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f61f800d800 tx=0x7f61f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:41.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f6203e00640 1 -- 172.21.15.100:0/169678722 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f8018020 con 0x7f620c0733c0 2024-04-23T14:32:41.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f6203e00640 1 -- 172.21.15.100:0/169678722 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61f80096c0 con 0x7f620c0733c0 2024-04-23T14:32:41.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f6213000640 1 -- 172.21.15.100:0/169678722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f620c154cf0 con 0x7f620c0733c0 2024-04-23T14:32:41.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f6203e00640 1 -- 172.21.15.100:0/169678722 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f8009ab0 con 0x7f620c0733c0 2024-04-23T14:32:41.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f6213000640 1 -- 172.21.15.100:0/169678722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f620c0cea10 con 0x7f620c0733c0 2024-04-23T14:32:41.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.291+0000 7f6213000640 1 -- 172.21.15.100:0/169678722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f620c068ad0 con 0x7f620c0733c0 2024-04-23T14:32:41.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.295+0000 7f6203e00640 1 -- 172.21.15.100:0/169678722 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f61f801b020 con 0x7f620c0733c0 2024-04-23T14:32:41.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.295+0000 7f6203e00640 1 --2- 172.21.15.100:0/169678722 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f61ec068380 0x7f61ec06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.295+0000 7f6203e00640 1 -- 172.21.15.100:0/169678722 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f61f80871f0 con 0x7f620c0733c0 2024-04-23T14:32:41.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.295+0000 7f620be00640 1 --2- 172.21.15.100:0/169678722 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f61ec068380 0x7f61ec06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.295+0000 7f620be00640 1 --2- 172.21.15.100:0/169678722 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f61ec068380 0x7f61ec06a840 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0012f0 tx=0x7f61fc005ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:41.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.299+0000 7f6203e00640 1 -- 172.21.15.100:0/169678722 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f61f8022020 con 0x7f620c0733c0 2024-04-23T14:32:41.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.455+0000 7f6213000640 1 -- 172.21.15.100:0/169678722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f620c0cf330 con 0x7f620c0733c0 2024-04-23T14:32:41.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.459+0000 7f6203e00640 1 -- 172.21.15.100:0/169678722 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f61f8057460 con 0x7f620c0733c0 2024-04-23T14:32:41.458 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:41.459 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":24,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:14.037027+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:41.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.459+0000 7f6213000640 1 -- 172.21.15.100:0/169678722 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f61ec068380 msgr2=0x7f61ec06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.459+0000 7f6213000640 1 --2- 172.21.15.100:0/169678722 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f61ec068380 0x7f61ec06a840 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0012f0 tx=0x7f61fc005ec0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.459+0000 7f6213000640 1 -- 172.21.15.100:0/169678722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f620c0733c0 msgr2=0x7f620c1637a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.459+0000 7f6213000640 1 --2- 172.21.15.100:0/169678722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f620c0733c0 0x7f620c1637a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f61f800d800 tx=0x7f61f800dcc0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.459+0000 7f6213000640 1 -- 172.21.15.100:0/169678722 shutdown_connections 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.459+0000 7f6213000640 1 --2- 172.21.15.100:0/169678722 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f61ec068380 0x7f61ec06a840 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.459+0000 7f6213000640 1 --2- 172.21.15.100:0/169678722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f620c15c550 0x7f620c163ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.459+0000 7f6213000640 1 --2- 172.21.15.100:0/169678722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f620c0733c0 0x7f620c1637a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.459+0000 7f6213000640 1 --2- 172.21.15.100:0/169678722 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f620c06ec40 0x7f620c1534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.459+0000 7f6213000640 1 -- 172.21.15.100:0/169678722 >> 172.21.15.100:0/169678722 conn(0x7f620c0b3050 msgr2=0x7f620c157010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.459+0000 7f6213000640 1 -- 172.21.15.100:0/169678722 shutdown_connections 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.459+0000 7f6213000640 1 -- 172.21.15.100:0/169678722 wait complete. 2024-04-23T14:32:41.479 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:32:06.835560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:32:07.930489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-23T14:32:11.636814+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-23T14:32:41.479 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:41.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.619+0000 7fa5a0a00640 1 Processor -- start 2024-04-23T14:32:41.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.619+0000 7fa5a0a00640 1 -- start start 2024-04-23T14:32:41.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.619+0000 7fa5a0a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa59c06d450 0x7fa59c06d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.619+0000 7fa5a0a00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fa59c0cd460 0x7fa59c0b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.619+0000 7fa5a0a00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa59c06dd70 con 0x7fa59c0cea60 2024-04-23T14:32:41.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.619+0000 7fa5a0a00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fa59c0b1f90 con 0x7fa59c06d450 2024-04-23T14:32:41.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.619+0000 7fa5a0a00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fa59c0b2270 con 0x7fa59c0cd460 2024-04-23T14:32:41.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.619+0000 7fa599e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa59c06d450 0x7fa59c06d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.619+0000 7fa599400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fa59c0cea60 0x7fa59c0d0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:44984/0 (socket says 172.21.15.100:44984) 2024-04-23T14:32:41.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.619+0000 7fa599400640 1 -- 172.21.15.100:0/1826695691 learned_addr learned my addr 172.21.15.100:0/1826695691 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:41.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.619+0000 7fa59a800640 1 --2- 172.21.15.100:0/1826695691 >> v2:172.21.15.151:3300/0 conn(0x7fa59c0cd460 0x7fa59c0b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.619+0000 7fa599e00640 1 -- 172.21.15.100:0/1826695691 >> v2:172.21.15.151:3300/0 conn(0x7fa59c0cd460 msgr2=0x7fa59c0b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:41.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.619+0000 7fa599e00640 1 --2- 172.21.15.100:0/1826695691 >> v2:172.21.15.151:3300/0 conn(0x7fa59c0cd460 0x7fa59c0b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.619+0000 7fa599e00640 1 -- 172.21.15.100:0/1826695691 >> v1:172.21.15.100:6789/0 conn(0x7fa59c0cea60 legacy=0x7fa59c0d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.619+0000 7fa599e00640 1 -- 172.21.15.100:0/1826695691 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59c0b2550 con 0x7fa59c06d450 2024-04-23T14:32:41.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.619+0000 7fa598a00640 1 -- 172.21.15.100:0/1826695691 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1590333807 0 0) 0x7fa59c06dd70 con 0x7fa59c0cea60 2024-04-23T14:32:41.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.623+0000 7fa599e00640 1 --2- 172.21.15.100:0/1826695691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa59c06d450 0x7fa59c06d830 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0ce160 tx=0x7fa58c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6395d5227561f1de server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:41.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.623+0000 7fa598a00640 1 -- 172.21.15.100:0/1826695691 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c0b0070 con 0x7fa59c06d450 2024-04-23T14:32:41.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.623+0000 7fa598a00640 1 -- 172.21.15.100:0/1826695691 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa58c0ab600 con 0x7fa59c06d450 2024-04-23T14:32:41.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.623+0000 7fa598a00640 1 -- 172.21.15.100:0/1826695691 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c0ab9d0 con 0x7fa59c06d450 2024-04-23T14:32:41.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.623+0000 7fa5a0a00640 1 -- 172.21.15.100:0/1826695691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa59c06d450 msgr2=0x7fa59c06d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.623+0000 7fa5a0a00640 1 --2- 172.21.15.100:0/1826695691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa59c06d450 0x7fa59c06d830 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0ce160 tx=0x7fa58c01c3f0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.623+0000 7fa5a0a00640 1 -- 172.21.15.100:0/1826695691 shutdown_connections 2024-04-23T14:32:41.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.623+0000 7fa5a0a00640 1 --2- 172.21.15.100:0/1826695691 >> v2:172.21.15.151:3300/0 conn(0x7fa59c0cd460 0x7fa59c0b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.623+0000 7fa5a0a00640 1 --2- 172.21.15.100:0/1826695691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa59c06d450 0x7fa59c06d830 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.623+0000 7fa5a0a00640 1 -- 172.21.15.100:0/1826695691 >> 172.21.15.100:0/1826695691 conn(0x7fa59c0cee80 msgr2=0x7fa59c0cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:41.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.623+0000 7fa5a0a00640 1 -- 172.21.15.100:0/1826695691 shutdown_connections 2024-04-23T14:32:41.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.623+0000 7fa5a0a00640 1 -- 172.21.15.100:0/1826695691 wait complete. 2024-04-23T14:32:41.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa5a0a00640 1 Processor -- start 2024-04-23T14:32:41.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa5a0a00640 1 -- start start 2024-04-23T14:32:41.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa5a0a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa59c06d450 0x7fa59c0caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa5a0a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa59c0cd460 0x7fa59c143680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa5a0a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa59c0cea60 0x7fa59c157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa5a0a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c16b740 con 0x7fa59c0cea60 2024-04-23T14:32:41.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa5a0a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c16b5c0 con 0x7fa59c0cd460 2024-04-23T14:32:41.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa5a0a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa59c16b8c0 con 0x7fa59c06d450 2024-04-23T14:32:41.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa59a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa59c0cea60 0x7fa59c157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa59a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa59c0cea60 0x7fa59c157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53034/0 (socket says 172.21.15.100:53034) 2024-04-23T14:32:41.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa59a800640 1 -- 172.21.15.100:0/1372506655 learned_addr learned my addr 172.21.15.100:0/1372506655 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:41.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa599400640 1 --2- 172.21.15.100:0/1372506655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa59c0cd460 0x7fa59c143680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa599e00640 1 --2- 172.21.15.100:0/1372506655 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa59c06d450 0x7fa59c0caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa59a800640 1 -- 172.21.15.100:0/1372506655 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa59c06d450 msgr2=0x7fa59c0caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa59a800640 1 --2- 172.21.15.100:0/1372506655 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa59c06d450 0x7fa59c0caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa59a800640 1 -- 172.21.15.100:0/1372506655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa59c0cd460 msgr2=0x7fa59c143680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa59a800640 1 --2- 172.21.15.100:0/1372506655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa59c0cd460 0x7fa59c143680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa59a800640 1 -- 172.21.15.100:0/1372506655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59c157880 con 0x7fa59c0cea60 2024-04-23T14:32:41.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa599e00640 1 --2- 172.21.15.100:0/1372506655 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa59c06d450 0x7fa59c0caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:41.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa59a800640 1 --2- 172.21.15.100:0/1372506655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa59c0cea60 0x7fa59c157030 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa59400efc0 tx=0x7fa59400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:41.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa592000640 1 -- 172.21.15.100:0/1372506655 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594018020 con 0x7fa59c0cea60 2024-04-23T14:32:41.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa592000640 1 -- 172.21.15.100:0/1372506655 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa594014480 con 0x7fa59c0cea60 2024-04-23T14:32:41.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa5a0a00640 1 -- 172.21.15.100:0/1372506655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa59c157b60 con 0x7fa59c0cea60 2024-04-23T14:32:41.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa592000640 1 -- 172.21.15.100:0/1372506655 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594014870 con 0x7fa59c0cea60 2024-04-23T14:32:41.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.627+0000 7fa5a0a00640 1 -- 172.21.15.100:0/1372506655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa59c158120 con 0x7fa59c0cea60 2024-04-23T14:32:41.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.631+0000 7fa5a0a00640 1 -- 172.21.15.100:0/1372506655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa59c068ad0 con 0x7fa59c0cea60 2024-04-23T14:32:41.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.631+0000 7fa592000640 1 -- 172.21.15.100:0/1372506655 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa59401b020 con 0x7fa59c0cea60 2024-04-23T14:32:41.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.631+0000 7fa592000640 1 --2- 172.21.15.100:0/1372506655 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa57c0683b0 0x7fa57c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.631+0000 7fa592000640 1 -- 172.21.15.100:0/1372506655 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa594087740 con 0x7fa59c0cea60 2024-04-23T14:32:41.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.631+0000 7fa599e00640 1 --2- 172.21.15.100:0/1372506655 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa57c0683b0 0x7fa57c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.631+0000 7fa599e00640 1 --2- 172.21.15.100:0/1372506655 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa57c0683b0 0x7fa57c06a870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0ad000 tx=0x7fa58c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:41.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.635+0000 7fa592000640 1 -- 172.21.15.100:0/1372506655 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa594022070 con 0x7fa59c0cea60 2024-04-23T14:32:41.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.795+0000 7fa5a0a00640 1 -- 172.21.15.100:0/1372506655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa59c0cf470 con 0x7fa59c0cea60 2024-04-23T14:32:41.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.795+0000 7fa592000640 1 -- 172.21.15.100:0/1372506655 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fa594010040 con 0x7fa59c0cea60 2024-04-23T14:32:41.797 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:32:41.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.799+0000 7fa5a0a00640 1 -- 172.21.15.100:0/1372506655 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa57c0683b0 msgr2=0x7fa57c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.799+0000 7fa5a0a00640 1 --2- 172.21.15.100:0/1372506655 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa57c0683b0 0x7fa57c06a870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0ad000 tx=0x7fa58c008040 comp rx=0 tx=0).stop 2024-04-23T14:32:41.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.799+0000 7fa5a0a00640 1 -- 172.21.15.100:0/1372506655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa59c0cea60 msgr2=0x7fa59c157030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.799+0000 7fa5a0a00640 1 --2- 172.21.15.100:0/1372506655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa59c0cea60 0x7fa59c157030 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa59400efc0 tx=0x7fa59400c490 comp rx=0 tx=0).stop 2024-04-23T14:32:41.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.799+0000 7fa5a0a00640 1 -- 172.21.15.100:0/1372506655 shutdown_connections 2024-04-23T14:32:41.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.799+0000 7fa5a0a00640 1 --2- 172.21.15.100:0/1372506655 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa57c0683b0 0x7fa57c06a870 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.799+0000 7fa5a0a00640 1 --2- 172.21.15.100:0/1372506655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa59c0cea60 0x7fa59c157030 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.799+0000 7fa5a0a00640 1 --2- 172.21.15.100:0/1372506655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa59c0cd460 0x7fa59c143680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.799+0000 7fa5a0a00640 1 --2- 172.21.15.100:0/1372506655 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa59c06d450 0x7fa59c0caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.799+0000 7fa5a0a00640 1 -- 172.21.15.100:0/1372506655 >> 172.21.15.100:0/1372506655 conn(0x7fa59c0cee80 msgr2=0x7fa59c070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:41.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.799+0000 7fa5a0a00640 1 -- 172.21.15.100:0/1372506655 shutdown_connections 2024-04-23T14:32:41.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.799+0000 7fa5a0a00640 1 -- 172.21.15.100:0/1372506655 wait complete. 2024-04-23T14:32:41.819 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:32:41.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.955+0000 7f566f800640 1 Processor -- start 2024-04-23T14:32:41.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.955+0000 7f566f800640 1 -- start start 2024-04-23T14:32:41.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.955+0000 7f566f800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f566815c7a0 0x7f56680af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.955+0000 7f566f800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f56680cd300 0x7f56680ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.959+0000 7f566f800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f566806cbb0 con 0x7f566806bb20 2024-04-23T14:32:41.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.959+0000 7f566f800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f566806d450 con 0x7f56680cd300 2024-04-23T14:32:41.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.959+0000 7f566f800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f566806ce90 con 0x7f566815c7a0 2024-04-23T14:32:41.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.959+0000 7f566d800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f56680cd300 0x7f56680ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.959+0000 7f566d800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f56680cd300 0x7f56680ad430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53040/0 (socket says 172.21.15.100:53040) 2024-04-23T14:32:41.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.959+0000 7f566d800640 1 -- 172.21.15.100:0/4081073421 learned_addr learned my addr 172.21.15.100:0/4081073421 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:41.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.959+0000 7f565fe00640 1 --1- 172.21.15.100:0/4081073421 >> v1:172.21.15.100:6789/0 conn(0x7f566806bb20 0x7f56680ccbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:44996/0 (socket says 172.21.15.100:44996) 2024-04-23T14:32:41.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.959+0000 7f566ce00640 1 --2- 172.21.15.100:0/4081073421 >> v2:172.21.15.151:3300/0 conn(0x7f566815c7a0 0x7f56680af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.959+0000 7f566d800640 1 -- 172.21.15.100:0/4081073421 >> v2:172.21.15.151:3300/0 conn(0x7f566815c7a0 msgr2=0x7f56680af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:41.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.959+0000 7f566d800640 1 --2- 172.21.15.100:0/4081073421 >> v2:172.21.15.151:3300/0 conn(0x7f566815c7a0 0x7f56680af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.959+0000 7f566d800640 1 -- 172.21.15.100:0/4081073421 >> v1:172.21.15.100:6789/0 conn(0x7f566806bb20 legacy=0x7f56680ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.959+0000 7f566d800640 1 -- 172.21.15.100:0/4081073421 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566806d170 con 0x7f56680cd300 2024-04-23T14:32:41.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.959+0000 7f5667e00640 1 -- 172.21.15.100:0/4081073421 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1458502324 0 0) 0x7f566806cbb0 con 0x7f566806bb20 2024-04-23T14:32:41.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.959+0000 7f566d800640 1 --2- 172.21.15.100:0/4081073421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56680cd300 0x7f56680ad430 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f56680716b0 tx=0x7f566001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3315ec7e7c5bcfc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:41.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.959+0000 7f5667e00640 1 -- 172.21.15.100:0/4081073421 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56600a5620 con 0x7f56680cd300 2024-04-23T14:32:41.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.959+0000 7f5667e00640 1 -- 172.21.15.100:0/4081073421 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56600a57c0 con 0x7f56680cd300 2024-04-23T14:32:41.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.959+0000 7f5667e00640 1 -- 172.21.15.100:0/4081073421 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56600a5b90 con 0x7f56680cd300 2024-04-23T14:32:41.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566f800640 1 -- 172.21.15.100:0/4081073421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56680cd300 msgr2=0x7f56680ad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566f800640 1 --2- 172.21.15.100:0/4081073421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56680cd300 0x7f56680ad430 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f56680716b0 tx=0x7f566001c960 comp rx=0 tx=0).stop 2024-04-23T14:32:41.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566f800640 1 -- 172.21.15.100:0/4081073421 shutdown_connections 2024-04-23T14:32:41.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566f800640 1 --2- 172.21.15.100:0/4081073421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56680cd300 0x7f56680ad430 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566f800640 1 --2- 172.21.15.100:0/4081073421 >> v2:172.21.15.151:3300/0 conn(0x7f566815c7a0 0x7f56680af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566f800640 1 -- 172.21.15.100:0/4081073421 >> 172.21.15.100:0/4081073421 conn(0x7f56680b5200 msgr2=0x7f56680b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:41.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566f800640 1 -- 172.21.15.100:0/4081073421 shutdown_connections 2024-04-23T14:32:41.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566f800640 1 -- 172.21.15.100:0/4081073421 wait complete. 2024-04-23T14:32:41.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566f800640 1 Processor -- start 2024-04-23T14:32:41.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566f800640 1 -- start start 2024-04-23T14:32:41.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566f800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f566806bb20 0x7f5668155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566f800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56680cd300 0x7f566815e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566f800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f566815c7a0 0x7f56680cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566f800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f56680ae590 con 0x7f566815c7a0 2024-04-23T14:32:41.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566f800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f56680ae410 con 0x7f56680cd300 2024-04-23T14:32:41.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566f800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f56680ae710 con 0x7f566806bb20 2024-04-23T14:32:41.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f566815c7a0 0x7f56680cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f566815c7a0 0x7f56680cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53056/0 (socket says 172.21.15.100:53056) 2024-04-23T14:32:41.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566d800640 1 -- 172.21.15.100:0/691406024 learned_addr learned my addr 172.21.15.100:0/691406024 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:41.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f565fe00640 1 --2- 172.21.15.100:0/691406024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56680cd300 0x7f566815e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566ce00640 1 --2- 172.21.15.100:0/691406024 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f566806bb20 0x7f5668155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566d800640 1 -- 172.21.15.100:0/691406024 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f566806bb20 msgr2=0x7f5668155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566d800640 1 --2- 172.21.15.100:0/691406024 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f566806bb20 0x7f5668155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566d800640 1 -- 172.21.15.100:0/691406024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56680cd300 msgr2=0x7f566815e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566d800640 1 --2- 172.21.15.100:0/691406024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56680cd300 0x7f566815e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f566d800640 1 -- 172.21.15.100:0/691406024 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56680b2f50 con 0x7f566815c7a0 2024-04-23T14:32:41.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.963+0000 7f565fe00640 1 --2- 172.21.15.100:0/691406024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56680cd300 0x7f566815e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:41.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.967+0000 7f566d800640 1 --2- 172.21.15.100:0/691406024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f566815c7a0 0x7f56680cc0c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f56600af040 tx=0x7f5660019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:41.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.967+0000 7f5665600640 1 -- 172.21.15.100:0/691406024 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56600bc020 con 0x7f566815c7a0 2024-04-23T14:32:41.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.967+0000 7f5665600640 1 -- 172.21.15.100:0/691406024 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56600b88a0 con 0x7f566815c7a0 2024-04-23T14:32:41.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.967+0000 7f566f800640 1 -- 172.21.15.100:0/691406024 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5668073a50 con 0x7f566815c7a0 2024-04-23T14:32:41.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.967+0000 7f5665600640 1 -- 172.21.15.100:0/691406024 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660009510 con 0x7f566815c7a0 2024-04-23T14:32:41.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.967+0000 7f566f800640 1 -- 172.21.15.100:0/691406024 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56680ccd30 con 0x7f566815c7a0 2024-04-23T14:32:41.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.967+0000 7f566f800640 1 -- 172.21.15.100:0/691406024 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5668068ad0 con 0x7f566815c7a0 2024-04-23T14:32:41.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.967+0000 7f5665600640 1 -- 172.21.15.100:0/691406024 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f56600bf020 con 0x7f566815c7a0 2024-04-23T14:32:41.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.967+0000 7f5665600640 1 --2- 172.21.15.100:0/691406024 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5644068380 0x7f564406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.967+0000 7f5665600640 1 -- 172.21.15.100:0/691406024 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5660121e10 con 0x7f566815c7a0 2024-04-23T14:32:41.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.967+0000 7f566ce00640 1 --2- 172.21.15.100:0/691406024 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5644068380 0x7f564406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.971+0000 7f566ce00640 1 --2- 172.21.15.100:0/691406024 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5644068380 0x7f564406a840 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f565800b9d0 tx=0x7f5658008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:41.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:41.971+0000 7f5665600640 1 -- 172.21.15.100:0/691406024 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5660125030 con 0x7f566815c7a0 2024-04-23T14:32:42.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.131+0000 7f566f800640 1 -- 172.21.15.100:0/691406024 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5668074f20 con 0x7f566815c7a0 2024-04-23T14:32:42.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.131+0000 7f5665600640 1 -- 172.21.15.100:0/691406024 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f56600c6030 con 0x7f566815c7a0 2024-04-23T14:32:42.133 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:32:42.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.135+0000 7f566f800640 1 -- 172.21.15.100:0/691406024 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5644068380 msgr2=0x7f564406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.135+0000 7f566f800640 1 --2- 172.21.15.100:0/691406024 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5644068380 0x7f564406a840 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f565800b9d0 tx=0x7f5658008040 comp rx=0 tx=0).stop 2024-04-23T14:32:42.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.135+0000 7f566f800640 1 -- 172.21.15.100:0/691406024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f566815c7a0 msgr2=0x7f56680cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.135+0000 7f566f800640 1 --2- 172.21.15.100:0/691406024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f566815c7a0 0x7f56680cc0c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f56600af040 tx=0x7f5660019700 comp rx=0 tx=0).stop 2024-04-23T14:32:42.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.135+0000 7f566f800640 1 -- 172.21.15.100:0/691406024 shutdown_connections 2024-04-23T14:32:42.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.135+0000 7f566f800640 1 --2- 172.21.15.100:0/691406024 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5644068380 0x7f564406a840 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.135+0000 7f566f800640 1 --2- 172.21.15.100:0/691406024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f566815c7a0 0x7f56680cc0c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.135+0000 7f566f800640 1 --2- 172.21.15.100:0/691406024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56680cd300 0x7f566815e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.135+0000 7f566f800640 1 --2- 172.21.15.100:0/691406024 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f566806bb20 0x7f5668155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.135+0000 7f566f800640 1 -- 172.21.15.100:0/691406024 >> 172.21.15.100:0/691406024 conn(0x7f56680b5200 msgr2=0x7f56680b55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:42.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.135+0000 7f566f800640 1 -- 172.21.15.100:0/691406024 shutdown_connections 2024-04-23T14:32:42.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.135+0000 7f566f800640 1 -- 172.21.15.100:0/691406024 wait complete. 2024-04-23T14:32:42.153 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:32:42.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.291+0000 7fd006800640 1 Processor -- start 2024-04-23T14:32:42.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.291+0000 7fd006800640 1 -- start start 2024-04-23T14:32:42.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.291+0000 7fd006800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd00015ae60 0x7fd0000cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.291+0000 7fd006800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0000cacb0 0x7fd00006d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.291+0000 7fd006800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0000cfc70 con 0x7fd0000ca830 2024-04-23T14:32:42.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.291+0000 7fd006800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fd00015b240 con 0x7fd00015ae60 2024-04-23T14:32:42.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.291+0000 7fd006800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd0000cff10 con 0x7fd0000cacb0 2024-04-23T14:32:42.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.291+0000 7fcfff400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd00015ae60 0x7fd0000cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.291+0000 7fcfff400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd00015ae60 0x7fd0000cf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:44754/0 (socket says 172.21.15.100:44754) 2024-04-23T14:32:42.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.291+0000 7fcfff400640 1 -- 172.21.15.100:0/488936076 learned_addr learned my addr 172.21.15.100:0/488936076 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:42.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.291+0000 7fcfffe00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0000cacb0 0x7fd00006d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.291+0000 7fcfff400640 1 -- 172.21.15.100:0/488936076 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0000cacb0 msgr2=0x7fd00006d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:42.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.291+0000 7fcfff400640 1 --2- 172.21.15.100:0/488936076 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0000cacb0 0x7fd00006d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.291+0000 7fcfff400640 1 -- 172.21.15.100:0/488936076 >> v1:172.21.15.151:6789/0 conn(0x7fd0000ca830 legacy=0x7fd00006c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.291+0000 7fcfff400640 1 -- 172.21.15.100:0/488936076 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00006bbe0 con 0x7fd00015ae60 2024-04-23T14:32:42.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.295+0000 7fcfff400640 1 --2- 172.21.15.100:0/488936076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd00015ae60 0x7fd0000cf730 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009870 tx=0x7fcff001e6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebd6bf230fe5ed2a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:42.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.295+0000 7fcffe000640 1 -- 172.21.15.100:0/488936076 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff00ae070 con 0x7fd00015ae60 2024-04-23T14:32:42.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.295+0000 7fcffe000640 1 -- 172.21.15.100:0/488936076 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff00a8680 con 0x7fd00015ae60 2024-04-23T14:32:42.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.295+0000 7fcffe000640 1 -- 172.21.15.100:0/488936076 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff00a8a50 con 0x7fd00015ae60 2024-04-23T14:32:42.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.295+0000 7fd006800640 1 -- 172.21.15.100:0/488936076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd00015ae60 msgr2=0x7fd0000cf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.295+0000 7fd006800640 1 --2- 172.21.15.100:0/488936076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd00015ae60 0x7fd0000cf730 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009870 tx=0x7fcff001e6b0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.295+0000 7fd006800640 1 -- 172.21.15.100:0/488936076 shutdown_connections 2024-04-23T14:32:42.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.295+0000 7fd006800640 1 --2- 172.21.15.100:0/488936076 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0000cacb0 0x7fd00006d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.295+0000 7fd006800640 1 --2- 172.21.15.100:0/488936076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd00015ae60 0x7fd0000cf730 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.295+0000 7fd006800640 1 -- 172.21.15.100:0/488936076 >> 172.21.15.100:0/488936076 conn(0x7fd0000b5200 msgr2=0x7fd0000b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:42.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.295+0000 7fd006800640 1 -- 172.21.15.100:0/488936076 shutdown_connections 2024-04-23T14:32:42.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.295+0000 7fd006800640 1 -- 172.21.15.100:0/488936076 wait complete. 2024-04-23T14:32:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fd006800640 1 Processor -- start 2024-04-23T14:32:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fd006800640 1 -- start start 2024-04-23T14:32:42.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fd006800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0000ca830 0x7fd000099a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fd006800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0000cacb0 0x7fd0000cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fd006800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd00015ae60 0x7fd0000cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fd006800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd0000cd3b0 con 0x7fd00015ae60 2024-04-23T14:32:42.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fd006800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd0000cd230 con 0x7fd0000ca830 2024-04-23T14:32:42.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fd006800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd0000cd530 con 0x7fd0000cacb0 2024-04-23T14:32:42.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fcfffe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd00015ae60 0x7fd0000cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fcfffe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd00015ae60 0x7fd0000cbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53060/0 (socket says 172.21.15.100:53060) 2024-04-23T14:32:42.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fcfffe00640 1 -- 172.21.15.100:0/1052605973 learned_addr learned my addr 172.21.15.100:0/1052605973 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:42.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fcfff400640 1 --2- 172.21.15.100:0/1052605973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0000ca830 0x7fd000099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fcffea00640 1 --2- 172.21.15.100:0/1052605973 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0000cacb0 0x7fd0000cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fcfffe00640 1 -- 172.21.15.100:0/1052605973 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0000cacb0 msgr2=0x7fd0000cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fcfffe00640 1 --2- 172.21.15.100:0/1052605973 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0000cacb0 0x7fd0000cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fcfffe00640 1 -- 172.21.15.100:0/1052605973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0000ca830 msgr2=0x7fd000099a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fcfffe00640 1 --2- 172.21.15.100:0/1052605973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0000ca830 0x7fd000099a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fcfffe00640 1 -- 172.21.15.100:0/1052605973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0001545d0 con 0x7fd00015ae60 2024-04-23T14:32:42.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fcfffe00640 1 --2- 172.21.15.100:0/1052605973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd00015ae60 0x7fd0000cbf60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcff800efc0 tx=0x7fcff800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:42.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fcff7400640 1 -- 172.21.15.100:0/1052605973 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8018020 con 0x7fd00015ae60 2024-04-23T14:32:42.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fcff7400640 1 -- 172.21.15.100:0/1052605973 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff8014480 con 0x7fd00015ae60 2024-04-23T14:32:42.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fcff7400640 1 -- 172.21.15.100:0/1052605973 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8014870 con 0x7fd00015ae60 2024-04-23T14:32:42.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fd006800640 1 -- 172.21.15.100:0/1052605973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd000153810 con 0x7fd00015ae60 2024-04-23T14:32:42.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.299+0000 7fd006800640 1 -- 172.21.15.100:0/1052605973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd000153dd0 con 0x7fd00015ae60 2024-04-23T14:32:42.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.303+0000 7fd006800640 1 -- 172.21.15.100:0/1052605973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd000068ad0 con 0x7fd00015ae60 2024-04-23T14:32:42.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.303+0000 7fcff7400640 1 -- 172.21.15.100:0/1052605973 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcff801b020 con 0x7fd00015ae60 2024-04-23T14:32:42.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.303+0000 7fcff7400640 1 --2- 172.21.15.100:0/1052605973 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcfe00683b0 0x7fcfe006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.303+0000 7fcff7400640 1 -- 172.21.15.100:0/1052605973 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fcff8087980 con 0x7fd00015ae60 2024-04-23T14:32:42.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.303+0000 7fcfff400640 1 --2- 172.21.15.100:0/1052605973 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcfe00683b0 0x7fcfe006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.303+0000 7fcfff400640 1 --2- 172.21.15.100:0/1052605973 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcfe00683b0 0x7fcfe006a870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcff00b0000 tx=0x7fcff0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:42.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.307+0000 7fcff7400640 1 -- 172.21.15.100:0/1052605973 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcff8022030 con 0x7fd00015ae60 2024-04-23T14:32:42.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.463+0000 7fd006800640 1 -- 172.21.15.100:0/1052605973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd000152640 con 0x7fd00015ae60 2024-04-23T14:32:42.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.463+0000 7fcff7400640 1 -- 172.21.15.100:0/1052605973 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fcff8010040 con 0x7fd00015ae60 2024-04-23T14:32:42.463 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:32:42.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.463+0000 7fd006800640 1 -- 172.21.15.100:0/1052605973 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcfe00683b0 msgr2=0x7fcfe006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.463+0000 7fd006800640 1 --2- 172.21.15.100:0/1052605973 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcfe00683b0 0x7fcfe006a870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcff00b0000 tx=0x7fcff0009040 comp rx=0 tx=0).stop 2024-04-23T14:32:42.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.463+0000 7fd006800640 1 -- 172.21.15.100:0/1052605973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd00015ae60 msgr2=0x7fd0000cbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.463+0000 7fd006800640 1 --2- 172.21.15.100:0/1052605973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd00015ae60 0x7fd0000cbf60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcff800efc0 tx=0x7fcff800c490 comp rx=0 tx=0).stop 2024-04-23T14:32:42.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.463+0000 7fd006800640 1 -- 172.21.15.100:0/1052605973 shutdown_connections 2024-04-23T14:32:42.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.463+0000 7fd006800640 1 --2- 172.21.15.100:0/1052605973 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcfe00683b0 0x7fcfe006a870 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.463+0000 7fd006800640 1 --2- 172.21.15.100:0/1052605973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd00015ae60 0x7fd0000cbf60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.463+0000 7fd006800640 1 --2- 172.21.15.100:0/1052605973 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0000cacb0 0x7fd0000cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.463+0000 7fd006800640 1 --2- 172.21.15.100:0/1052605973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0000ca830 0x7fd000099a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.463+0000 7fd006800640 1 -- 172.21.15.100:0/1052605973 >> 172.21.15.100:0/1052605973 conn(0x7fd0000b5200 msgr2=0x7fd000155e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:42.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.463+0000 7fd006800640 1 -- 172.21.15.100:0/1052605973 shutdown_connections 2024-04-23T14:32:42.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.463+0000 7fd006800640 1 -- 172.21.15.100:0/1052605973 wait complete. 2024-04-23T14:32:42.483 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-23T14:32:42.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.619+0000 7f1e4a200640 1 Processor -- start 2024-04-23T14:32:42.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.619+0000 7f1e4a200640 1 -- start start 2024-04-23T14:32:42.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.619+0000 7f1e4a200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4415e860 0x7f1e440b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.619+0000 7f1e4a200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1e440adfb0 0x7f1e4406d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.619+0000 7f1e4a200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e440ae3e0 con 0x7f1e440adb90 2024-04-23T14:32:42.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.619+0000 7f1e4a200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f1e4406e5f0 con 0x7f1e440adfb0 2024-04-23T14:32:42.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.619+0000 7f1e4a200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1e4406e8b0 con 0x7f1e4415e860 2024-04-23T14:32:42.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.619+0000 7f1e43e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1e440adfb0 0x7f1e4406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.619+0000 7f1e43e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1e440adfb0 0x7f1e4406d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53066/0 (socket says 172.21.15.100:53066) 2024-04-23T14:32:42.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.619+0000 7f1e42a00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1e440adb90 0x7f1e4406f950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:44998/0 (socket says 172.21.15.100:44998) 2024-04-23T14:32:42.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.619+0000 7f1e43e00640 1 -- 172.21.15.100:0/1647372840 learned_addr learned my addr 172.21.15.100:0/1647372840 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:42.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.619+0000 7f1e43400640 1 --2- 172.21.15.100:0/1647372840 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4415e860 0x7f1e440b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.623+0000 7f1e43e00640 1 -- 172.21.15.100:0/1647372840 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4415e860 msgr2=0x7f1e440b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:42.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.623+0000 7f1e43e00640 1 --2- 172.21.15.100:0/1647372840 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4415e860 0x7f1e440b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.623+0000 7f1e43e00640 1 -- 172.21.15.100:0/1647372840 >> v1:172.21.15.100:6789/0 conn(0x7f1e440adb90 legacy=0x7f1e4406f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.623+0000 7f1e43e00640 1 -- 172.21.15.100:0/1647372840 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4406eb90 con 0x7f1e440adfb0 2024-04-23T14:32:42.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.623+0000 7f1e42000640 1 -- 172.21.15.100:0/1647372840 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2800424524 0 0) 0x7f1e440ae3e0 con 0x7f1e440adb90 2024-04-23T14:32:42.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.623+0000 7f1e43e00640 1 --2- 172.21.15.100:0/1647372840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e440adfb0 0x7f1e4406d350 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c009e50 tx=0x7f1e3c01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0338957ee30cf6e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:42.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.623+0000 7f1e42000640 1 -- 172.21.15.100:0/1647372840 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c0a57e0 con 0x7f1e440adfb0 2024-04-23T14:32:42.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.623+0000 7f1e42000640 1 -- 172.21.15.100:0/1647372840 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e3c0a5980 con 0x7f1e440adfb0 2024-04-23T14:32:42.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.623+0000 7f1e42000640 1 -- 172.21.15.100:0/1647372840 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c0a5d50 con 0x7f1e440adfb0 2024-04-23T14:32:42.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.623+0000 7f1e4a200640 1 -- 172.21.15.100:0/1647372840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e440adfb0 msgr2=0x7f1e4406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.623+0000 7f1e4a200640 1 --2- 172.21.15.100:0/1647372840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e440adfb0 0x7f1e4406d350 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c009e50 tx=0x7f1e3c01ca00 comp rx=0 tx=0).stop 2024-04-23T14:32:42.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.623+0000 7f1e4a200640 1 -- 172.21.15.100:0/1647372840 shutdown_connections 2024-04-23T14:32:42.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.623+0000 7f1e4a200640 1 --2- 172.21.15.100:0/1647372840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e440adfb0 0x7f1e4406d350 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.623+0000 7f1e4a200640 1 --2- 172.21.15.100:0/1647372840 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4415e860 0x7f1e440b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.623+0000 7f1e4a200640 1 -- 172.21.15.100:0/1647372840 >> 172.21.15.100:0/1647372840 conn(0x7f1e44072590 msgr2=0x7f1e440729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:42.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.623+0000 7f1e4a200640 1 -- 172.21.15.100:0/1647372840 shutdown_connections 2024-04-23T14:32:42.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.623+0000 7f1e4a200640 1 -- 172.21.15.100:0/1647372840 wait complete. 2024-04-23T14:32:42.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e4a200640 1 Processor -- start 2024-04-23T14:32:42.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e4a200640 1 -- start start 2024-04-23T14:32:42.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e4a200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e440adb90 0x7f1e44142a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e4a200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e440adfb0 0x7f1e4415b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e4a200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4415e860 0x7f1e44149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e4a200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1e44073790 con 0x7f1e440adfb0 2024-04-23T14:32:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e4a200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e440b3ff0 con 0x7f1e440adb90 2024-04-23T14:32:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e4a200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1e4415ec40 con 0x7f1e4415e860 2024-04-23T14:32:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e42a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e440adfb0 0x7f1e4415b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e42a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e440adfb0 0x7f1e4415b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53072/0 (socket says 172.21.15.100:53072) 2024-04-23T14:32:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e42a00640 1 -- 172.21.15.100:0/2693609502 learned_addr learned my addr 172.21.15.100:0/2693609502 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e43400640 1 --2- 172.21.15.100:0/2693609502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e440adb90 0x7f1e44142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e43e00640 1 --2- 172.21.15.100:0/2693609502 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4415e860 0x7f1e44149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e42a00640 1 -- 172.21.15.100:0/2693609502 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4415e860 msgr2=0x7f1e44149820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e42a00640 1 --2- 172.21.15.100:0/2693609502 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4415e860 0x7f1e44149820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e42a00640 1 -- 172.21.15.100:0/2693609502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e440adb90 msgr2=0x7f1e44142a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e42a00640 1 --2- 172.21.15.100:0/2693609502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e440adb90 0x7f1e44142a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e42a00640 1 -- 172.21.15.100:0/2693609502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e44155400 con 0x7f1e440adfb0 2024-04-23T14:32:42.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e43e00640 1 --2- 172.21.15.100:0/2693609502 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4415e860 0x7f1e44149820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:42.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e42a00640 1 --2- 172.21.15.100:0/2693609502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e440adfb0 0x7f1e4415b680 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000c4b0 tx=0x7f1e3000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:42.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e3b400640 1 -- 172.21.15.100:0/2693609502 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e30018070 con 0x7f1e440adfb0 2024-04-23T14:32:42.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e3b400640 1 -- 172.21.15.100:0/2693609502 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e30007500 con 0x7f1e440adfb0 2024-04-23T14:32:42.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e4a200640 1 -- 172.21.15.100:0/2693609502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e441556e0 con 0x7f1e440adfb0 2024-04-23T14:32:42.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e3b400640 1 -- 172.21.15.100:0/2693609502 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e30004a30 con 0x7f1e440adfb0 2024-04-23T14:32:42.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.627+0000 7f1e4a200640 1 -- 172.21.15.100:0/2693609502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e4415bbc0 con 0x7f1e440adfb0 2024-04-23T14:32:42.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.631+0000 7f1e4a200640 1 -- 172.21.15.100:0/2693609502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e44068ad0 con 0x7f1e440adfb0 2024-04-23T14:32:42.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.631+0000 7f1e3b400640 1 -- 172.21.15.100:0/2693609502 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1e3001b070 con 0x7f1e440adfb0 2024-04-23T14:32:42.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.631+0000 7f1e3b400640 1 --2- 172.21.15.100:0/2693609502 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e24068380 0x7f1e2406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.631+0000 7f1e3b400640 1 -- 172.21.15.100:0/2693609502 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1e30085f40 con 0x7f1e440adfb0 2024-04-23T14:32:42.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.631+0000 7f1e43400640 1 --2- 172.21.15.100:0/2693609502 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e24068380 0x7f1e2406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.631+0000 7f1e43400640 1 --2- 172.21.15.100:0/2693609502 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e24068380 0x7f1e2406a840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1e34009b60 tx=0x7f1e34009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.635+0000 7f1e3b400640 1 -- 172.21.15.100:0/2693609502 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1e30055b60 con 0x7f1e440adfb0 2024-04-23T14:32:42.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.795+0000 7f1e4a200640 1 -- 172.21.15.100:0/2693609502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1e441621e0 con 0x7f1e440adfb0 2024-04-23T14:32:42.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.795+0000 7f1e3b400640 1 -- 172.21.15.100:0/2693609502 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f1e30059e60 con 0x7f1e440adfb0 2024-04-23T14:32:42.797 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:32:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.799+0000 7f1e4a200640 1 -- 172.21.15.100:0/2693609502 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e24068380 msgr2=0x7f1e2406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.799+0000 7f1e4a200640 1 --2- 172.21.15.100:0/2693609502 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e24068380 0x7f1e2406a840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1e34009b60 tx=0x7f1e34009190 comp rx=0 tx=0).stop 2024-04-23T14:32:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.799+0000 7f1e4a200640 1 -- 172.21.15.100:0/2693609502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e440adfb0 msgr2=0x7f1e4415b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.799+0000 7f1e4a200640 1 --2- 172.21.15.100:0/2693609502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e440adfb0 0x7f1e4415b680 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000c4b0 tx=0x7f1e3000c970 comp rx=0 tx=0).stop 2024-04-23T14:32:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.799+0000 7f1e4a200640 1 -- 172.21.15.100:0/2693609502 shutdown_connections 2024-04-23T14:32:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.799+0000 7f1e4a200640 1 --2- 172.21.15.100:0/2693609502 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e24068380 0x7f1e2406a840 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.799+0000 7f1e4a200640 1 --2- 172.21.15.100:0/2693609502 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4415e860 0x7f1e44149820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.799+0000 7f1e4a200640 1 --2- 172.21.15.100:0/2693609502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e440adfb0 0x7f1e4415b680 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.799+0000 7f1e4a200640 1 --2- 172.21.15.100:0/2693609502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e440adb90 0x7f1e44142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.799+0000 7f1e4a200640 1 -- 172.21.15.100:0/2693609502 >> 172.21.15.100:0/2693609502 conn(0x7f1e44072590 msgr2=0x7f1e44158960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.799+0000 7f1e4a200640 1 -- 172.21.15.100:0/2693609502 shutdown_connections 2024-04-23T14:32:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.799+0000 7f1e4a200640 1 -- 172.21.15.100:0/2693609502 wait complete. 2024-04-23T14:32:42.819 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:32:42.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.951+0000 7f030e000640 1 Processor -- start 2024-04-23T14:32:42.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.951+0000 7f030e000640 1 -- start start 2024-04-23T14:32:42.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.955+0000 7f030e000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f03080cea80 0x7f03080cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.955+0000 7f030e000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0308073cc0 0x7f03080cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.955+0000 7f030e000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03080cfc80 con 0x7f0308145400 2024-04-23T14:32:42.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.955+0000 7f030e000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f030815da90 con 0x7f0308073cc0 2024-04-23T14:32:42.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.955+0000 7f030e000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f03080cdcc0 con 0x7f03080cea80 2024-04-23T14:32:42.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.955+0000 7f0307e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0308073cc0 0x7f03080cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.955+0000 7f0307e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0308073cc0 0x7f03080cf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53084/0 (socket says 172.21.15.100:53084) 2024-04-23T14:32:42.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.955+0000 7f0307e00640 1 -- 172.21.15.100:0/2316515952 learned_addr learned my addr 172.21.15.100:0/2316515952 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:42.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.955+0000 7f0307400640 1 --1- 172.21.15.100:0/2316515952 >> v1:172.21.15.100:6789/0 conn(0x7f0308145400 0x7f03080735b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:45002/0 (socket says 172.21.15.100:45002) 2024-04-23T14:32:42.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.955+0000 7f0306a00640 1 --2- 172.21.15.100:0/2316515952 >> v2:172.21.15.151:3300/0 conn(0x7f03080cea80 0x7f03080cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.955+0000 7f0307e00640 1 -- 172.21.15.100:0/2316515952 >> v2:172.21.15.151:3300/0 conn(0x7f03080cea80 msgr2=0x7f03080cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:42.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.955+0000 7f0307e00640 1 --2- 172.21.15.100:0/2316515952 >> v2:172.21.15.151:3300/0 conn(0x7f03080cea80 0x7f03080cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.955+0000 7f0307e00640 1 -- 172.21.15.100:0/2316515952 >> v1:172.21.15.100:6789/0 conn(0x7f0308145400 legacy=0x7f03080735b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.955+0000 7f0307e00640 1 -- 172.21.15.100:0/2316515952 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03080cdfa0 con 0x7f0308073cc0 2024-04-23T14:32:42.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f0307e00640 1 --2- 172.21.15.100:0/2316515952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0308073cc0 0x7f03080cf5e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0308074150 tx=0x7f030001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4797b48753402b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:42.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f0306000640 1 -- 172.21.15.100:0/2316515952 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03000ab610 con 0x7f0308073cc0 2024-04-23T14:32:42.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f0306000640 1 -- 172.21.15.100:0/2316515952 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03000b0070 con 0x7f0308073cc0 2024-04-23T14:32:42.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f0306000640 1 -- 172.21.15.100:0/2316515952 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03000ab9d0 con 0x7f0308073cc0 2024-04-23T14:32:42.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f030e000640 1 -- 172.21.15.100:0/2316515952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0308073cc0 msgr2=0x7f03080cf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f030e000640 1 --2- 172.21.15.100:0/2316515952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0308073cc0 0x7f03080cf5e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0308074150 tx=0x7f030001c520 comp rx=0 tx=0).stop 2024-04-23T14:32:42.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f030e000640 1 -- 172.21.15.100:0/2316515952 shutdown_connections 2024-04-23T14:32:42.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f030e000640 1 --2- 172.21.15.100:0/2316515952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0308073cc0 0x7f03080cf5e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f030e000640 1 --2- 172.21.15.100:0/2316515952 >> v2:172.21.15.151:3300/0 conn(0x7f03080cea80 0x7f03080cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f030e000640 1 -- 172.21.15.100:0/2316515952 >> 172.21.15.100:0/2316515952 conn(0x7f03080715f0 msgr2=0x7f0308072ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:42.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f030e000640 1 -- 172.21.15.100:0/2316515952 shutdown_connections 2024-04-23T14:32:42.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f030e000640 1 -- 172.21.15.100:0/2316515952 wait complete. 2024-04-23T14:32:42.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f030e000640 1 Processor -- start 2024-04-23T14:32:42.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f030e000640 1 -- start start 2024-04-23T14:32:42.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f030e000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0308073cc0 0x7f030806f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f030e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03080cea80 0x7f030813a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f030e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0308145400 0x7f030815b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f030e000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f030815d620 con 0x7f0308145400 2024-04-23T14:32:42.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f030e000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f030815d4a0 con 0x7f03080cea80 2024-04-23T14:32:42.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f030e000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f030815d7a0 con 0x7f0308073cc0 2024-04-23T14:32:42.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f0307e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0308145400 0x7f030815b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f0307e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0308145400 0x7f030815b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53098/0 (socket says 172.21.15.100:53098) 2024-04-23T14:32:42.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.959+0000 7f0307e00640 1 -- 172.21.15.100:0/3752216998 learned_addr learned my addr 172.21.15.100:0/3752216998 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:42.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.963+0000 7f0306a00640 1 --2- 172.21.15.100:0/3752216998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03080cea80 0x7f030813a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.963+0000 7f0307400640 1 --2- 172.21.15.100:0/3752216998 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0308073cc0 0x7f030806f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.963+0000 7f0307e00640 1 -- 172.21.15.100:0/3752216998 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0308073cc0 msgr2=0x7f030806f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.963+0000 7f0307e00640 1 --2- 172.21.15.100:0/3752216998 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0308073cc0 0x7f030806f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.963+0000 7f0307e00640 1 -- 172.21.15.100:0/3752216998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03080cea80 msgr2=0x7f030813a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.963+0000 7f0307e00640 1 --2- 172.21.15.100:0/3752216998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03080cea80 0x7f030813a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.963+0000 7f0307e00640 1 -- 172.21.15.100:0/3752216998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0308140530 con 0x7f0308145400 2024-04-23T14:32:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.963+0000 7f0307400640 1 --2- 172.21.15.100:0/3752216998 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0308073cc0 0x7f030806f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.963+0000 7f0307e00640 1 --2- 172.21.15.100:0/3752216998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0308145400 0x7f030815b0e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f03000ad040 tx=0x7f030001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.963+0000 7f02ff400640 1 -- 172.21.15.100:0/3752216998 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03000be5f0 con 0x7f0308145400 2024-04-23T14:32:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.963+0000 7f02ff400640 1 -- 172.21.15.100:0/3752216998 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03000be790 con 0x7f0308145400 2024-04-23T14:32:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.963+0000 7f030e000640 1 -- 172.21.15.100:0/3752216998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03081407a0 con 0x7f0308145400 2024-04-23T14:32:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.963+0000 7f02ff400640 1 -- 172.21.15.100:0/3752216998 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03000beb40 con 0x7f0308145400 2024-04-23T14:32:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.963+0000 7f030e000640 1 -- 172.21.15.100:0/3752216998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f030813aa80 con 0x7f0308145400 2024-04-23T14:32:42.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.963+0000 7f030e000640 1 -- 172.21.15.100:0/3752216998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03080691e0 con 0x7f0308145400 2024-04-23T14:32:42.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.963+0000 7f02ff400640 1 -- 172.21.15.100:0/3752216998 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f03000b43f0 con 0x7f0308145400 2024-04-23T14:32:42.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.963+0000 7f02ff400640 1 --2- 172.21.15.100:0/3752216998 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f02e80683b0 0x7f02e806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.963+0000 7f02ff400640 1 -- 172.21.15.100:0/3752216998 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0300121b20 con 0x7f0308145400 2024-04-23T14:32:42.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.963+0000 7f0307400640 1 --2- 172.21.15.100:0/3752216998 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f02e80683b0 0x7f02e806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.967+0000 7f0307400640 1 --2- 172.21.15.100:0/3752216998 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f02e80683b0 0x7f02e806a870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f02f80012f0 tx=0x7f02f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:42.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:42.967+0000 7f02ff400640 1 -- 172.21.15.100:0/3752216998 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f03000c4060 con 0x7f0308145400 2024-04-23T14:32:43.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.155+0000 7f030e000640 1 -- 172.21.15.100:0/3752216998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0308139250 con 0x7f0308145400 2024-04-23T14:32:43.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.155+0000 7f02ff400640 1 -- 172.21.15.100:0/3752216998 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f03000a8050 con 0x7f0308145400 2024-04-23T14:32:43.158 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:43.158 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1456958549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1456958549},{"type":"v1","addr":"172.21.15.100:6836","nonce":1456958549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3655609551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3655609551},{"type":"v1","addr":"172.21.15.100:6837","nonce":3655609551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/4242180972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":4242180972},{"type":"v1","addr":"172.21.15.151:6834","nonce":4242180972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:32:08.976576+0000","modified":"2024-04-23T14:32:34.057740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1533322173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1533322173},{"type":"v1","addr":"172.21.15.151:6835","nonce":1533322173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-23T14:32:43.158 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 9 2024-04-23T14:32:43.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.159+0000 7f030e000640 1 -- 172.21.15.100:0/3752216998 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f02e80683b0 msgr2=0x7f02e806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.159+0000 7f030e000640 1 --2- 172.21.15.100:0/3752216998 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f02e80683b0 0x7f02e806a870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f02f80012f0 tx=0x7f02f8009040 comp rx=0 tx=0).stop 2024-04-23T14:32:43.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.159+0000 7f030e000640 1 -- 172.21.15.100:0/3752216998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0308145400 msgr2=0x7f030815b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.159+0000 7f030e000640 1 --2- 172.21.15.100:0/3752216998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0308145400 0x7f030815b0e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f03000ad040 tx=0x7f030001ad60 comp rx=0 tx=0).stop 2024-04-23T14:32:43.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.159+0000 7f030e000640 1 -- 172.21.15.100:0/3752216998 shutdown_connections 2024-04-23T14:32:43.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.159+0000 7f030e000640 1 --2- 172.21.15.100:0/3752216998 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f02e80683b0 0x7f02e806a870 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.159+0000 7f030e000640 1 --2- 172.21.15.100:0/3752216998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0308145400 0x7f030815b0e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.159+0000 7f030e000640 1 --2- 172.21.15.100:0/3752216998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03080cea80 0x7f030813a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.159+0000 7f030e000640 1 --2- 172.21.15.100:0/3752216998 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0308073cc0 0x7f030806f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.159+0000 7f030e000640 1 -- 172.21.15.100:0/3752216998 >> 172.21.15.100:0/3752216998 conn(0x7f03080715f0 msgr2=0x7f0308139a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:43.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.159+0000 7f030e000640 1 -- 172.21.15.100:0/3752216998 shutdown_connections 2024-04-23T14:32:43.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.159+0000 7f030e000640 1 -- 172.21.15.100:0/3752216998 wait complete. 2024-04-23T14:32:43.178 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:43.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d7200640 1 Processor -- start 2024-04-23T14:32:43.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d7200640 1 -- start start 2024-04-23T14:32:43.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d7200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f75d80d1d60 0x7f75d815c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d7200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75d8153840 con 0x7f75d80cc940 2024-04-23T14:32:43.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d7200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75d815cfe0 con 0x7f75d8153460 2024-04-23T14:32:43.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d7200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f75d815d2c0 con 0x7f75d80d1d60 2024-04-23T14:32:43.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d6800640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f75d80cc940 0x7f75d80b2250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:45006/0 (socket says 172.21.15.100:45006) 2024-04-23T14:32:43.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d6800640 1 -- 172.21.15.100:0/4220101247 learned_addr learned my addr 172.21.15.100:0/4220101247 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:43.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75cd400640 1 --2- 172.21.15.100:0/4220101247 >> v2:172.21.15.151:3300/0 conn(0x7f75d80d1d60 0x7f75d815c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d5400640 1 -- 172.21.15.100:0/4220101247 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1056319048 0 0) 0x7f75d0001120 con 0x7f75d80cc940 2024-04-23T14:32:43.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d5400640 1 -- 172.21.15.100:0/4220101247 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75bc003530 con 0x7f75d80cc940 2024-04-23T14:32:43.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d5400640 1 -- 172.21.15.100:0/4220101247 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1665804513 0 0) 0x7f75bc003530 con 0x7f75d80cc940 2024-04-23T14:32:43.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d5400640 1 -- 172.21.15.100:0/4220101247 >> v2:172.21.15.151:3300/0 conn(0x7f75d80d1d60 msgr2=0x7f75d815c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:43.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d5400640 1 --2- 172.21.15.100:0/4220101247 >> v2:172.21.15.151:3300/0 conn(0x7f75d80d1d60 0x7f75d815c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d5400640 1 -- 172.21.15.100:0/4220101247 >> v1:172.21.15.151:6789/0 conn(0x7f75d8153460 legacy=0x7f75d80d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d5400640 1 -- 172.21.15.100:0/4220101247 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d815d5a0 con 0x7f75d80cc940 2024-04-23T14:32:43.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d5400640 1 -- 172.21.15.100:0/4220101247 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f75d0001850 con 0x7f75d80cc940 2024-04-23T14:32:43.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75cd400640 1 --2- 172.21.15.100:0/4220101247 >> v2:172.21.15.151:3300/0 conn(0x7f75d80d1d60 0x7f75d815c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:43.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d5400640 1 -- 172.21.15.100:0/4220101247 >> v1:172.21.15.100:6789/0 conn(0x7f75d80cc940 legacy=0x7f75d80b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d5400640 1 --2- 172.21.15.100:0/4220101247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75bc0039c0 0x7f75bc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d5400640 1 --2- 172.21.15.100:0/4220101247 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75bc004810 0x7f75bc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d5400640 1 --2- 172.21.15.100:0/4220101247 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75bc0051d0 0x7f75bc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d5400640 1 -- 172.21.15.100:0/4220101247 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f75d815d2c0 con 0x7f75bc0039c0 2024-04-23T14:32:43.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d5400640 1 -- 172.21.15.100:0/4220101247 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc00c170 con 0x7f75bc004810 2024-04-23T14:32:43.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d5400640 1 -- 172.21.15.100:0/4220101247 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f75bc00c370 con 0x7f75bc0051d0 2024-04-23T14:32:43.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d5e00640 1 --2- 172.21.15.100:0/4220101247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75bc0039c0 0x7f75bc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75cd400640 1 --2- 172.21.15.100:0/4220101247 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75bc004810 0x7f75bc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d6800640 1 --2- 172.21.15.100:0/4220101247 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75bc0051d0 0x7f75bc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d5e00640 1 -- 172.21.15.100:0/4220101247 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75bc0051d0 msgr2=0x7f75bc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:43.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d5e00640 1 --2- 172.21.15.100:0/4220101247 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75bc0051d0 0x7f75bc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d5e00640 1 -- 172.21.15.100:0/4220101247 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75bc004810 msgr2=0x7f75bc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:43.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d5e00640 1 --2- 172.21.15.100:0/4220101247 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75bc004810 0x7f75bc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d5e00640 1 -- 172.21.15.100:0/4220101247 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75bc003530 con 0x7f75bc0039c0 2024-04-23T14:32:43.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75cd400640 1 --2- 172.21.15.100:0/4220101247 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75bc004810 0x7f75bc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:43.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.315+0000 7f75d6800640 1 -- 172.21.15.100:0/4220101247 reap_dead start 2024-04-23T14:32:43.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.319+0000 7f75d5e00640 1 --2- 172.21.15.100:0/4220101247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75bc0039c0 0x7f75bc003de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f75c8001e70 tx=0x7f75c801c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d154382f9b1bf370 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:43.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.319+0000 7f75d5400640 1 -- 172.21.15.100:0/4220101247 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8001a90 con 0x7f75bc0039c0 2024-04-23T14:32:43.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.319+0000 7f75d5400640 1 -- 172.21.15.100:0/4220101247 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c8003b80 con 0x7f75bc0039c0 2024-04-23T14:32:43.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.319+0000 7f75d5400640 1 -- 172.21.15.100:0/4220101247 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8005080 con 0x7f75bc0039c0 2024-04-23T14:32:43.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.319+0000 7f75d7200640 1 -- 172.21.15.100:0/4220101247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75bc0039c0 msgr2=0x7f75bc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.319+0000 7f75d7200640 1 --2- 172.21.15.100:0/4220101247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75bc0039c0 0x7f75bc003de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f75c8001e70 tx=0x7f75c801c750 comp rx=0 tx=0).stop 2024-04-23T14:32:43.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.319+0000 7f75d7200640 1 -- 172.21.15.100:0/4220101247 shutdown_connections 2024-04-23T14:32:43.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.319+0000 7f75d7200640 1 --2- 172.21.15.100:0/4220101247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75bc0039c0 0x7f75bc003de0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.319+0000 7f75d7200640 1 -- 172.21.15.100:0/4220101247 >> 172.21.15.100:0/4220101247 conn(0x7f75d80b3e60 msgr2=0x7f75d80b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:43.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.319+0000 7f75d7200640 1 -- 172.21.15.100:0/4220101247 shutdown_connections 2024-04-23T14:32:43.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.319+0000 7f75d7200640 1 -- 172.21.15.100:0/4220101247 wait complete. 2024-04-23T14:32:43.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75d7200640 1 Processor -- start 2024-04-23T14:32:43.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75d7200640 1 -- start start 2024-04-23T14:32:43.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75d7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75bc0039c0 0x7f75d8150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75d7200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75d80b6190 0x7f75d80b6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75d7200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75d80c7390 0x7f75d80c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75d7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f75d80b2230 con 0x7f75bc0039c0 2024-04-23T14:32:43.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75d7200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f75d815cd50 con 0x7f75d80c7390 2024-04-23T14:32:43.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75d7200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f75d80b23b0 con 0x7f75d80b6190 2024-04-23T14:32:43.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75d5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75bc0039c0 0x7f75d8150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75d5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75bc0039c0 0x7f75d8150c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53114/0 (socket says 172.21.15.100:53114) 2024-04-23T14:32:43.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75d5e00640 1 -- 172.21.15.100:0/3033696928 learned_addr learned my addr 172.21.15.100:0/3033696928 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:43.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75cd400640 1 --2- 172.21.15.100:0/3033696928 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75d80b6190 0x7f75d80b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75d6800640 1 --2- 172.21.15.100:0/3033696928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75d80c7390 0x7f75d80c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75d5e00640 1 -- 172.21.15.100:0/3033696928 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75d80b6190 msgr2=0x7f75d80b6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75d5e00640 1 --2- 172.21.15.100:0/3033696928 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75d80b6190 0x7f75d80b6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75d5e00640 1 -- 172.21.15.100:0/3033696928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75d80c7390 msgr2=0x7f75d80c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75d5e00640 1 --2- 172.21.15.100:0/3033696928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75d80c7390 0x7f75d80c7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75d5e00640 1 -- 172.21.15.100:0/3033696928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d81568c0 con 0x7f75bc0039c0 2024-04-23T14:32:43.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75d6800640 1 --2- 172.21.15.100:0/3033696928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75d80c7390 0x7f75d80c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:43.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75d5e00640 1 --2- 172.21.15.100:0/3033696928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75bc0039c0 0x7f75d8150c20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f75c801c580 tx=0x7f75bc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:43.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75cea00640 1 -- 172.21.15.100:0/3033696928 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8001a90 con 0x7f75bc0039c0 2024-04-23T14:32:43.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75cea00640 1 -- 172.21.15.100:0/3033696928 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c80bc070 con 0x7f75bc0039c0 2024-04-23T14:32:43.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75cea00640 1 -- 172.21.15.100:0/3033696928 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8005710 con 0x7f75bc0039c0 2024-04-23T14:32:43.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75d7200640 1 -- 172.21.15.100:0/3033696928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d80b6d00 con 0x7f75bc0039c0 2024-04-23T14:32:43.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75d7200640 1 -- 172.21.15.100:0/3033696928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d80b72c0 con 0x7f75bc0039c0 2024-04-23T14:32:43.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.323+0000 7f75d7200640 1 -- 172.21.15.100:0/3033696928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d8075c20 con 0x7f75bc0039c0 2024-04-23T14:32:43.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.327+0000 7f75cea00640 1 -- 172.21.15.100:0/3033696928 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f75c80b1040 con 0x7f75bc0039c0 2024-04-23T14:32:43.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.327+0000 7f75cea00640 1 --2- 172.21.15.100:0/3033696928 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f75b40683b0 0x7f75b406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.327+0000 7f75cea00640 1 -- 172.21.15.100:0/3033696928 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f75c81212a0 con 0x7f75bc0039c0 2024-04-23T14:32:43.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.327+0000 7f75cd400640 1 --2- 172.21.15.100:0/3033696928 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f75b40683b0 0x7f75b406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.327+0000 7f75cd400640 1 --2- 172.21.15.100:0/3033696928 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f75b40683b0 0x7f75b406a870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f75d81f8f50 tx=0x7f75c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:43.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.331+0000 7f75cea00640 1 -- 172.21.15.100:0/3033696928 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f75c80ad020 con 0x7f75bc0039c0 2024-04-23T14:32:43.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.487+0000 7f75d7200640 1 -- 172.21.15.100:0/3033696928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f75d80cfec0 con 0x7f75bc0039c0 2024-04-23T14:32:43.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.487+0000 7f75cea00640 1 -- 172.21.15.100:0/3033696928 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f75c8016490 con 0x7f75bc0039c0 2024-04-23T14:32:43.489 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:43.489 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":24,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:14.037027+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.491+0000 7f75d7200640 1 -- 172.21.15.100:0/3033696928 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f75b40683b0 msgr2=0x7f75b406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.491+0000 7f75d7200640 1 --2- 172.21.15.100:0/3033696928 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f75b40683b0 0x7f75b406a870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f75d81f8f50 tx=0x7f75c0008040 comp rx=0 tx=0).stop 2024-04-23T14:32:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.491+0000 7f75d7200640 1 -- 172.21.15.100:0/3033696928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75bc0039c0 msgr2=0x7f75d8150c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.491+0000 7f75d7200640 1 --2- 172.21.15.100:0/3033696928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75bc0039c0 0x7f75d8150c20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f75c801c580 tx=0x7f75bc005840 comp rx=0 tx=0).stop 2024-04-23T14:32:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.491+0000 7f75d7200640 1 -- 172.21.15.100:0/3033696928 shutdown_connections 2024-04-23T14:32:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.491+0000 7f75d7200640 1 --2- 172.21.15.100:0/3033696928 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f75b40683b0 0x7f75b406a870 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.491+0000 7f75d7200640 1 --2- 172.21.15.100:0/3033696928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75d80c7390 0x7f75d80c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.491+0000 7f75d7200640 1 --2- 172.21.15.100:0/3033696928 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75d80b6190 0x7f75d80b6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.491+0000 7f75d7200640 1 --2- 172.21.15.100:0/3033696928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75bc0039c0 0x7f75d8150c20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.491+0000 7f75d7200640 1 -- 172.21.15.100:0/3033696928 >> 172.21.15.100:0/3033696928 conn(0x7f75d80b3e60 msgr2=0x7f75d8151a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.491+0000 7f75d7200640 1 -- 172.21.15.100:0/3033696928 shutdown_connections 2024-04-23T14:32:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.491+0000 7f75d7200640 1 -- 172.21.15.100:0/3033696928 wait complete. 2024-04-23T14:32:43.509 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:32:06.835560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:32:07.930489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-23T14:32:11.636814+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-23T14:32:43.510 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:43.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.647+0000 7fde1f200640 1 Processor -- start 2024-04-23T14:32:43.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.647+0000 7fde1f200640 1 -- start start 2024-04-23T14:32:43.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.647+0000 7fde1f200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fde1815c600 0x7fde180d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.647+0000 7fde1f200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fde180cea20 0x7fde180cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.647+0000 7fde1f200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde180cf570 con 0x7fde180d1eb0 2024-04-23T14:32:43.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.647+0000 7fde1f200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fde180d0e90 con 0x7fde1815c600 2024-04-23T14:32:43.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.647+0000 7fde1f200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fde180d1170 con 0x7fde180cea20 2024-04-23T14:32:43.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.647+0000 7fde17e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fde1815c600 0x7fde180d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.647+0000 7fde0f400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fde180d1eb0 0x7fde180b52b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:45022/0 (socket says 172.21.15.100:45022) 2024-04-23T14:32:43.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.647+0000 7fde0f400640 1 -- 172.21.15.100:0/3488448983 learned_addr learned my addr 172.21.15.100:0/3488448983 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:43.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.647+0000 7fde1d200640 1 --2- 172.21.15.100:0/3488448983 >> v2:172.21.15.151:3300/0 conn(0x7fde180cea20 0x7fde180cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.647+0000 7fde17e00640 1 -- 172.21.15.100:0/3488448983 >> v2:172.21.15.151:3300/0 conn(0x7fde180cea20 msgr2=0x7fde180cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:43.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.647+0000 7fde17e00640 1 --2- 172.21.15.100:0/3488448983 >> v2:172.21.15.151:3300/0 conn(0x7fde180cea20 0x7fde180cee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.647+0000 7fde17e00640 1 -- 172.21.15.100:0/3488448983 >> v1:172.21.15.100:6789/0 conn(0x7fde180d1eb0 legacy=0x7fde180b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.647+0000 7fde17e00640 1 -- 172.21.15.100:0/3488448983 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde180d1450 con 0x7fde1815c600 2024-04-23T14:32:43.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.651+0000 7fde17e00640 1 --2- 172.21.15.100:0/3488448983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde1815c600 0x7fde180d1970 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fde180cc9e0 tx=0x7fde0801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7f4f7aac7b57a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:43.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.651+0000 7fde17400640 1 -- 172.21.15.100:0/3488448983 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde080b2070 con 0x7fde1815c600 2024-04-23T14:32:43.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.651+0000 7fde17400640 1 -- 172.21.15.100:0/3488448983 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde080a57f0 con 0x7fde1815c600 2024-04-23T14:32:43.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.651+0000 7fde17400640 1 -- 172.21.15.100:0/3488448983 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde080a5bc0 con 0x7fde1815c600 2024-04-23T14:32:43.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.651+0000 7fde1f200640 1 -- 172.21.15.100:0/3488448983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde1815c600 msgr2=0x7fde180d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.651+0000 7fde1f200640 1 --2- 172.21.15.100:0/3488448983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde1815c600 0x7fde180d1970 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fde180cc9e0 tx=0x7fde0801c930 comp rx=0 tx=0).stop 2024-04-23T14:32:43.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.651+0000 7fde1f200640 1 -- 172.21.15.100:0/3488448983 shutdown_connections 2024-04-23T14:32:43.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.651+0000 7fde1f200640 1 --2- 172.21.15.100:0/3488448983 >> v2:172.21.15.151:3300/0 conn(0x7fde180cea20 0x7fde180cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.651+0000 7fde1f200640 1 --2- 172.21.15.100:0/3488448983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde1815c600 0x7fde180d1970 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.651+0000 7fde1f200640 1 -- 172.21.15.100:0/3488448983 >> 172.21.15.100:0/3488448983 conn(0x7fde180719d0 msgr2=0x7fde18071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:43.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.651+0000 7fde1f200640 1 -- 172.21.15.100:0/3488448983 shutdown_connections 2024-04-23T14:32:43.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.651+0000 7fde1f200640 1 -- 172.21.15.100:0/3488448983 wait complete. 2024-04-23T14:32:43.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde1f200640 1 Processor -- start 2024-04-23T14:32:43.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde1f200640 1 -- start start 2024-04-23T14:32:43.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde1f200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde180cea20 0x7fde18166340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde1f200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde180d1eb0 0x7fde18152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde1f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde1815c600 0x7fde1806ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde1f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fde181696e0 con 0x7fde1815c600 2024-04-23T14:32:43.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde1f200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde18169560 con 0x7fde180d1eb0 2024-04-23T14:32:43.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde1f200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fde18169860 con 0x7fde180cea20 2024-04-23T14:32:43.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde1d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde1815c600 0x7fde1806ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde1d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde1815c600 0x7fde1806ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53146/0 (socket says 172.21.15.100:53146) 2024-04-23T14:32:43.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde1d200640 1 -- 172.21.15.100:0/535669067 learned_addr learned my addr 172.21.15.100:0/535669067 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:43.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde17e00640 1 --2- 172.21.15.100:0/535669067 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde180cea20 0x7fde18166340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde0f400640 1 --2- 172.21.15.100:0/535669067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde180d1eb0 0x7fde18152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde1d200640 1 -- 172.21.15.100:0/535669067 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde180cea20 msgr2=0x7fde18166340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde1d200640 1 --2- 172.21.15.100:0/535669067 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde180cea20 0x7fde18166340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde1d200640 1 -- 172.21.15.100:0/535669067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde180d1eb0 msgr2=0x7fde18152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde1d200640 1 --2- 172.21.15.100:0/535669067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde180d1eb0 0x7fde18152900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde1d200640 1 -- 172.21.15.100:0/535669067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde181658a0 con 0x7fde1815c600 2024-04-23T14:32:43.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde1d200640 1 --2- 172.21.15.100:0/535669067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde1815c600 0x7fde1806ed70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fde1000d800 tx=0x7fde1000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:43.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde14c00640 1 -- 172.21.15.100:0/535669067 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1000b6c0 con 0x7fde1815c600 2024-04-23T14:32:43.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde14c00640 1 -- 172.21.15.100:0/535669067 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde10019070 con 0x7fde1815c600 2024-04-23T14:32:43.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde1f200640 1 -- 172.21.15.100:0/535669067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde1806fad0 con 0x7fde1815c600 2024-04-23T14:32:43.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde14c00640 1 -- 172.21.15.100:0/535669067 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1000baa0 con 0x7fde1815c600 2024-04-23T14:32:43.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.655+0000 7fde1f200640 1 -- 172.21.15.100:0/535669067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde18165b40 con 0x7fde1815c600 2024-04-23T14:32:43.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.659+0000 7fde1f200640 1 -- 172.21.15.100:0/535669067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde18068ad0 con 0x7fde1815c600 2024-04-23T14:32:43.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.659+0000 7fde14c00640 1 -- 172.21.15.100:0/535669067 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fde1000bc40 con 0x7fde1815c600 2024-04-23T14:32:43.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.659+0000 7fde14c00640 1 --2- 172.21.15.100:0/535669067 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fddf4068380 0x7fddf406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.659+0000 7fde14c00640 1 -- 172.21.15.100:0/535669067 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fde10087200 con 0x7fde1815c600 2024-04-23T14:32:43.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.659+0000 7fde17e00640 1 --2- 172.21.15.100:0/535669067 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fddf4068380 0x7fddf406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.659+0000 7fde17e00640 1 --2- 172.21.15.100:0/535669067 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fddf4068380 0x7fddf406a840 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fde080af000 tx=0x7fde080a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:43.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.663+0000 7fde14c00640 1 -- 172.21.15.100:0/535669067 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fde100573e0 con 0x7fde1815c600 2024-04-23T14:32:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.823+0000 7fde1f200640 1 -- 172.21.15.100:0/535669067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fde18161040 con 0x7fde1815c600 2024-04-23T14:32:43.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.823+0000 7fde14c00640 1 -- 172.21.15.100:0/535669067 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fde10056ce0 con 0x7fde1815c600 2024-04-23T14:32:43.824 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:32:43.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.827+0000 7fde1f200640 1 -- 172.21.15.100:0/535669067 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fddf4068380 msgr2=0x7fddf406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.827+0000 7fde1f200640 1 --2- 172.21.15.100:0/535669067 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fddf4068380 0x7fddf406a840 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fde080af000 tx=0x7fde080a7070 comp rx=0 tx=0).stop 2024-04-23T14:32:43.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.827+0000 7fde1f200640 1 -- 172.21.15.100:0/535669067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde1815c600 msgr2=0x7fde1806ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.827+0000 7fde1f200640 1 --2- 172.21.15.100:0/535669067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde1815c600 0x7fde1806ed70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fde1000d800 tx=0x7fde1000dcc0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.827+0000 7fde1f200640 1 -- 172.21.15.100:0/535669067 shutdown_connections 2024-04-23T14:32:43.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.827+0000 7fde1f200640 1 --2- 172.21.15.100:0/535669067 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fddf4068380 0x7fddf406a840 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.827+0000 7fde1f200640 1 --2- 172.21.15.100:0/535669067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde1815c600 0x7fde1806ed70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.827+0000 7fde1f200640 1 --2- 172.21.15.100:0/535669067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde180d1eb0 0x7fde18152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.827+0000 7fde1f200640 1 --2- 172.21.15.100:0/535669067 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde180cea20 0x7fde18166340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.827+0000 7fde1f200640 1 -- 172.21.15.100:0/535669067 >> 172.21.15.100:0/535669067 conn(0x7fde180719d0 msgr2=0x7fde18155410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:43.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.827+0000 7fde1f200640 1 -- 172.21.15.100:0/535669067 shutdown_connections 2024-04-23T14:32:43.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.827+0000 7fde1f200640 1 -- 172.21.15.100:0/535669067 wait complete. 2024-04-23T14:32:43.845 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:32:43.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.983+0000 7fad27200640 1 Processor -- start 2024-04-23T14:32:43.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.983+0000 7fad27200640 1 -- start start 2024-04-23T14:32:43.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.983+0000 7fad27200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fad2015e860 0x7fad200b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.983+0000 7fad27200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fad200adfe0 0x7fad2006d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.983+0000 7fad27200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad2006e5f0 con 0x7fad200adb90 2024-04-23T14:32:43.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.983+0000 7fad27200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fad200ae410 con 0x7fad200adfe0 2024-04-23T14:32:43.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.983+0000 7fad27200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fad2006e8b0 con 0x7fad2015e860 2024-04-23T14:32:43.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.983+0000 7fad25200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fad200adfe0 0x7fad2006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.983+0000 7fad25200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fad200adfe0 0x7fad2006d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53152/0 (socket says 172.21.15.100:53152) 2024-04-23T14:32:43.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.983+0000 7fad25200640 1 -- 172.21.15.100:0/2684596406 learned_addr learned my addr 172.21.15.100:0/2684596406 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:43.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.983+0000 7fad1fe00640 1 --2- 172.21.15.100:0/2684596406 >> v2:172.21.15.151:3300/0 conn(0x7fad2015e860 0x7fad200b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.983+0000 7fad25200640 1 -- 172.21.15.100:0/2684596406 >> v2:172.21.15.151:3300/0 conn(0x7fad2015e860 msgr2=0x7fad200b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:43.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.983+0000 7fad25200640 1 --2- 172.21.15.100:0/2684596406 >> v2:172.21.15.151:3300/0 conn(0x7fad2015e860 0x7fad200b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.983+0000 7fad25200640 1 -- 172.21.15.100:0/2684596406 >> v1:172.21.15.151:6789/0 conn(0x7fad200adb90 legacy=0x7fad2006f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.983+0000 7fad25200640 1 -- 172.21.15.100:0/2684596406 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2006eb90 con 0x7fad200adfe0 2024-04-23T14:32:43.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.987+0000 7fad25200640 1 --2- 172.21.15.100:0/2684596406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad200adfe0 0x7fad2006d350 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fad2006e030 tx=0x7fad1801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ca6b95cf92e95d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:43.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.987+0000 7fad1f400640 1 -- 172.21.15.100:0/2684596406 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad180ab600 con 0x7fad200adfe0 2024-04-23T14:32:43.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.987+0000 7fad1f400640 1 -- 172.21.15.100:0/2684596406 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad180b0070 con 0x7fad200adfe0 2024-04-23T14:32:43.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.987+0000 7fad1f400640 1 -- 172.21.15.100:0/2684596406 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad180ab9c0 con 0x7fad200adfe0 2024-04-23T14:32:43.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.987+0000 7fad27200640 1 -- 172.21.15.100:0/2684596406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad200adfe0 msgr2=0x7fad2006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.987+0000 7fad27200640 1 --2- 172.21.15.100:0/2684596406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad200adfe0 0x7fad2006d350 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fad2006e030 tx=0x7fad1801c4f0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.987+0000 7fad27200640 1 -- 172.21.15.100:0/2684596406 shutdown_connections 2024-04-23T14:32:43.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.987+0000 7fad27200640 1 --2- 172.21.15.100:0/2684596406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad200adfe0 0x7fad2006d350 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.987+0000 7fad27200640 1 --2- 172.21.15.100:0/2684596406 >> v2:172.21.15.151:3300/0 conn(0x7fad2015e860 0x7fad200b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.987+0000 7fad27200640 1 -- 172.21.15.100:0/2684596406 >> 172.21.15.100:0/2684596406 conn(0x7fad20072590 msgr2=0x7fad200729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:43.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.987+0000 7fad27200640 1 -- 172.21.15.100:0/2684596406 shutdown_connections 2024-04-23T14:32:43.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.987+0000 7fad27200640 1 -- 172.21.15.100:0/2684596406 wait complete. 2024-04-23T14:32:43.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad27200640 1 Processor -- start 2024-04-23T14:32:43.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad27200640 1 -- start start 2024-04-23T14:32:43.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad27200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fad200adb90 0x7fad2014e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad27200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad200adfe0 0x7fad200b1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad27200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fad2015e860 0x7fad2014b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad27200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fad20073790 con 0x7fad200adfe0 2024-04-23T14:32:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad27200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fad200b3ff0 con 0x7fad200adb90 2024-04-23T14:32:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad27200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fad2015ec40 con 0x7fad2015e860 2024-04-23T14:32:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad17400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad200adfe0 0x7fad200b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad17400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad200adfe0 0x7fad200b1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53156/0 (socket says 172.21.15.100:53156) 2024-04-23T14:32:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad17400640 1 -- 172.21.15.100:0/414370969 learned_addr learned my addr 172.21.15.100:0/414370969 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad1fe00640 1 --2- 172.21.15.100:0/414370969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fad200adb90 0x7fad2014e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad25200640 1 --2- 172.21.15.100:0/414370969 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fad2015e860 0x7fad2014b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad17400640 1 -- 172.21.15.100:0/414370969 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fad2015e860 msgr2=0x7fad2014b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad17400640 1 --2- 172.21.15.100:0/414370969 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fad2015e860 0x7fad2014b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad17400640 1 -- 172.21.15.100:0/414370969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fad200adb90 msgr2=0x7fad2014e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad17400640 1 --2- 172.21.15.100:0/414370969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fad200adb90 0x7fad2014e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad17400640 1 -- 172.21.15.100:0/414370969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad200b21e0 con 0x7fad200adfe0 2024-04-23T14:32:43.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad17400640 1 --2- 172.21.15.100:0/414370969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad200adfe0 0x7fad200b1790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fad0800c790 tx=0x7fad0800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:43.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad1cc00640 1 -- 172.21.15.100:0/414370969 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad08016020 con 0x7fad200adfe0 2024-04-23T14:32:43.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad1cc00640 1 -- 172.21.15.100:0/414370969 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad08006d60 con 0x7fad200adfe0 2024-04-23T14:32:43.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad1cc00640 1 -- 172.21.15.100:0/414370969 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad08003bc0 con 0x7fad200adfe0 2024-04-23T14:32:43.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad27200640 1 -- 172.21.15.100:0/414370969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad200b24c0 con 0x7fad200adfe0 2024-04-23T14:32:43.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad27200640 1 -- 172.21.15.100:0/414370969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad2014fc70 con 0x7fad200adfe0 2024-04-23T14:32:43.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.991+0000 7fad27200640 1 -- 172.21.15.100:0/414370969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad20068ad0 con 0x7fad200adfe0 2024-04-23T14:32:43.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.995+0000 7fad1cc00640 1 -- 172.21.15.100:0/414370969 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fad0800a7f0 con 0x7fad200adfe0 2024-04-23T14:32:43.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.995+0000 7fad1cc00640 1 --2- 172.21.15.100:0/414370969 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7facfc0683b0 0x7facfc06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.995+0000 7fad1cc00640 1 -- 172.21.15.100:0/414370969 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fad08086900 con 0x7fad200adfe0 2024-04-23T14:32:43.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.995+0000 7fad1fe00640 1 --2- 172.21.15.100:0/414370969 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7facfc0683b0 0x7facfc06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.995+0000 7fad1fe00640 1 --2- 172.21.15.100:0/414370969 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7facfc0683b0 0x7facfc06a870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fad10009ac0 tx=0x7fad10009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:43.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:43.999+0000 7fad1cc00640 1 -- 172.21.15.100:0/414370969 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fad08020020 con 0x7fad200adfe0 2024-04-23T14:32:44.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.155+0000 7fad27200640 1 -- 172.21.15.100:0/414370969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fad2014fe40 con 0x7fad200adfe0 2024-04-23T14:32:44.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.155+0000 7fad1cc00640 1 -- 172.21.15.100:0/414370969 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fad08056b70 con 0x7fad200adfe0 2024-04-23T14:32:44.155 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:32:44.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.155+0000 7fad27200640 1 -- 172.21.15.100:0/414370969 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7facfc0683b0 msgr2=0x7facfc06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.155+0000 7fad27200640 1 --2- 172.21.15.100:0/414370969 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7facfc0683b0 0x7facfc06a870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fad10009ac0 tx=0x7fad10009190 comp rx=0 tx=0).stop 2024-04-23T14:32:44.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.155+0000 7fad27200640 1 -- 172.21.15.100:0/414370969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad200adfe0 msgr2=0x7fad200b1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.155+0000 7fad27200640 1 --2- 172.21.15.100:0/414370969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad200adfe0 0x7fad200b1790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fad0800c790 tx=0x7fad0800cc50 comp rx=0 tx=0).stop 2024-04-23T14:32:44.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.155+0000 7fad27200640 1 -- 172.21.15.100:0/414370969 shutdown_connections 2024-04-23T14:32:44.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.155+0000 7fad27200640 1 --2- 172.21.15.100:0/414370969 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7facfc0683b0 0x7facfc06a870 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.155+0000 7fad27200640 1 --2- 172.21.15.100:0/414370969 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fad2015e860 0x7fad2014b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.155+0000 7fad27200640 1 --2- 172.21.15.100:0/414370969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad200adfe0 0x7fad200b1790 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.155+0000 7fad27200640 1 --2- 172.21.15.100:0/414370969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fad200adb90 0x7fad2014e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.155+0000 7fad27200640 1 -- 172.21.15.100:0/414370969 >> 172.21.15.100:0/414370969 conn(0x7fad20072590 msgr2=0x7fad2014bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:44.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.155+0000 7fad27200640 1 -- 172.21.15.100:0/414370969 shutdown_connections 2024-04-23T14:32:44.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.155+0000 7fad27200640 1 -- 172.21.15.100:0/414370969 wait complete. 2024-04-23T14:32:44.175 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:32:44.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.311+0000 7f02c4400640 1 Processor -- start 2024-04-23T14:32:44.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.311+0000 7f02c4400640 1 -- start start 2024-04-23T14:32:44.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.315+0000 7f02c4400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02bc15e9c0 0x7f02bc0cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.315+0000 7f02c4400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f02bc06d450 0x7f02bc06d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.315+0000 7f02c4400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02bc0ccaf0 con 0x7f02bc0718f0 2024-04-23T14:32:44.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.315+0000 7f02c4400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f02bc15eda0 con 0x7f02bc06d450 2024-04-23T14:32:44.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.315+0000 7f02c4400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f02bc0ccdd0 con 0x7f02bc15e9c0 2024-04-23T14:32:44.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.315+0000 7f02c1000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f02bc06d450 0x7f02bc06d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:44.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.315+0000 7f02c1000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f02bc06d450 0x7f02bc06d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53158/0 (socket says 172.21.15.100:53158) 2024-04-23T14:32:44.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.315+0000 7f02c1000640 1 -- 172.21.15.100:0/4191811447 learned_addr learned my addr 172.21.15.100:0/4191811447 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:44.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.315+0000 7f02c1a00640 1 --2- 172.21.15.100:0/4191811447 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02bc15e9c0 0x7f02bc0cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:44.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.315+0000 7f02c1000640 1 -- 172.21.15.100:0/4191811447 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02bc15e9c0 msgr2=0x7f02bc0cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:44.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.315+0000 7f02c1000640 1 --2- 172.21.15.100:0/4191811447 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02bc15e9c0 0x7f02bc0cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.315+0000 7f02c1000640 1 -- 172.21.15.100:0/4191811447 >> v1:172.21.15.151:6789/0 conn(0x7f02bc0718f0 legacy=0x7f02bc0d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:44.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.315+0000 7f02c1000640 1 -- 172.21.15.100:0/4191811447 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02bc0cd0b0 con 0x7f02bc06d450 2024-04-23T14:32:44.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.315+0000 7f02c1000640 1 --2- 172.21.15.100:0/4191811447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02bc06d450 0x7f02bc06d8b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009870 tx=0x7f02a801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1ac7d100008180 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:44.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02bbe00640 1 -- 172.21.15.100:0/4191811447 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a80b1070 con 0x7f02bc06d450 2024-04-23T14:32:44.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02bbe00640 1 -- 172.21.15.100:0/4191811447 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02a80ac620 con 0x7f02bc06d450 2024-04-23T14:32:44.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02bbe00640 1 -- 172.21.15.100:0/4191811447 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a80ac9f0 con 0x7f02bc06d450 2024-04-23T14:32:44.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c4400640 1 -- 172.21.15.100:0/4191811447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02bc06d450 msgr2=0x7f02bc06d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c4400640 1 --2- 172.21.15.100:0/4191811447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02bc06d450 0x7f02bc06d8b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009870 tx=0x7f02a801c460 comp rx=0 tx=0).stop 2024-04-23T14:32:44.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c4400640 1 -- 172.21.15.100:0/4191811447 shutdown_connections 2024-04-23T14:32:44.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c4400640 1 --2- 172.21.15.100:0/4191811447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02bc06d450 0x7f02bc06d8b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c4400640 1 --2- 172.21.15.100:0/4191811447 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02bc15e9c0 0x7f02bc0cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c4400640 1 -- 172.21.15.100:0/4191811447 >> 172.21.15.100:0/4191811447 conn(0x7f02bc0d2400 msgr2=0x7f02bc0d2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:44.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c4400640 1 -- 172.21.15.100:0/4191811447 shutdown_connections 2024-04-23T14:32:44.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c4400640 1 -- 172.21.15.100:0/4191811447 wait complete. 2024-04-23T14:32:44.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c4400640 1 Processor -- start 2024-04-23T14:32:44.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c4400640 1 -- start start 2024-04-23T14:32:44.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c4400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02bc06d450 0x7f02bc155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c4400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02bc0718f0 0x7f02bc0cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c4400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02bc15e9c0 0x7f02bc0cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c4400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc15feb0 con 0x7f02bc0718f0 2024-04-23T14:32:44.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c4400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc15fd30 con 0x7f02bc15e9c0 2024-04-23T14:32:44.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c4400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f02bc0d0e20 con 0x7f02bc06d450 2024-04-23T14:32:44.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02bc0718f0 0x7f02bc0cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:44.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02bc0718f0 0x7f02bc0cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53174/0 (socket says 172.21.15.100:53174) 2024-04-23T14:32:44.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c1000640 1 -- 172.21.15.100:0/3606924069 learned_addr learned my addr 172.21.15.100:0/3606924069 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:44.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c1a00640 1 --2- 172.21.15.100:0/3606924069 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02bc06d450 0x7f02bc155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:44.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c2400640 1 --2- 172.21.15.100:0/3606924069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02bc15e9c0 0x7f02bc0cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:44.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c1000640 1 -- 172.21.15.100:0/3606924069 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02bc06d450 msgr2=0x7f02bc155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c1000640 1 --2- 172.21.15.100:0/3606924069 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02bc06d450 0x7f02bc155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c1000640 1 -- 172.21.15.100:0/3606924069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02bc15e9c0 msgr2=0x7f02bc0cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c1000640 1 --2- 172.21.15.100:0/3606924069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02bc15e9c0 0x7f02bc0cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c1000640 1 -- 172.21.15.100:0/3606924069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02bc157480 con 0x7f02bc0718f0 2024-04-23T14:32:44.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.319+0000 7f02c2400640 1 --2- 172.21.15.100:0/3606924069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02bc15e9c0 0x7f02bc0cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:44.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.323+0000 7f02c1000640 1 --2- 172.21.15.100:0/3606924069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02bc0718f0 0x7f02bc0cb300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f02a80af040 tx=0x7f02a8019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:44.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.323+0000 7f02b9600640 1 -- 172.21.15.100:0/3606924069 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a80b1040 con 0x7f02bc0718f0 2024-04-23T14:32:44.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.323+0000 7f02b9600640 1 -- 172.21.15.100:0/3606924069 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02a80ac420 con 0x7f02bc0718f0 2024-04-23T14:32:44.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.323+0000 7f02b9600640 1 -- 172.21.15.100:0/3606924069 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a80b5680 con 0x7f02bc0718f0 2024-04-23T14:32:44.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.323+0000 7f02c4400640 1 -- 172.21.15.100:0/3606924069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02bc157760 con 0x7f02bc0718f0 2024-04-23T14:32:44.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.323+0000 7f02c4400640 1 -- 172.21.15.100:0/3606924069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02bc157d20 con 0x7f02bc0718f0 2024-04-23T14:32:44.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.323+0000 7f02c4400640 1 -- 172.21.15.100:0/3606924069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02bc068ad0 con 0x7f02bc0718f0 2024-04-23T14:32:44.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.323+0000 7f02b9600640 1 -- 172.21.15.100:0/3606924069 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f02a80bc020 con 0x7f02bc0718f0 2024-04-23T14:32:44.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.323+0000 7f02b9600640 1 --2- 172.21.15.100:0/3606924069 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f029c068380 0x7f029c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.323+0000 7f02b9600640 1 -- 172.21.15.100:0/3606924069 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f02a81201d0 con 0x7f02bc0718f0 2024-04-23T14:32:44.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.323+0000 7f02c1a00640 1 --2- 172.21.15.100:0/3606924069 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f029c068380 0x7f029c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:44.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.327+0000 7f02c1a00640 1 --2- 172.21.15.100:0/3606924069 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f029c068380 0x7f029c06a840 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f02bc1fb550 tx=0x7f02b00098d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:44.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.327+0000 7f02b9600640 1 -- 172.21.15.100:0/3606924069 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f02a80c3020 con 0x7f02bc0718f0 2024-04-23T14:32:44.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.483+0000 7f02c4400640 1 -- 172.21.15.100:0/3606924069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f02bc0b36a0 con 0x7f02bc0718f0 2024-04-23T14:32:44.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.483+0000 7f02b9600640 1 -- 172.21.15.100:0/3606924069 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f02a8016110 con 0x7f02bc0718f0 2024-04-23T14:32:44.485 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:32:44.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.487+0000 7f02c4400640 1 -- 172.21.15.100:0/3606924069 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f029c068380 msgr2=0x7f029c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.487+0000 7f02c4400640 1 --2- 172.21.15.100:0/3606924069 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f029c068380 0x7f029c06a840 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f02bc1fb550 tx=0x7f02b00098d0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.487+0000 7f02c4400640 1 -- 172.21.15.100:0/3606924069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02bc0718f0 msgr2=0x7f02bc0cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.487+0000 7f02c4400640 1 --2- 172.21.15.100:0/3606924069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02bc0718f0 0x7f02bc0cb300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f02a80af040 tx=0x7f02a8019200 comp rx=0 tx=0).stop 2024-04-23T14:32:44.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.487+0000 7f02c4400640 1 -- 172.21.15.100:0/3606924069 shutdown_connections 2024-04-23T14:32:44.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.487+0000 7f02c4400640 1 --2- 172.21.15.100:0/3606924069 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f029c068380 0x7f029c06a840 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.487+0000 7f02c4400640 1 --2- 172.21.15.100:0/3606924069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02bc15e9c0 0x7f02bc0cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.487+0000 7f02c4400640 1 --2- 172.21.15.100:0/3606924069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02bc0718f0 0x7f02bc0cb300 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.487+0000 7f02c4400640 1 --2- 172.21.15.100:0/3606924069 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f02bc06d450 0x7f02bc155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.487+0000 7f02c4400640 1 -- 172.21.15.100:0/3606924069 >> 172.21.15.100:0/3606924069 conn(0x7f02bc0d2400 msgr2=0x7f02bc0d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:44.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.487+0000 7f02c4400640 1 -- 172.21.15.100:0/3606924069 shutdown_connections 2024-04-23T14:32:44.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.487+0000 7f02c4400640 1 -- 172.21.15.100:0/3606924069 wait complete. 2024-04-23T14:32:44.505 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-23T14:32:44.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.643+0000 7f6d90400640 1 Processor -- start 2024-04-23T14:32:44.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.643+0000 7f6d90400640 1 -- start start 2024-04-23T14:32:44.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.643+0000 7f6d90400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f6d8815c7a0 0x7f6d880af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.643+0000 7f6d90400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6d880cd300 0x7f6d880ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.643+0000 7f6d90400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d8806cbb0 con 0x7f6d8806bb20 2024-04-23T14:32:44.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.643+0000 7f6d90400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f6d8806d450 con 0x7f6d8815c7a0 2024-04-23T14:32:44.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.643+0000 7f6d90400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f6d8806ce90 con 0x7f6d880cd300 2024-04-23T14:32:44.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.643+0000 7f6d8d000640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f6d8806bb20 0x7f6d880ccbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:45028/0 (socket says 172.21.15.100:45028) 2024-04-23T14:32:44.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.643+0000 7f6d8d000640 1 -- 172.21.15.100:0/1805693688 learned_addr learned my addr 172.21.15.100:0/1805693688 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:44.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.647+0000 7f6d8e400640 1 --2- 172.21.15.100:0/1805693688 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6d880cd300 0x7f6d880ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:44.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.647+0000 7f6d8da00640 1 --2- 172.21.15.100:0/1805693688 >> v2:172.21.15.151:3300/0 conn(0x7f6d8815c7a0 0x7f6d880af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:44.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.647+0000 7f6d87e00640 1 -- 172.21.15.100:0/1805693688 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4273569440 0 0) 0x7f6d8806cbb0 con 0x7f6d8806bb20 2024-04-23T14:32:44.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.647+0000 7f6d87e00640 1 -- 172.21.15.100:0/1805693688 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d70003530 con 0x7f6d8806bb20 2024-04-23T14:32:44.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.647+0000 7f6d87e00640 1 -- 172.21.15.100:0/1805693688 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3525767552 0 0) 0x7f6d70003530 con 0x7f6d8806bb20 2024-04-23T14:32:44.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.647+0000 7f6d8da00640 1 -- 172.21.15.100:0/1805693688 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6d880cd300 msgr2=0x7f6d880ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:44.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.647+0000 7f6d8da00640 1 --2- 172.21.15.100:0/1805693688 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6d880cd300 0x7f6d880ad430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.647+0000 7f6d8da00640 1 -- 172.21.15.100:0/1805693688 >> v1:172.21.15.100:6789/0 conn(0x7f6d8806bb20 legacy=0x7f6d880ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.647+0000 7f6d8da00640 1 -- 172.21.15.100:0/1805693688 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8806d170 con 0x7f6d8815c7a0 2024-04-23T14:32:44.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.647+0000 7f6d8da00640 1 --2- 172.21.15.100:0/1805693688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d8815c7a0 0x7f6d880af190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0097c0 tx=0x7f6d7c01c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba316edc9e07ef10 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:44.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.647+0000 7f6d87e00640 1 -- 172.21.15.100:0/1805693688 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c0b0070 con 0x7f6d8815c7a0 2024-04-23T14:32:44.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.647+0000 7f6d87e00640 1 -- 172.21.15.100:0/1805693688 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d7c0ab400 con 0x7f6d8815c7a0 2024-04-23T14:32:44.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.647+0000 7f6d87e00640 1 -- 172.21.15.100:0/1805693688 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d74001680 con 0x7f6d8815c7a0 2024-04-23T14:32:44.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.651+0000 7f6d90400640 1 -- 172.21.15.100:0/1805693688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d8815c7a0 msgr2=0x7f6d880af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.651+0000 7f6d90400640 1 --2- 172.21.15.100:0/1805693688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d8815c7a0 0x7f6d880af190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0097c0 tx=0x7f6d7c01c330 comp rx=0 tx=0).stop 2024-04-23T14:32:44.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.651+0000 7f6d90400640 1 -- 172.21.15.100:0/1805693688 shutdown_connections 2024-04-23T14:32:44.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.651+0000 7f6d90400640 1 --2- 172.21.15.100:0/1805693688 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6d880cd300 0x7f6d880ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.651+0000 7f6d90400640 1 --2- 172.21.15.100:0/1805693688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d8815c7a0 0x7f6d880af190 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.651+0000 7f6d90400640 1 -- 172.21.15.100:0/1805693688 >> 172.21.15.100:0/1805693688 conn(0x7f6d880b5200 msgr2=0x7f6d880b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:44.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.651+0000 7f6d90400640 1 -- 172.21.15.100:0/1805693688 shutdown_connections 2024-04-23T14:32:44.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.651+0000 7f6d90400640 1 -- 172.21.15.100:0/1805693688 wait complete. 2024-04-23T14:32:44.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.651+0000 7f6d90400640 1 Processor -- start 2024-04-23T14:32:44.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.651+0000 7f6d90400640 1 -- start start 2024-04-23T14:32:44.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.651+0000 7f6d90400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d8806bb20 0x7f6d88149950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.651+0000 7f6d90400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d880cd300 0x7f6d881492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.651+0000 7f6d90400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6d8815c7a0 0x7f6d8814d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.651+0000 7f6d90400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6d880ae550 con 0x7f6d8806bb20 2024-04-23T14:32:44.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.651+0000 7f6d90400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6d880ae3d0 con 0x7f6d880cd300 2024-04-23T14:32:44.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.651+0000 7f6d90400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f6d880ae6d0 con 0x7f6d8815c7a0 2024-04-23T14:32:44.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.651+0000 7f6d8da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d8806bb20 0x7f6d88149950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:44.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.651+0000 7f6d8da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d8806bb20 0x7f6d88149950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53176/0 (socket says 172.21.15.100:53176) 2024-04-23T14:32:44.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.651+0000 7f6d8da00640 1 -- 172.21.15.100:0/1326158234 learned_addr learned my addr 172.21.15.100:0/1326158234 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:44.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.651+0000 7f6d8da00640 1 -- 172.21.15.100:0/1326158234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6d8815c7a0 msgr2=0x7f6d8814d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.651+0000 7f6d8da00640 1 --2- 172.21.15.100:0/1326158234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6d8815c7a0 0x7f6d8814d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.655+0000 7f6d8da00640 1 -- 172.21.15.100:0/1326158234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d880cd300 msgr2=0x7f6d881492b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.655+0000 7f6d8da00640 1 --2- 172.21.15.100:0/1326158234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d880cd300 0x7f6d881492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.655+0000 7f6d8da00640 1 -- 172.21.15.100:0/1326158234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8814e030 con 0x7f6d8806bb20 2024-04-23T14:32:44.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.655+0000 7f6d8da00640 1 --2- 172.21.15.100:0/1326158234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d8806bb20 0x7f6d88149950 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0ae040 tx=0x7f6d7c0190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:44.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.655+0000 7f6d85600640 1 -- 172.21.15.100:0/1326158234 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c0b0040 con 0x7f6d8806bb20 2024-04-23T14:32:44.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.655+0000 7f6d85600640 1 -- 172.21.15.100:0/1326158234 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d7c0b9070 con 0x7f6d8806bb20 2024-04-23T14:32:44.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.655+0000 7f6d85600640 1 -- 172.21.15.100:0/1326158234 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c0b4870 con 0x7f6d8806bb20 2024-04-23T14:32:44.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.655+0000 7f6d90400640 1 -- 172.21.15.100:0/1326158234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d8814e310 con 0x7f6d8806bb20 2024-04-23T14:32:44.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.655+0000 7f6d90400640 1 -- 172.21.15.100:0/1326158234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d8814e8d0 con 0x7f6d8806bb20 2024-04-23T14:32:44.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.655+0000 7f6d90400640 1 -- 172.21.15.100:0/1326158234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d88068ad0 con 0x7f6d8806bb20 2024-04-23T14:32:44.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.655+0000 7f6d85600640 1 -- 172.21.15.100:0/1326158234 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6d7c0b4a10 con 0x7f6d8806bb20 2024-04-23T14:32:44.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.655+0000 7f6d85600640 1 --2- 172.21.15.100:0/1326158234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6d68068380 0x7f6d6806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.655+0000 7f6d85600640 1 -- 172.21.15.100:0/1326158234 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6d7c120a70 con 0x7f6d8806bb20 2024-04-23T14:32:44.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.655+0000 7f6d8d000640 1 --2- 172.21.15.100:0/1326158234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6d68068380 0x7f6d6806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:44.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.659+0000 7f6d8d000640 1 --2- 172.21.15.100:0/1326158234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6d68068380 0x7f6d6806a840 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6d740011f0 tx=0x7f6d74001160 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:44.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.659+0000 7f6d85600640 1 -- 172.21.15.100:0/1326158234 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6d7c015fb0 con 0x7f6d8806bb20 2024-04-23T14:32:44.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.831+0000 7f6d90400640 1 -- 172.21.15.100:0/1326158234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6d880cf250 con 0x7f6d8806bb20 2024-04-23T14:32:44.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.835+0000 7f6d85600640 1 -- 172.21.15.100:0/1326158234 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f6d7c0f0d80 con 0x7f6d8806bb20 2024-04-23T14:32:44.834 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:32:44.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.835+0000 7f6d90400640 1 -- 172.21.15.100:0/1326158234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6d68068380 msgr2=0x7f6d6806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.835+0000 7f6d90400640 1 --2- 172.21.15.100:0/1326158234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6d68068380 0x7f6d6806a840 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6d740011f0 tx=0x7f6d74001160 comp rx=0 tx=0).stop 2024-04-23T14:32:44.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.835+0000 7f6d90400640 1 -- 172.21.15.100:0/1326158234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d8806bb20 msgr2=0x7f6d88149950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.835+0000 7f6d90400640 1 --2- 172.21.15.100:0/1326158234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d8806bb20 0x7f6d88149950 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0ae040 tx=0x7f6d7c0190d0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.835+0000 7f6d90400640 1 -- 172.21.15.100:0/1326158234 shutdown_connections 2024-04-23T14:32:44.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.835+0000 7f6d90400640 1 --2- 172.21.15.100:0/1326158234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6d68068380 0x7f6d6806a840 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.835+0000 7f6d90400640 1 --2- 172.21.15.100:0/1326158234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6d8815c7a0 0x7f6d8814d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.835+0000 7f6d90400640 1 --2- 172.21.15.100:0/1326158234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d880cd300 0x7f6d881492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.835+0000 7f6d90400640 1 --2- 172.21.15.100:0/1326158234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d8806bb20 0x7f6d88149950 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.835+0000 7f6d90400640 1 -- 172.21.15.100:0/1326158234 >> 172.21.15.100:0/1326158234 conn(0x7f6d880b5200 msgr2=0x7f6d880cf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:44.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.835+0000 7f6d90400640 1 -- 172.21.15.100:0/1326158234 shutdown_connections 2024-04-23T14:32:44.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.835+0000 7f6d90400640 1 -- 172.21.15.100:0/1326158234 wait complete. 2024-04-23T14:32:44.856 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:32:44.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.991+0000 7f670c800640 1 Processor -- start 2024-04-23T14:32:44.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.991+0000 7f670c800640 1 -- start start 2024-04-23T14:32:44.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.991+0000 7f670c800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f670406d450 0x7f670406d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.991+0000 7f670c800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f670406dd70 con 0x7f67040cd460 2024-04-23T14:32:44.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.991+0000 7f670c800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67040b1f90 con 0x7f67040cea60 2024-04-23T14:32:44.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.991+0000 7f670c800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f67040b2270 con 0x7f670406d450 2024-04-23T14:32:44.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.991+0000 7f670a800640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f67040cd460 0x7f67040b2d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:45030/0 (socket says 172.21.15.100:45030) 2024-04-23T14:32:44.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.991+0000 7f670a800640 1 -- 172.21.15.100:0/3203536646 learned_addr learned my addr 172.21.15.100:0/3203536646 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:44.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.991+0000 7f6709e00640 1 --2- 172.21.15.100:0/3203536646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f670406d450 0x7f670406d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:44.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.991+0000 7f6708a00640 1 -- 172.21.15.100:0/3203536646 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1595461061 0 0) 0x7f670406dd70 con 0x7f67040cd460 2024-04-23T14:32:44.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.991+0000 7f6708a00640 1 -- 172.21.15.100:0/3203536646 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66ec003530 con 0x7f67040cd460 2024-04-23T14:32:44.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.991+0000 7f6708a00640 1 -- 172.21.15.100:0/3203536646 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3661262977 0 0) 0x7f67040b1f90 con 0x7f67040cea60 2024-04-23T14:32:44.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.991+0000 7f6708a00640 1 -- 172.21.15.100:0/3203536646 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f670406dd70 con 0x7f67040cea60 2024-04-23T14:32:44.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.991+0000 7f6709e00640 1 -- 172.21.15.100:0/3203536646 >> v1:172.21.15.151:6789/0 conn(0x7f67040cea60 legacy=0x7f67040d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.991+0000 7f6709e00640 1 -- 172.21.15.100:0/3203536646 >> v1:172.21.15.100:6789/0 conn(0x7f67040cd460 legacy=0x7f67040b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.991+0000 7f6709e00640 1 -- 172.21.15.100:0/3203536646 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67040b2550 con 0x7f670406d450 2024-04-23T14:32:44.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.995+0000 7f6709e00640 1 --2- 172.21.15.100:0/3203536646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f670406d450 0x7f670406d830 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f66f80097f0 tx=0x7f66f801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55f1ad6b039b40af server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:44.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.995+0000 7f6708a00640 1 -- 172.21.15.100:0/3203536646 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f80b0070 con 0x7f670406d450 2024-04-23T14:32:44.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.995+0000 7f6708a00640 1 -- 172.21.15.100:0/3203536646 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66f80ab600 con 0x7f670406d450 2024-04-23T14:32:44.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.995+0000 7f6708a00640 1 -- 172.21.15.100:0/3203536646 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f80ab9d0 con 0x7f670406d450 2024-04-23T14:32:44.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.995+0000 7f670c800640 1 -- 172.21.15.100:0/3203536646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f670406d450 msgr2=0x7f670406d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.995+0000 7f670c800640 1 --2- 172.21.15.100:0/3203536646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f670406d450 0x7f670406d830 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f66f80097f0 tx=0x7f66f801c450 comp rx=0 tx=0).stop 2024-04-23T14:32:44.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.995+0000 7f670c800640 1 -- 172.21.15.100:0/3203536646 shutdown_connections 2024-04-23T14:32:44.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.995+0000 7f670c800640 1 --2- 172.21.15.100:0/3203536646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f670406d450 0x7f670406d830 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.995+0000 7f670c800640 1 -- 172.21.15.100:0/3203536646 >> 172.21.15.100:0/3203536646 conn(0x7f67040cee80 msgr2=0x7f67040cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:44.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.995+0000 7f670c800640 1 -- 172.21.15.100:0/3203536646 shutdown_connections 2024-04-23T14:32:44.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.995+0000 7f670c800640 1 -- 172.21.15.100:0/3203536646 wait complete. 2024-04-23T14:32:44.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f670c800640 1 Processor -- start 2024-04-23T14:32:44.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f670c800640 1 -- start start 2024-04-23T14:32:45.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f670c800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f670406d450 0x7f670415bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f670c800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f67040cd460 0x7f67040c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f670c800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67040cea60 0x7f67040c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f670c800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f670416cd50 con 0x7f67040cea60 2024-04-23T14:32:45.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f670c800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f670416cbd0 con 0x7f670406d450 2024-04-23T14:32:45.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f670c800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f670416ced0 con 0x7f67040cd460 2024-04-23T14:32:45.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f670a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67040cea60 0x7f67040c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f670a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67040cea60 0x7f67040c4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53184/0 (socket says 172.21.15.100:53184) 2024-04-23T14:32:45.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f670a800640 1 -- 172.21.15.100:0/2738447207 learned_addr learned my addr 172.21.15.100:0/2738447207 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:45.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f6709e00640 1 --2- 172.21.15.100:0/2738447207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f670406d450 0x7f670415bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f6709400640 1 --2- 172.21.15.100:0/2738447207 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f67040cd460 0x7f67040c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f670a800640 1 -- 172.21.15.100:0/2738447207 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f67040cd460 msgr2=0x7f67040c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f670a800640 1 --2- 172.21.15.100:0/2738447207 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f67040cd460 0x7f67040c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f670a800640 1 -- 172.21.15.100:0/2738447207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f670406d450 msgr2=0x7f670415bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f670a800640 1 --2- 172.21.15.100:0/2738447207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f670406d450 0x7f670415bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f670a800640 1 -- 172.21.15.100:0/2738447207 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670415c5f0 con 0x7f67040cea60 2024-04-23T14:32:45.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f670a800640 1 --2- 172.21.15.100:0/2738447207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67040cea60 0x7f67040c4a50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f670000cb20 tx=0x7f670000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:45.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f66fe000640 1 -- 172.21.15.100:0/2738447207 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6700017070 con 0x7f67040cea60 2024-04-23T14:32:45.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f66fe000640 1 -- 172.21.15.100:0/2738447207 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f670000aca0 con 0x7f67040cea60 2024-04-23T14:32:45.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f66fe000640 1 -- 172.21.15.100:0/2738447207 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6700012600 con 0x7f67040cea60 2024-04-23T14:32:45.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f670c800640 1 -- 172.21.15.100:0/2738447207 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670415c8d0 con 0x7f67040cea60 2024-04-23T14:32:45.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:44.999+0000 7f670c800640 1 -- 172.21.15.100:0/2738447207 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f670415ce90 con 0x7f67040cea60 2024-04-23T14:32:45.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.003+0000 7f670c800640 1 -- 172.21.15.100:0/2738447207 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6704068ad0 con 0x7f67040cea60 2024-04-23T14:32:45.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.003+0000 7f66fe000640 1 -- 172.21.15.100:0/2738447207 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f670001a070 con 0x7f67040cea60 2024-04-23T14:32:45.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.003+0000 7f66fe000640 1 --2- 172.21.15.100:0/2738447207 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f66e80683b0 0x7f66e806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.003+0000 7f66fe000640 1 -- 172.21.15.100:0/2738447207 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6700086920 con 0x7f67040cea60 2024-04-23T14:32:45.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.003+0000 7f6709e00640 1 --2- 172.21.15.100:0/2738447207 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f66e80683b0 0x7f66e806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.003+0000 7f6709e00640 1 --2- 172.21.15.100:0/2738447207 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f66e80683b0 0x7f66e806a870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f66f80ad000 tx=0x7f66f80a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:45.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.007+0000 7f66fe000640 1 -- 172.21.15.100:0/2738447207 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6700056540 con 0x7f67040cea60 2024-04-23T14:32:45.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.199+0000 7f670c800640 1 -- 172.21.15.100:0/2738447207 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67040cce10 con 0x7f67040cea60 2024-04-23T14:32:45.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.199+0000 7f66fe000640 1 -- 172.21.15.100:0/2738447207 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f670005a840 con 0x7f67040cea60 2024-04-23T14:32:45.200 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:45.200 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1456958549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1456958549},{"type":"v1","addr":"172.21.15.100:6836","nonce":1456958549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3655609551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3655609551},{"type":"v1","addr":"172.21.15.100:6837","nonce":3655609551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/4242180972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":4242180972},{"type":"v1","addr":"172.21.15.151:6834","nonce":4242180972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:32:08.976576+0000","modified":"2024-04-23T14:32:34.057740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1533322173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1533322173},{"type":"v1","addr":"172.21.15.151:6835","nonce":1533322173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-23T14:32:45.200 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 9 2024-04-23T14:32:45.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.199+0000 7f670c800640 1 -- 172.21.15.100:0/2738447207 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f66e80683b0 msgr2=0x7f66e806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.199+0000 7f670c800640 1 --2- 172.21.15.100:0/2738447207 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f66e80683b0 0x7f66e806a870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f66f80ad000 tx=0x7f66f80a6000 comp rx=0 tx=0).stop 2024-04-23T14:32:45.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.199+0000 7f670c800640 1 -- 172.21.15.100:0/2738447207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67040cea60 msgr2=0x7f67040c4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.199+0000 7f670c800640 1 --2- 172.21.15.100:0/2738447207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67040cea60 0x7f67040c4a50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f670000cb20 tx=0x7f670000a060 comp rx=0 tx=0).stop 2024-04-23T14:32:45.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.199+0000 7f670c800640 1 -- 172.21.15.100:0/2738447207 shutdown_connections 2024-04-23T14:32:45.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.199+0000 7f670c800640 1 --2- 172.21.15.100:0/2738447207 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f66e80683b0 0x7f66e806a870 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.199+0000 7f670c800640 1 --2- 172.21.15.100:0/2738447207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67040cea60 0x7f67040c4a50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.199+0000 7f670c800640 1 --2- 172.21.15.100:0/2738447207 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f67040cd460 0x7f67040c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.199+0000 7f670c800640 1 --2- 172.21.15.100:0/2738447207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f670406d450 0x7f670415bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.199+0000 7f670c800640 1 -- 172.21.15.100:0/2738447207 >> 172.21.15.100:0/2738447207 conn(0x7f67040cee80 msgr2=0x7f67040d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:45.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.203+0000 7f670c800640 1 -- 172.21.15.100:0/2738447207 shutdown_connections 2024-04-23T14:32:45.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.203+0000 7f670c800640 1 -- 172.21.15.100:0/2738447207 wait complete. 2024-04-23T14:32:45.222 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:45.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.355+0000 7f2637200640 1 Processor -- start 2024-04-23T14:32:45.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.355+0000 7f2637200640 1 -- start start 2024-04-23T14:32:45.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.355+0000 7f2637200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f263015e730 0x7f26300cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.355+0000 7f2637200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f26300d0760 0x7f26300ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.355+0000 7f2637200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f263006f450 con 0x7f263015eb10 2024-04-23T14:32:45.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.355+0000 7f2637200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f263006f170 con 0x7f26300d0760 2024-04-23T14:32:45.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.355+0000 7f2637200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f2630162590 con 0x7f263015e730 2024-04-23T14:32:45.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.359+0000 7f2635200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f26300d0760 0x7f26300ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.359+0000 7f2627400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f263015eb10 0x7f26300d0050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:45032/0 (socket says 172.21.15.100:45032) 2024-04-23T14:32:45.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.359+0000 7f2627400640 1 -- 172.21.15.100:0/3029304175 learned_addr learned my addr 172.21.15.100:0/3029304175 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:45.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.359+0000 7f262fe00640 1 --2- 172.21.15.100:0/3029304175 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f263015e730 0x7f26300cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.359+0000 7f2635200640 1 -- 172.21.15.100:0/3029304175 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f263015e730 msgr2=0x7f26300cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:45.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.359+0000 7f2635200640 1 --2- 172.21.15.100:0/3029304175 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f263015e730 0x7f26300cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.359+0000 7f2635200640 1 -- 172.21.15.100:0/3029304175 >> v1:172.21.15.100:6789/0 conn(0x7f263015eb10 legacy=0x7f26300d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.359+0000 7f2635200640 1 -- 172.21.15.100:0/3029304175 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2630162870 con 0x7f26300d0760 2024-04-23T14:32:45.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.359+0000 7f262f400640 1 -- 172.21.15.100:0/3029304175 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2664828871 0 0) 0x7f263006f450 con 0x7f263015eb10 2024-04-23T14:32:45.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.359+0000 7f2635200640 1 --2- 172.21.15.100:0/3029304175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26300d0760 0x7f26300ccad0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2628009d90 tx=0x7f262801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d8ad2ef3438f2b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:45.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.359+0000 7f262f400640 1 -- 172.21.15.100:0/3029304175 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26280b2070 con 0x7f26300d0760 2024-04-23T14:32:45.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.359+0000 7f262f400640 1 -- 172.21.15.100:0/3029304175 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26280a58b0 con 0x7f26300d0760 2024-04-23T14:32:45.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.359+0000 7f262f400640 1 -- 172.21.15.100:0/3029304175 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26280a5c80 con 0x7f26300d0760 2024-04-23T14:32:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2637200640 1 -- 172.21.15.100:0/3029304175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26300d0760 msgr2=0x7f26300ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2637200640 1 --2- 172.21.15.100:0/3029304175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26300d0760 0x7f26300ccad0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2628009d90 tx=0x7f262801c8e0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2637200640 1 -- 172.21.15.100:0/3029304175 shutdown_connections 2024-04-23T14:32:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2637200640 1 --2- 172.21.15.100:0/3029304175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26300d0760 0x7f26300ccad0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2637200640 1 --2- 172.21.15.100:0/3029304175 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f263015e730 0x7f26300cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2637200640 1 -- 172.21.15.100:0/3029304175 >> 172.21.15.100:0/3029304175 conn(0x7f26300b4050 msgr2=0x7f26300b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2637200640 1 -- 172.21.15.100:0/3029304175 shutdown_connections 2024-04-23T14:32:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2637200640 1 -- 172.21.15.100:0/3029304175 wait complete. 2024-04-23T14:32:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2637200640 1 Processor -- start 2024-04-23T14:32:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2637200640 1 -- start start 2024-04-23T14:32:45.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2637200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f26300d0760 0x7f26300c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2637200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f263015e730 0x7f26300b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2637200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f263015eb10 0x7f26300b6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2637200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f263016bb70 con 0x7f263015e730 2024-04-23T14:32:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2637200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f263016b9f0 con 0x7f263015eb10 2024-04-23T14:32:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2637200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f263016bcf0 con 0x7f26300d0760 2024-04-23T14:32:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2627400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f263015e730 0x7f26300b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2627400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f263015e730 0x7f26300b6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53198/0 (socket says 172.21.15.100:53198) 2024-04-23T14:32:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2627400640 1 -- 172.21.15.100:0/4007152466 learned_addr learned my addr 172.21.15.100:0/4007152466 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2635200640 1 --2- 172.21.15.100:0/4007152466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f263015eb10 0x7f26300b6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f262fe00640 1 --2- 172.21.15.100:0/4007152466 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f26300d0760 0x7f26300c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2627400640 1 -- 172.21.15.100:0/4007152466 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f26300d0760 msgr2=0x7f26300c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2627400640 1 --2- 172.21.15.100:0/4007152466 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f26300d0760 0x7f26300c5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2627400640 1 -- 172.21.15.100:0/4007152466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f263015eb10 msgr2=0x7f26300b6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2627400640 1 --2- 172.21.15.100:0/4007152466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f263015eb10 0x7f26300b6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2627400640 1 -- 172.21.15.100:0/4007152466 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26300d3350 con 0x7f263015e730 2024-04-23T14:32:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.363+0000 7f2635200640 1 --2- 172.21.15.100:0/4007152466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f263015eb10 0x7f26300b6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.367+0000 7f2627400640 1 --2- 172.21.15.100:0/4007152466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f263015e730 0x7f26300b6000 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f261800c4e0 tx=0x7f261800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:45.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.367+0000 7f262cc00640 1 -- 172.21.15.100:0/4007152466 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618018070 con 0x7f263015e730 2024-04-23T14:32:45.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.367+0000 7f262cc00640 1 -- 172.21.15.100:0/4007152466 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2618007500 con 0x7f263015e730 2024-04-23T14:32:45.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.367+0000 7f262cc00640 1 -- 172.21.15.100:0/4007152466 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618004a30 con 0x7f263015e730 2024-04-23T14:32:45.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.367+0000 7f2637200640 1 -- 172.21.15.100:0/4007152466 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26300c64f0 con 0x7f263015e730 2024-04-23T14:32:45.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.367+0000 7f2637200640 1 -- 172.21.15.100:0/4007152466 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26300c6a50 con 0x7f263015e730 2024-04-23T14:32:45.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.367+0000 7f2637200640 1 -- 172.21.15.100:0/4007152466 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2630068ad0 con 0x7f263015e730 2024-04-23T14:32:45.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.367+0000 7f262cc00640 1 -- 172.21.15.100:0/4007152466 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2618004bd0 con 0x7f263015e730 2024-04-23T14:32:45.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.367+0000 7f262cc00640 1 --2- 172.21.15.100:0/4007152466 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f260c068380 0x7f260c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.367+0000 7f262cc00640 1 -- 172.21.15.100:0/4007152466 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2618085fb0 con 0x7f263015e730 2024-04-23T14:32:45.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.367+0000 7f262fe00640 1 --2- 172.21.15.100:0/4007152466 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f260c068380 0x7f260c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.371+0000 7f262fe00640 1 --2- 172.21.15.100:0/4007152466 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f260c068380 0x7f260c06a840 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2620009ac0 tx=0x7f2620009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:45.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.371+0000 7f262cc00640 1 -- 172.21.15.100:0/4007152466 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f26300c64f0 con 0x7f263015e730 2024-04-23T14:32:45.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.531+0000 7f2637200640 1 -- 172.21.15.100:0/4007152466 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26300b2950 con 0x7f263015e730 2024-04-23T14:32:45.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.531+0000 7f262cc00640 1 -- 172.21.15.100:0/4007152466 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f2618055c90 con 0x7f263015e730 2024-04-23T14:32:45.531 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:45.532 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":24,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:14.037027+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:45.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.531+0000 7f2637200640 1 -- 172.21.15.100:0/4007152466 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f260c068380 msgr2=0x7f260c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.531+0000 7f2637200640 1 --2- 172.21.15.100:0/4007152466 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f260c068380 0x7f260c06a840 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2620009ac0 tx=0x7f2620009190 comp rx=0 tx=0).stop 2024-04-23T14:32:45.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.531+0000 7f2637200640 1 -- 172.21.15.100:0/4007152466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f263015e730 msgr2=0x7f26300b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.531+0000 7f2637200640 1 --2- 172.21.15.100:0/4007152466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f263015e730 0x7f26300b6000 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f261800c4e0 tx=0x7f261800c9a0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.531+0000 7f2637200640 1 -- 172.21.15.100:0/4007152466 shutdown_connections 2024-04-23T14:32:45.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.531+0000 7f2637200640 1 --2- 172.21.15.100:0/4007152466 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f260c068380 0x7f260c06a840 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.531+0000 7f2637200640 1 --2- 172.21.15.100:0/4007152466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f263015eb10 0x7f26300b6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.531+0000 7f2637200640 1 --2- 172.21.15.100:0/4007152466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f263015e730 0x7f26300b6000 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.531+0000 7f2637200640 1 --2- 172.21.15.100:0/4007152466 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f26300d0760 0x7f26300c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.531+0000 7f2637200640 1 -- 172.21.15.100:0/4007152466 >> 172.21.15.100:0/4007152466 conn(0x7f26300b4050 msgr2=0x7f26300b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:45.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.531+0000 7f2637200640 1 -- 172.21.15.100:0/4007152466 shutdown_connections 2024-04-23T14:32:45.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.531+0000 7f2637200640 1 -- 172.21.15.100:0/4007152466 wait complete. 2024-04-23T14:32:45.552 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:32:45.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.687+0000 7f6812600640 1 Processor -- start 2024-04-23T14:32:45.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.687+0000 7f6812600640 1 -- start start 2024-04-23T14:32:45.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.691+0000 7f6812600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f680c06d350 0x7f680c073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.691+0000 7f6812600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f680c15fa90 0x7f680c06d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.691+0000 7f6812600640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f680c162390 0x7f680c162810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.691+0000 7f6812600640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f680c06eaf0 con 0x7f680c06d350 2024-04-23T14:32:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.691+0000 7f6812600640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f680c06e530 con 0x7f680c162390 2024-04-23T14:32:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.691+0000 7f6812600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f680c073880 con 0x7f680c15fa90 2024-04-23T14:32:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.691+0000 7f680b400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f680c06d350 0x7f680c073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.691+0000 7f680b400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f680c06d350 0x7f680c073340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53212/0 (socket says 172.21.15.100:53212) 2024-04-23T14:32:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.691+0000 7f680b400640 1 -- 172.21.15.100:0/2879171750 learned_addr learned my addr 172.21.15.100:0/2879171750 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.691+0000 7f680be00640 1 --2- 172.21.15.100:0/2879171750 >> v2:172.21.15.151:3300/0 conn(0x7f680c162390 0x7f680c162810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.691+0000 7f680aa00640 1 --2- 172.21.15.100:0/2879171750 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f680c15fa90 0x7f680c06d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.691+0000 7f680b400640 1 -- 172.21.15.100:0/2879171750 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f680c15fa90 msgr2=0x7f680c06d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.691+0000 7f680b400640 1 --2- 172.21.15.100:0/2879171750 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f680c15fa90 0x7f680c06d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.691+0000 7f680b400640 1 -- 172.21.15.100:0/2879171750 >> v2:172.21.15.151:3300/0 conn(0x7f680c162390 msgr2=0x7f680c162810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.691+0000 7f680b400640 1 --2- 172.21.15.100:0/2879171750 >> v2:172.21.15.151:3300/0 conn(0x7f680c162390 0x7f680c162810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.691+0000 7f680b400640 1 -- 172.21.15.100:0/2879171750 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680c06dc70 con 0x7f680c06d350 2024-04-23T14:32:45.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.695+0000 7f680b400640 1 --2- 172.21.15.100:0/2879171750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f680c06d350 0x7f680c073340 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f680c068110 tx=0x7f67fc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6c924cbc407183 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:45.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.695+0000 7f680a000640 1 -- 172.21.15.100:0/2879171750 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc0ab640 con 0x7f680c06d350 2024-04-23T14:32:45.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.695+0000 7f680a000640 1 -- 172.21.15.100:0/2879171750 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67fc0b0070 con 0x7f680c06d350 2024-04-23T14:32:45.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.695+0000 7f680a000640 1 -- 172.21.15.100:0/2879171750 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc0aba00 con 0x7f680c06d350 2024-04-23T14:32:45.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.695+0000 7f6812600640 1 -- 172.21.15.100:0/2879171750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f680c06d350 msgr2=0x7f680c073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.695+0000 7f6812600640 1 --2- 172.21.15.100:0/2879171750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f680c06d350 0x7f680c073340 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f680c068110 tx=0x7f67fc01c460 comp rx=0 tx=0).stop 2024-04-23T14:32:45.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.695+0000 7f6812600640 1 -- 172.21.15.100:0/2879171750 shutdown_connections 2024-04-23T14:32:45.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.695+0000 7f6812600640 1 --2- 172.21.15.100:0/2879171750 >> v2:172.21.15.151:3300/0 conn(0x7f680c162390 0x7f680c162810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.695+0000 7f6812600640 1 --2- 172.21.15.100:0/2879171750 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f680c15fa90 0x7f680c06d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.695+0000 7f6812600640 1 --2- 172.21.15.100:0/2879171750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f680c06d350 0x7f680c073340 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.695+0000 7f6812600640 1 -- 172.21.15.100:0/2879171750 >> 172.21.15.100:0/2879171750 conn(0x7f680c072600 msgr2=0x7f680c072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:45.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.695+0000 7f6812600640 1 -- 172.21.15.100:0/2879171750 shutdown_connections 2024-04-23T14:32:45.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.695+0000 7f6812600640 1 -- 172.21.15.100:0/2879171750 wait complete. 2024-04-23T14:32:45.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.695+0000 7f6812600640 1 Processor -- start 2024-04-23T14:32:45.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.695+0000 7f6812600640 1 -- start start 2024-04-23T14:32:45.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f6812600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f680c06d350 0x7f680c13de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f6812600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f680c15fa90 0x7f680c158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f6812600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f680c162390 0x7f680c158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f6812600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f680c06f790 con 0x7f680c15fa90 2024-04-23T14:32:45.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f6812600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f680c163160 con 0x7f680c162390 2024-04-23T14:32:45.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f6812600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f680c06f910 con 0x7f680c06d350 2024-04-23T14:32:45.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f680aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f680c15fa90 0x7f680c158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f680aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f680c15fa90 0x7f680c158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53216/0 (socket says 172.21.15.100:53216) 2024-04-23T14:32:45.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f680aa00640 1 -- 172.21.15.100:0/504901645 learned_addr learned my addr 172.21.15.100:0/504901645 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:45.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f680be00640 1 --2- 172.21.15.100:0/504901645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f680c162390 0x7f680c158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f680b400640 1 --2- 172.21.15.100:0/504901645 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f680c06d350 0x7f680c13de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f680aa00640 1 -- 172.21.15.100:0/504901645 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f680c06d350 msgr2=0x7f680c13de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f680aa00640 1 --2- 172.21.15.100:0/504901645 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f680c06d350 0x7f680c13de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f680aa00640 1 -- 172.21.15.100:0/504901645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f680c162390 msgr2=0x7f680c158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f680aa00640 1 --2- 172.21.15.100:0/504901645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f680c162390 0x7f680c158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f680aa00640 1 -- 172.21.15.100:0/504901645 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680c140680 con 0x7f680c15fa90 2024-04-23T14:32:45.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f680aa00640 1 --2- 172.21.15.100:0/504901645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f680c15fa90 0x7f680c158630 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f67f800d800 tx=0x7f67f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:45.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f6803400640 1 -- 172.21.15.100:0/504901645 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f8018030 con 0x7f680c15fa90 2024-04-23T14:32:45.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f6803400640 1 -- 172.21.15.100:0/504901645 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67f8004020 con 0x7f680c15fa90 2024-04-23T14:32:45.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f6812600640 1 -- 172.21.15.100:0/504901645 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680c140920 con 0x7f680c15fa90 2024-04-23T14:32:45.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f6803400640 1 -- 172.21.15.100:0/504901645 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f8004210 con 0x7f680c15fa90 2024-04-23T14:32:45.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f6812600640 1 -- 172.21.15.100:0/504901645 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f680c140ee0 con 0x7f680c15fa90 2024-04-23T14:32:45.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f6812600640 1 -- 172.21.15.100:0/504901645 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f680c068ad0 con 0x7f680c15fa90 2024-04-23T14:32:45.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f6803400640 1 -- 172.21.15.100:0/504901645 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f67f801b070 con 0x7f680c15fa90 2024-04-23T14:32:45.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.699+0000 7f6803400640 1 --2- 172.21.15.100:0/504901645 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f67ec0683b0 0x7f67ec06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.703+0000 7f6803400640 1 -- 172.21.15.100:0/504901645 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f67f8086d80 con 0x7f680c15fa90 2024-04-23T14:32:45.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.703+0000 7f680b400640 1 --2- 172.21.15.100:0/504901645 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f67ec0683b0 0x7f67ec06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.703+0000 7f680b400640 1 --2- 172.21.15.100:0/504901645 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f67ec0683b0 0x7f67ec06a870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0af000 tx=0x7f67fc0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:45.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.703+0000 7f6803400640 1 -- 172.21.15.100:0/504901645 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f67f8056880 con 0x7f680c15fa90 2024-04-23T14:32:45.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.903+0000 7f6812600640 1 -- 172.21.15.100:0/504901645 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f680c073120 con 0x7f680c15fa90 2024-04-23T14:32:45.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.903+0000 7f6803400640 1 -- 172.21.15.100:0/504901645 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f67f805ab80 con 0x7f680c15fa90 2024-04-23T14:32:45.904 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:45.904 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1456958549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1456958549},{"type":"v1","addr":"172.21.15.100:6836","nonce":1456958549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3655609551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3655609551},{"type":"v1","addr":"172.21.15.100:6837","nonce":3655609551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/4242180972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":4242180972},{"type":"v1","addr":"172.21.15.151:6834","nonce":4242180972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:32:08.976576+0000","modified":"2024-04-23T14:32:34.057740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1533322173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1533322173},{"type":"v1","addr":"172.21.15.151:6835","nonce":1533322173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-23T14:32:45.904 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 9 2024-04-23T14:32:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.903+0000 7f6812600640 1 -- 172.21.15.100:0/504901645 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f67ec0683b0 msgr2=0x7f67ec06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.903+0000 7f6812600640 1 --2- 172.21.15.100:0/504901645 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f67ec0683b0 0x7f67ec06a870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0af000 tx=0x7f67fc0a7000 comp rx=0 tx=0).stop 2024-04-23T14:32:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.903+0000 7f6812600640 1 -- 172.21.15.100:0/504901645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f680c15fa90 msgr2=0x7f680c158630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.903+0000 7f6812600640 1 --2- 172.21.15.100:0/504901645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f680c15fa90 0x7f680c158630 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f67f800d800 tx=0x7f67f800dcc0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.903+0000 7f6812600640 1 -- 172.21.15.100:0/504901645 shutdown_connections 2024-04-23T14:32:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.903+0000 7f6812600640 1 --2- 172.21.15.100:0/504901645 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f67ec0683b0 0x7f67ec06a870 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.903+0000 7f6812600640 1 --2- 172.21.15.100:0/504901645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f680c162390 0x7f680c158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.903+0000 7f6812600640 1 --2- 172.21.15.100:0/504901645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f680c15fa90 0x7f680c158630 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.903+0000 7f6812600640 1 --2- 172.21.15.100:0/504901645 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f680c06d350 0x7f680c13de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.903+0000 7f6812600640 1 -- 172.21.15.100:0/504901645 >> 172.21.15.100:0/504901645 conn(0x7f680c072600 msgr2=0x7f680c1613e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.903+0000 7f6812600640 1 -- 172.21.15.100:0/504901645 shutdown_connections 2024-04-23T14:32:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:45.907+0000 7f6812600640 1 -- 172.21.15.100:0/504901645 wait complete. 2024-04-23T14:32:46.327 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:32:08.976576+0000', 'modified': '2024-04-23T14:32:34.057740+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4279}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4279': {'gid': 4279, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.151:6835/1533322173', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6833', 'nonce': 1533322173}, {'type': 'v1', 'addr': '172.21.15.151:6835', 'nonce': 1533322173}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4279, 'qdb_cluster': [4279]} 2024-04-23T14:32:46.327 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-23T14:32:46.327 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-23T14:32:46.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.463+0000 7f4f2b600640 1 Processor -- start 2024-04-23T14:32:46.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.463+0000 7f4f2b600640 1 -- start start 2024-04-23T14:32:46.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.463+0000 7f4f2b600640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f4f2406f9d0 0x7f4f2406c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.463+0000 7f4f2b600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f2415f360 0x7f4f240729e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.463+0000 7f4f2b600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f2415f9b0 con 0x7f4f24072550 2024-04-23T14:32:46.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.463+0000 7f4f2b600640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f4f2406bc50 con 0x7f4f2406f9d0 2024-04-23T14:32:46.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.463+0000 7f4f2b600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4f2406bf30 con 0x7f4f2415f360 2024-04-23T14:32:46.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.463+0000 7f4f23e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f4f24072550 0x7f4f2415ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:45040/0 (socket says 172.21.15.100:45040) 2024-04-23T14:32:46.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.463+0000 7f4f23e00640 1 -- 172.21.15.100:0/1533225215 learned_addr learned my addr 172.21.15.100:0/1533225215 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:46.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f28c00640 1 --2- 172.21.15.100:0/1533225215 >> v2:172.21.15.151:3300/0 conn(0x7f4f2406f9d0 0x7f4f2406c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f29600640 1 --2- 172.21.15.100:0/1533225215 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f2415f360 0x7f4f240729e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f23400640 1 -- 172.21.15.100:0/1533225215 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2121748174 0 0) 0x7f4f2415f9b0 con 0x7f4f24072550 2024-04-23T14:32:46.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f23400640 1 -- 172.21.15.100:0/1533225215 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f08003530 con 0x7f4f24072550 2024-04-23T14:32:46.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f23400640 1 -- 172.21.15.100:0/1533225215 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3372264849 0 0) 0x7f4f08003530 con 0x7f4f24072550 2024-04-23T14:32:46.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f23400640 1 -- 172.21.15.100:0/1533225215 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f2415f360 msgr2=0x7f4f240729e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:46.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f23400640 1 --2- 172.21.15.100:0/1533225215 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f2415f360 0x7f4f240729e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f23400640 1 -- 172.21.15.100:0/1533225215 >> v2:172.21.15.151:3300/0 conn(0x7f4f2406f9d0 msgr2=0x7f4f2406c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:46.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f23400640 1 --2- 172.21.15.100:0/1533225215 >> v2:172.21.15.151:3300/0 conn(0x7f4f2406f9d0 0x7f4f2406c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f23400640 1 -- 172.21.15.100:0/1533225215 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f2406c210 con 0x7f4f24072550 2024-04-23T14:32:46.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f23400640 1 -- 172.21.15.100:0/1533225215 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f4f10001680 con 0x7f4f24072550 2024-04-23T14:32:46.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f23400640 1 -- 172.21.15.100:0/1533225215 >> v1:172.21.15.100:6789/0 conn(0x7f4f24072550 legacy=0x7f4f2415ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:46.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f23400640 1 --2- 172.21.15.100:0/1533225215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f080039c0 0x7f4f08003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f23400640 1 --2- 172.21.15.100:0/1533225215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f08004810 0x7f4f08004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f23400640 1 --2- 172.21.15.100:0/1533225215 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f2415f360 0x7f4f080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f23400640 1 -- 172.21.15.100:0/1533225215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2406bc50 con 0x7f4f08004810 2024-04-23T14:32:46.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f23400640 1 -- 172.21.15.100:0/1533225215 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2406bf30 con 0x7f4f080039c0 2024-04-23T14:32:46.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f23400640 1 -- 172.21.15.100:0/1533225215 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4f0800ba00 con 0x7f4f2415f360 2024-04-23T14:32:46.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f23e00640 1 --2- 172.21.15.100:0/1533225215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f08004810 0x7f4f08004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f28c00640 1 --2- 172.21.15.100:0/1533225215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f080039c0 0x7f4f08003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f29600640 1 --2- 172.21.15.100:0/1533225215 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f2415f360 0x7f4f080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f23e00640 1 -- 172.21.15.100:0/1533225215 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f2415f360 msgr2=0x7f4f080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:46.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f23e00640 1 --2- 172.21.15.100:0/1533225215 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f2415f360 0x7f4f080051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f23e00640 1 -- 172.21.15.100:0/1533225215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f080039c0 msgr2=0x7f4f08003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:46.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f23e00640 1 --2- 172.21.15.100:0/1533225215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f080039c0 0x7f4f08003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f23e00640 1 -- 172.21.15.100:0/1533225215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f08003530 con 0x7f4f08004810 2024-04-23T14:32:46.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.467+0000 7f4f28c00640 1 --2- 172.21.15.100:0/1533225215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f080039c0 0x7f4f08003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:46.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.471+0000 7f4f23e00640 1 --2- 172.21.15.100:0/1533225215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f08004810 0x7f4f08004c90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000c240 tx=0x7f4f1001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdbbd3129551201 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:46.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.471+0000 7f4f23400640 1 -- 172.21.15.100:0/1533225215 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10007d40 con 0x7f4f08004810 2024-04-23T14:32:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.471+0000 7f4f23400640 1 -- 172.21.15.100:0/1533225215 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f100a9020 con 0x7f4f08004810 2024-04-23T14:32:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.471+0000 7f4f23400640 1 -- 172.21.15.100:0/1533225215 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10005b00 con 0x7f4f08004810 2024-04-23T14:32:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.471+0000 7f4f2b600640 1 -- 172.21.15.100:0/1533225215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f08004810 msgr2=0x7f4f08004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.471+0000 7f4f2b600640 1 --2- 172.21.15.100:0/1533225215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f08004810 0x7f4f08004c90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000c240 tx=0x7f4f1001cca0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.471+0000 7f4f29600640 1 -- 172.21.15.100:0/1533225215 reap_dead start 2024-04-23T14:32:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.471+0000 7f4f2b600640 1 -- 172.21.15.100:0/1533225215 shutdown_connections 2024-04-23T14:32:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.471+0000 7f4f2b600640 1 -- 172.21.15.100:0/1533225215 >> 172.21.15.100:0/1533225215 conn(0x7f4f240b2600 msgr2=0x7f4f240b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.471+0000 7f4f2b600640 1 -- 172.21.15.100:0/1533225215 shutdown_connections 2024-04-23T14:32:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.471+0000 7f4f2b600640 1 -- 172.21.15.100:0/1533225215 wait complete. 2024-04-23T14:32:46.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.471+0000 7f4f2b600640 1 Processor -- start 2024-04-23T14:32:46.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.471+0000 7f4f2b600640 1 -- start start 2024-04-23T14:32:46.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.471+0000 7f4f2b600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f08004810 0x7f4f240cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.471+0000 7f4f2b600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f2414b020 0x7f4f240cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.471+0000 7f4f2b600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f240d0310 0x7f4f240cd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.471+0000 7f4f2b600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4f240729e0 con 0x7f4f2414b020 2024-04-23T14:32:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.471+0000 7f4f2b600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2406ffc0 con 0x7f4f240d0310 2024-04-23T14:32:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.471+0000 7f4f2b600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4f24072b60 con 0x7f4f08004810 2024-04-23T14:32:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.471+0000 7f4f23e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f2414b020 0x7f4f240cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.475+0000 7f4f23e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f2414b020 0x7f4f240cfdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53224/0 (socket says 172.21.15.100:53224) 2024-04-23T14:32:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.475+0000 7f4f23e00640 1 -- 172.21.15.100:0/2241956094 learned_addr learned my addr 172.21.15.100:0/2241956094 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.475+0000 7f4f23e00640 1 -- 172.21.15.100:0/2241956094 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f08004810 msgr2=0x7f4f240cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.475+0000 7f4f28c00640 1 --2- 172.21.15.100:0/2241956094 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f08004810 0x7f4f240cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.475+0000 7f4f23e00640 1 --2- 172.21.15.100:0/2241956094 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f08004810 0x7f4f240cf4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.475+0000 7f4f29600640 1 --2- 172.21.15.100:0/2241956094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f240d0310 0x7f4f240cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.475+0000 7f4f23e00640 1 -- 172.21.15.100:0/2241956094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f240d0310 msgr2=0x7f4f240cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.475+0000 7f4f23e00640 1 --2- 172.21.15.100:0/2241956094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f240d0310 0x7f4f240cd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.475+0000 7f4f23e00640 1 -- 172.21.15.100:0/2241956094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f240ce410 con 0x7f4f2414b020 2024-04-23T14:32:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.475+0000 7f4f29600640 1 --2- 172.21.15.100:0/2241956094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f240d0310 0x7f4f240cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:46.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.475+0000 7f4f23e00640 1 --2- 172.21.15.100:0/2241956094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f2414b020 0x7f4f240cfdd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4f10007040 tx=0x7f4f08005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:46.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.475+0000 7f4f20c00640 1 -- 172.21.15.100:0/2241956094 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10007d40 con 0x7f4f2414b020 2024-04-23T14:32:46.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.475+0000 7f4f20c00640 1 -- 172.21.15.100:0/2241956094 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f100b3070 con 0x7f4f2414b020 2024-04-23T14:32:46.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.475+0000 7f4f2b600640 1 -- 172.21.15.100:0/2241956094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f240ce6f0 con 0x7f4f2414b020 2024-04-23T14:32:46.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.475+0000 7f4f20c00640 1 -- 172.21.15.100:0/2241956094 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10001c80 con 0x7f4f2414b020 2024-04-23T14:32:46.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.475+0000 7f4f2b600640 1 -- 172.21.15.100:0/2241956094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f240cecb0 con 0x7f4f2414b020 2024-04-23T14:32:46.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.475+0000 7f4f20c00640 1 -- 172.21.15.100:0/2241956094 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4f10004c10 con 0x7f4f2414b020 2024-04-23T14:32:46.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.475+0000 7f4f20c00640 1 --2- 172.21.15.100:0/2241956094 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4f040683b0 0x7f4f0406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f20c00640 1 -- 172.21.15.100:0/2241956094 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4f10120b20 con 0x7f4f2414b020 2024-04-23T14:32:46.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f28c00640 1 --2- 172.21.15.100:0/2241956094 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4f040683b0 0x7f4f0406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f2eb1f000 1 Processor -- start 2024-04-23T14:32:46.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f2eb1f000 1 -- start start 2024-04-23T14:32:46.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f28c00640 1 --2- 172.21.15.100:0/2241956094 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4f040683b0 0x7f4f0406a870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4f14007520 tx=0x7f4f1400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:46.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f2eb1f000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645af65b0 0x564645af6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f2eb1f000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x564645af6ed0 0x564645aa9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f2eb1f000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564645aaa140 0x564645aaa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f2eb1f000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5646459936f0 con 0x564645af65b0 2024-04-23T14:32:46.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f2eb1f000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5646458fa4d0 con 0x564645aaa140 2024-04-23T14:32:46.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f2eb1f000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x564645894bf0 con 0x564645af6ed0 2024-04-23T14:32:46.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f28c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645af65b0 0x564645af6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f28c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645af65b0 0x564645af6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53232/0 (socket says 172.21.15.100:53232) 2024-04-23T14:32:46.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f28c00640 1 -- 172.21.15.100:0/1304599133 learned_addr learned my addr 172.21.15.100:0/1304599133 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:46.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f29600640 1 --2- 172.21.15.100:0/1304599133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564645aaa140 0x564645aaa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f23e00640 1 --2- 172.21.15.100:0/1304599133 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x564645af6ed0 0x564645aa9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f28c00640 1 -- 172.21.15.100:0/1304599133 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x564645af6ed0 msgr2=0x564645aa9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:46.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f28c00640 1 --2- 172.21.15.100:0/1304599133 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x564645af6ed0 0x564645aa9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f28c00640 1 -- 172.21.15.100:0/1304599133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564645aaa140 msgr2=0x564645aaa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:46.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f28c00640 1 --2- 172.21.15.100:0/1304599133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564645aaa140 0x564645aaa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f28c00640 1 -- 172.21.15.100:0/1304599133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564645894e40 con 0x564645af65b0 2024-04-23T14:32:46.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f28c00640 1 --2- 172.21.15.100:0/1304599133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645af65b0 0x564645af6990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4f140187f0 tx=0x7f4f14015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba93b302834ed2cb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:46.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f19600640 1 -- 172.21.15.100:0/1304599133 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1401e050 con 0x564645af65b0 2024-04-23T14:32:46.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f19600640 1 -- 172.21.15.100:0/1304599133 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f14028070 con 0x564645af65b0 2024-04-23T14:32:46.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f19600640 1 -- 172.21.15.100:0/1304599133 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f14018e30 con 0x564645af65b0 2024-04-23T14:32:46.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/1304599133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645af65b0 msgr2=0x564645af6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:46.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f2eb1f000 1 --2- 172.21.15.100:0/1304599133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645af65b0 0x564645af6990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4f140187f0 tx=0x7f4f14015000 comp rx=0 tx=0).stop 2024-04-23T14:32:46.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/1304599133 shutdown_connections 2024-04-23T14:32:46.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f2eb1f000 1 --2- 172.21.15.100:0/1304599133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564645aaa140 0x564645aaa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f2eb1f000 1 --2- 172.21.15.100:0/1304599133 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x564645af6ed0 0x564645aa9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f2eb1f000 1 --2- 172.21.15.100:0/1304599133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645af65b0 0x564645af6990 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/1304599133 >> 172.21.15.100:0/1304599133 conn(0x564645ad7350 msgr2=0x564645ad9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:46.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/1304599133 shutdown_connections 2024-04-23T14:32:46.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.479+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/1304599133 wait complete. 2024-04-23T14:32:46.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f2eb1f000 1 -- start start 2024-04-23T14:32:46.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f2eb1f000 1 Processor -- start 2024-04-23T14:32:46.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f2eb1f000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645aaa140 0x564645ab0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f2eb1f000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x564645af65b0 0x564645ab0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f2eb1f000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564645af6ed0 0x564645ab0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f2eb1f000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5646459b6e50 con 0x564645aaa140 2024-04-23T14:32:46.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f2eb1f000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5646458b7a60 con 0x564645af6ed0 2024-04-23T14:32:46.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f2eb1f000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x5646458b5cf0 con 0x564645af65b0 2024-04-23T14:32:46.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f2eb1f000 10 client.0 _ll_register_callbacks cb 0x564645a97a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:32:46.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f28c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645aaa140 0x564645ab0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f28c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645aaa140 0x564645ab0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53238/0 (socket says 172.21.15.100:53238) 2024-04-23T14:32:46.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f28c00640 1 -- 172.21.15.100:0/3537660398 learned_addr learned my addr 172.21.15.100:0/3537660398 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:46.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f29600640 1 --2- 172.21.15.100:0/3537660398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564645af6ed0 0x564645ab0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f23e00640 1 --2- 172.21.15.100:0/3537660398 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x564645af65b0 0x564645ab0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f28c00640 1 -- 172.21.15.100:0/3537660398 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x564645af65b0 msgr2=0x564645ab0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f28c00640 1 --2- 172.21.15.100:0/3537660398 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x564645af65b0 0x564645ab0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f28c00640 1 -- 172.21.15.100:0/3537660398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564645af6ed0 msgr2=0x564645ab0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f28c00640 1 --2- 172.21.15.100:0/3537660398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564645af6ed0 0x564645ab0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f28c00640 1 -- 172.21.15.100:0/3537660398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646458c5e00 con 0x564645aaa140 2024-04-23T14:32:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f23e00640 1 --2- 172.21.15.100:0/3537660398 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x564645af65b0 0x564645ab0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f28c00640 1 --2- 172.21.15.100:0/3537660398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645aaa140 0x564645ab0250 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4f14025040 tx=0x7f4f14004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2250720a159cbb63 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f0e000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-04-23T14:32:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f0e000640 1 -- 172.21.15.100:0/3537660398 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f14023a70 con 0x564645aaa140 2024-04-23T14:32:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f0e000640 1 -- 172.21.15.100:0/3537660398 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4f14023c10 con 0x564645aaa140 2024-04-23T14:32:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/3537660398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f14028070 con 0x564645aaa140 2024-04-23T14:32:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f0e000640 1 -- 172.21.15.100:0/3537660398 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f4f1401e020 con 0x564645aaa140 2024-04-23T14:32:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f2eb1f000 10 client.4812 fetch_fsmap learned FSMap version 9 2024-04-23T14:32:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/3537660398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x56464597c670 con 0x564645aaa140 2024-04-23T14:32:46.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f0e000640 1 -- 172.21.15.100:0/3537660398 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f14028070 con 0x564645aaa140 2024-04-23T14:32:46.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f0e000640 1 -- 172.21.15.100:0/3537660398 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f4f14036430 con 0x564645aaa140 2024-04-23T14:32:46.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f2eb1f000 10 client.4812 fetch_fsmap finished waiting for FSMap version 9 2024-04-23T14:32:46.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f2eb1f000 10 client.4812 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-23T14:32:46.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f2eb1f000 20 client.4812 populate_metadata read hostname 'smithi100' 2024-04-23T14:32:46.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f2eb1f000 1 --2- 172.21.15.100:0/3537660398 >> [v2:172.21.15.151:6833/1533322173,v1:172.21.15.151:6835/1533322173] conn(0x564645ab1570 0x564645b07d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f2eb1f000 4 client.4812 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:32:46.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/3537660398 --> [v2:172.21.15.151:6833/1533322173,v1:172.21.15.151:6835/1533322173] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564645893bc0 con 0x564645ab1570 2024-04-23T14:32:46.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.483+0000 7f4f23e00640 1 --2- 172.21.15.100:0/3537660398 >> [v2:172.21.15.151:6833/1533322173,v1:172.21.15.151:6835/1533322173] conn(0x564645ab1570 0x564645b07d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.487+0000 7f4f23e00640 1 --2- 172.21.15.100:0/3537660398 >> [v2:172.21.15.151:6833/1533322173,v1:172.21.15.151:6835/1533322173] conn(0x564645ab1570 0x564645b07d30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62ae99d8df33ef8c server_cookie=75d4710b454350ef in_seq=0 out_seq=0 2024-04-23T14:32:46.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.487+0000 7f4f0e000640 10 client.4812 ms_handle_connect on v2:172.21.15.151:6833/1533322173 2024-04-23T14:32:46.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.487+0000 7f4f0e000640 1 -- 172.21.15.100:0/3537660398 <== mds.0 v2:172.21.15.151:6833/1533322173 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564645893bc0 con 0x564645ab1570 2024-04-23T14:32:46.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.487+0000 7f4f0e000640 10 client.4812 handle_command_reply: tid=0 2024-04-23T14:32:46.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.487+0000 7f4f2eb1f000 1 client.4812 shutdown 2024-04-23T14:32:46.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.487+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/3537660398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645aaa140 msgr2=0x564645ab0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:46.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.487+0000 7f4f2eb1f000 1 --2- 172.21.15.100:0/3537660398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645aaa140 0x564645ab0250 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4f14025040 tx=0x7f4f14004a50 comp rx=0 tx=0).stop 2024-04-23T14:32:46.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.487+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/3537660398 shutdown_connections 2024-04-23T14:32:46.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.487+0000 7f4f2eb1f000 1 --2- 172.21.15.100:0/3537660398 >> [v2:172.21.15.151:6833/1533322173,v1:172.21.15.151:6835/1533322173] conn(0x564645ab1570 0x564645b07d30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.487+0000 7f4f2eb1f000 1 --2- 172.21.15.100:0/3537660398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564645af6ed0 0x564645ab0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.487+0000 7f4f2eb1f000 1 --2- 172.21.15.100:0/3537660398 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x564645af65b0 0x564645ab0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.487+0000 7f4f0e000640 0 client.4812 ms_handle_reset on v2:172.21.15.151:6833/1533322173 2024-04-23T14:32:46.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.487+0000 7f4f2eb1f000 1 --2- 172.21.15.100:0/3537660398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645aaa140 0x564645ab0250 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.487+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/3537660398 >> 172.21.15.100:0/3537660398 conn(0x564645ad7350 msgr2=0x564645ad9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:46.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.487+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/3537660398 shutdown_connections 2024-04-23T14:32:46.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.487+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/3537660398 wait complete. 2024-04-23T14:32:46.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.487+0000 7f4f2eb1f000 20 client.4812 trim_cache size 0 max 16384 2024-04-23T14:32:46.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.503+0000 7f4f2eb1f000 1 Processor -- start 2024-04-23T14:32:46.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.503+0000 7f4f2eb1f000 1 -- start start 2024-04-23T14:32:46.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645aaa140 0x564645aaf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564645af65b0 0x564645aaf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x564645af6ed0 0x564645af4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4f242019f0 con 0x564645aaa140 2024-04-23T14:32:46.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4f241598f0 con 0x564645af65b0 2024-04-23T14:32:46.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4f242013a0 con 0x564645af6ed0 2024-04-23T14:32:46.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f28c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645aaa140 0x564645aaf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f28c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645aaa140 0x564645aaf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53248/0 (socket says 172.21.15.100:53248) 2024-04-23T14:32:46.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f28c00640 1 -- 172.21.15.100:0/3394277262 learned_addr learned my addr 172.21.15.100:0/3394277262 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:46.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f29600640 1 --2- 172.21.15.100:0/3394277262 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x564645af6ed0 0x564645af4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f23e00640 1 --2- 172.21.15.100:0/3394277262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564645af65b0 0x564645aaf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f28c00640 1 -- 172.21.15.100:0/3394277262 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x564645af6ed0 msgr2=0x564645af4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:46.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f28c00640 1 --2- 172.21.15.100:0/3394277262 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x564645af6ed0 0x564645af4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f28c00640 1 -- 172.21.15.100:0/3394277262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564645af65b0 msgr2=0x564645aaf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:46.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f28c00640 1 --2- 172.21.15.100:0/3394277262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564645af65b0 0x564645aaf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f28c00640 1 -- 172.21.15.100:0/3394277262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564645953340 con 0x564645aaa140 2024-04-23T14:32:46.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f29600640 1 --2- 172.21.15.100:0/3394277262 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x564645af6ed0 0x564645af4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:46.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f23e00640 1 --2- 172.21.15.100:0/3394277262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564645af65b0 0x564645aaf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:46.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f28c00640 1 --2- 172.21.15.100:0/3394277262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645aaa140 0x564645aaf0e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4f14007f10 tx=0x7f4f14021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bbe5f4c2fd944a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:46.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f18c00640 1 -- 172.21.15.100:0/3394277262 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f14036930 con 0x564645aaa140 2024-04-23T14:32:46.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f18c00640 1 -- 172.21.15.100:0/3394277262 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f14044070 con 0x564645aaa140 2024-04-23T14:32:46.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f18c00640 1 -- 172.21.15.100:0/3394277262 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f140238f0 con 0x564645aaa140 2024-04-23T14:32:46.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/3394277262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645aaa140 msgr2=0x564645aaf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:46.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 --2- 172.21.15.100:0/3394277262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645aaa140 0x564645aaf0e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4f14007f10 tx=0x7f4f14021000 comp rx=0 tx=0).stop 2024-04-23T14:32:46.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/3394277262 shutdown_connections 2024-04-23T14:32:46.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 --2- 172.21.15.100:0/3394277262 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x564645af6ed0 0x564645af4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 --2- 172.21.15.100:0/3394277262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564645af65b0 0x564645aaf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 --2- 172.21.15.100:0/3394277262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645aaa140 0x564645aaf0e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/3394277262 >> 172.21.15.100:0/3394277262 conn(0x564645ad7350 msgr2=0x564645aae9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:46.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/3394277262 shutdown_connections 2024-04-23T14:32:46.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/3394277262 wait complete. 2024-04-23T14:32:46.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 -- start start 2024-04-23T14:32:46.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 Processor -- start 2024-04-23T14:32:46.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564645aaa140 0x564645b08d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x564645af65b0 0x564645b092c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645af6ed0 0x564645b09800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5646458feb00 con 0x564645af6ed0 2024-04-23T14:32:46.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5646458f97e0 con 0x564645aaa140 2024-04-23T14:32:46.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x5646459076e0 con 0x564645af65b0 2024-04-23T14:32:46.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f2eb1f000 10 client.0 _ll_register_callbacks cb 0x7f4f140186f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:32:46.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f29600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645af6ed0 0x564645b09800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f29600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645af6ed0 0x564645b09800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53252/0 (socket says 172.21.15.100:53252) 2024-04-23T14:32:46.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f29600640 1 -- 172.21.15.100:0/1276272523 learned_addr learned my addr 172.21.15.100:0/1276272523 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:46.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f28c00640 1 --2- 172.21.15.100:0/1276272523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564645aaa140 0x564645b08d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.507+0000 7f4f23e00640 1 --2- 172.21.15.100:0/1276272523 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x564645af65b0 0x564645b092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f28c00640 1 -- 172.21.15.100:0/1276272523 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x564645af65b0 msgr2=0x564645b092c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:46.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f28c00640 1 --2- 172.21.15.100:0/1276272523 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x564645af65b0 0x564645b092c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f28c00640 1 -- 172.21.15.100:0/1276272523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645af6ed0 msgr2=0x564645b09800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:46.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f28c00640 1 --2- 172.21.15.100:0/1276272523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645af6ed0 0x564645b09800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f28c00640 1 -- 172.21.15.100:0/1276272523 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56464588c7c0 con 0x564645aaa140 2024-04-23T14:32:46.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f29600640 1 --2- 172.21.15.100:0/1276272523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645af6ed0 0x564645b09800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:46.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f28c00640 1 --2- 172.21.15.100:0/1276272523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564645aaa140 0x564645b08d80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4f14046000 tx=0x7f4f14004ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64973a8c76ae818a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:46.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f0e000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-23T14:32:46.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f0e000640 1 -- 172.21.15.100:0/1276272523 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f14023850 con 0x564645aaa140 2024-04-23T14:32:46.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/1276272523 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f14044070 con 0x564645aaa140 2024-04-23T14:32:46.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f0e000640 1 -- 172.21.15.100:0/1276272523 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4f140239f0 con 0x564645aaa140 2024-04-23T14:32:46.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f0e000640 1 -- 172.21.15.100:0/1276272523 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f4f14056050 con 0x564645aaa140 2024-04-23T14:32:46.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2eb1f000 10 client.4768 fetch_fsmap learned FSMap version 9 2024-04-23T14:32:46.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/1276272523 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x56464589f650 con 0x564645aaa140 2024-04-23T14:32:46.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f0e000640 1 -- 172.21.15.100:0/1276272523 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f14044070 con 0x564645aaa140 2024-04-23T14:32:46.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f0e000640 1 -- 172.21.15.100:0/1276272523 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f4f14059020 con 0x564645aaa140 2024-04-23T14:32:46.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2eb1f000 10 client.4768 fetch_fsmap finished waiting for FSMap version 9 2024-04-23T14:32:46.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2eb1f000 10 client.4768 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-23T14:32:46.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2eb1f000 20 client.4768 populate_metadata read hostname 'smithi100' 2024-04-23T14:32:46.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2eb1f000 1 --2- 172.21.15.100:0/1276272523 >> [v2:172.21.15.151:6833/1533322173,v1:172.21.15.151:6835/1533322173] conn(0x564645b09d40 0x564645b0a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2eb1f000 4 client.4768 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:32:46.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/1276272523 --> [v2:172.21.15.151:6833/1533322173,v1:172.21.15.151:6835/1533322173] -- command(tid 0: {"prefix": "status"}) -- 0x5646458cf4e0 con 0x564645b09d40 2024-04-23T14:32:46.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f23e00640 1 --2- 172.21.15.100:0/1276272523 >> [v2:172.21.15.151:6833/1533322173,v1:172.21.15.151:6835/1533322173] conn(0x564645b09d40 0x564645b0a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f23e00640 1 --2- 172.21.15.100:0/1276272523 >> [v2:172.21.15.151:6833/1533322173,v1:172.21.15.151:6835/1533322173] conn(0x564645b09d40 0x564645b0a120 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c2a2b995880d779 server_cookie=8d270b684821e064 in_seq=0 out_seq=0 2024-04-23T14:32:46.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f0e000640 10 client.4768 ms_handle_connect on v2:172.21.15.151:6833/1533322173 2024-04-23T14:32:46.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f0e000640 1 -- 172.21.15.100:0/1276272523 <== mds.0 v2:172.21.15.151:6833/1533322173 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x5646458cf4e0 con 0x564645b09d40 2024-04-23T14:32:46.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f0e000640 10 client.4768 handle_command_reply: tid=0 2024-04-23T14:32:46.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2eb1f000 1 client.4768 shutdown 2024-04-23T14:32:46.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/1276272523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564645aaa140 msgr2=0x564645b08d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:46.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2eb1f000 1 --2- 172.21.15.100:0/1276272523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564645aaa140 0x564645b08d80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4f14046000 tx=0x7f4f14004ac0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/1276272523 shutdown_connections 2024-04-23T14:32:46.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2eb1f000 1 --2- 172.21.15.100:0/1276272523 >> [v2:172.21.15.151:6833/1533322173,v1:172.21.15.151:6835/1533322173] conn(0x564645b09d40 0x564645b0a120 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2eb1f000 1 --2- 172.21.15.100:0/1276272523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564645af6ed0 0x564645b09800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2eb1f000 1 --2- 172.21.15.100:0/1276272523 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x564645af65b0 0x564645b092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2eb1f000 1 --2- 172.21.15.100:0/1276272523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564645aaa140 0x564645b08d80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f0e000640 0 client.4768 ms_handle_reset on v2:172.21.15.151:6833/1533322173 2024-04-23T14:32:46.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/1276272523 >> 172.21.15.100:0/1276272523 conn(0x564645ad7350 msgr2=0x564645aaf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:46.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/1276272523 shutdown_connections 2024-04-23T14:32:46.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2eb1f000 1 -- 172.21.15.100:0/1276272523 wait complete. 2024-04-23T14:32:46.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2eb1f000 20 client.4768 trim_cache size 0 max 16384 2024-04-23T14:32:46.519 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:32:46.519 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "167d2af1-01c4-4f87-8bce-0de22791f823", 2024-04-23T14:32:46.519 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-04-23T14:32:46.520 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 4279, 2024-04-23T14:32:46.520 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-04-23T14:32:46.520 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-04-23T14:32:46.520 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-04-23T14:32:46.520 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 36.910395452000003, 2024-04-23T14:32:46.520 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 9, 2024-04-23T14:32:46.520 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 24, 2024-04-23T14:32:46.520 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 24, 2024-04-23T14:32:46.520 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 44.295605244999997 2024-04-23T14:32:46.520 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:32:46.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2b600640 1 -- 172.21.15.100:0/2241956094 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4f040683b0 msgr2=0x7f4f0406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:46.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2b600640 1 --2- 172.21.15.100:0/2241956094 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4f040683b0 0x7f4f0406a870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4f14007520 tx=0x7f4f1400b040 comp rx=0 tx=0).stop 2024-04-23T14:32:46.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2b600640 1 -- 172.21.15.100:0/2241956094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f2414b020 msgr2=0x7f4f240cfdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:46.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.511+0000 7f4f2b600640 1 --2- 172.21.15.100:0/2241956094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f2414b020 0x7f4f240cfdd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4f10007040 tx=0x7f4f08005840 comp rx=0 tx=0).stop 2024-04-23T14:32:46.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.515+0000 7f4f2b600640 1 -- 172.21.15.100:0/2241956094 shutdown_connections 2024-04-23T14:32:46.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.515+0000 7f4f2b600640 1 --2- 172.21.15.100:0/2241956094 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4f040683b0 0x7f4f0406a870 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.515+0000 7f4f2b600640 1 --2- 172.21.15.100:0/2241956094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f240d0310 0x7f4f240cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.515+0000 7f4f2b600640 1 --2- 172.21.15.100:0/2241956094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f2414b020 0x7f4f240cfdd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.515+0000 7f4f2b600640 1 --2- 172.21.15.100:0/2241956094 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f08004810 0x7f4f240cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.515+0000 7f4f2b600640 1 -- 172.21.15.100:0/2241956094 >> 172.21.15.100:0/2241956094 conn(0x7f4f240b2600 msgr2=0x7f4f240b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:46.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.515+0000 7f4f2b600640 1 -- 172.21.15.100:0/2241956094 shutdown_connections 2024-04-23T14:32:46.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:46.515+0000 7f4f2b600640 1 -- 172.21.15.100:0/2241956094 wait complete. 2024-04-23T14:32:46.532 INFO:teuthology.run_tasks:Running task sequential... 2024-04-23T14:32:46.541 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-23T14:32:46.542 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-23T14:32:46.542 INFO:teuthology.task.print:Enabling mgr modules 2024-04-23T14:32:46.542 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-23T14:32:46.590 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-23T14:32:46.591 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-23T14:32:46.591 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-23T14:32:46.591 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-23T14:32:46.591 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:32:46.591 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-23T14:32:46.591 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-23T14:32:46.591 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-23T14:32:46.591 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-23T14:32:46.591 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-23T14:32:46.591 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-23T14:32:46.591 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-23T14:32:46.592 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-23T14:32:46.592 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-23T14:32:46.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:46.592 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-23T14:32:46.601 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-23T14:32:46.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:46.601 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-23T14:32:46.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:46.667 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-04-23T14:32:46.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:46.837 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-04-23T14:32:46.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:46.847 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link delete ceph-brx 2024-04-23T14:32:46.912 INFO:teuthology.orchestra.run.smithi151.stderr:Cannot find device "ceph-brx" 2024-04-23T14:32:46.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:32:46.915 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-23T14:32:46.915 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:32:47.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.047+0000 7f7c62600640 1 Processor -- start 2024-04-23T14:32:47.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.047+0000 7f7c62600640 1 -- start start 2024-04-23T14:32:47.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c62600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7c5c073610 0x7f7c5c0adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:47.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c62600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c5c0af080 con 0x7f7c5c15b640 2024-04-23T14:32:47.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c62600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c5c0afbc0 con 0x7f7c5c0b2a10 2024-04-23T14:32:47.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c62600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7c5c15ba20 con 0x7f7c5c073610 2024-04-23T14:32:47.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5b400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f7c5c15b640 0x7f7c5c072f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:45042/0 (socket says 172.21.15.100:45042) 2024-04-23T14:32:47.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5b400640 1 -- 172.21.15.100:0/1019722062 learned_addr learned my addr 172.21.15.100:0/1019722062 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:47.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5be00640 1 --2- 172.21.15.100:0/1019722062 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7c5c073610 0x7f7c5c0adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:47.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5a000640 1 -- 172.21.15.100:0/1019722062 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3899049227 0 0) 0x7f7c5c0af080 con 0x7f7c5c15b640 2024-04-23T14:32:47.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5a000640 1 -- 172.21.15.100:0/1019722062 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c40003530 con 0x7f7c5c15b640 2024-04-23T14:32:47.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5a000640 1 -- 172.21.15.100:0/1019722062 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1568551856 0 0) 0x7f7c40003530 con 0x7f7c5c15b640 2024-04-23T14:32:47.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5a000640 1 -- 172.21.15.100:0/1019722062 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7c5c073610 msgr2=0x7f7c5c0adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:47.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5a000640 1 --2- 172.21.15.100:0/1019722062 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7c5c073610 0x7f7c5c0adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:47.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5a000640 1 -- 172.21.15.100:0/1019722062 >> v1:172.21.15.151:6789/0 conn(0x7f7c5c0b2a10 legacy=0x7f7c5c06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:47.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5a000640 1 -- 172.21.15.100:0/1019722062 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5c0af320 con 0x7f7c5c15b640 2024-04-23T14:32:47.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5a000640 1 -- 172.21.15.100:0/1019722062 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f7c4c001680 con 0x7f7c5c15b640 2024-04-23T14:32:47.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5a000640 1 -- 172.21.15.100:0/1019722062 >> v1:172.21.15.100:6789/0 conn(0x7f7c5c15b640 legacy=0x7f7c5c072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:47.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5a000640 1 --2- 172.21.15.100:0/1019722062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c400039f0 0x7f7c40003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:47.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5a000640 1 --2- 172.21.15.100:0/1019722062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c400043a0 0x7f7c40004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:47.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5a000640 1 --2- 172.21.15.100:0/1019722062 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7c5c073610 0x7f7c400050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:47.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5a000640 1 -- 172.21.15.100:0/1019722062 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c15ba20 con 0x7f7c400043a0 2024-04-23T14:32:47.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5a000640 1 -- 172.21.15.100:0/1019722062 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4000ba00 con 0x7f7c400039f0 2024-04-23T14:32:47.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5a000640 1 -- 172.21.15.100:0/1019722062 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7c4000bc00 con 0x7f7c5c073610 2024-04-23T14:32:47.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5aa00640 1 --2- 172.21.15.100:0/1019722062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c400043a0 0x7f7c40004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:47.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5be00640 1 --2- 172.21.15.100:0/1019722062 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7c5c073610 0x7f7c400050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:47.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5b400640 1 --2- 172.21.15.100:0/1019722062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c400039f0 0x7f7c40003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:47.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5aa00640 1 -- 172.21.15.100:0/1019722062 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7c5c073610 msgr2=0x7f7c400050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:47.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5aa00640 1 --2- 172.21.15.100:0/1019722062 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7c5c073610 0x7f7c400050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:47.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5aa00640 1 -- 172.21.15.100:0/1019722062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c400039f0 msgr2=0x7f7c40003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:47.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5aa00640 1 --2- 172.21.15.100:0/1019722062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c400039f0 0x7f7c40003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:47.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.051+0000 7f7c5aa00640 1 -- 172.21.15.100:0/1019722062 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c40003530 con 0x7f7c400043a0 2024-04-23T14:32:47.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.055+0000 7f7c5aa00640 1 --2- 172.21.15.100:0/1019722062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c400043a0 0x7f7c40004870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c075380 tx=0x7f7c480212c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=815518406e0412b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:47.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.055+0000 7f7c5a000640 1 -- 172.21.15.100:0/1019722062 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c480b6070 con 0x7f7c400043a0 2024-04-23T14:32:47.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.055+0000 7f7c5a000640 1 -- 172.21.15.100:0/1019722062 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c480b1620 con 0x7f7c400043a0 2024-04-23T14:32:47.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.055+0000 7f7c5a000640 1 -- 172.21.15.100:0/1019722062 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c480014e0 con 0x7f7c400043a0 2024-04-23T14:32:47.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.055+0000 7f7c62600640 1 -- 172.21.15.100:0/1019722062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c400043a0 msgr2=0x7f7c40004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:47.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.055+0000 7f7c62600640 1 --2- 172.21.15.100:0/1019722062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c400043a0 0x7f7c40004870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c075380 tx=0x7f7c480212c0 comp rx=0 tx=0).stop 2024-04-23T14:32:47.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.055+0000 7f7c5be00640 1 -- 172.21.15.100:0/1019722062 reap_dead start 2024-04-23T14:32:47.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.055+0000 7f7c62600640 1 -- 172.21.15.100:0/1019722062 shutdown_connections 2024-04-23T14:32:47.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.055+0000 7f7c62600640 1 -- 172.21.15.100:0/1019722062 >> 172.21.15.100:0/1019722062 conn(0x7f7c5c0b1e80 msgr2=0x7f7c5c0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:47.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.055+0000 7f7c62600640 1 -- 172.21.15.100:0/1019722062 shutdown_connections 2024-04-23T14:32:47.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.055+0000 7f7c62600640 1 -- 172.21.15.100:0/1019722062 wait complete. 2024-04-23T14:32:47.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c62600640 1 Processor -- start 2024-04-23T14:32:47.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c62600640 1 -- start start 2024-04-23T14:32:47.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c62600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c400043a0 0x7f7c5c1f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:47.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c62600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c5c14cf60 0x7f7c5c14c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:47.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c62600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7c5c14f450 0x7f7c5c14f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:47.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c62600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c0cc750 con 0x7f7c400043a0 2024-04-23T14:32:47.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c62600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c0af8e0 con 0x7f7c5c14cf60 2024-04-23T14:32:47.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c62600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7c5c0cc8d0 con 0x7f7c5c14f450 2024-04-23T14:32:47.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c5b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c400043a0 0x7f7c5c1f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:47.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c5b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c400043a0 0x7f7c5c1f3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53264/0 (socket says 172.21.15.100:53264) 2024-04-23T14:32:47.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c5b400640 1 -- 172.21.15.100:0/3116617964 learned_addr learned my addr 172.21.15.100:0/3116617964 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:47.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c5aa00640 1 --2- 172.21.15.100:0/3116617964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c5c14cf60 0x7f7c5c14c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:47.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c5be00640 1 --2- 172.21.15.100:0/3116617964 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7c5c14f450 0x7f7c5c14f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:47.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c5b400640 1 -- 172.21.15.100:0/3116617964 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7c5c14f450 msgr2=0x7f7c5c14f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:47.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c5b400640 1 --2- 172.21.15.100:0/3116617964 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7c5c14f450 0x7f7c5c14f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:47.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c5b400640 1 -- 172.21.15.100:0/3116617964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c5c14cf60 msgr2=0x7f7c5c14c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:47.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c5b400640 1 --2- 172.21.15.100:0/3116617964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c5c14cf60 0x7f7c5c14c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:47.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c5b400640 1 -- 172.21.15.100:0/3116617964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5c14d6d0 con 0x7f7c400043a0 2024-04-23T14:32:47.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c5b400640 1 --2- 172.21.15.100:0/3116617964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c400043a0 0x7f7c5c1f3c70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0072f0 tx=0x7f7c4c0077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:47.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c53400640 1 -- 172.21.15.100:0/3116617964 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c01a070 con 0x7f7c400043a0 2024-04-23T14:32:47.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c53400640 1 -- 172.21.15.100:0/3116617964 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c4c00f9c0 con 0x7f7c400043a0 2024-04-23T14:32:47.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c62600640 1 -- 172.21.15.100:0/3116617964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5c14d9b0 con 0x7f7c400043a0 2024-04-23T14:32:47.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c53400640 1 -- 172.21.15.100:0/3116617964 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c00fdb0 con 0x7f7c400043a0 2024-04-23T14:32:47.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c62600640 1 -- 172.21.15.100:0/3116617964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c5c14df70 con 0x7f7c400043a0 2024-04-23T14:32:47.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.059+0000 7f7c62600640 1 -- 172.21.15.100:0/3116617964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c5c068ad0 con 0x7f7c400043a0 2024-04-23T14:32:47.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.063+0000 7f7c53400640 1 -- 172.21.15.100:0/3116617964 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7c4c01d070 con 0x7f7c400043a0 2024-04-23T14:32:47.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.063+0000 7f7c53400640 1 --2- 172.21.15.100:0/3116617964 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7c3c068380 0x7f7c3c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:47.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.063+0000 7f7c53400640 1 -- 172.21.15.100:0/3116617964 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7c4c0866e0 con 0x7f7c400043a0 2024-04-23T14:32:47.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.063+0000 7f7c5aa00640 1 --2- 172.21.15.100:0/3116617964 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7c3c068380 0x7f7c3c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:47.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.063+0000 7f7c5aa00640 1 --2- 172.21.15.100:0/3116617964 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7c3c068380 0x7f7c3c06a840 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7c480b3000 tx=0x7f7c48005040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:47.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.067+0000 7f7c53400640 1 -- 172.21.15.100:0/3116617964 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7c4c056270 con 0x7f7c400043a0 2024-04-23T14:32:47.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.255+0000 7f7c62600640 1 -- 172.21.15.100:0/3116617964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7c5c14cd00 con 0x7f7c400043a0 2024-04-23T14:32:47.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.255+0000 7f7c53400640 1 -- 172.21.15.100:0/3116617964 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f7c4c05a570 con 0x7f7c400043a0 2024-04-23T14:32:47.258 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-23T14:32:47.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.259+0000 7f7c62600640 1 -- 172.21.15.100:0/3116617964 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7c3c068380 msgr2=0x7f7c3c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:47.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.259+0000 7f7c62600640 1 --2- 172.21.15.100:0/3116617964 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7c3c068380 0x7f7c3c06a840 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7c480b3000 tx=0x7f7c48005040 comp rx=0 tx=0).stop 2024-04-23T14:32:47.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.259+0000 7f7c62600640 1 -- 172.21.15.100:0/3116617964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c400043a0 msgr2=0x7f7c5c1f3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:47.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.259+0000 7f7c62600640 1 --2- 172.21.15.100:0/3116617964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c400043a0 0x7f7c5c1f3c70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0072f0 tx=0x7f7c4c0077b0 comp rx=0 tx=0).stop 2024-04-23T14:32:47.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.259+0000 7f7c62600640 1 -- 172.21.15.100:0/3116617964 shutdown_connections 2024-04-23T14:32:47.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.259+0000 7f7c62600640 1 --2- 172.21.15.100:0/3116617964 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7c3c068380 0x7f7c3c06a840 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:47.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.259+0000 7f7c62600640 1 --2- 172.21.15.100:0/3116617964 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7c5c14f450 0x7f7c5c14f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:47.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.259+0000 7f7c62600640 1 --2- 172.21.15.100:0/3116617964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c5c14cf60 0x7f7c5c14c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:47.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.259+0000 7f7c62600640 1 --2- 172.21.15.100:0/3116617964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c400043a0 0x7f7c5c1f3c70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:47.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.259+0000 7f7c62600640 1 -- 172.21.15.100:0/3116617964 >> 172.21.15.100:0/3116617964 conn(0x7f7c5c0b1e80 msgr2=0x7f7c5c0adfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:47.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.259+0000 7f7c62600640 1 -- 172.21.15.100:0/3116617964 shutdown_connections 2024-04-23T14:32:47.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47.259+0000 7f7c62600640 1 -- 172.21.15.100:0/3116617964 wait complete. 2024-04-23T14:32:47.278 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:32:47.278 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:32:47.278 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-04-23T14:32:47.279 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:32:47.279 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:32:47.279 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:32:47.279 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:32:47.279 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:32:47.279 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:32:47.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:47.279 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-04-23T14:32:47.288 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:32:47.288 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:32:47.288 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:32:47.288 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:47.288 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-23T14:32:47.288 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:47.288 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:32:47.288 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:47.288 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-04-23T14:32:47.289 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:32:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-04-23T14:32:47.289 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:32:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:47.289 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:32:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-23T14:32:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42416sec preferred_lft 42416sec 2024-04-23T14:32:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-04-23T14:32:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:47.289 INFO:teuthology.orchestra.run.smithi100.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-23T14:32:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 02:42:ff:fc:d1:cc brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-23T14:32:47.290 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:47.290 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-23T14:32:47.291 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:47.291 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-23T14:32:47.291 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add name ceph-brx type bridge 2024-04-23T14:32:47.291 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr flush dev ceph-brx 2024-04-23T14:32:47.291 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set ceph-brx up 2024-04-23T14:32:47.291 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-23T14:32:47.291 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-23T14:32:47.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:47.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:47.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:47.447 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:32:47.466 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-04-23T14:32:47.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:47.468 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-04-23T14:32:47.520 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-04-23T14:32:47.521 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:32:47.521 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-23T14:32:47.521 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-23T14:32:47.521 INFO:teuthology.orchestra.run.smithi100.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-23T14:32:47.521 INFO:teuthology.orchestra.run.smithi100.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-23T14:32:47.521 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-23T14:32:47.521 INFO:teuthology.orchestra.run.smithi100.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:32:47.521 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:32:47.521 INFO:teuthology.orchestra.run.smithi100.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:32:47.521 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:32:47.522 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:47.522 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-23T14:32:47.522 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-23T14:32:47.522 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-23T14:32:47.522 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-23T14:32:47.522 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-23T14:32:47.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:47.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:47.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:47.653 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-23T14:32:47.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:47.663 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2024-04-23T14:32:47.719 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:47.719 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-23T14:32:47.719 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:32:47.719 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-23T14:32:47.719 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-23T14:32:47.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:47.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:47.847 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-23T14:32:47.848 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:47.848 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-23T14:32:47.848 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-23T14:32:47.848 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-23T14:32:47.848 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-23T14:32:47.848 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-23T14:32:47.848 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-23T14:32:47.848 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-23T14:32:47.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:47.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:47.987 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:47.987 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-23T14:32:47.987 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-04-23T14:32:47.987 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.0 master ceph-brx 2024-04-23T14:32:47.987 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-23T14:32:48.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:48.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:48.111 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:32:48.111 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:32:48.111 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:32:48.120 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:32:48.120 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:32:48.121 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:32:48.173 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-23T14:32:48.233 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:32:48.289 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-23T14:32:48.289 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-23T14:32:48.289 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-23T14:32:48.289 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-23T14:32:48.289 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-23T14:32:48.289 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-23T14:32:48.289 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-23T14:32:48.289 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-23T14:32:48.289 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-23T14:32:48.289 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-23T14:32:48.289 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-23T14:32:48.290 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-23T14:32:48.290 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:32:48.290 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-23T14:32:48.290 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-23T14:32:48.290 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-23T14:32:48.290 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-23T14:32:48.290 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-23T14:32:48.290 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-23T14:32:48.290 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-23T14:32:48.290 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:32:48.290 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-23T14:32:48.290 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-23T14:32:48.291 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-23T14:32:48.291 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:32:48.291 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:32:48.291 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:32:48.291 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:32:48.291 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-23T14:32:48.291 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:32:48.291 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:32:48.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:48.291 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-23T14:32:48.337 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:32:48.337 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:32:48.380 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-23T14:32:48.400 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:32:48.451 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-23T14:32:48.451 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-23T14:32:48.451 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-23T14:32:48.451 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-23T14:32:48.451 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-23T14:32:48.451 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-23T14:32:48.452 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-23T14:32:48.452 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-23T14:32:48.452 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-23T14:32:48.452 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-23T14:32:48.452 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-23T14:32:48.452 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-23T14:32:48.452 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:32:48.452 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-23T14:32:48.452 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-23T14:32:48.452 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-23T14:32:48.452 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-23T14:32:48.452 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-23T14:32:48.452 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-23T14:32:48.452 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-23T14:32:48.452 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:32:48.453 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-23T14:32:48.453 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-23T14:32:48.453 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-23T14:32:48.453 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:32:48.453 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:32:48.453 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:32:48.453 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:32:48.453 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-23T14:32:48.453 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:32:48.453 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:32:48.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:48.454 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-23T14:32:48.473 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:2024-04-23T14:32:48.471+0000 7f9b994dc580 -1 init, newargv = 0x560a79a71be0 newargc=15 2024-04-23T14:32:48.473 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[35885]: starting ceph client 2024-04-23T14:32:48.481 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[35885]: starting fuse 2024-04-23T14:32:48.482 INFO:teuthology.orchestra.run.smithi100.stdout:44 2024-04-23T14:32:48.482 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-23T14:32:48.483 DEBUG:teuthology.orchestra.run.smithi100:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:32:48.483 DEBUG:teuthology.orchestra.run.smithi100:> import glob 2024-04-23T14:32:48.483 DEBUG:teuthology.orchestra.run.smithi100:> import re 2024-04-23T14:32:48.483 DEBUG:teuthology.orchestra.run.smithi100:> import os 2024-04-23T14:32:48.483 DEBUG:teuthology.orchestra.run.smithi100:> import subprocess 2024-04-23T14:32:48.483 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:32:48.483 DEBUG:teuthology.orchestra.run.smithi100:> def _find_admin_socket(client_name): 2024-04-23T14:32:48.483 DEBUG:teuthology.orchestra.run.smithi100:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:32:48.483 DEBUG:teuthology.orchestra.run.smithi100:> files = glob.glob(asok_path) 2024-04-23T14:32:48.483 DEBUG:teuthology.orchestra.run.smithi100:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:32:48.483 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:32:48.483 DEBUG:teuthology.orchestra.run.smithi100:> # Given a non-glob path, it better be there 2024-04-23T14:32:48.483 DEBUG:teuthology.orchestra.run.smithi100:> if "*" not in asok_path: 2024-04-23T14:32:48.483 DEBUG:teuthology.orchestra.run.smithi100:> assert(len(files) == 1) 2024-04-23T14:32:48.483 DEBUG:teuthology.orchestra.run.smithi100:> return files[0] 2024-04-23T14:32:48.484 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:32:48.484 DEBUG:teuthology.orchestra.run.smithi100:> for f in files: 2024-04-23T14:32:48.484 DEBUG:teuthology.orchestra.run.smithi100:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:32:48.484 DEBUG:teuthology.orchestra.run.smithi100:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:32:48.484 DEBUG:teuthology.orchestra.run.smithi100:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:32:48.484 DEBUG:teuthology.orchestra.run.smithi100:> contents = proc_f.read() 2024-04-23T14:32:48.484 DEBUG:teuthology.orchestra.run.smithi100:> if mountpoint in contents: 2024-04-23T14:32:48.484 DEBUG:teuthology.orchestra.run.smithi100:> return f 2024-04-23T14:32:48.484 DEBUG:teuthology.orchestra.run.smithi100:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:32:48.484 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:32:48.484 DEBUG:teuthology.orchestra.run.smithi100:> print(_find_admin_socket("client.0")) 2024-04-23T14:32:48.484 DEBUG:teuthology.orchestra.run.smithi100:> ' 2024-04-23T14:32:48.609 INFO:teuthology.orchestra.run.smithi100.stdout:/var/run/ceph/ceph-client.0.35885.asok 2024-04-23T14:32:48.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:48.621 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.35885.asok 2024-04-23T14:32:48.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:48.621 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.35885.asok status 2024-04-23T14:32:48.803 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:32:48.803 INFO:teuthology.orchestra.run.smithi100.stdout: "metadata": { 2024-04-23T14:32:48.803 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:32:48.803 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:32:48.803 INFO:teuthology.orchestra.run.smithi100.stdout: "entity_id": "0", 2024-04-23T14:32:48.803 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": "smithi100", 2024-04-23T14:32:48.803 INFO:teuthology.orchestra.run.smithi100.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:32:48.803 INFO:teuthology.orchestra.run.smithi100.stdout: "pid": "35885", 2024-04-23T14:32:48.803 INFO:teuthology.orchestra.run.smithi100.stdout: "root": "/" 2024-04-23T14:32:48.804 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:32:48.804 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_count": 0, 2024-04-23T14:32:48.804 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_pinned_count": 0, 2024-04-23T14:32:48.804 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 4833, 2024-04-23T14:32:48.804 INFO:teuthology.orchestra.run.smithi100.stdout: "inst": { 2024-04-23T14:32:48.804 INFO:teuthology.orchestra.run.smithi100.stdout: "name": { 2024-04-23T14:32:48.804 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "client", 2024-04-23T14:32:48.804 INFO:teuthology.orchestra.run.smithi100.stdout: "num": 4833 2024-04-23T14:32:48.804 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:32:48.804 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-23T14:32:48.804 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-23T14:32:48.804 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:32:48.804 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3571547789 2024-04-23T14:32:48.804 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:32:48.804 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:32:48.805 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-23T14:32:48.805 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-23T14:32:48.805 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:32:48.805 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3571547789 2024-04-23T14:32:48.805 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:32:48.805 INFO:teuthology.orchestra.run.smithi100.stdout: "inst_str": "client.4833 192.168.0.1:0/3571547789", 2024-04-23T14:32:48.805 INFO:teuthology.orchestra.run.smithi100.stdout: "addr_str": "192.168.0.1:0/3571547789", 2024-04-23T14:32:48.805 INFO:teuthology.orchestra.run.smithi100.stdout: "inode_count": 1, 2024-04-23T14:32:48.805 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_epoch": 9, 2024-04-23T14:32:48.805 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch": 24, 2024-04-23T14:32:48.805 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:32:48.805 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklisted": false, 2024-04-23T14:32:48.805 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs" 2024-04-23T14:32:48.805 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:32:48.818 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:32:48.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.951+0000 7f39ed600640 1 Processor -- start 2024-04-23T14:32:48.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.955+0000 7f39ed600640 1 -- start start 2024-04-23T14:32:48.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.955+0000 7f39ed600640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f39e815c7e0 0x7f39e80d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:48.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.955+0000 7f39ed600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39e80d04a0 con 0x7f39e815d600 2024-04-23T14:32:48.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.955+0000 7f39ed600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39e80d0780 con 0x7f39e80cca50 2024-04-23T14:32:48.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.955+0000 7f39ed600640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f39e815db00 con 0x7f39e815c7e0 2024-04-23T14:32:48.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.955+0000 7f39e7400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f39e815d600 0x7f39e80d17a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42688/0 (socket says 172.21.15.100:42688) 2024-04-23T14:32:48.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.955+0000 7f39e7400640 1 -- 172.21.15.100:0/3057479399 learned_addr learned my addr 172.21.15.100:0/3057479399 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:48.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.955+0000 7f39e6000640 1 --2- 172.21.15.100:0/3057479399 >> v2:172.21.15.151:3300/0 conn(0x7f39e815c7e0 0x7f39e80d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:48.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.955+0000 7f39e5600640 1 -- 172.21.15.100:0/3057479399 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3778072433 0 0) 0x7f39e80d04a0 con 0x7f39e815d600 2024-04-23T14:32:48.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.955+0000 7f39e5600640 1 -- 172.21.15.100:0/3057479399 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39cc003530 con 0x7f39e815d600 2024-04-23T14:32:48.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.955+0000 7f39e6000640 1 -- 172.21.15.100:0/3057479399 >> v1:172.21.15.151:6789/0 conn(0x7f39e80cca50 legacy=0x7f39e80cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:48.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.955+0000 7f39e6000640 1 -- 172.21.15.100:0/3057479399 >> v1:172.21.15.100:6789/0 conn(0x7f39e815d600 legacy=0x7f39e80d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:48.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.955+0000 7f39e6000640 1 -- 172.21.15.100:0/3057479399 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e80d0a60 con 0x7f39e815c7e0 2024-04-23T14:32:48.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.959+0000 7f39e6000640 1 --2- 172.21.15.100:0/3057479399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39e815c7e0 0x7f39e80d1260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f39d40097c0 tx=0x7f39d401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b757e9707be0d546 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:48.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.959+0000 7f39e5600640 1 -- 172.21.15.100:0/3057479399 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d40ab600 con 0x7f39e815c7e0 2024-04-23T14:32:48.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.959+0000 7f39e5600640 1 -- 172.21.15.100:0/3057479399 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39d40b0070 con 0x7f39e815c7e0 2024-04-23T14:32:48.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.959+0000 7f39e5600640 1 -- 172.21.15.100:0/3057479399 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d40ab9a0 con 0x7f39e815c7e0 2024-04-23T14:32:48.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.959+0000 7f39ed600640 1 -- 172.21.15.100:0/3057479399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39e815c7e0 msgr2=0x7f39e80d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:48.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.959+0000 7f39ed600640 1 --2- 172.21.15.100:0/3057479399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39e815c7e0 0x7f39e80d1260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f39d40097c0 tx=0x7f39d401c3e0 comp rx=0 tx=0).stop 2024-04-23T14:32:48.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.959+0000 7f39ed600640 1 -- 172.21.15.100:0/3057479399 shutdown_connections 2024-04-23T14:32:48.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.959+0000 7f39ed600640 1 --2- 172.21.15.100:0/3057479399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39e815c7e0 0x7f39e80d1260 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:48.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.959+0000 7f39ed600640 1 -- 172.21.15.100:0/3057479399 >> 172.21.15.100:0/3057479399 conn(0x7f39e80cee80 msgr2=0x7f39e80cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:48.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.959+0000 7f39ed600640 1 -- 172.21.15.100:0/3057479399 shutdown_connections 2024-04-23T14:32:48.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.959+0000 7f39ed600640 1 -- 172.21.15.100:0/3057479399 wait complete. 2024-04-23T14:32:48.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.959+0000 7f39ed600640 1 Processor -- start 2024-04-23T14:32:48.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.959+0000 7f39ed600640 1 -- start start 2024-04-23T14:32:48.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39ed600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f39e80cca50 0x7f39e8149180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:48.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39ed600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e815c7e0 0x7f39e806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:48.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39ed600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39e815d600 0x7f39e8157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:48.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39ed600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f39e816ac60 con 0x7f39e815c7e0 2024-04-23T14:32:48.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39ed600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f39e816aae0 con 0x7f39e815d600 2024-04-23T14:32:48.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39ed600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f39e816ade0 con 0x7f39e80cca50 2024-04-23T14:32:48.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39e6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e815c7e0 0x7f39e806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:48.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39e6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e815c7e0 0x7f39e806e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50826/0 (socket says 172.21.15.100:50826) 2024-04-23T14:32:48.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39e6000640 1 -- 172.21.15.100:0/1843273066 learned_addr learned my addr 172.21.15.100:0/1843273066 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:48.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39e6a00640 1 --2- 172.21.15.100:0/1843273066 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f39e80cca50 0x7f39e8149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:48.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39e7400640 1 --2- 172.21.15.100:0/1843273066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39e815d600 0x7f39e8157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:48.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39e6000640 1 -- 172.21.15.100:0/1843273066 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f39e80cca50 msgr2=0x7f39e8149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:48.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39e6000640 1 --2- 172.21.15.100:0/1843273066 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f39e80cca50 0x7f39e8149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:48.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39e6000640 1 -- 172.21.15.100:0/1843273066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39e815d600 msgr2=0x7f39e8157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:48.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39e6000640 1 --2- 172.21.15.100:0/1843273066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39e815d600 0x7f39e8157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:48.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39e6000640 1 -- 172.21.15.100:0/1843273066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e806d5f0 con 0x7f39e815c7e0 2024-04-23T14:32:48.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39e7400640 1 --2- 172.21.15.100:0/1843273066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39e815d600 0x7f39e8157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:48.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39e6000640 1 --2- 172.21.15.100:0/1843273066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e815c7e0 0x7f39e806e3b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f39d40097c0 tx=0x7f39d4019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:48.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39dea00640 1 -- 172.21.15.100:0/1843273066 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d40c1070 con 0x7f39e815c7e0 2024-04-23T14:32:48.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39dea00640 1 -- 172.21.15.100:0/1843273066 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39d4003820 con 0x7f39e815c7e0 2024-04-23T14:32:48.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39dea00640 1 -- 172.21.15.100:0/1843273066 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4003bd0 con 0x7f39e815c7e0 2024-04-23T14:32:48.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39ed600640 1 -- 172.21.15.100:0/1843273066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e806d8d0 con 0x7f39e815c7e0 2024-04-23T14:32:48.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39ed600640 1 -- 172.21.15.100:0/1843273066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e806de90 con 0x7f39e815c7e0 2024-04-23T14:32:48.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.963+0000 7f39ed600640 1 -- 172.21.15.100:0/1843273066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e8068ad0 con 0x7f39e815c7e0 2024-04-23T14:32:48.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.967+0000 7f39dea00640 1 -- 172.21.15.100:0/1843273066 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f39d40c4070 con 0x7f39e815c7e0 2024-04-23T14:32:48.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.967+0000 7f39dea00640 1 --2- 172.21.15.100:0/1843273066 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f39c8068380 0x7f39c806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:48.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.967+0000 7f39dea00640 1 -- 172.21.15.100:0/1843273066 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6735+0+0 (secure 0 0 0) 0x7f39d4120d80 con 0x7f39e815c7e0 2024-04-23T14:32:48.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.967+0000 7f39e6a00640 1 --2- 172.21.15.100:0/1843273066 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f39c8068380 0x7f39c806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:48.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.967+0000 7f39e6a00640 1 --2- 172.21.15.100:0/1843273066 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f39c8068380 0x7f39c806a840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f39e814c4b0 tx=0x7f39d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:48.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:48.971+0000 7f39dea00640 1 -- 172.21.15.100:0/1843273066 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f39d4016090 con 0x7f39e815c7e0 2024-04-23T14:32:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:49.159+0000 7f39ed600640 1 -- 172.21.15.100:0/1843273066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f39e80cd2b0 con 0x7f39e815c7e0 2024-04-23T14:32:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:49.159+0000 7f39dea00640 1 -- 172.21.15.100:0/1843273066 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f39d40f0850 con 0x7f39e815c7e0 2024-04-23T14:32:49.161 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-23T14:32:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:49.163+0000 7f39ed600640 1 -- 172.21.15.100:0/1843273066 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f39c8068380 msgr2=0x7f39c806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:49.163+0000 7f39ed600640 1 --2- 172.21.15.100:0/1843273066 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f39c8068380 0x7f39c806a840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f39e814c4b0 tx=0x7f39d8009040 comp rx=0 tx=0).stop 2024-04-23T14:32:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:49.163+0000 7f39ed600640 1 -- 172.21.15.100:0/1843273066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e815c7e0 msgr2=0x7f39e806e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:49.163+0000 7f39ed600640 1 --2- 172.21.15.100:0/1843273066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e815c7e0 0x7f39e806e3b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f39d40097c0 tx=0x7f39d4019180 comp rx=0 tx=0).stop 2024-04-23T14:32:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:49.163+0000 7f39ed600640 1 -- 172.21.15.100:0/1843273066 shutdown_connections 2024-04-23T14:32:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:49.163+0000 7f39ed600640 1 --2- 172.21.15.100:0/1843273066 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f39c8068380 0x7f39c806a840 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:49.163+0000 7f39ed600640 1 --2- 172.21.15.100:0/1843273066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39e815d600 0x7f39e8157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:49.163+0000 7f39ed600640 1 --2- 172.21.15.100:0/1843273066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e815c7e0 0x7f39e806e3b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:49.163+0000 7f39ed600640 1 --2- 172.21.15.100:0/1843273066 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f39e80cca50 0x7f39e8149180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:49.163+0000 7f39ed600640 1 -- 172.21.15.100:0/1843273066 >> 172.21.15.100:0/1843273066 conn(0x7f39e80cee80 msgr2=0x7f39e80b56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:49.163+0000 7f39ed600640 1 -- 172.21.15.100:0/1843273066 shutdown_connections 2024-04-23T14:32:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:49.163+0000 7f39ed600640 1 -- 172.21.15.100:0/1843273066 wait complete. 2024-04-23T14:32:49.184 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:32:49.184 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:32:49.184 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-04-23T14:32:49.184 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-23T14:32:49.184 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-23T14:32:49.184 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:32:49.184 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:32:49.184 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:32:49.184 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-23T14:32:49.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:49.185 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-04-23T14:32:49.193 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:32:49.193 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:32:49.193 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:32:49.193 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:49.193 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-23T14:32:49.194 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:49.194 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:32:49.194 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:49.194 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-04-23T14:32:49.194 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:32:49.194 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:49.194 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-04-23T14:32:49.194 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:32:49.194 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:49.194 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:32:49.194 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:49.194 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-23T14:32:49.194 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42414sec preferred_lft 42414sec 2024-04-23T14:32:49.195 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-04-23T14:32:49.195 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:49.195 INFO:teuthology.orchestra.run.smithi100.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-23T14:32:49.195 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 02:42:ff:fc:d1:cc brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:49.195 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-23T14:32:49.195 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:49.195 INFO:teuthology.orchestra.run.smithi100.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:32:49.195 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:49.195 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:32:49.195 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:49.195 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2024-04-23T14:32:49.195 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:49.195 INFO:teuthology.orchestra.run.smithi100.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:32:49.195 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:32:49.196 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2024-04-23T14:32:49.196 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:49.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:49.197 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:32:49.254 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-04-23T14:32:49.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:49.257 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-04-23T14:32:49.309 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-04-23T14:32:49.309 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:32:49.309 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-23T14:32:49.309 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-23T14:32:49.309 INFO:teuthology.orchestra.run.smithi100.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-23T14:32:49.309 INFO:teuthology.orchestra.run.smithi100.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-23T14:32:49.309 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-23T14:32:49.310 INFO:teuthology.orchestra.run.smithi100.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:32:49.310 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:32:49.310 INFO:teuthology.orchestra.run.smithi100.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:32:49.310 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:32:49.310 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:49.310 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-23T14:32:49.310 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-23T14:32:49.310 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-23T14:32:49.310 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-23T14:32:49.310 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-23T14:32:49.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:49.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:49.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:49.443 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-23T14:32:49.451 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:32:49.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:49.452 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2024-04-23T14:32:49.502 INFO:teuthology.orchestra.run.smithi100.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-23T14:32:49.503 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:49.503 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-23T14:32:49.503 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:32:49.503 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-23T14:32:49.503 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-23T14:32:49.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:49.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:49.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:49.634 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-23T14:32:49.654 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:32:49.654 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:32:49.654 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:32:49.654 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:49.654 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-23T14:32:49.654 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:49.654 INFO:teuthology.orchestra.run.smithi100.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:32:49.654 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether c6:c9:e2:eb:1d:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-23T14:32:49.655 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-23T14:32:49.655 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:49.655 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c4c9:e2ff:feeb:1d50/64 scope link 2024-04-23T14:32:49.655 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:49.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:49.656 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-23T14:32:49.721 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:32:49.721 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:32:49.722 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:32:49.722 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:49.722 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-23T14:32:49.722 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:49.722 INFO:teuthology.orchestra.run.smithi100.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:32:49.722 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether c6:c9:e2:eb:1d:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-23T14:32:49.722 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-23T14:32:49.722 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:49.722 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c4c9:e2ff:feeb:1d50/64 scope link 2024-04-23T14:32:49.722 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:49.724 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-23T14:32:49.724 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:49.724 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-23T14:32:49.724 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-23T14:32:49.724 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-23T14:32:49.724 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-23T14:32:49.724 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-23T14:32:49.724 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-23T14:32:49.724 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-23T14:32:49.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:49.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:49.909 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:49.909 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-23T14:32:49.909 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.1 up 2024-04-23T14:32:49.909 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.1 master ceph-brx 2024-04-23T14:32:49.909 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-23T14:32:49.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:49.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:49.992 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-23T14:32:49.992 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:32:49.992 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-23T14:32:50.001 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-23T14:32:50.001 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:32:50.001 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-23T14:32:50.049 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-23T14:32:50.109 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:32:50.163 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-23T14:32:50.163 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-23T14:32:50.163 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-23T14:32:50.163 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-23T14:32:50.163 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-23T14:32:50.163 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-23T14:32:50.163 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-23T14:32:50.163 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-23T14:32:50.163 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-23T14:32:50.163 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-23T14:32:50.164 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-23T14:32:50.164 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-23T14:32:50.164 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:32:50.164 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-23T14:32:50.164 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-23T14:32:50.164 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-23T14:32:50.164 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-23T14:32:50.164 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-23T14:32:50.164 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-23T14:32:50.164 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-23T14:32:50.164 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:32:50.164 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-23T14:32:50.164 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-23T14:32:50.164 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-23T14:32:50.165 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:32:50.165 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:32:50.165 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:32:50.165 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:32:50.165 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-23T14:32:50.165 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:32:50.165 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:32:50.165 INFO:teuthology.orchestra.run.smithi100.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-23T14:32:50.165 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:32:50.165 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:32:50.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:50.166 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-23T14:32:50.221 INFO:teuthology.orchestra.run.smithi100.stdout:44 2024-04-23T14:32:50.221 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [44] 2024-04-23T14:32:50.222 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-23T14:32:50.264 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-23T14:32:50.284 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:32:50.335 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-23T14:32:50.335 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-23T14:32:50.335 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-23T14:32:50.335 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-23T14:32:50.335 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-23T14:32:50.336 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-23T14:32:50.336 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-23T14:32:50.336 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-23T14:32:50.336 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-23T14:32:50.336 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-23T14:32:50.336 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-23T14:32:50.336 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-23T14:32:50.336 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:32:50.336 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-23T14:32:50.336 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-23T14:32:50.336 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-23T14:32:50.336 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-23T14:32:50.336 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-23T14:32:50.336 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-23T14:32:50.336 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-23T14:32:50.337 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:32:50.337 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-23T14:32:50.337 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-23T14:32:50.337 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-23T14:32:50.337 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:32:50.337 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:32:50.337 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:32:50.337 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:32:50.337 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-23T14:32:50.337 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:32:50.337 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:32:50.337 INFO:teuthology.orchestra.run.smithi100.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-23T14:32:50.337 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:32:50.337 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:32:50.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:50.338 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-23T14:32:50.358 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi100.stderr:2024-04-23T14:32:50.355+0000 7f8b26a62580 -1 init, newargv = 0x55e6a1ed0cf0 newargc=15 2024-04-23T14:32:50.358 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi100.stderr:ceph-fuse[36024]: starting ceph client 2024-04-23T14:32:50.364 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi100.stderr:ceph-fuse[36024]: starting fuse 2024-04-23T14:32:50.366 INFO:teuthology.orchestra.run.smithi100.stdout:44 2024-04-23T14:32:50.366 INFO:teuthology.orchestra.run.smithi100.stdout:45 2024-04-23T14:32:50.366 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44, 45] 2024-04-23T14:32:50.366 DEBUG:teuthology.orchestra.run.smithi100:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:32:50.366 DEBUG:teuthology.orchestra.run.smithi100:> import glob 2024-04-23T14:32:50.367 DEBUG:teuthology.orchestra.run.smithi100:> import re 2024-04-23T14:32:50.367 DEBUG:teuthology.orchestra.run.smithi100:> import os 2024-04-23T14:32:50.367 DEBUG:teuthology.orchestra.run.smithi100:> import subprocess 2024-04-23T14:32:50.367 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:32:50.367 DEBUG:teuthology.orchestra.run.smithi100:> def _find_admin_socket(client_name): 2024-04-23T14:32:50.367 DEBUG:teuthology.orchestra.run.smithi100:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-23T14:32:50.367 DEBUG:teuthology.orchestra.run.smithi100:> files = glob.glob(asok_path) 2024-04-23T14:32:50.367 DEBUG:teuthology.orchestra.run.smithi100:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-23T14:32:50.367 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:32:50.367 DEBUG:teuthology.orchestra.run.smithi100:> # Given a non-glob path, it better be there 2024-04-23T14:32:50.367 DEBUG:teuthology.orchestra.run.smithi100:> if "*" not in asok_path: 2024-04-23T14:32:50.367 DEBUG:teuthology.orchestra.run.smithi100:> assert(len(files) == 1) 2024-04-23T14:32:50.367 DEBUG:teuthology.orchestra.run.smithi100:> return files[0] 2024-04-23T14:32:50.368 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:32:50.368 DEBUG:teuthology.orchestra.run.smithi100:> for f in files: 2024-04-23T14:32:50.368 DEBUG:teuthology.orchestra.run.smithi100:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:32:50.368 DEBUG:teuthology.orchestra.run.smithi100:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:32:50.368 DEBUG:teuthology.orchestra.run.smithi100:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:32:50.368 DEBUG:teuthology.orchestra.run.smithi100:> contents = proc_f.read() 2024-04-23T14:32:50.368 DEBUG:teuthology.orchestra.run.smithi100:> if mountpoint in contents: 2024-04-23T14:32:50.368 DEBUG:teuthology.orchestra.run.smithi100:> return f 2024-04-23T14:32:50.368 DEBUG:teuthology.orchestra.run.smithi100:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:32:50.368 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:32:50.368 DEBUG:teuthology.orchestra.run.smithi100:> print(_find_admin_socket("client.2")) 2024-04-23T14:32:50.368 DEBUG:teuthology.orchestra.run.smithi100:> ' 2024-04-23T14:32:50.488 INFO:teuthology.orchestra.run.smithi100.stdout:/var/run/ceph/ceph-client.2.36024.asok 2024-04-23T14:32:50.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:50.499 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.36024.asok 2024-04-23T14:32:50.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:50.499 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.36024.asok status 2024-04-23T14:32:50.668 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:32:50.668 INFO:teuthology.orchestra.run.smithi100.stdout: "metadata": { 2024-04-23T14:32:50.668 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:32:50.668 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:32:50.668 INFO:teuthology.orchestra.run.smithi100.stdout: "entity_id": "2", 2024-04-23T14:32:50.668 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": "smithi100", 2024-04-23T14:32:50.668 INFO:teuthology.orchestra.run.smithi100.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-23T14:32:50.668 INFO:teuthology.orchestra.run.smithi100.stdout: "pid": "36024", 2024-04-23T14:32:50.668 INFO:teuthology.orchestra.run.smithi100.stdout: "root": "/" 2024-04-23T14:32:50.668 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:32:50.668 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_count": 0, 2024-04-23T14:32:50.668 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_pinned_count": 0, 2024-04-23T14:32:50.669 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 4845, 2024-04-23T14:32:50.669 INFO:teuthology.orchestra.run.smithi100.stdout: "inst": { 2024-04-23T14:32:50.669 INFO:teuthology.orchestra.run.smithi100.stdout: "name": { 2024-04-23T14:32:50.669 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "client", 2024-04-23T14:32:50.669 INFO:teuthology.orchestra.run.smithi100.stdout: "num": 4845 2024-04-23T14:32:50.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:32:50.669 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-23T14:32:50.669 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-23T14:32:50.669 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.2:0", 2024-04-23T14:32:50.669 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2784586121 2024-04-23T14:32:50.669 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:32:50.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:32:50.669 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-23T14:32:50.669 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-23T14:32:50.670 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.2:0", 2024-04-23T14:32:50.670 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2784586121 2024-04-23T14:32:50.670 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:32:50.670 INFO:teuthology.orchestra.run.smithi100.stdout: "inst_str": "client.4845 192.168.0.2:0/2784586121", 2024-04-23T14:32:50.670 INFO:teuthology.orchestra.run.smithi100.stdout: "addr_str": "192.168.0.2:0/2784586121", 2024-04-23T14:32:50.670 INFO:teuthology.orchestra.run.smithi100.stdout: "inode_count": 1, 2024-04-23T14:32:50.670 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_epoch": 9, 2024-04-23T14:32:50.670 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch": 26, 2024-04-23T14:32:50.670 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:32:50.670 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklisted": false, 2024-04-23T14:32:50.670 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs" 2024-04-23T14:32:50.670 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:32:50.683 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:32:50.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.818+0000 7fe694000640 1 Processor -- start 2024-04-23T14:32:50.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.818+0000 7fe694000640 1 -- start start 2024-04-23T14:32:50.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.822+0000 7fe694000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe68c0b2a10 0x7fe68c0adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:50.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.822+0000 7fe694000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe68c0af080 con 0x7fe68c15b640 2024-04-23T14:32:50.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.822+0000 7fe694000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe68c0afbc0 con 0x7fe68c073610 2024-04-23T14:32:50.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.822+0000 7fe694000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe68c15ba20 con 0x7fe68c0b2a10 2024-04-23T14:32:50.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.822+0000 7fe691600640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe68c15b640 0x7fe68c072f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42698/0 (socket says 172.21.15.100:42698) 2024-04-23T14:32:50.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.822+0000 7fe691600640 1 -- 172.21.15.100:0/1858931771 learned_addr learned my addr 172.21.15.100:0/1858931771 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:50.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.822+0000 7fe690c00640 1 --2- 172.21.15.100:0/1858931771 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe68c0b2a10 0x7fe68c0adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:50.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.822+0000 7fe68be00640 1 -- 172.21.15.100:0/1858931771 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1537234264 0 0) 0x7fe68c0af080 con 0x7fe68c15b640 2024-04-23T14:32:50.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.822+0000 7fe68be00640 1 -- 172.21.15.100:0/1858931771 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe674003530 con 0x7fe68c15b640 2024-04-23T14:32:50.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.822+0000 7fe68be00640 1 -- 172.21.15.100:0/1858931771 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1439000685 0 0) 0x7fe68c0afbc0 con 0x7fe68c073610 2024-04-23T14:32:50.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.822+0000 7fe68be00640 1 -- 172.21.15.100:0/1858931771 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe68c0af080 con 0x7fe68c073610 2024-04-23T14:32:50.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.822+0000 7fe68be00640 1 -- 172.21.15.100:0/1858931771 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1487216110 0 0) 0x7fe674003530 con 0x7fe68c15b640 2024-04-23T14:32:50.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.822+0000 7fe690c00640 1 -- 172.21.15.100:0/1858931771 >> v1:172.21.15.151:6789/0 conn(0x7fe68c073610 legacy=0x7fe68c06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:50.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.822+0000 7fe690c00640 1 -- 172.21.15.100:0/1858931771 >> v1:172.21.15.100:6789/0 conn(0x7fe68c15b640 legacy=0x7fe68c072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:50.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.822+0000 7fe690c00640 1 -- 172.21.15.100:0/1858931771 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68c0af320 con 0x7fe68c0b2a10 2024-04-23T14:32:50.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.826+0000 7fe690c00640 1 --2- 172.21.15.100:0/1858931771 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe68c0b2a10 0x7fe68c0adfc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe6780097f0 tx=0x7fe67801c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f067dd9a246fa53f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:50.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.826+0000 7fe68be00640 1 -- 172.21.15.100:0/1858931771 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6780b0070 con 0x7fe68c0b2a10 2024-04-23T14:32:50.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.826+0000 7fe68be00640 1 -- 172.21.15.100:0/1858931771 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe678004030 con 0x7fe68c0b2a10 2024-04-23T14:32:50.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.826+0000 7fe68be00640 1 -- 172.21.15.100:0/1858931771 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe680001680 con 0x7fe68c0b2a10 2024-04-23T14:32:50.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.826+0000 7fe694000640 1 -- 172.21.15.100:0/1858931771 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe68c0b2a10 msgr2=0x7fe68c0adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:50.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.826+0000 7fe694000640 1 --2- 172.21.15.100:0/1858931771 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe68c0b2a10 0x7fe68c0adfc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe6780097f0 tx=0x7fe67801c3d0 comp rx=0 tx=0).stop 2024-04-23T14:32:50.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.826+0000 7fe694000640 1 -- 172.21.15.100:0/1858931771 shutdown_connections 2024-04-23T14:32:50.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.826+0000 7fe694000640 1 --2- 172.21.15.100:0/1858931771 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe68c0b2a10 0x7fe68c0adfc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:50.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.826+0000 7fe694000640 1 -- 172.21.15.100:0/1858931771 >> 172.21.15.100:0/1858931771 conn(0x7fe68c0b1e80 msgr2=0x7fe68c0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:50.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.826+0000 7fe694000640 1 -- 172.21.15.100:0/1858931771 shutdown_connections 2024-04-23T14:32:50.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.826+0000 7fe694000640 1 -- 172.21.15.100:0/1858931771 wait complete. 2024-04-23T14:32:50.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.826+0000 7fe694000640 1 Processor -- start 2024-04-23T14:32:50.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.826+0000 7fe694000640 1 -- start start 2024-04-23T14:32:50.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe694000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe68c073610 0x7fe68c1494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:50.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe694000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe68c0b2a10 0x7fe68c0b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:50.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe694000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe68c15b640 0x7fe68c14b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:50.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe694000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1691c0 con 0x7fe68c15b640 2024-04-23T14:32:50.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe694000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c06f160 con 0x7fe68c0b2a10 2024-04-23T14:32:50.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe694000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe68c169340 con 0x7fe68c073610 2024-04-23T14:32:50.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe692000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe68c15b640 0x7fe68c14b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:50.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe692000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe68c15b640 0x7fe68c14b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50830/0 (socket says 172.21.15.100:50830) 2024-04-23T14:32:50.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe692000640 1 -- 172.21.15.100:0/2644425950 learned_addr learned my addr 172.21.15.100:0/2644425950 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:50.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe690c00640 1 --2- 172.21.15.100:0/2644425950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe68c0b2a10 0x7fe68c0b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:50.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe691600640 1 --2- 172.21.15.100:0/2644425950 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe68c073610 0x7fe68c1494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:50.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe692000640 1 -- 172.21.15.100:0/2644425950 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe68c073610 msgr2=0x7fe68c1494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:50.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe692000640 1 --2- 172.21.15.100:0/2644425950 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe68c073610 0x7fe68c1494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:50.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe692000640 1 -- 172.21.15.100:0/2644425950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe68c0b2a10 msgr2=0x7fe68c0b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:50.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe692000640 1 --2- 172.21.15.100:0/2644425950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe68c0b2a10 0x7fe68c0b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:50.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe692000640 1 -- 172.21.15.100:0/2644425950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68c0b17d0 con 0x7fe68c15b640 2024-04-23T14:32:50.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe692000640 1 --2- 172.21.15.100:0/2644425950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe68c15b640 0x7fe68c14b060 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe68400c6b0 tx=0x7fe68400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:50.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe689600640 1 -- 172.21.15.100:0/2644425950 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684003b80 con 0x7fe68c15b640 2024-04-23T14:32:50.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe689600640 1 -- 172.21.15.100:0/2644425950 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe684003d20 con 0x7fe68c15b640 2024-04-23T14:32:50.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe689600640 1 -- 172.21.15.100:0/2644425950 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684005310 con 0x7fe68c15b640 2024-04-23T14:32:50.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe694000640 1 -- 172.21.15.100:0/2644425950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68c0b1210 con 0x7fe68c15b640 2024-04-23T14:32:50.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe694000640 1 -- 172.21.15.100:0/2644425950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68c150710 con 0x7fe68c15b640 2024-04-23T14:32:50.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe694000640 1 -- 172.21.15.100:0/2644425950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe68c068ad0 con 0x7fe68c15b640 2024-04-23T14:32:50.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.830+0000 7fe689600640 1 -- 172.21.15.100:0/2644425950 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe6840054b0 con 0x7fe68c15b640 2024-04-23T14:32:50.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.834+0000 7fe689600640 1 --2- 172.21.15.100:0/2644425950 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe66c068380 0x7fe66c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:50.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.834+0000 7fe689600640 1 -- 172.21.15.100:0/2644425950 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe684086bc0 con 0x7fe68c15b640 2024-04-23T14:32:50.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.834+0000 7fe691600640 1 --2- 172.21.15.100:0/2644425950 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe66c068380 0x7fe66c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:50.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.834+0000 7fe691600640 1 --2- 172.21.15.100:0/2644425950 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe66c068380 0x7fe66c06a840 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe6800011f0 tx=0x7fe6800016a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:50.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:50.838+0000 7fe689600640 1 -- 172.21.15.100:0/2644425950 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe684056690 con 0x7fe68c15b640 2024-04-23T14:32:51.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:51.026+0000 7fe694000640 1 -- 172.21.15.100:0/2644425950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe68c0b1c40 con 0x7fe68c15b640 2024-04-23T14:32:51.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:51.026+0000 7fe689600640 1 -- 172.21.15.100:0/2644425950 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fe68405a990 con 0x7fe68c15b640 2024-04-23T14:32:51.030 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-23T14:32:51.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:51.030+0000 7fe694000640 1 -- 172.21.15.100:0/2644425950 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe66c068380 msgr2=0x7fe66c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:51.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:51.030+0000 7fe694000640 1 --2- 172.21.15.100:0/2644425950 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe66c068380 0x7fe66c06a840 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe6800011f0 tx=0x7fe6800016a0 comp rx=0 tx=0).stop 2024-04-23T14:32:51.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:51.030+0000 7fe694000640 1 -- 172.21.15.100:0/2644425950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe68c15b640 msgr2=0x7fe68c14b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:51.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:51.030+0000 7fe694000640 1 --2- 172.21.15.100:0/2644425950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe68c15b640 0x7fe68c14b060 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe68400c6b0 tx=0x7fe68400cb70 comp rx=0 tx=0).stop 2024-04-23T14:32:51.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:51.030+0000 7fe694000640 1 -- 172.21.15.100:0/2644425950 shutdown_connections 2024-04-23T14:32:51.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:51.030+0000 7fe694000640 1 --2- 172.21.15.100:0/2644425950 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe66c068380 0x7fe66c06a840 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:51.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:51.030+0000 7fe694000640 1 --2- 172.21.15.100:0/2644425950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe68c15b640 0x7fe68c14b060 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:51.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:51.030+0000 7fe694000640 1 --2- 172.21.15.100:0/2644425950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe68c0b2a10 0x7fe68c0b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:51.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:51.030+0000 7fe694000640 1 --2- 172.21.15.100:0/2644425950 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe68c073610 0x7fe68c1494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:51.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:51.030+0000 7fe694000640 1 -- 172.21.15.100:0/2644425950 >> 172.21.15.100:0/2644425950 conn(0x7fe68c0b1e80 msgr2=0x7fe68c0ae590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:51.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:51.030+0000 7fe694000640 1 -- 172.21.15.100:0/2644425950 shutdown_connections 2024-04-23T14:32:51.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:51.030+0000 7fe694000640 1 -- 172.21.15.100:0/2644425950 wait complete. 2024-04-23T14:32:51.051 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:32:51.051 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:32:51.051 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi151.front.sepia.ceph.com 2024-04-23T14:32:51.052 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-23T14:32:51.052 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-23T14:32:51.052 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:32:51.052 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:32:51.052 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:32:51.052 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-23T14:32:51.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:51.052 DEBUG:teuthology.orchestra.run.smithi151:> ip addr 2024-04-23T14:32:51.061 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:32:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:32:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:32:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-04-23T14:32:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:51.061 INFO:teuthology.orchestra.run.smithi151.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:32:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0c brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f0 2024-04-23T14:32:51.061 INFO:teuthology.orchestra.run.smithi151.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:32:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0d brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f1 2024-04-23T14:32:51.061 INFO:teuthology.orchestra.run.smithi151.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:32:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:62 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:51.061 INFO:teuthology.orchestra.run.smithi151.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:32:51.062 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:63 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:51.062 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.21.15.151/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-23T14:32:51.062 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft 42408sec preferred_lft 42408sec 2024-04-23T14:32:51.062 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::ec4:7aff:fe88:6f63/64 scope link 2024-04-23T14:32:51.062 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:51.062 INFO:teuthology.orchestra.run.smithi151.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-23T14:32:51.062 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 02:42:4e:28:80:72 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:51.062 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-23T14:32:51.062 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:51.063 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-23T14:32:51.063 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:51.063 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-04-23T14:32:51.063 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link add name ceph-brx type bridge 2024-04-23T14:32:51.063 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip addr flush dev ceph-brx 2024-04-23T14:32:51.063 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set ceph-brx up 2024-04-23T14:32:51.063 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-23T14:32:51.063 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-04-23T14:32:51.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:32:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:51.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:32:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:51.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:51.215 DEBUG:teuthology.orchestra.run.smithi151:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:32:51.273 INFO:teuthology.orchestra.run.smithi151.stdout:1 2024-04-23T14:32:51.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:51.275 DEBUG:teuthology.orchestra.run.smithi151:> route 2024-04-23T14:32:51.358 INFO:teuthology.orchestra.run.smithi151.stdout:Kernel IP routing table 2024-04-23T14:32:51.358 INFO:teuthology.orchestra.run.smithi151.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:32:51.358 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-23T14:32:51.358 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-23T14:32:51.358 INFO:teuthology.orchestra.run.smithi151.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-23T14:32:51.358 INFO:teuthology.orchestra.run.smithi151.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-23T14:32:51.359 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-23T14:32:51.359 INFO:teuthology.orchestra.run.smithi151.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:32:51.359 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:32:51.359 INFO:teuthology.orchestra.run.smithi151.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:32:51.359 INFO:teuthology.orchestra.run.smithi151.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:32:51.359 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:51.359 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-04-23T14:32:51.359 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-23T14:32:51.359 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-23T14:32:51.359 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-23T14:32:51.359 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-04-23T14:32:51.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:32:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:51.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:32:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:51.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:51.492 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-04-23T14:32:51.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:51.502 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list-id 2024-04-23T14:32:51.552 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:51.552 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-04-23T14:32:51.552 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:32:51.552 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-23T14:32:51.552 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-04-23T14:32:51.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:32:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:51.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:32:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:51.677 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-23T14:32:51.677 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:51.677 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-04-23T14:32:51.677 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-23T14:32:51.678 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-23T14:32:51.678 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-23T14:32:51.678 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-23T14:32:51.678 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-23T14:32:51.678 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-04-23T14:32:51.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:32:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:51.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:32:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:51.894 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:51.894 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-04-23T14:32:51.894 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set brx.0 up 2024-04-23T14:32:51.894 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set dev brx.0 master ceph-brx 2024-04-23T14:32:51.894 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-04-23T14:32:51.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:32:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:51.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:32:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:51.979 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-23T14:32:51.979 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:32:51.979 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-23T14:32:51.987 INFO:teuthology.orchestra.run.smithi151.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-23T14:32:51.987 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:32:51.988 DEBUG:teuthology.orchestra.run.smithi151:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-23T14:32:52.043 DEBUG:teuthology.orchestra.run.smithi151:> sudo modprobe fuse 2024-04-23T14:32:52.103 DEBUG:teuthology.orchestra.run.smithi151:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:32:52.153 INFO:teuthology.orchestra.run.smithi151.stdout:/sys 2024-04-23T14:32:52.153 INFO:teuthology.orchestra.run.smithi151.stdout:/proc 2024-04-23T14:32:52.153 INFO:teuthology.orchestra.run.smithi151.stdout:/dev 2024-04-23T14:32:52.153 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/pts 2024-04-23T14:32:52.153 INFO:teuthology.orchestra.run.smithi151.stdout:/run 2024-04-23T14:32:52.153 INFO:teuthology.orchestra.run.smithi151.stdout:/ 2024-04-23T14:32:52.153 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/security 2024-04-23T14:32:52.153 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/shm 2024-04-23T14:32:52.153 INFO:teuthology.orchestra.run.smithi151.stdout:/run/lock 2024-04-23T14:32:52.153 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/cgroup 2024-04-23T14:32:52.153 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/pstore 2024-04-23T14:32:52.153 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/bpf 2024-04-23T14:32:52.153 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:32:52.154 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/hugepages 2024-04-23T14:32:52.154 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/mqueue 2024-04-23T14:32:52.154 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/debug 2024-04-23T14:32:52.154 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/tracing 2024-04-23T14:32:52.154 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/fs/nfsd 2024-04-23T14:32:52.154 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/fuse/connections 2024-04-23T14:32:52.154 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/config 2024-04-23T14:32:52.154 INFO:teuthology.orchestra.run.smithi151.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:32:52.154 INFO:teuthology.orchestra.run.smithi151.stdout:/run/rpc_pipefs 2024-04-23T14:32:52.154 INFO:teuthology.orchestra.run.smithi151.stdout:/run/user/1000 2024-04-23T14:32:52.154 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph 2024-04-23T14:32:52.154 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-4 2024-04-23T14:32:52.154 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-5 2024-04-23T14:32:52.154 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-6 2024-04-23T14:32:52.155 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-7 2024-04-23T14:32:52.155 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns 2024-04-23T14:32:52.155 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:32:52.155 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:32:52.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:52.156 DEBUG:teuthology.orchestra.run.smithi151:> ls /sys/fs/fuse/connections 2024-04-23T14:32:52.211 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:32:52.211 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-23T14:32:52.254 DEBUG:teuthology.orchestra.run.smithi151:> sudo modprobe fuse 2024-04-23T14:32:52.274 DEBUG:teuthology.orchestra.run.smithi151:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:32:52.325 INFO:teuthology.orchestra.run.smithi151.stdout:/sys 2024-04-23T14:32:52.325 INFO:teuthology.orchestra.run.smithi151.stdout:/proc 2024-04-23T14:32:52.325 INFO:teuthology.orchestra.run.smithi151.stdout:/dev 2024-04-23T14:32:52.325 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/pts 2024-04-23T14:32:52.325 INFO:teuthology.orchestra.run.smithi151.stdout:/run 2024-04-23T14:32:52.325 INFO:teuthology.orchestra.run.smithi151.stdout:/ 2024-04-23T14:32:52.325 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/security 2024-04-23T14:32:52.325 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/shm 2024-04-23T14:32:52.325 INFO:teuthology.orchestra.run.smithi151.stdout:/run/lock 2024-04-23T14:32:52.325 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/cgroup 2024-04-23T14:32:52.325 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/pstore 2024-04-23T14:32:52.325 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/bpf 2024-04-23T14:32:52.326 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:32:52.326 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/hugepages 2024-04-23T14:32:52.326 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/mqueue 2024-04-23T14:32:52.326 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/debug 2024-04-23T14:32:52.326 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/tracing 2024-04-23T14:32:52.326 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/fs/nfsd 2024-04-23T14:32:52.326 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/fuse/connections 2024-04-23T14:32:52.326 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/config 2024-04-23T14:32:52.326 INFO:teuthology.orchestra.run.smithi151.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:32:52.326 INFO:teuthology.orchestra.run.smithi151.stdout:/run/rpc_pipefs 2024-04-23T14:32:52.326 INFO:teuthology.orchestra.run.smithi151.stdout:/run/user/1000 2024-04-23T14:32:52.326 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph 2024-04-23T14:32:52.326 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-4 2024-04-23T14:32:52.326 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-5 2024-04-23T14:32:52.326 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-6 2024-04-23T14:32:52.326 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-7 2024-04-23T14:32:52.327 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns 2024-04-23T14:32:52.327 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:32:52.327 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:32:52.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:52.328 DEBUG:teuthology.orchestra.run.smithi151:> ls /sys/fs/fuse/connections 2024-04-23T14:32:52.347 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi151.stderr:2024-04-23T14:32:52.342+0000 7ff5dae48580 -1 init, newargv = 0x55807f489ce0 newargc=15 2024-04-23T14:32:52.348 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi151.stderr:ceph-fuse[33222]: starting ceph client 2024-04-23T14:32:52.357 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi151.stderr:ceph-fuse[33222]: starting fuse 2024-04-23T14:32:53.458 DEBUG:teuthology.orchestra.run.smithi151:> sudo modprobe fuse 2024-04-23T14:32:53.483 DEBUG:teuthology.orchestra.run.smithi151:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:32:53.534 INFO:teuthology.orchestra.run.smithi151.stdout:/sys 2024-04-23T14:32:53.534 INFO:teuthology.orchestra.run.smithi151.stdout:/proc 2024-04-23T14:32:53.534 INFO:teuthology.orchestra.run.smithi151.stdout:/dev 2024-04-23T14:32:53.534 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/pts 2024-04-23T14:32:53.534 INFO:teuthology.orchestra.run.smithi151.stdout:/run 2024-04-23T14:32:53.534 INFO:teuthology.orchestra.run.smithi151.stdout:/ 2024-04-23T14:32:53.534 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/security 2024-04-23T14:32:53.534 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/shm 2024-04-23T14:32:53.534 INFO:teuthology.orchestra.run.smithi151.stdout:/run/lock 2024-04-23T14:32:53.534 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/cgroup 2024-04-23T14:32:53.535 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/pstore 2024-04-23T14:32:53.535 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/bpf 2024-04-23T14:32:53.535 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:32:53.535 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/hugepages 2024-04-23T14:32:53.535 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/mqueue 2024-04-23T14:32:53.535 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/debug 2024-04-23T14:32:53.535 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/tracing 2024-04-23T14:32:53.535 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/fs/nfsd 2024-04-23T14:32:53.535 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/fuse/connections 2024-04-23T14:32:53.535 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/config 2024-04-23T14:32:53.535 INFO:teuthology.orchestra.run.smithi151.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:32:53.535 INFO:teuthology.orchestra.run.smithi151.stdout:/run/rpc_pipefs 2024-04-23T14:32:53.535 INFO:teuthology.orchestra.run.smithi151.stdout:/run/user/1000 2024-04-23T14:32:53.535 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph 2024-04-23T14:32:53.535 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-4 2024-04-23T14:32:53.535 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-5 2024-04-23T14:32:53.536 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-6 2024-04-23T14:32:53.536 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-7 2024-04-23T14:32:53.536 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns 2024-04-23T14:32:53.536 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:32:53.536 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:32:53.536 INFO:teuthology.orchestra.run.smithi151.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-23T14:32:53.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:53.537 DEBUG:teuthology.orchestra.run.smithi151:> ls /sys/fs/fuse/connections 2024-04-23T14:32:53.587 INFO:teuthology.orchestra.run.smithi151.stdout:44 2024-04-23T14:32:53.587 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-23T14:32:53.587 DEBUG:teuthology.orchestra.run.smithi151:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:32:53.588 DEBUG:teuthology.orchestra.run.smithi151:> import glob 2024-04-23T14:32:53.588 DEBUG:teuthology.orchestra.run.smithi151:> import re 2024-04-23T14:32:53.588 DEBUG:teuthology.orchestra.run.smithi151:> import os 2024-04-23T14:32:53.588 DEBUG:teuthology.orchestra.run.smithi151:> import subprocess 2024-04-23T14:32:53.588 DEBUG:teuthology.orchestra.run.smithi151:> 2024-04-23T14:32:53.588 DEBUG:teuthology.orchestra.run.smithi151:> def _find_admin_socket(client_name): 2024-04-23T14:32:53.588 DEBUG:teuthology.orchestra.run.smithi151:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-23T14:32:53.588 DEBUG:teuthology.orchestra.run.smithi151:> files = glob.glob(asok_path) 2024-04-23T14:32:53.588 DEBUG:teuthology.orchestra.run.smithi151:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-23T14:32:53.588 DEBUG:teuthology.orchestra.run.smithi151:> 2024-04-23T14:32:53.588 DEBUG:teuthology.orchestra.run.smithi151:> # Given a non-glob path, it better be there 2024-04-23T14:32:53.588 DEBUG:teuthology.orchestra.run.smithi151:> if "*" not in asok_path: 2024-04-23T14:32:53.588 DEBUG:teuthology.orchestra.run.smithi151:> assert(len(files) == 1) 2024-04-23T14:32:53.588 DEBUG:teuthology.orchestra.run.smithi151:> return files[0] 2024-04-23T14:32:53.588 DEBUG:teuthology.orchestra.run.smithi151:> 2024-04-23T14:32:53.588 DEBUG:teuthology.orchestra.run.smithi151:> for f in files: 2024-04-23T14:32:53.589 DEBUG:teuthology.orchestra.run.smithi151:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:32:53.589 DEBUG:teuthology.orchestra.run.smithi151:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:32:53.589 DEBUG:teuthology.orchestra.run.smithi151:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:32:53.589 DEBUG:teuthology.orchestra.run.smithi151:> contents = proc_f.read() 2024-04-23T14:32:53.589 DEBUG:teuthology.orchestra.run.smithi151:> if mountpoint in contents: 2024-04-23T14:32:53.589 DEBUG:teuthology.orchestra.run.smithi151:> return f 2024-04-23T14:32:53.589 DEBUG:teuthology.orchestra.run.smithi151:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:32:53.589 DEBUG:teuthology.orchestra.run.smithi151:> 2024-04-23T14:32:53.589 DEBUG:teuthology.orchestra.run.smithi151:> print(_find_admin_socket("client.1")) 2024-04-23T14:32:53.589 DEBUG:teuthology.orchestra.run.smithi151:> ' 2024-04-23T14:32:53.725 INFO:teuthology.orchestra.run.smithi151.stdout:/var/run/ceph/ceph-client.1.33222.asok 2024-04-23T14:32:53.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:32:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:53.738 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33222.asok 2024-04-23T14:32:53.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:53.738 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33222.asok status 2024-04-23T14:32:53.908 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-23T14:32:53.908 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": { 2024-04-23T14:32:53.908 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:32:53.908 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:32:53.908 INFO:teuthology.orchestra.run.smithi151.stdout: "entity_id": "1", 2024-04-23T14:32:53.908 INFO:teuthology.orchestra.run.smithi151.stdout: "hostname": "smithi151", 2024-04-23T14:32:53.908 INFO:teuthology.orchestra.run.smithi151.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-23T14:32:53.908 INFO:teuthology.orchestra.run.smithi151.stdout: "pid": "33222", 2024-04-23T14:32:53.908 INFO:teuthology.orchestra.run.smithi151.stdout: "root": "/" 2024-04-23T14:32:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-23T14:32:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "dentry_count": 0, 2024-04-23T14:32:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "dentry_pinned_count": 0, 2024-04-23T14:32:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 4798, 2024-04-23T14:32:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "inst": { 2024-04-23T14:32:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "name": { 2024-04-23T14:32:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "client", 2024-04-23T14:32:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "num": 4798 2024-04-23T14:32:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-23T14:32:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": { 2024-04-23T14:32:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v1", 2024-04-23T14:32:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:32:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 2383156621 2024-04-23T14:32:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-23T14:32:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-23T14:32:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": { 2024-04-23T14:32:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v1", 2024-04-23T14:32:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:32:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 2383156621 2024-04-23T14:32:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-23T14:32:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "inst_str": "client.4798 192.168.0.1:0/2383156621", 2024-04-23T14:32:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "addr_str": "192.168.0.1:0/2383156621", 2024-04-23T14:32:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "inode_count": 1, 2024-04-23T14:32:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_epoch": 9, 2024-04-23T14:32:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_epoch": 26, 2024-04-23T14:32:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:32:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "blocklisted": false, 2024-04-23T14:32:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs" 2024-04-23T14:32:53.910 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-23T14:32:53.924 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:32:54.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.066+0000 7f7301400640 1 Processor -- start 2024-04-23T14:32:54.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.066+0000 7f7301400640 1 -- start start 2024-04-23T14:32:54.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.066+0000 7f7301400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f72fc15e5a0 0x7f72fc1621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:54.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.066+0000 7f7301400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72fc0ced00 0x7f72fc0cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:54.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.066+0000 7f7301400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72fc162a20 con 0x7f72fc0b3930 2024-04-23T14:32:54.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.066+0000 7f7301400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f72fc0cf6b0 con 0x7f72fc15e5a0 2024-04-23T14:32:54.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.066+0000 7f7301400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f72fc162d00 con 0x7f72fc0ced00 2024-04-23T14:32:54.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.066+0000 7f7300a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72fc0ced00 0x7f72fc0cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:54.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.066+0000 7f72fb400640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f72fc0b3930 0x7f72fc06df80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:50432/0 (socket says 172.21.15.100:50432) 2024-04-23T14:32:54.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.066+0000 7f72fb400640 1 -- 172.21.15.100:0/1685930591 learned_addr learned my addr 172.21.15.100:0/1685930591 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:54.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.066+0000 7f72fbe00640 1 --2- 172.21.15.100:0/1685930591 >> v2:172.21.15.151:3300/0 conn(0x7f72fc15e5a0 0x7f72fc1621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:54.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.066+0000 7f72faa00640 1 -- 172.21.15.100:0/1685930591 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2857255623 0 0) 0x7f72fc162a20 con 0x7f72fc0b3930 2024-04-23T14:32:54.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.066+0000 7f7300a00640 1 -- 172.21.15.100:0/1685930591 >> v2:172.21.15.151:3300/0 conn(0x7f72fc15e5a0 msgr2=0x7f72fc1621d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:54.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.066+0000 7f7300a00640 1 --2- 172.21.15.100:0/1685930591 >> v2:172.21.15.151:3300/0 conn(0x7f72fc15e5a0 0x7f72fc1621d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.066+0000 7f7300a00640 1 -- 172.21.15.100:0/1685930591 >> v1:172.21.15.151:6789/0 conn(0x7f72fc0b3930 legacy=0x7f72fc06df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:54.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.066+0000 7f7300a00640 1 -- 172.21.15.100:0/1685930591 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72fc162fe0 con 0x7f72fc0ced00 2024-04-23T14:32:54.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.070+0000 7f7300a00640 1 --2- 172.21.15.100:0/1685930591 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72fc0ced00 0x7f72fc0cf170 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f72f40097c0 tx=0x7f72f401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f8ae2c8b032cb1c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:54.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.070+0000 7f72faa00640 1 -- 172.21.15.100:0/1685930591 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f40a7070 con 0x7f72fc0ced00 2024-04-23T14:32:54.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.070+0000 7f72faa00640 1 -- 172.21.15.100:0/1685930591 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72f40b0070 con 0x7f72fc0ced00 2024-04-23T14:32:54.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.070+0000 7f72faa00640 1 -- 172.21.15.100:0/1685930591 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f40ab890 con 0x7f72fc0ced00 2024-04-23T14:32:54.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.070+0000 7f7301400640 1 -- 172.21.15.100:0/1685930591 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72fc0ced00 msgr2=0x7f72fc0cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:54.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.070+0000 7f7301400640 1 --2- 172.21.15.100:0/1685930591 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72fc0ced00 0x7f72fc0cf170 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f72f40097c0 tx=0x7f72f401c470 comp rx=0 tx=0).stop 2024-04-23T14:32:54.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.070+0000 7f7301400640 1 -- 172.21.15.100:0/1685930591 shutdown_connections 2024-04-23T14:32:54.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.070+0000 7f7301400640 1 --2- 172.21.15.100:0/1685930591 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72fc0ced00 0x7f72fc0cf170 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.070+0000 7f7301400640 1 --2- 172.21.15.100:0/1685930591 >> v2:172.21.15.151:3300/0 conn(0x7f72fc15e5a0 0x7f72fc1621d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.070+0000 7f7301400640 1 -- 172.21.15.100:0/1685930591 >> 172.21.15.100:0/1685930591 conn(0x7f72fc0b3e60 msgr2=0x7f72fc0b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:54.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.070+0000 7f7301400640 1 -- 172.21.15.100:0/1685930591 shutdown_connections 2024-04-23T14:32:54.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.070+0000 7f7301400640 1 -- 172.21.15.100:0/1685930591 wait complete. 2024-04-23T14:32:54.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f7301400640 1 Processor -- start 2024-04-23T14:32:54.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f7301400640 1 -- start start 2024-04-23T14:32:54.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f7301400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72fc0b3930 0x7f72fc0c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:54.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f7301400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72fc0ced00 0x7f72fc1f7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:54.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f7301400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72fc15e5a0 0x7f72fc0cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:54.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f7301400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc15e980 con 0x7f72fc15e5a0 2024-04-23T14:32:54.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f7301400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc06e690 con 0x7f72fc0ced00 2024-04-23T14:32:54.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f7301400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f72fc16b570 con 0x7f72fc0b3930 2024-04-23T14:32:54.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f7300a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72fc15e5a0 0x7f72fc0cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:54.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f7300a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72fc15e5a0 0x7f72fc0cb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50836/0 (socket says 172.21.15.100:50836) 2024-04-23T14:32:54.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f7300a00640 1 -- 172.21.15.100:0/213372360 learned_addr learned my addr 172.21.15.100:0/213372360 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:54.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f72fb400640 1 --2- 172.21.15.100:0/213372360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72fc0ced00 0x7f72fc1f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:54.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f72fbe00640 1 --2- 172.21.15.100:0/213372360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72fc0b3930 0x7f72fc0c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:54.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f7300a00640 1 -- 172.21.15.100:0/213372360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72fc0b3930 msgr2=0x7f72fc0c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:54.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f7300a00640 1 --2- 172.21.15.100:0/213372360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72fc0b3930 0x7f72fc0c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f7300a00640 1 -- 172.21.15.100:0/213372360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72fc0ced00 msgr2=0x7f72fc1f7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:54.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f7300a00640 1 --2- 172.21.15.100:0/213372360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72fc0ced00 0x7f72fc1f7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f7300a00640 1 -- 172.21.15.100:0/213372360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72fc0b4240 con 0x7f72fc15e5a0 2024-04-23T14:32:54.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f72fbe00640 1 --2- 172.21.15.100:0/213372360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72fc0b3930 0x7f72fc0c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:54.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f7300a00640 1 --2- 172.21.15.100:0/213372360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72fc15e5a0 0x7f72fc0cb3d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f72f401a0a0 tx=0x7f72f401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:54.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f72f3e00640 1 -- 172.21.15.100:0/213372360 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f40abe60 con 0x7f72fc15e5a0 2024-04-23T14:32:54.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f72f3e00640 1 -- 172.21.15.100:0/213372360 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72f40bf610 con 0x7f72fc15e5a0 2024-04-23T14:32:54.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f72f3e00640 1 -- 172.21.15.100:0/213372360 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f40bf9e0 con 0x7f72fc15e5a0 2024-04-23T14:32:54.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f7301400640 1 -- 172.21.15.100:0/213372360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72fc1f8920 con 0x7f72fc15e5a0 2024-04-23T14:32:54.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.074+0000 7f7301400640 1 -- 172.21.15.100:0/213372360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72fc1f8380 con 0x7f72fc15e5a0 2024-04-23T14:32:54.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.078+0000 7f7301400640 1 -- 172.21.15.100:0/213372360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72fc075c20 con 0x7f72fc15e5a0 2024-04-23T14:32:54.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.078+0000 7f72f3e00640 1 -- 172.21.15.100:0/213372360 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f72f40bd020 con 0x7f72fc15e5a0 2024-04-23T14:32:54.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.078+0000 7f72f3e00640 1 --2- 172.21.15.100:0/213372360 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f72dc068380 0x7f72dc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:54.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.078+0000 7f72f3e00640 1 -- 172.21.15.100:0/213372360 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f72f4121600 con 0x7f72fc15e5a0 2024-04-23T14:32:54.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.078+0000 7f72fbe00640 1 --2- 172.21.15.100:0/213372360 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f72dc068380 0x7f72dc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:54.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.078+0000 7f72fbe00640 1 --2- 172.21.15.100:0/213372360 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f72dc068380 0x7f72dc06a840 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f72fc06edf0 tx=0x7f72ec00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.082+0000 7f72f3e00640 1 -- 172.21.15.100:0/213372360 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f72f4016090 con 0x7f72fc15e5a0 2024-04-23T14:32:54.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.274+0000 7f7301400640 1 -- 172.21.15.100:0/213372360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f72fc0b2b50 con 0x7f72fc15e5a0 2024-04-23T14:32:54.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.274+0000 7f72f3e00640 1 -- 172.21.15.100:0/213372360 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f72f40f1280 con 0x7f72fc15e5a0 2024-04-23T14:32:54.276 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-23T14:32:54.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.274+0000 7f7301400640 1 -- 172.21.15.100:0/213372360 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f72dc068380 msgr2=0x7f72dc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:54.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.274+0000 7f7301400640 1 --2- 172.21.15.100:0/213372360 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f72dc068380 0x7f72dc06a840 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f72fc06edf0 tx=0x7f72ec00b040 comp rx=0 tx=0).stop 2024-04-23T14:32:54.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.274+0000 7f7301400640 1 -- 172.21.15.100:0/213372360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72fc15e5a0 msgr2=0x7f72fc0cb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:54.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.274+0000 7f7301400640 1 --2- 172.21.15.100:0/213372360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72fc15e5a0 0x7f72fc0cb3d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f72f401a0a0 tx=0x7f72f401acb0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.274+0000 7f7301400640 1 -- 172.21.15.100:0/213372360 shutdown_connections 2024-04-23T14:32:54.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.274+0000 7f7301400640 1 --2- 172.21.15.100:0/213372360 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f72dc068380 0x7f72dc06a840 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.274+0000 7f7301400640 1 --2- 172.21.15.100:0/213372360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72fc15e5a0 0x7f72fc0cb3d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.274+0000 7f7301400640 1 --2- 172.21.15.100:0/213372360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72fc0ced00 0x7f72fc1f7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.274+0000 7f7301400640 1 --2- 172.21.15.100:0/213372360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f72fc0b3930 0x7f72fc0c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.274+0000 7f7301400640 1 -- 172.21.15.100:0/213372360 >> 172.21.15.100:0/213372360 conn(0x7f72fc0b3e60 msgr2=0x7f72fc070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:54.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.274+0000 7f7301400640 1 -- 172.21.15.100:0/213372360 shutdown_connections 2024-04-23T14:32:54.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:54.278+0000 7f7301400640 1 -- 172.21.15.100:0/213372360 wait complete. 2024-04-23T14:32:54.297 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:32:54.297 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:32:54.297 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi151.front.sepia.ceph.com 2024-04-23T14:32:54.297 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-23T14:32:54.297 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-23T14:32:54.297 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:32:54.297 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:32:54.297 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:32:54.297 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-23T14:32:54.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:54.297 DEBUG:teuthology.orchestra.run.smithi151:> ip addr 2024-04-23T14:32:54.307 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:32:54.307 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:32:54.307 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:32:54.307 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:54.307 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-04-23T14:32:54.307 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:54.307 INFO:teuthology.orchestra.run.smithi151.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:32:54.307 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0c brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:54.307 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f0 2024-04-23T14:32:54.307 INFO:teuthology.orchestra.run.smithi151.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:32:54.307 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0d brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:54.307 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f1 2024-04-23T14:32:54.307 INFO:teuthology.orchestra.run.smithi151.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:32:54.307 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:62 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:54.308 INFO:teuthology.orchestra.run.smithi151.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:32:54.308 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:63 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:54.308 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.21.15.151/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-23T14:32:54.308 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft 42405sec preferred_lft 42405sec 2024-04-23T14:32:54.308 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::ec4:7aff:fe88:6f63/64 scope link 2024-04-23T14:32:54.308 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:54.308 INFO:teuthology.orchestra.run.smithi151.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-23T14:32:54.308 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 02:42:4e:28:80:72 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:54.308 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-23T14:32:54.308 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:54.308 INFO:teuthology.orchestra.run.smithi151.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:32:54.308 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-23T14:32:54.308 INFO:teuthology.orchestra.run.smithi151.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:32:54.308 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:54.308 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-23T14:32:54.309 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:54.309 INFO:teuthology.orchestra.run.smithi151.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:32:54.309 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:32:54.309 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-23T14:32:54.309 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:54.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:54.310 DEBUG:teuthology.orchestra.run.smithi151:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:32:54.373 INFO:teuthology.orchestra.run.smithi151.stdout:1 2024-04-23T14:32:54.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:54.375 DEBUG:teuthology.orchestra.run.smithi151:> route 2024-04-23T14:32:54.436 INFO:teuthology.orchestra.run.smithi151.stdout:Kernel IP routing table 2024-04-23T14:32:54.436 INFO:teuthology.orchestra.run.smithi151.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:32:54.436 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-23T14:32:54.436 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-23T14:32:54.436 INFO:teuthology.orchestra.run.smithi151.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-23T14:32:54.436 INFO:teuthology.orchestra.run.smithi151.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-23T14:32:54.436 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-23T14:32:54.436 INFO:teuthology.orchestra.run.smithi151.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:32:54.436 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:32:54.436 INFO:teuthology.orchestra.run.smithi151.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:32:54.436 INFO:teuthology.orchestra.run.smithi151.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:32:54.437 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:54.437 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-04-23T14:32:54.437 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-23T14:32:54.437 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-23T14:32:54.437 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-23T14:32:54.437 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-04-23T14:32:54.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:32:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:54.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:32:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:54.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:54.573 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-04-23T14:32:54.581 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-23T14:32:54.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:54.582 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list-id 2024-04-23T14:32:54.634 INFO:teuthology.orchestra.run.smithi151.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-23T14:32:54.635 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:54.635 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-04-23T14:32:54.635 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-23T14:32:54.635 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-23T14:32:54.635 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-04-23T14:32:54.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:32:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:54.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:32:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:54.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:54.766 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-23T14:32:54.827 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:32:54.828 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:32:54.828 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:32:54.828 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:54.828 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-04-23T14:32:54.828 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:54.828 INFO:teuthology.orchestra.run.smithi151.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:32:54.828 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 12:b2:83:5c:fe:ef brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-23T14:32:54.828 INFO:teuthology.orchestra.run.smithi151.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-23T14:32:54.828 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:54.828 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::10b2:83ff:fe5c:feef/64 scope link 2024-04-23T14:32:54.828 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:54.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:54.830 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-23T14:32:54.895 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:32:54.895 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:32:54.895 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:32:54.895 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:54.895 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-04-23T14:32:54.895 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:54.895 INFO:teuthology.orchestra.run.smithi151.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:32:54.895 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 12:b2:83:5c:fe:ef brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-23T14:32:54.895 INFO:teuthology.orchestra.run.smithi151.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-23T14:32:54.895 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:54.896 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::10b2:83ff:fe5c:feef/64 scope link 2024-04-23T14:32:54.896 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:32:54.897 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-23T14:32:54.897 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:54.897 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-04-23T14:32:54.897 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-23T14:32:54.897 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-23T14:32:54.897 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-23T14:32:54.897 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-23T14:32:54.897 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-23T14:32:54.898 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-04-23T14:32:54.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:32:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:55.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:32:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:55.078 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:55.078 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-04-23T14:32:55.078 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set brx.1 up 2024-04-23T14:32:55.078 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set dev brx.1 master ceph-brx 2024-04-23T14:32:55.078 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-04-23T14:32:55.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:32:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:55.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:32:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:55.160 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-23T14:32:55.160 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:32:55.160 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-23T14:32:55.168 INFO:teuthology.orchestra.run.smithi151.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-23T14:32:55.169 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:32:55.169 DEBUG:teuthology.orchestra.run.smithi151:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-23T14:32:55.218 DEBUG:teuthology.orchestra.run.smithi151:> sudo modprobe fuse 2024-04-23T14:32:55.279 DEBUG:teuthology.orchestra.run.smithi151:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:32:55.332 INFO:teuthology.orchestra.run.smithi151.stdout:/sys 2024-04-23T14:32:55.332 INFO:teuthology.orchestra.run.smithi151.stdout:/proc 2024-04-23T14:32:55.332 INFO:teuthology.orchestra.run.smithi151.stdout:/dev 2024-04-23T14:32:55.332 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/pts 2024-04-23T14:32:55.332 INFO:teuthology.orchestra.run.smithi151.stdout:/run 2024-04-23T14:32:55.333 INFO:teuthology.orchestra.run.smithi151.stdout:/ 2024-04-23T14:32:55.333 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/security 2024-04-23T14:32:55.333 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/shm 2024-04-23T14:32:55.333 INFO:teuthology.orchestra.run.smithi151.stdout:/run/lock 2024-04-23T14:32:55.333 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/cgroup 2024-04-23T14:32:55.333 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/pstore 2024-04-23T14:32:55.333 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/bpf 2024-04-23T14:32:55.333 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:32:55.333 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/hugepages 2024-04-23T14:32:55.333 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/mqueue 2024-04-23T14:32:55.333 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/debug 2024-04-23T14:32:55.333 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/tracing 2024-04-23T14:32:55.333 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/fs/nfsd 2024-04-23T14:32:55.333 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/fuse/connections 2024-04-23T14:32:55.333 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/config 2024-04-23T14:32:55.334 INFO:teuthology.orchestra.run.smithi151.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:32:55.334 INFO:teuthology.orchestra.run.smithi151.stdout:/run/rpc_pipefs 2024-04-23T14:32:55.334 INFO:teuthology.orchestra.run.smithi151.stdout:/run/user/1000 2024-04-23T14:32:55.334 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph 2024-04-23T14:32:55.334 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-4 2024-04-23T14:32:55.334 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-5 2024-04-23T14:32:55.334 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-6 2024-04-23T14:32:55.334 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-7 2024-04-23T14:32:55.334 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns 2024-04-23T14:32:55.334 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:32:55.334 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:32:55.334 INFO:teuthology.orchestra.run.smithi151.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-23T14:32:55.334 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-23T14:32:55.335 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-23T14:32:55.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:55.335 DEBUG:teuthology.orchestra.run.smithi151:> ls /sys/fs/fuse/connections 2024-04-23T14:32:55.387 INFO:teuthology.orchestra.run.smithi151.stdout:44 2024-04-23T14:32:55.387 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [44] 2024-04-23T14:32:55.387 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-23T14:32:55.434 DEBUG:teuthology.orchestra.run.smithi151:> sudo modprobe fuse 2024-04-23T14:32:55.454 DEBUG:teuthology.orchestra.run.smithi151:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:32:55.504 INFO:teuthology.orchestra.run.smithi151.stdout:/sys 2024-04-23T14:32:55.505 INFO:teuthology.orchestra.run.smithi151.stdout:/proc 2024-04-23T14:32:55.505 INFO:teuthology.orchestra.run.smithi151.stdout:/dev 2024-04-23T14:32:55.505 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/pts 2024-04-23T14:32:55.505 INFO:teuthology.orchestra.run.smithi151.stdout:/run 2024-04-23T14:32:55.505 INFO:teuthology.orchestra.run.smithi151.stdout:/ 2024-04-23T14:32:55.505 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/security 2024-04-23T14:32:55.505 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/shm 2024-04-23T14:32:55.505 INFO:teuthology.orchestra.run.smithi151.stdout:/run/lock 2024-04-23T14:32:55.505 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/cgroup 2024-04-23T14:32:55.505 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/pstore 2024-04-23T14:32:55.505 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/bpf 2024-04-23T14:32:55.505 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:32:55.505 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/hugepages 2024-04-23T14:32:55.506 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/mqueue 2024-04-23T14:32:55.506 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/debug 2024-04-23T14:32:55.506 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/tracing 2024-04-23T14:32:55.506 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/fs/nfsd 2024-04-23T14:32:55.506 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/fuse/connections 2024-04-23T14:32:55.506 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/config 2024-04-23T14:32:55.506 INFO:teuthology.orchestra.run.smithi151.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:32:55.506 INFO:teuthology.orchestra.run.smithi151.stdout:/run/rpc_pipefs 2024-04-23T14:32:55.506 INFO:teuthology.orchestra.run.smithi151.stdout:/run/user/1000 2024-04-23T14:32:55.506 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph 2024-04-23T14:32:55.506 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-4 2024-04-23T14:32:55.506 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-5 2024-04-23T14:32:55.506 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-6 2024-04-23T14:32:55.506 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-7 2024-04-23T14:32:55.507 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns 2024-04-23T14:32:55.507 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:32:55.507 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:32:55.507 INFO:teuthology.orchestra.run.smithi151.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-23T14:32:55.507 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-23T14:32:55.507 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-23T14:32:55.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:55.508 DEBUG:teuthology.orchestra.run.smithi151:> ls /sys/fs/fuse/connections 2024-04-23T14:32:55.528 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi151.stderr:2024-04-23T14:32:55.526+0000 7fbfa8323580 -1 init, newargv = 0x564172cbf050 newargc=15 2024-04-23T14:32:55.528 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi151.stderr:ceph-fuse[33341]: starting ceph client 2024-04-23T14:32:55.535 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi151.stderr:ceph-fuse[33341]: starting fuse 2024-04-23T14:32:55.537 INFO:teuthology.orchestra.run.smithi151.stdout:44 2024-04-23T14:32:55.537 INFO:teuthology.orchestra.run.smithi151.stdout:45 2024-04-23T14:32:55.537 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44, 45] 2024-04-23T14:32:55.537 DEBUG:teuthology.orchestra.run.smithi151:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:32:55.537 DEBUG:teuthology.orchestra.run.smithi151:> import glob 2024-04-23T14:32:55.537 DEBUG:teuthology.orchestra.run.smithi151:> import re 2024-04-23T14:32:55.537 DEBUG:teuthology.orchestra.run.smithi151:> import os 2024-04-23T14:32:55.537 DEBUG:teuthology.orchestra.run.smithi151:> import subprocess 2024-04-23T14:32:55.538 DEBUG:teuthology.orchestra.run.smithi151:> 2024-04-23T14:32:55.538 DEBUG:teuthology.orchestra.run.smithi151:> def _find_admin_socket(client_name): 2024-04-23T14:32:55.538 DEBUG:teuthology.orchestra.run.smithi151:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-23T14:32:55.538 DEBUG:teuthology.orchestra.run.smithi151:> files = glob.glob(asok_path) 2024-04-23T14:32:55.538 DEBUG:teuthology.orchestra.run.smithi151:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-23T14:32:55.538 DEBUG:teuthology.orchestra.run.smithi151:> 2024-04-23T14:32:55.538 DEBUG:teuthology.orchestra.run.smithi151:> # Given a non-glob path, it better be there 2024-04-23T14:32:55.538 DEBUG:teuthology.orchestra.run.smithi151:> if "*" not in asok_path: 2024-04-23T14:32:55.538 DEBUG:teuthology.orchestra.run.smithi151:> assert(len(files) == 1) 2024-04-23T14:32:55.538 DEBUG:teuthology.orchestra.run.smithi151:> return files[0] 2024-04-23T14:32:55.538 DEBUG:teuthology.orchestra.run.smithi151:> 2024-04-23T14:32:55.538 DEBUG:teuthology.orchestra.run.smithi151:> for f in files: 2024-04-23T14:32:55.538 DEBUG:teuthology.orchestra.run.smithi151:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:32:55.538 DEBUG:teuthology.orchestra.run.smithi151:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:32:55.538 DEBUG:teuthology.orchestra.run.smithi151:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:32:55.538 DEBUG:teuthology.orchestra.run.smithi151:> contents = proc_f.read() 2024-04-23T14:32:55.538 DEBUG:teuthology.orchestra.run.smithi151:> if mountpoint in contents: 2024-04-23T14:32:55.539 DEBUG:teuthology.orchestra.run.smithi151:> return f 2024-04-23T14:32:55.539 DEBUG:teuthology.orchestra.run.smithi151:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:32:55.539 DEBUG:teuthology.orchestra.run.smithi151:> 2024-04-23T14:32:55.539 DEBUG:teuthology.orchestra.run.smithi151:> print(_find_admin_socket("client.3")) 2024-04-23T14:32:55.539 DEBUG:teuthology.orchestra.run.smithi151:> ' 2024-04-23T14:32:55.662 INFO:teuthology.orchestra.run.smithi151.stdout:/var/run/ceph/ceph-client.3.33341.asok 2024-04-23T14:32:55.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-23T14:32:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:55.673 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33341.asok 2024-04-23T14:32:55.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:55.673 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33341.asok status 2024-04-23T14:32:55.847 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-23T14:32:55.847 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": { 2024-04-23T14:32:55.847 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:32:55.847 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:32:55.847 INFO:teuthology.orchestra.run.smithi151.stdout: "entity_id": "3", 2024-04-23T14:32:55.847 INFO:teuthology.orchestra.run.smithi151.stdout: "hostname": "smithi151", 2024-04-23T14:32:55.847 INFO:teuthology.orchestra.run.smithi151.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-23T14:32:55.847 INFO:teuthology.orchestra.run.smithi151.stdout: "pid": "33341", 2024-04-23T14:32:55.847 INFO:teuthology.orchestra.run.smithi151.stdout: "root": "/" 2024-04-23T14:32:55.847 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-23T14:32:55.847 INFO:teuthology.orchestra.run.smithi151.stdout: "dentry_count": 0, 2024-04-23T14:32:55.847 INFO:teuthology.orchestra.run.smithi151.stdout: "dentry_pinned_count": 0, 2024-04-23T14:32:55.848 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 4813, 2024-04-23T14:32:55.848 INFO:teuthology.orchestra.run.smithi151.stdout: "inst": { 2024-04-23T14:32:55.848 INFO:teuthology.orchestra.run.smithi151.stdout: "name": { 2024-04-23T14:32:55.848 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "client", 2024-04-23T14:32:55.848 INFO:teuthology.orchestra.run.smithi151.stdout: "num": 4813 2024-04-23T14:32:55.848 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-23T14:32:55.848 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": { 2024-04-23T14:32:55.848 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v1", 2024-04-23T14:32:55.848 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "192.168.0.2:0", 2024-04-23T14:32:55.848 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 3729286367 2024-04-23T14:32:55.848 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-23T14:32:55.848 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-23T14:32:55.848 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": { 2024-04-23T14:32:55.848 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v1", 2024-04-23T14:32:55.848 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "192.168.0.2:0", 2024-04-23T14:32:55.849 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 3729286367 2024-04-23T14:32:55.849 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-23T14:32:55.849 INFO:teuthology.orchestra.run.smithi151.stdout: "inst_str": "client.4813 192.168.0.2:0/3729286367", 2024-04-23T14:32:55.849 INFO:teuthology.orchestra.run.smithi151.stdout: "addr_str": "192.168.0.2:0/3729286367", 2024-04-23T14:32:55.849 INFO:teuthology.orchestra.run.smithi151.stdout: "inode_count": 1, 2024-04-23T14:32:55.849 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_epoch": 9, 2024-04-23T14:32:55.849 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_epoch": 26, 2024-04-23T14:32:55.849 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:32:55.849 INFO:teuthology.orchestra.run.smithi151.stdout: "blocklisted": false, 2024-04-23T14:32:55.849 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs" 2024-04-23T14:32:55.849 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-23T14:32:55.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:55.862 DEBUG:teuthology.orchestra.run.smithi100:> stat --file-system '--printf=%T 2024-04-23T14:32:55.862 DEBUG:teuthology.orchestra.run.smithi100:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:32:55.871 INFO:teuthology.orchestra.run.smithi100.stdout:fuseblk 2024-04-23T14:32:55.872 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:32:55.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:55.872 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:32:55.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:55.938 DEBUG:teuthology.orchestra.run.smithi100:> stat --file-system '--printf=%T 2024-04-23T14:32:55.938 DEBUG:teuthology.orchestra.run.smithi100:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-23T14:32:55.990 INFO:teuthology.orchestra.run.smithi100.stdout:fuseblk 2024-04-23T14:32:55.991 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-23T14:32:55.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:55.991 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-23T14:32:56.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:56.054 DEBUG:teuthology.orchestra.run.smithi151:> stat --file-system '--printf=%T 2024-04-23T14:32:56.054 DEBUG:teuthology.orchestra.run.smithi151:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-23T14:32:56.063 INFO:teuthology.orchestra.run.smithi151.stdout:fuseblk 2024-04-23T14:32:56.063 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-23T14:32:56.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:56.064 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-23T14:32:56.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:56.127 DEBUG:teuthology.orchestra.run.smithi151:> stat --file-system '--printf=%T 2024-04-23T14:32:56.127 DEBUG:teuthology.orchestra.run.smithi151:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-23T14:32:56.184 INFO:teuthology.orchestra.run.smithi151.stdout:fuseblk 2024-04-23T14:32:56.184 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-23T14:32:56.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:56.184 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-23T14:32:56.247 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-23T14:32:56.270 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:56.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.410+0000 7f0bc9200640 1 Processor -- start 2024-04-23T14:32:56.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.410+0000 7f0bc9200640 1 -- start start 2024-04-23T14:32:56.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.414+0000 7f0bc9200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc415c470 0x7f0bc40b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:56.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.414+0000 7f0bc9200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f0bc40ad870 0x7f0bc406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:56.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.414+0000 7f0bc9200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc4160320 con 0x7f0bc4160de0 2024-04-23T14:32:56.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.414+0000 7f0bc9200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f0bc40cc7e0 con 0x7f0bc40ad870 2024-04-23T14:32:56.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.414+0000 7f0bc9200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc41605e0 con 0x7f0bc415c470 2024-04-23T14:32:56.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.414+0000 7f0bc1e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc415c470 0x7f0bc40b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:56.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.414+0000 7f0bc1e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc415c470 0x7f0bc40b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:42776/0 (socket says 172.21.15.100:42776) 2024-04-23T14:32:56.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.414+0000 7f0bc1e00640 1 -- 172.21.15.100:0/628342063 learned_addr learned my addr 172.21.15.100:0/628342063 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:56.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.414+0000 7f0bc2800640 1 --2- 172.21.15.100:0/628342063 >> v2:172.21.15.151:3300/0 conn(0x7f0bc40ad870 0x7f0bc406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:56.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.414+0000 7f0bb9400640 1 --1- 172.21.15.100:0/628342063 >> v1:172.21.15.151:6789/0 conn(0x7f0bc4160de0 0x7f0bc40ad160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:50448/0 (socket says 172.21.15.100:50448) 2024-04-23T14:32:56.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.414+0000 7f0bc1e00640 1 -- 172.21.15.100:0/628342063 >> v2:172.21.15.151:3300/0 conn(0x7f0bc40ad870 msgr2=0x7f0bc406bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:56.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.414+0000 7f0bc1e00640 1 --2- 172.21.15.100:0/628342063 >> v2:172.21.15.151:3300/0 conn(0x7f0bc40ad870 0x7f0bc406bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:56.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.414+0000 7f0bc1e00640 1 -- 172.21.15.100:0/628342063 >> v1:172.21.15.151:6789/0 conn(0x7f0bc4160de0 legacy=0x7f0bc40ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:56.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.414+0000 7f0bc1e00640 1 -- 172.21.15.100:0/628342063 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc41608c0 con 0x7f0bc415c470 2024-04-23T14:32:56.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.418+0000 7f0bc1e00640 1 --2- 172.21.15.100:0/628342063 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc415c470 0x7f0bc40b29a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009e50 tx=0x7f0bb401c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5821e544d256d6c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:56.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.418+0000 7f0bc1400640 1 -- 172.21.15.100:0/628342063 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb40a5770 con 0x7f0bc415c470 2024-04-23T14:32:56.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.418+0000 7f0bc1400640 1 -- 172.21.15.100:0/628342063 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb40a5910 con 0x7f0bc415c470 2024-04-23T14:32:56.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.418+0000 7f0bc1400640 1 -- 172.21.15.100:0/628342063 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb40a5ce0 con 0x7f0bc415c470 2024-04-23T14:32:56.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.418+0000 7f0bc9200640 1 -- 172.21.15.100:0/628342063 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc415c470 msgr2=0x7f0bc40b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:56.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.418+0000 7f0bc9200640 1 --2- 172.21.15.100:0/628342063 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc415c470 0x7f0bc40b29a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009e50 tx=0x7f0bb401c9a0 comp rx=0 tx=0).stop 2024-04-23T14:32:56.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.418+0000 7f0bc9200640 1 -- 172.21.15.100:0/628342063 shutdown_connections 2024-04-23T14:32:56.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.418+0000 7f0bc9200640 1 --2- 172.21.15.100:0/628342063 >> v2:172.21.15.151:3300/0 conn(0x7f0bc40ad870 0x7f0bc406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:56.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.418+0000 7f0bc9200640 1 --2- 172.21.15.100:0/628342063 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc415c470 0x7f0bc40b29a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:56.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.418+0000 7f0bc9200640 1 -- 172.21.15.100:0/628342063 >> 172.21.15.100:0/628342063 conn(0x7f0bc40b1e80 msgr2=0x7f0bc40b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:56.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.418+0000 7f0bc9200640 1 -- 172.21.15.100:0/628342063 shutdown_connections 2024-04-23T14:32:56.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.418+0000 7f0bc9200640 1 -- 172.21.15.100:0/628342063 wait complete. 2024-04-23T14:32:56.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.418+0000 7f0bc9200640 1 Processor -- start 2024-04-23T14:32:56.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.418+0000 7f0bc9200640 1 -- start start 2024-04-23T14:32:56.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bc9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bc40ad870 0x7f0bc40ce360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:56.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bc9200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc415c470 0x7f0bc4154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:56.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bc9200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bc4160de0 0x7f0bc4154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:56.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bc9200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc415ca60 con 0x7f0bc40ad870 2024-04-23T14:32:56.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bc9200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc40cc580 con 0x7f0bc4160de0 2024-04-23T14:32:56.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bc9200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc4169540 con 0x7f0bc415c470 2024-04-23T14:32:56.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bc1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bc40ad870 0x7f0bc40ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:56.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bc1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bc40ad870 0x7f0bc40ce360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50852/0 (socket says 172.21.15.100:50852) 2024-04-23T14:32:56.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bc1e00640 1 -- 172.21.15.100:0/3708061780 learned_addr learned my addr 172.21.15.100:0/3708061780 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:56.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bb9400640 1 --2- 172.21.15.100:0/3708061780 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc415c470 0x7f0bc4154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:56.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bc2800640 1 --2- 172.21.15.100:0/3708061780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bc4160de0 0x7f0bc4154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:56.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bc1e00640 1 -- 172.21.15.100:0/3708061780 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc415c470 msgr2=0x7f0bc4154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:56.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bc1e00640 1 --2- 172.21.15.100:0/3708061780 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc415c470 0x7f0bc4154020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:56.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bc1e00640 1 -- 172.21.15.100:0/3708061780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bc4160de0 msgr2=0x7f0bc4154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:56.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bc1e00640 1 --2- 172.21.15.100:0/3708061780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bc4160de0 0x7f0bc4154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:56.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bc1e00640 1 -- 172.21.15.100:0/3708061780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc40d0eb0 con 0x7f0bc40ad870 2024-04-23T14:32:56.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bb9400640 1 --2- 172.21.15.100:0/3708061780 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc415c470 0x7f0bc4154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:56.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bc2800640 1 --2- 172.21.15.100:0/3708061780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bc4160de0 0x7f0bc4154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:56.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bc1e00640 1 --2- 172.21.15.100:0/3708061780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bc40ad870 0x7f0bc40ce360 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40af040 tx=0x7f0bb4019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:56.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bbaa00640 1 -- 172.21.15.100:0/3708061780 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb40bc020 con 0x7f0bc40ad870 2024-04-23T14:32:56.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bbaa00640 1 -- 172.21.15.100:0/3708061780 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb40b8670 con 0x7f0bc40ad870 2024-04-23T14:32:56.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bbaa00640 1 -- 172.21.15.100:0/3708061780 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb40b8a60 con 0x7f0bc40ad870 2024-04-23T14:32:56.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bc9200640 1 -- 172.21.15.100:0/3708061780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc40ce8a0 con 0x7f0bc40ad870 2024-04-23T14:32:56.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bc9200640 1 -- 172.21.15.100:0/3708061780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc415f820 con 0x7f0bc40ad870 2024-04-23T14:32:56.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bc9200640 1 -- 172.21.15.100:0/3708061780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc4068ad0 con 0x7f0bc40ad870 2024-04-23T14:32:56.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.422+0000 7f0bbaa00640 1 -- 172.21.15.100:0/3708061780 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0bb40bf020 con 0x7f0bc40ad870 2024-04-23T14:32:56.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.426+0000 7f0bbaa00640 1 --2- 172.21.15.100:0/3708061780 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0ba0068380 0x7f0ba006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:56.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.426+0000 7f0bbaa00640 1 -- 172.21.15.100:0/3708061780 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f0bb4121da0 con 0x7f0bc40ad870 2024-04-23T14:32:56.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.426+0000 7f0bb9400640 1 --2- 172.21.15.100:0/3708061780 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0ba0068380 0x7f0ba006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:56.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.426+0000 7f0bb9400640 1 --2- 172.21.15.100:0/3708061780 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0ba0068380 0x7f0ba006a840 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40b29f0 tx=0x7f0bac007cf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:56.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.430+0000 7f0bbaa00640 1 -- 172.21.15.100:0/3708061780 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0bb40c6030 con 0x7f0bc40ad870 2024-04-23T14:32:56.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.586+0000 7f0bc9200640 1 -- 172.21.15.100:0/3708061780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0bc4154e00 con 0x7f0bc40ad870 2024-04-23T14:32:56.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.586+0000 7f0bbaa00640 1 -- 172.21.15.100:0/3708061780 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18859 (secure 0 0 0) 0x7f0bb4016620 con 0x7f0bc40ad870 2024-04-23T14:32:56.589 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:56.589 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:49.784608+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:56.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.590+0000 7f0bc9200640 1 -- 172.21.15.100:0/3708061780 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0ba0068380 msgr2=0x7f0ba006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:56.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.590+0000 7f0bc9200640 1 --2- 172.21.15.100:0/3708061780 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0ba0068380 0x7f0ba006a840 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40b29f0 tx=0x7f0bac007cf0 comp rx=0 tx=0).stop 2024-04-23T14:32:56.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.590+0000 7f0bc9200640 1 -- 172.21.15.100:0/3708061780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bc40ad870 msgr2=0x7f0bc40ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:56.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.590+0000 7f0bc9200640 1 --2- 172.21.15.100:0/3708061780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bc40ad870 0x7f0bc40ce360 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40af040 tx=0x7f0bb4019740 comp rx=0 tx=0).stop 2024-04-23T14:32:56.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.590+0000 7f0bc9200640 1 -- 172.21.15.100:0/3708061780 shutdown_connections 2024-04-23T14:32:56.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.590+0000 7f0bc9200640 1 --2- 172.21.15.100:0/3708061780 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0ba0068380 0x7f0ba006a840 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:56.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.590+0000 7f0bc9200640 1 --2- 172.21.15.100:0/3708061780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bc4160de0 0x7f0bc4154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:56.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.590+0000 7f0bc9200640 1 --2- 172.21.15.100:0/3708061780 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc415c470 0x7f0bc4154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:56.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.590+0000 7f0bc9200640 1 --2- 172.21.15.100:0/3708061780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bc40ad870 0x7f0bc40ce360 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:56.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.590+0000 7f0bc9200640 1 -- 172.21.15.100:0/3708061780 >> 172.21.15.100:0/3708061780 conn(0x7f0bc40b1e80 msgr2=0x7f0bc41452f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:56.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.590+0000 7f0bc9200640 1 -- 172.21.15.100:0/3708061780 shutdown_connections 2024-04-23T14:32:56.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.590+0000 7f0bc9200640 1 -- 172.21.15.100:0/3708061780 wait complete. 2024-04-23T14:32:56.612 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:32:06.835560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:32:07.930489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-23T14:32:11.636814+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-23T14:32:56.613 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:56.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.750+0000 7fd6b2600640 1 Processor -- start 2024-04-23T14:32:56.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.750+0000 7fd6b2600640 1 -- start start 2024-04-23T14:32:56.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.750+0000 7fd6b2600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6ac06f8c0 0x7fd6ac06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:56.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.750+0000 7fd6b2600640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd6ac0d0170 0x7fd6ac0ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:56.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.750+0000 7fd6b2600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6ac06dd60 con 0x7fd6ac0ce620 2024-04-23T14:32:56.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.750+0000 7fd6b2600640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fd6ac0ad050 con 0x7fd6ac0d0170 2024-04-23T14:32:56.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.750+0000 7fd6b2600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd6ac0ad8f0 con 0x7fd6ac06f8c0 2024-04-23T14:32:56.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.750+0000 7fd6ab400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6ac06f8c0 0x7fd6ac06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:56.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.750+0000 7fd6ab400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6ac06f8c0 0x7fd6ac06c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:42798/0 (socket says 172.21.15.100:42798) 2024-04-23T14:32:56.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.750+0000 7fd6ab400640 1 -- 172.21.15.100:0/71030499 learned_addr learned my addr 172.21.15.100:0/71030499 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:56.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.750+0000 7fd6aaa00640 1 --2- 172.21.15.100:0/71030499 >> v2:172.21.15.151:3300/0 conn(0x7fd6ac0d0170 0x7fd6ac0ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:56.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.750+0000 7fd6ab400640 1 -- 172.21.15.100:0/71030499 >> v2:172.21.15.151:3300/0 conn(0x7fd6ac0d0170 msgr2=0x7fd6ac0ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:56.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.750+0000 7fd6ab400640 1 --2- 172.21.15.100:0/71030499 >> v2:172.21.15.151:3300/0 conn(0x7fd6ac0d0170 0x7fd6ac0ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:56.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.750+0000 7fd6ab400640 1 -- 172.21.15.100:0/71030499 >> v1:172.21.15.151:6789/0 conn(0x7fd6ac0ce620 legacy=0x7fd6ac15f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:56.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.750+0000 7fd6ab400640 1 -- 172.21.15.100:0/71030499 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ac0d09c0 con 0x7fd6ac06f8c0 2024-04-23T14:32:56.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.754+0000 7fd6ab400640 1 --2- 172.21.15.100:0/71030499 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6ac06f8c0 0x7fd6ac06c6c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009870 tx=0x7fd69c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83ceec8b7cde1592 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:56.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.754+0000 7fd6aa000640 1 -- 172.21.15.100:0/71030499 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0b2070 con 0x7fd6ac06f8c0 2024-04-23T14:32:56.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.754+0000 7fd6aa000640 1 -- 172.21.15.100:0/71030499 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd69c0ad640 con 0x7fd6ac06f8c0 2024-04-23T14:32:56.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.754+0000 7fd6aa000640 1 -- 172.21.15.100:0/71030499 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0ada10 con 0x7fd6ac06f8c0 2024-04-23T14:32:56.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.754+0000 7fd6b2600640 1 -- 172.21.15.100:0/71030499 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6ac06f8c0 msgr2=0x7fd6ac06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:56.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.754+0000 7fd6b2600640 1 --2- 172.21.15.100:0/71030499 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6ac06f8c0 0x7fd6ac06c6c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009870 tx=0x7fd69c01c560 comp rx=0 tx=0).stop 2024-04-23T14:32:56.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.754+0000 7fd6b2600640 1 -- 172.21.15.100:0/71030499 shutdown_connections 2024-04-23T14:32:56.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.754+0000 7fd6b2600640 1 --2- 172.21.15.100:0/71030499 >> v2:172.21.15.151:3300/0 conn(0x7fd6ac0d0170 0x7fd6ac0ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:56.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.754+0000 7fd6b2600640 1 --2- 172.21.15.100:0/71030499 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6ac06f8c0 0x7fd6ac06c6c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:56.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.754+0000 7fd6b2600640 1 -- 172.21.15.100:0/71030499 >> 172.21.15.100:0/71030499 conn(0x7fd6ac0b5200 msgr2=0x7fd6ac0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:56.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.754+0000 7fd6b2600640 1 -- 172.21.15.100:0/71030499 shutdown_connections 2024-04-23T14:32:56.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.754+0000 7fd6b2600640 1 -- 172.21.15.100:0/71030499 wait complete. 2024-04-23T14:32:56.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6b2600640 1 Processor -- start 2024-04-23T14:32:56.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6b2600640 1 -- start start 2024-04-23T14:32:56.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6b2600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ac06f8c0 0x7fd6ac0c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:56.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6b2600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6ac0ce620 0x7fd6ac158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:56.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6b2600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6ac0d0170 0x7fd6ac1587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:56.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6b2600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac0d06d0 con 0x7fd6ac06f8c0 2024-04-23T14:32:56.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6b2600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac0d0550 con 0x7fd6ac0d0170 2024-04-23T14:32:56.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6b2600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd6ac168400 con 0x7fd6ac0ce620 2024-04-23T14:32:56.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6ab400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ac06f8c0 0x7fd6ac0c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:56.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6aaa00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6ac0ce620 0x7fd6ac158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:56.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6ab400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ac06f8c0 0x7fd6ac0c6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50854/0 (socket says 172.21.15.100:50854) 2024-04-23T14:32:56.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6ab400640 1 -- 172.21.15.100:0/1082534650 learned_addr learned my addr 172.21.15.100:0/1082534650 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:56.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6abe00640 1 --2- 172.21.15.100:0/1082534650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6ac0d0170 0x7fd6ac1587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:56.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6ab400640 1 -- 172.21.15.100:0/1082534650 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6ac0ce620 msgr2=0x7fd6ac158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:56.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6ab400640 1 --2- 172.21.15.100:0/1082534650 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6ac0ce620 0x7fd6ac158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:56.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6ab400640 1 -- 172.21.15.100:0/1082534650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6ac0d0170 msgr2=0x7fd6ac1587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:56.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6ab400640 1 --2- 172.21.15.100:0/1082534650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6ac0d0170 0x7fd6ac1587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:56.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6ab400640 1 -- 172.21.15.100:0/1082534650 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ac0c9150 con 0x7fd6ac06f8c0 2024-04-23T14:32:56.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6abe00640 1 --2- 172.21.15.100:0/1082534650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6ac0d0170 0x7fd6ac1587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:56.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6ab400640 1 --2- 172.21.15.100:0/1082534650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ac06f8c0 0x7fd6ac0c6c30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0b0040 tx=0x7fd69c019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:56.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6a3400640 1 -- 172.21.15.100:0/1082534650 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0bd070 con 0x7fd6ac06f8c0 2024-04-23T14:32:56.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6a3400640 1 -- 172.21.15.100:0/1082534650 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd69c0b8430 con 0x7fd6ac06f8c0 2024-04-23T14:32:56.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6a3400640 1 -- 172.21.15.100:0/1082534650 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0b2040 con 0x7fd6ac06f8c0 2024-04-23T14:32:56.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6b2600640 1 -- 172.21.15.100:0/1082534650 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6ac0c9430 con 0x7fd6ac06f8c0 2024-04-23T14:32:56.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.758+0000 7fd6b2600640 1 -- 172.21.15.100:0/1082534650 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6ac0c99f0 con 0x7fd6ac06f8c0 2024-04-23T14:32:56.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.762+0000 7fd6b2600640 1 -- 172.21.15.100:0/1082534650 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6ac068ad0 con 0x7fd6ac06f8c0 2024-04-23T14:32:56.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.762+0000 7fd6a3400640 1 -- 172.21.15.100:0/1082534650 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd69c0c0070 con 0x7fd6ac06f8c0 2024-04-23T14:32:56.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.762+0000 7fd6a3400640 1 --2- 172.21.15.100:0/1082534650 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd68c068380 0x7fd68c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:56.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.762+0000 7fd6a3400640 1 -- 172.21.15.100:0/1082534650 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd69c120560 con 0x7fd6ac06f8c0 2024-04-23T14:32:56.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.762+0000 7fd6aaa00640 1 --2- 172.21.15.100:0/1082534650 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd68c068380 0x7fd68c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:56.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.762+0000 7fd6aaa00640 1 --2- 172.21.15.100:0/1082534650 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd68c068380 0x7fd68c06a840 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac15eba0 tx=0x7fd698009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:56.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.766+0000 7fd6a3400640 1 -- 172.21.15.100:0/1082534650 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd69c016150 con 0x7fd6ac06f8c0 2024-04-23T14:32:56.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.926+0000 7fd6b2600640 1 -- 172.21.15.100:0/1082534650 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd6ac06cc00 con 0x7fd6ac06f8c0 2024-04-23T14:32:56.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.926+0000 7fd6a3400640 1 -- 172.21.15.100:0/1082534650 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fd69c0f0210 con 0x7fd6ac06f8c0 2024-04-23T14:32:56.928 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:32:56.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.930+0000 7fd6b2600640 1 -- 172.21.15.100:0/1082534650 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd68c068380 msgr2=0x7fd68c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:56.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.930+0000 7fd6b2600640 1 --2- 172.21.15.100:0/1082534650 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd68c068380 0x7fd68c06a840 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac15eba0 tx=0x7fd698009190 comp rx=0 tx=0).stop 2024-04-23T14:32:56.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.930+0000 7fd6b2600640 1 -- 172.21.15.100:0/1082534650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ac06f8c0 msgr2=0x7fd6ac0c6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:56.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.930+0000 7fd6b2600640 1 --2- 172.21.15.100:0/1082534650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ac06f8c0 0x7fd6ac0c6c30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0b0040 tx=0x7fd69c019300 comp rx=0 tx=0).stop 2024-04-23T14:32:56.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.930+0000 7fd6b2600640 1 -- 172.21.15.100:0/1082534650 shutdown_connections 2024-04-23T14:32:56.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.930+0000 7fd6b2600640 1 --2- 172.21.15.100:0/1082534650 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd68c068380 0x7fd68c06a840 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:56.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.930+0000 7fd6b2600640 1 --2- 172.21.15.100:0/1082534650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6ac0d0170 0x7fd6ac1587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:56.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.930+0000 7fd6b2600640 1 --2- 172.21.15.100:0/1082534650 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd6ac0ce620 0x7fd6ac158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:56.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.930+0000 7fd6b2600640 1 --2- 172.21.15.100:0/1082534650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ac06f8c0 0x7fd6ac0c6c30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:56.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.930+0000 7fd6b2600640 1 -- 172.21.15.100:0/1082534650 >> 172.21.15.100:0/1082534650 conn(0x7fd6ac0b5200 msgr2=0x7fd6ac06e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:56.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.930+0000 7fd6b2600640 1 -- 172.21.15.100:0/1082534650 shutdown_connections 2024-04-23T14:32:56.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:56.930+0000 7fd6b2600640 1 -- 172.21.15.100:0/1082534650 wait complete. 2024-04-23T14:32:56.949 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:32:57.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f4874000640 1 Processor -- start 2024-04-23T14:32:57.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f4874000640 1 -- start start 2024-04-23T14:32:57.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f4874000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f486c0adfe0 0x7f486c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:57.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f4874000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f486c06e5f0 con 0x7f486c0adb90 2024-04-23T14:32:57.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f4874000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f486c06e8b0 con 0x7f486c15e860 2024-04-23T14:32:57.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f4874000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f486c0ae410 con 0x7f486c0adfe0 2024-04-23T14:32:57.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f4870c00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f486c0adb90 0x7f486c06f950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42700/0 (socket says 172.21.15.100:42700) 2024-04-23T14:32:57.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f4870c00640 1 -- 172.21.15.100:0/2247333858 learned_addr learned my addr 172.21.15.100:0/2247333858 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:57.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f4872000640 1 --2- 172.21.15.100:0/2247333858 >> v2:172.21.15.151:3300/0 conn(0x7f486c0adfe0 0x7f486c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:57.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f486be00640 1 -- 172.21.15.100:0/2247333858 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 106319529 0 0) 0x7f486c06e5f0 con 0x7f486c0adb90 2024-04-23T14:32:57.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f486be00640 1 -- 172.21.15.100:0/2247333858 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4854003530 con 0x7f486c0adb90 2024-04-23T14:32:57.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f486be00640 1 -- 172.21.15.100:0/2247333858 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1794604135 0 0) 0x7f4854003530 con 0x7f486c0adb90 2024-04-23T14:32:57.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f486be00640 1 -- 172.21.15.100:0/2247333858 >> v2:172.21.15.151:3300/0 conn(0x7f486c0adfe0 msgr2=0x7f486c06d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:57.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f486be00640 1 --2- 172.21.15.100:0/2247333858 >> v2:172.21.15.151:3300/0 conn(0x7f486c0adfe0 0x7f486c06d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f486be00640 1 -- 172.21.15.100:0/2247333858 >> v1:172.21.15.151:6789/0 conn(0x7f486c15e860 legacy=0x7f486c0b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:57.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f486be00640 1 -- 172.21.15.100:0/2247333858 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c06eb90 con 0x7f486c0adb90 2024-04-23T14:32:57.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f486be00640 1 -- 172.21.15.100:0/2247333858 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f4858001680 con 0x7f486c0adb90 2024-04-23T14:32:57.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f486be00640 1 -- 172.21.15.100:0/2247333858 >> v1:172.21.15.100:6789/0 conn(0x7f486c0adb90 legacy=0x7f486c06f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f486be00640 1 --2- 172.21.15.100:0/2247333858 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f48540039c0 0x7f4854003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f486be00640 1 --2- 172.21.15.100:0/2247333858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48540043c0 0x7f4854004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f486be00640 1 --2- 172.21.15.100:0/2247333858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4854005090 0x7f485400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f486be00640 1 -- 172.21.15.100:0/2247333858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f486c0ae410 con 0x7f4854005090 2024-04-23T14:32:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f486be00640 1 -- 172.21.15.100:0/2247333858 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f485400bff0 con 0x7f48540043c0 2024-04-23T14:32:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f486be00640 1 -- 172.21.15.100:0/2247333858 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f485400c1f0 con 0x7f48540039c0 2024-04-23T14:32:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f4872000640 1 --2- 172.21.15.100:0/2247333858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4854005090 0x7f485400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f4870c00640 1 --2- 172.21.15.100:0/2247333858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48540043c0 0x7f4854004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f4871600640 1 --2- 172.21.15.100:0/2247333858 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f48540039c0 0x7f4854003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f4872000640 1 -- 172.21.15.100:0/2247333858 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f48540039c0 msgr2=0x7f4854003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f4872000640 1 --2- 172.21.15.100:0/2247333858 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f48540039c0 0x7f4854003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f4872000640 1 -- 172.21.15.100:0/2247333858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48540043c0 msgr2=0x7f4854004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f4872000640 1 --2- 172.21.15.100:0/2247333858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48540043c0 0x7f4854004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f4872000640 1 -- 172.21.15.100:0/2247333858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4854003530 con 0x7f4854005090 2024-04-23T14:32:57.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.086+0000 7f4871600640 1 --2- 172.21.15.100:0/2247333858 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f48540039c0 0x7f4854003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:57.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.090+0000 7f4872000640 1 -- 172.21.15.100:0/2247333858 reap_dead start 2024-04-23T14:32:57.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.090+0000 7f4872000640 1 --2- 172.21.15.100:0/2247333858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4854005090 0x7f485400b9d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4864009fc0 tx=0x7f486401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9bb17f9cd9a233 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:57.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.090+0000 7f486be00640 1 -- 172.21.15.100:0/2247333858 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4864009c10 con 0x7f4854005090 2024-04-23T14:32:57.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.090+0000 7f486be00640 1 -- 172.21.15.100:0/2247333858 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4864003be0 con 0x7f4854005090 2024-04-23T14:32:57.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.090+0000 7f486be00640 1 -- 172.21.15.100:0/2247333858 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4864004020 con 0x7f4854005090 2024-04-23T14:32:57.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.090+0000 7f4874000640 1 -- 172.21.15.100:0/2247333858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4854005090 msgr2=0x7f485400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:57.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.090+0000 7f4874000640 1 --2- 172.21.15.100:0/2247333858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4854005090 0x7f485400b9d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4864009fc0 tx=0x7f486401c9a0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.090+0000 7f4874000640 1 -- 172.21.15.100:0/2247333858 shutdown_connections 2024-04-23T14:32:57.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.090+0000 7f4874000640 1 --2- 172.21.15.100:0/2247333858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4854005090 0x7f485400b9d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.090+0000 7f4874000640 1 -- 172.21.15.100:0/2247333858 >> 172.21.15.100:0/2247333858 conn(0x7f486c072590 msgr2=0x7f486c0729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:57.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.090+0000 7f4874000640 1 -- 172.21.15.100:0/2247333858 shutdown_connections 2024-04-23T14:32:57.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.090+0000 7f4874000640 1 -- 172.21.15.100:0/2247333858 wait complete. 2024-04-23T14:32:57.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4874000640 1 Processor -- start 2024-04-23T14:32:57.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4874000640 1 -- start start 2024-04-23T14:32:57.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4874000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4854005090 0x7f486c13cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:57.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4874000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f486c0aefa0 0x7f486c0b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:57.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4874000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f486c0b0860 0x7f486c15fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:57.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4874000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f486c06f430 con 0x7f486c0b0860 2024-04-23T14:32:57.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4874000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f486c06f150 con 0x7f4854005090 2024-04-23T14:32:57.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4874000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f486c073790 con 0x7f486c0aefa0 2024-04-23T14:32:57.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4872000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f486c0b0860 0x7f486c15fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:57.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4872000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f486c0b0860 0x7f486c15fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50870/0 (socket says 172.21.15.100:50870) 2024-04-23T14:32:57.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4872000640 1 -- 172.21.15.100:0/2835247765 learned_addr learned my addr 172.21.15.100:0/2835247765 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:57.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4871600640 1 --2- 172.21.15.100:0/2835247765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4854005090 0x7f486c13cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:57.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4870c00640 1 --2- 172.21.15.100:0/2835247765 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f486c0aefa0 0x7f486c0b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:57.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4872000640 1 -- 172.21.15.100:0/2835247765 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f486c0aefa0 msgr2=0x7f486c0b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:57.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4872000640 1 --2- 172.21.15.100:0/2835247765 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f486c0aefa0 0x7f486c0b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4872000640 1 -- 172.21.15.100:0/2835247765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4854005090 msgr2=0x7f486c13cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:57.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4872000640 1 --2- 172.21.15.100:0/2835247765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4854005090 0x7f486c13cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4872000640 1 -- 172.21.15.100:0/2835247765 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c0b0f40 con 0x7f486c0b0860 2024-04-23T14:32:57.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4872000640 1 --2- 172.21.15.100:0/2835247765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f486c0b0860 0x7f486c15fc90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4864097d30 tx=0x7f4864009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:57.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4869600640 1 -- 172.21.15.100:0/2835247765 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4864009c10 con 0x7f486c0b0860 2024-04-23T14:32:57.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4869600640 1 -- 172.21.15.100:0/2835247765 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48640bd070 con 0x7f486c0b0860 2024-04-23T14:32:57.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4869600640 1 -- 172.21.15.100:0/2835247765 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48640aba60 con 0x7f486c0b0860 2024-04-23T14:32:57.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4874000640 1 -- 172.21.15.100:0/2835247765 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486c0b1220 con 0x7f486c0b0860 2024-04-23T14:32:57.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.094+0000 7f4874000640 1 -- 172.21.15.100:0/2835247765 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f486c0b17e0 con 0x7f486c0b0860 2024-04-23T14:32:57.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.098+0000 7f4874000640 1 -- 172.21.15.100:0/2835247765 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f486c068ad0 con 0x7f486c0b0860 2024-04-23T14:32:57.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.098+0000 7f4869600640 1 -- 172.21.15.100:0/2835247765 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f48640b4040 con 0x7f486c0b0860 2024-04-23T14:32:57.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.098+0000 7f4869600640 1 --2- 172.21.15.100:0/2835247765 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f484c0683b0 0x7f484c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:57.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.098+0000 7f4869600640 1 -- 172.21.15.100:0/2835247765 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f4864120ec0 con 0x7f486c0b0860 2024-04-23T14:32:57.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.098+0000 7f4871600640 1 --2- 172.21.15.100:0/2835247765 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f484c0683b0 0x7f484c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:57.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.098+0000 7f4871600640 1 --2- 172.21.15.100:0/2835247765 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f484c0683b0 0x7f484c06a870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4860007bd0 tx=0x7f4860009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:57.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.102+0000 7f4869600640 1 -- 172.21.15.100:0/2835247765 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f48640169d0 con 0x7f486c0b0860 2024-04-23T14:32:57.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.258+0000 7f4874000640 1 -- 172.21.15.100:0/2835247765 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f486c1621e0 con 0x7f486c0b0860 2024-04-23T14:32:57.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.258+0000 7f4869600640 1 -- 172.21.15.100:0/2835247765 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f48640f5e20 con 0x7f486c0b0860 2024-04-23T14:32:57.261 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:32:57.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.262+0000 7f4874000640 1 -- 172.21.15.100:0/2835247765 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f484c0683b0 msgr2=0x7f484c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:57.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.262+0000 7f4874000640 1 --2- 172.21.15.100:0/2835247765 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f484c0683b0 0x7f484c06a870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4860007bd0 tx=0x7f4860009040 comp rx=0 tx=0).stop 2024-04-23T14:32:57.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.262+0000 7f4874000640 1 -- 172.21.15.100:0/2835247765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f486c0b0860 msgr2=0x7f486c15fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:57.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.262+0000 7f4874000640 1 --2- 172.21.15.100:0/2835247765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f486c0b0860 0x7f486c15fc90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4864097d30 tx=0x7f4864009750 comp rx=0 tx=0).stop 2024-04-23T14:32:57.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.262+0000 7f4874000640 1 -- 172.21.15.100:0/2835247765 shutdown_connections 2024-04-23T14:32:57.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.262+0000 7f4874000640 1 --2- 172.21.15.100:0/2835247765 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f484c0683b0 0x7f484c06a870 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.262+0000 7f4874000640 1 --2- 172.21.15.100:0/2835247765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f486c0b0860 0x7f486c15fc90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.262+0000 7f4874000640 1 --2- 172.21.15.100:0/2835247765 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f486c0aefa0 0x7f486c0b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.262+0000 7f4874000640 1 --2- 172.21.15.100:0/2835247765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4854005090 0x7f486c13cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.262+0000 7f4874000640 1 -- 172.21.15.100:0/2835247765 >> 172.21.15.100:0/2835247765 conn(0x7f486c072590 msgr2=0x7f486c160ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:57.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.262+0000 7f4874000640 1 -- 172.21.15.100:0/2835247765 shutdown_connections 2024-04-23T14:32:57.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.262+0000 7f4874000640 1 -- 172.21.15.100:0/2835247765 wait complete. 2024-04-23T14:32:57.282 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:32:57.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.418+0000 7f87af000640 1 Processor -- start 2024-04-23T14:32:57.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.418+0000 7f87af000640 1 -- start start 2024-04-23T14:32:57.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.418+0000 7f87af000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f87a806bb20 0x7f87a80ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:57.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.418+0000 7f87af000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f87a80cd130 0x7f87a80ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:57.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.418+0000 7f87af000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87a80cd540 con 0x7f87a815c7a0 2024-04-23T14:32:57.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.418+0000 7f87af000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f87a806d450 con 0x7f87a806bb20 2024-04-23T14:32:57.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.418+0000 7f87af000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f87a806cbb0 con 0x7f87a80cd130 2024-04-23T14:32:57.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.418+0000 7f879fe00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f87a806bb20 0x7f87a80ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:57.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.418+0000 7f879fe00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f87a806bb20 0x7f87a80ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50876/0 (socket says 172.21.15.100:50876) 2024-04-23T14:32:57.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.418+0000 7f879fe00640 1 -- 172.21.15.100:0/155440589 learned_addr learned my addr 172.21.15.100:0/155440589 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:57.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.422+0000 7f87ad000640 1 --2- 172.21.15.100:0/155440589 >> v2:172.21.15.151:3300/0 conn(0x7f87a80cd130 0x7f87a80ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:57.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.422+0000 7f879fe00640 1 -- 172.21.15.100:0/155440589 >> v2:172.21.15.151:3300/0 conn(0x7f87a80cd130 msgr2=0x7f87a80ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:57.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.422+0000 7f879fe00640 1 --2- 172.21.15.100:0/155440589 >> v2:172.21.15.151:3300/0 conn(0x7f87a80cd130 0x7f87a80ad430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.422+0000 7f879fe00640 1 -- 172.21.15.100:0/155440589 >> v1:172.21.15.151:6789/0 conn(0x7f87a815c7a0 legacy=0x7f87a80af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:57.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.422+0000 7f879fe00640 1 -- 172.21.15.100:0/155440589 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a806ce90 con 0x7f87a806bb20 2024-04-23T14:32:57.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.422+0000 7f879fe00640 1 --2- 172.21.15.100:0/155440589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87a806bb20 0x7f87a80ccbf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f87a80cd8f0 tx=0x7f879001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e78020017cc6f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:57.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.422+0000 7f87a7400640 1 -- 172.21.15.100:0/155440589 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87900b0070 con 0x7f87a806bb20 2024-04-23T14:32:57.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.422+0000 7f87a7400640 1 -- 172.21.15.100:0/155440589 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87900ab600 con 0x7f87a806bb20 2024-04-23T14:32:57.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.422+0000 7f87a7400640 1 -- 172.21.15.100:0/155440589 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87900ab9d0 con 0x7f87a806bb20 2024-04-23T14:32:57.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.422+0000 7f87af000640 1 -- 172.21.15.100:0/155440589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87a806bb20 msgr2=0x7f87a80ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:57.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.422+0000 7f87af000640 1 --2- 172.21.15.100:0/155440589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87a806bb20 0x7f87a80ccbf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f87a80cd8f0 tx=0x7f879001c490 comp rx=0 tx=0).stop 2024-04-23T14:32:57.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f87af000640 1 -- 172.21.15.100:0/155440589 shutdown_connections 2024-04-23T14:32:57.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f87af000640 1 --2- 172.21.15.100:0/155440589 >> v2:172.21.15.151:3300/0 conn(0x7f87a80cd130 0x7f87a80ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f87af000640 1 --2- 172.21.15.100:0/155440589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87a806bb20 0x7f87a80ccbf0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f87af000640 1 -- 172.21.15.100:0/155440589 >> 172.21.15.100:0/155440589 conn(0x7f87a80b5200 msgr2=0x7f87a80b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:57.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f87af000640 1 -- 172.21.15.100:0/155440589 shutdown_connections 2024-04-23T14:32:57.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f87af000640 1 -- 172.21.15.100:0/155440589 wait complete. 2024-04-23T14:32:57.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f87af000640 1 Processor -- start 2024-04-23T14:32:57.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f87af000640 1 -- start start 2024-04-23T14:32:57.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f87af000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f87a806bb20 0x7f87a80c3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:57.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f87af000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87a80cd130 0x7f87a81577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:57.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f87af000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87a815c7a0 0x7f87a8096980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:57.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f87af000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f87a80ae550 con 0x7f87a80cd130 2024-04-23T14:32:57.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f87af000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f87a80ae3d0 con 0x7f87a815c7a0 2024-04-23T14:32:57.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f87af000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f87a80ae6d0 con 0x7f87a806bb20 2024-04-23T14:32:57.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f879fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87a80cd130 0x7f87a81577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:57.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f879fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87a80cd130 0x7f87a81577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50892/0 (socket says 172.21.15.100:50892) 2024-04-23T14:32:57.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f879fe00640 1 -- 172.21.15.100:0/3768954561 learned_addr learned my addr 172.21.15.100:0/3768954561 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:57.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f87a7e00640 1 --2- 172.21.15.100:0/3768954561 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f87a806bb20 0x7f87a80c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:57.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f87ad000640 1 --2- 172.21.15.100:0/3768954561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87a815c7a0 0x7f87a8096980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:57.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f879fe00640 1 -- 172.21.15.100:0/3768954561 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f87a806bb20 msgr2=0x7f87a80c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:57.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f879fe00640 1 --2- 172.21.15.100:0/3768954561 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f87a806bb20 0x7f87a80c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f879fe00640 1 -- 172.21.15.100:0/3768954561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87a815c7a0 msgr2=0x7f87a8096980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:57.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f879fe00640 1 --2- 172.21.15.100:0/3768954561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87a815c7a0 0x7f87a8096980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f879fe00640 1 -- 172.21.15.100:0/3768954561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a8073ee0 con 0x7f87a80cd130 2024-04-23T14:32:57.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f87a7e00640 1 --2- 172.21.15.100:0/3768954561 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f87a806bb20 0x7f87a80c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:57.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f879fe00640 1 --2- 172.21.15.100:0/3768954561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87a80cd130 0x7f87a81577c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f87900ae040 tx=0x7f8790019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:57.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f87a4c00640 1 -- 172.21.15.100:0/3768954561 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87900b0040 con 0x7f87a80cd130 2024-04-23T14:32:57.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f87a4c00640 1 -- 172.21.15.100:0/3768954561 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87900b9070 con 0x7f87a80cd130 2024-04-23T14:32:57.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.426+0000 7f87af000640 1 -- 172.21.15.100:0/3768954561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a8156ce0 con 0x7f87a80cd130 2024-04-23T14:32:57.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.430+0000 7f87a4c00640 1 -- 172.21.15.100:0/3768954561 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87900b38a0 con 0x7f87a80cd130 2024-04-23T14:32:57.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.430+0000 7f87af000640 1 -- 172.21.15.100:0/3768954561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a8157d00 con 0x7f87a80cd130 2024-04-23T14:32:57.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.430+0000 7f87af000640 1 -- 172.21.15.100:0/3768954561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a8156ae0 con 0x7f87a80cd130 2024-04-23T14:32:57.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.430+0000 7f87a4c00640 1 -- 172.21.15.100:0/3768954561 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f87900b3a40 con 0x7f87a80cd130 2024-04-23T14:32:57.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.430+0000 7f87a4c00640 1 --2- 172.21.15.100:0/3768954561 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f87840683b0 0x7f878406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:57.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.430+0000 7f87a4c00640 1 -- 172.21.15.100:0/3768954561 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f8790120d20 con 0x7f87a80cd130 2024-04-23T14:32:57.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.430+0000 7f87a7e00640 1 --2- 172.21.15.100:0/3768954561 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f87840683b0 0x7f878406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:57.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.430+0000 7f87a7e00640 1 --2- 172.21.15.100:0/3768954561 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f87840683b0 0x7f878406a870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8798001710 tx=0x7f8798009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:57.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.434+0000 7f87a4c00640 1 -- 172.21.15.100:0/3768954561 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8790016110 con 0x7f87a80cd130 2024-04-23T14:32:57.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.590+0000 7f87af000640 1 -- 172.21.15.100:0/3768954561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f87a8068ad0 con 0x7f87a80cd130 2024-04-23T14:32:57.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.594+0000 7f87a4c00640 1 -- 172.21.15.100:0/3768954561 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f87900f0e40 con 0x7f87a80cd130 2024-04-23T14:32:57.594 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:32:57.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.594+0000 7f87af000640 1 -- 172.21.15.100:0/3768954561 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f87840683b0 msgr2=0x7f878406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:57.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.594+0000 7f87af000640 1 --2- 172.21.15.100:0/3768954561 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f87840683b0 0x7f878406a870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8798001710 tx=0x7f8798009040 comp rx=0 tx=0).stop 2024-04-23T14:32:57.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.594+0000 7f87af000640 1 -- 172.21.15.100:0/3768954561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87a80cd130 msgr2=0x7f87a81577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:57.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.594+0000 7f87af000640 1 --2- 172.21.15.100:0/3768954561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87a80cd130 0x7f87a81577c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f87900ae040 tx=0x7f8790019230 comp rx=0 tx=0).stop 2024-04-23T14:32:57.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.594+0000 7f87af000640 1 -- 172.21.15.100:0/3768954561 shutdown_connections 2024-04-23T14:32:57.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.594+0000 7f87af000640 1 --2- 172.21.15.100:0/3768954561 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f87840683b0 0x7f878406a870 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.594+0000 7f87af000640 1 --2- 172.21.15.100:0/3768954561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87a815c7a0 0x7f87a8096980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.594+0000 7f87af000640 1 --2- 172.21.15.100:0/3768954561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87a80cd130 0x7f87a81577c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.594+0000 7f87af000640 1 --2- 172.21.15.100:0/3768954561 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f87a806bb20 0x7f87a80c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.594+0000 7f87af000640 1 -- 172.21.15.100:0/3768954561 >> 172.21.15.100:0/3768954561 conn(0x7f87a80b5200 msgr2=0x7f87a80cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:57.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.594+0000 7f87af000640 1 -- 172.21.15.100:0/3768954561 shutdown_connections 2024-04-23T14:32:57.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.594+0000 7f87af000640 1 -- 172.21.15.100:0/3768954561 wait complete. 2024-04-23T14:32:57.616 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-23T14:32:57.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.754+0000 7f2610e00640 1 Processor -- start 2024-04-23T14:32:57.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.754+0000 7f2610e00640 1 -- start start 2024-04-23T14:32:57.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.754+0000 7f2610e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f260c06bb20 0x7f260c0ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:57.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.754+0000 7f2610e00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f260c0cd540 con 0x7f260c15c7a0 2024-04-23T14:32:57.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.754+0000 7f2610e00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f260c06cbb0 con 0x7f260c0cd130 2024-04-23T14:32:57.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.754+0000 7f2610e00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f260c06d450 con 0x7f260c06bb20 2024-04-23T14:32:57.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.758+0000 7f2609e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f260c15c7a0 0x7f260c0af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42716/0 (socket says 172.21.15.100:42716) 2024-04-23T14:32:57.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.758+0000 7f2609400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f260c06bb20 0x7f260c0ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:57.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.758+0000 7f2609e00640 1 -- 172.21.15.100:0/631811148 learned_addr learned my addr 172.21.15.100:0/631811148 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:57.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.758+0000 7f2609400640 1 -- 172.21.15.100:0/631811148 >> v1:172.21.15.151:6789/0 conn(0x7f260c0cd130 legacy=0x7f260c0ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:57.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.758+0000 7f2609400640 1 -- 172.21.15.100:0/631811148 >> v1:172.21.15.100:6789/0 conn(0x7f260c15c7a0 legacy=0x7f260c0af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:57.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.758+0000 7f2609400640 1 -- 172.21.15.100:0/631811148 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260c06ce90 con 0x7f260c06bb20 2024-04-23T14:32:57.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.758+0000 7f2608a00640 1 -- 172.21.15.100:0/631811148 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1571315799 0 0) 0x7f25fc0014a0 con 0x7f260c15c7a0 2024-04-23T14:32:57.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.758+0000 7f2609400640 1 --2- 172.21.15.100:0/631811148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260c06bb20 0x7f260c0ccbf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f25f80097c0 tx=0x7f25f801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54aa3c1615f5fdcd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:57.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.758+0000 7f2608a00640 1 -- 172.21.15.100:0/631811148 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f80b0070 con 0x7f260c06bb20 2024-04-23T14:32:57.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.758+0000 7f2608a00640 1 -- 172.21.15.100:0/631811148 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f80ab600 con 0x7f260c06bb20 2024-04-23T14:32:57.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.758+0000 7f2608a00640 1 -- 172.21.15.100:0/631811148 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f80a4e60 con 0x7f260c06bb20 2024-04-23T14:32:57.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2610e00640 1 -- 172.21.15.100:0/631811148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260c06bb20 msgr2=0x7f260c0ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:57.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2610e00640 1 --2- 172.21.15.100:0/631811148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260c06bb20 0x7f260c0ccbf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f25f80097c0 tx=0x7f25f801c3d0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2610e00640 1 -- 172.21.15.100:0/631811148 shutdown_connections 2024-04-23T14:32:57.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2610e00640 1 --2- 172.21.15.100:0/631811148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260c06bb20 0x7f260c0ccbf0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2610e00640 1 -- 172.21.15.100:0/631811148 >> 172.21.15.100:0/631811148 conn(0x7f260c0b5200 msgr2=0x7f260c0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:57.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2610e00640 1 -- 172.21.15.100:0/631811148 shutdown_connections 2024-04-23T14:32:57.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2610e00640 1 -- 172.21.15.100:0/631811148 wait complete. 2024-04-23T14:32:57.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2610e00640 1 Processor -- start 2024-04-23T14:32:57.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2610e00640 1 -- start start 2024-04-23T14:32:57.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2610e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f260c06bb20 0x7f260c158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:57.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2610e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260c0cd130 0x7f260c073ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:57.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2610e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f260c15c7a0 0x7f260c14f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:57.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2610e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f260c0ae720 con 0x7f260c0cd130 2024-04-23T14:32:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2610e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f260c0ae5a0 con 0x7f260c06bb20 2024-04-23T14:32:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2610e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f260c16a210 con 0x7f260c15c7a0 2024-04-23T14:32:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2609400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260c0cd130 0x7f260c073ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2609400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260c0cd130 0x7f260c073ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50916/0 (socket says 172.21.15.100:50916) 2024-04-23T14:32:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2609400640 1 -- 172.21.15.100:0/3210274601 learned_addr learned my addr 172.21.15.100:0/3210274601 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f260a800640 1 --2- 172.21.15.100:0/3210274601 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f260c15c7a0 0x7f260c14f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2609e00640 1 --2- 172.21.15.100:0/3210274601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f260c06bb20 0x7f260c158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2609400640 1 -- 172.21.15.100:0/3210274601 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f260c15c7a0 msgr2=0x7f260c14f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2609400640 1 --2- 172.21.15.100:0/3210274601 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f260c15c7a0 0x7f260c14f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2609400640 1 -- 172.21.15.100:0/3210274601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f260c06bb20 msgr2=0x7f260c158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2609400640 1 --2- 172.21.15.100:0/3210274601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f260c06bb20 0x7f260c158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f2609400640 1 -- 172.21.15.100:0/3210274601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260c0cc810 con 0x7f260c0cd130 2024-04-23T14:32:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.762+0000 7f260a800640 1 --2- 172.21.15.100:0/3210274601 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f260c15c7a0 0x7f260c14f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.766+0000 7f2609400640 1 --2- 172.21.15.100:0/3210274601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260c0cd130 0x7f260c073ac0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f25f8018fe0 tx=0x7f25f8019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.766+0000 7f2602000640 1 -- 172.21.15.100:0/3210274601 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f80b8070 con 0x7f260c0cd130 2024-04-23T14:32:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.766+0000 7f2602000640 1 -- 172.21.15.100:0/3210274601 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f80a4e60 con 0x7f260c0cd130 2024-04-23T14:32:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.766+0000 7f2610e00640 1 -- 172.21.15.100:0/3210274601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f260c0cbc90 con 0x7f260c0cd130 2024-04-23T14:32:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.766+0000 7f2602000640 1 -- 172.21.15.100:0/3210274601 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f80b0040 con 0x7f260c0cd130 2024-04-23T14:32:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.766+0000 7f2610e00640 1 -- 172.21.15.100:0/3210274601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f260c0cc250 con 0x7f260c0cd130 2024-04-23T14:32:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.766+0000 7f2610e00640 1 -- 172.21.15.100:0/3210274601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f260c068ad0 con 0x7f260c0cd130 2024-04-23T14:32:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.766+0000 7f2602000640 1 -- 172.21.15.100:0/3210274601 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f25f80bb070 con 0x7f260c0cd130 2024-04-23T14:32:57.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.766+0000 7f2602000640 1 --2- 172.21.15.100:0/3210274601 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f25ec068380 0x7f25ec06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:57.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.766+0000 7f2602000640 1 -- 172.21.15.100:0/3210274601 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f25f8121150 con 0x7f260c0cd130 2024-04-23T14:32:57.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.766+0000 7f2609e00640 1 --2- 172.21.15.100:0/3210274601 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f25ec068380 0x7f25ec06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:57.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.766+0000 7f2609e00640 1 --2- 172.21.15.100:0/3210274601 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f25ec068380 0x7f25ec06a840 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0012f0 tx=0x7f25fc005ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:57.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.770+0000 7f2602000640 1 -- 172.21.15.100:0/3210274601 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f25f8016050 con 0x7f260c0cd130 2024-04-23T14:32:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.930+0000 7f2610e00640 1 -- 172.21.15.100:0/3210274601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f260c0d0a70 con 0x7f260c0cd130 2024-04-23T14:32:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.930+0000 7f2602000640 1 -- 172.21.15.100:0/3210274601 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f25f80f1270 con 0x7f260c0cd130 2024-04-23T14:32:57.933 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:32:57.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.934+0000 7f2610e00640 1 -- 172.21.15.100:0/3210274601 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f25ec068380 msgr2=0x7f25ec06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:57.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.934+0000 7f2610e00640 1 --2- 172.21.15.100:0/3210274601 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f25ec068380 0x7f25ec06a840 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0012f0 tx=0x7f25fc005ec0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.934+0000 7f2610e00640 1 -- 172.21.15.100:0/3210274601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260c0cd130 msgr2=0x7f260c073ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:57.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.934+0000 7f2610e00640 1 --2- 172.21.15.100:0/3210274601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260c0cd130 0x7f260c073ac0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f25f8018fe0 tx=0x7f25f8019170 comp rx=0 tx=0).stop 2024-04-23T14:32:57.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.934+0000 7f2610e00640 1 -- 172.21.15.100:0/3210274601 shutdown_connections 2024-04-23T14:32:57.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.934+0000 7f2610e00640 1 --2- 172.21.15.100:0/3210274601 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f25ec068380 0x7f25ec06a840 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.934+0000 7f2610e00640 1 --2- 172.21.15.100:0/3210274601 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f260c15c7a0 0x7f260c14f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.934+0000 7f2610e00640 1 --2- 172.21.15.100:0/3210274601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260c0cd130 0x7f260c073ac0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.934+0000 7f2610e00640 1 --2- 172.21.15.100:0/3210274601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f260c06bb20 0x7f260c158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:57.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.934+0000 7f2610e00640 1 -- 172.21.15.100:0/3210274601 >> 172.21.15.100:0/3210274601 conn(0x7f260c0b5200 msgr2=0x7f260c0b55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:57.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.934+0000 7f2610e00640 1 -- 172.21.15.100:0/3210274601 shutdown_connections 2024-04-23T14:32:57.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:57.934+0000 7f2610e00640 1 -- 172.21.15.100:0/3210274601 wait complete. 2024-04-23T14:32:57.953 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:58.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.086+0000 7f1327a00640 1 Processor -- start 2024-04-23T14:32:58.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.086+0000 7f1327a00640 1 -- start start 2024-04-23T14:32:58.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.086+0000 7f1327a00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f13200ad870 0x7f13200b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:58.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.086+0000 7f1327a00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1320160320 con 0x7f132015c470 2024-04-23T14:32:58.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.086+0000 7f1327a00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13200cc7e0 con 0x7f1320160de0 2024-04-23T14:32:58.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.086+0000 7f1327a00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f13201605e0 con 0x7f13200ad870 2024-04-23T14:32:58.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.086+0000 7f1325000640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f132015c470 0x7f13200ad160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42722/0 (socket says 172.21.15.100:42722) 2024-04-23T14:32:58.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.086+0000 7f1325000640 1 -- 172.21.15.100:0/821855936 learned_addr learned my addr 172.21.15.100:0/821855936 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:58.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.090+0000 7f131fe00640 1 --2- 172.21.15.100:0/821855936 >> v2:172.21.15.151:3300/0 conn(0x7f13200ad870 0x7f13200b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:58.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.090+0000 7f131f400640 1 -- 172.21.15.100:0/821855936 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2242474019 0 0) 0x7f1320160320 con 0x7f132015c470 2024-04-23T14:32:58.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.090+0000 7f131f400640 1 -- 172.21.15.100:0/821855936 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1308003530 con 0x7f132015c470 2024-04-23T14:32:58.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.090+0000 7f131f400640 1 -- 172.21.15.100:0/821855936 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 100335596 0 0) 0x7f1308003530 con 0x7f132015c470 2024-04-23T14:32:58.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.090+0000 7f131fe00640 1 -- 172.21.15.100:0/821855936 >> v1:172.21.15.151:6789/0 conn(0x7f1320160de0 legacy=0x7f132006bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:58.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.090+0000 7f131fe00640 1 -- 172.21.15.100:0/821855936 >> v1:172.21.15.100:6789/0 conn(0x7f132015c470 legacy=0x7f13200ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:58.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.090+0000 7f131fe00640 1 -- 172.21.15.100:0/821855936 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13201608c0 con 0x7f13200ad870 2024-04-23T14:32:58.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.090+0000 7f131fe00640 1 --2- 172.21.15.100:0/821855936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13200ad870 0x7f13200b29a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f130c0097c0 tx=0x7f130c01c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92c3f9c5b578e753 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:58.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.090+0000 7f131f400640 1 -- 172.21.15.100:0/821855936 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c0b0070 con 0x7f13200ad870 2024-04-23T14:32:58.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.090+0000 7f131f400640 1 -- 172.21.15.100:0/821855936 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f130c0a4e50 con 0x7f13200ad870 2024-04-23T14:32:58.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.094+0000 7f131f400640 1 -- 172.21.15.100:0/821855936 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314001680 con 0x7f13200ad870 2024-04-23T14:32:58.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.094+0000 7f1327a00640 1 -- 172.21.15.100:0/821855936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13200ad870 msgr2=0x7f13200b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:58.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.094+0000 7f1327a00640 1 --2- 172.21.15.100:0/821855936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13200ad870 0x7f13200b29a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f130c0097c0 tx=0x7f130c01c330 comp rx=0 tx=0).stop 2024-04-23T14:32:58.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.094+0000 7f1327a00640 1 -- 172.21.15.100:0/821855936 shutdown_connections 2024-04-23T14:32:58.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.094+0000 7f1327a00640 1 --2- 172.21.15.100:0/821855936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13200ad870 0x7f13200b29a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.094+0000 7f1327a00640 1 -- 172.21.15.100:0/821855936 >> 172.21.15.100:0/821855936 conn(0x7f13200b1e80 msgr2=0x7f13200b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:58.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.094+0000 7f1327a00640 1 -- 172.21.15.100:0/821855936 shutdown_connections 2024-04-23T14:32:58.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.094+0000 7f1327a00640 1 -- 172.21.15.100:0/821855936 wait complete. 2024-04-23T14:32:58.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.094+0000 7f1327a00640 1 Processor -- start 2024-04-23T14:32:58.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.094+0000 7f1327a00640 1 -- start start 2024-04-23T14:32:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.094+0000 7f1327a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13200ad870 0x7f1320099910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.094+0000 7f1327a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f132015c470 0x7f13200726a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.094+0000 7f1327a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1320160de0 0x7f1320153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.094+0000 7f1327a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f132015ca60 con 0x7f1320160de0 2024-04-23T14:32:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.094+0000 7f1327a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13200cc5c0 con 0x7f13200ad870 2024-04-23T14:32:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.094+0000 7f1327a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f132016a550 con 0x7f132015c470 2024-04-23T14:32:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.094+0000 7f1325a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1320160de0 0x7f1320153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.094+0000 7f1325a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1320160de0 0x7f1320153170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50922/0 (socket says 172.21.15.100:50922) 2024-04-23T14:32:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.094+0000 7f1325a00640 1 -- 172.21.15.100:0/3786287242 learned_addr learned my addr 172.21.15.100:0/3786287242 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:58.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.098+0000 7f131fe00640 1 --2- 172.21.15.100:0/3786287242 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f132015c470 0x7f13200726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:58.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.098+0000 7f1325000640 1 --2- 172.21.15.100:0/3786287242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13200ad870 0x7f1320099910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:58.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.098+0000 7f1325a00640 1 -- 172.21.15.100:0/3786287242 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f132015c470 msgr2=0x7f13200726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:58.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.098+0000 7f1325a00640 1 --2- 172.21.15.100:0/3786287242 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f132015c470 0x7f13200726a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.098+0000 7f1325a00640 1 -- 172.21.15.100:0/3786287242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13200ad870 msgr2=0x7f1320099910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:58.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.098+0000 7f1325a00640 1 --2- 172.21.15.100:0/3786287242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13200ad870 0x7f1320099910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.098+0000 7f1325a00640 1 -- 172.21.15.100:0/3786287242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1320072be0 con 0x7f1320160de0 2024-04-23T14:32:58.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.098+0000 7f1325000640 1 --2- 172.21.15.100:0/3786287242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13200ad870 0x7f1320099910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:58.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.098+0000 7f1325a00640 1 --2- 172.21.15.100:0/3786287242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1320160de0 0x7f1320153170 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f131800c730 tx=0x7f131800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:58.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.098+0000 7f131cc00640 1 -- 172.21.15.100:0/3786287242 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1318003b80 con 0x7f1320160de0 2024-04-23T14:32:58.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.098+0000 7f131cc00640 1 -- 172.21.15.100:0/3786287242 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1318003d20 con 0x7f1320160de0 2024-04-23T14:32:58.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.098+0000 7f131cc00640 1 -- 172.21.15.100:0/3786287242 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1318005310 con 0x7f1320160de0 2024-04-23T14:32:58.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.098+0000 7f1327a00640 1 -- 172.21.15.100:0/3786287242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13201f8860 con 0x7f1320160de0 2024-04-23T14:32:58.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.098+0000 7f1327a00640 1 -- 172.21.15.100:0/3786287242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1320073140 con 0x7f1320160de0 2024-04-23T14:32:58.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.098+0000 7f1327a00640 1 -- 172.21.15.100:0/3786287242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1320068ad0 con 0x7f1320160de0 2024-04-23T14:32:58.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.098+0000 7f131cc00640 1 -- 172.21.15.100:0/3786287242 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f13180056c0 con 0x7f1320160de0 2024-04-23T14:32:58.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.098+0000 7f131cc00640 1 --2- 172.21.15.100:0/3786287242 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1300068380 0x7f130006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:58.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.098+0000 7f131cc00640 1 -- 172.21.15.100:0/3786287242 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1318086e10 con 0x7f1320160de0 2024-04-23T14:32:58.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.098+0000 7f1325000640 1 --2- 172.21.15.100:0/3786287242 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1300068380 0x7f130006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:58.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.102+0000 7f1325000640 1 --2- 172.21.15.100:0/3786287242 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1300068380 0x7f130006a840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f13140011f0 tx=0x7f1314009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:58.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.102+0000 7f131cc00640 1 -- 172.21.15.100:0/3786287242 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1320072be0 con 0x7f1320160de0 2024-04-23T14:32:58.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.262+0000 7f1327a00640 1 -- 172.21.15.100:0/3786287242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1320073cd0 con 0x7f1320160de0 2024-04-23T14:32:58.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.262+0000 7f131cc00640 1 -- 172.21.15.100:0/3786287242 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18859 (secure 0 0 0) 0x7f1318056fd0 con 0x7f1320160de0 2024-04-23T14:32:58.265 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:58.265 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:49.784608+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:58.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.266+0000 7f1327a00640 1 -- 172.21.15.100:0/3786287242 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1300068380 msgr2=0x7f130006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:58.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.266+0000 7f1327a00640 1 --2- 172.21.15.100:0/3786287242 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1300068380 0x7f130006a840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f13140011f0 tx=0x7f1314009040 comp rx=0 tx=0).stop 2024-04-23T14:32:58.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.266+0000 7f1327a00640 1 -- 172.21.15.100:0/3786287242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1320160de0 msgr2=0x7f1320153170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:58.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.266+0000 7f1327a00640 1 --2- 172.21.15.100:0/3786287242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1320160de0 0x7f1320153170 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f131800c730 tx=0x7f131800cbf0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.266+0000 7f1327a00640 1 -- 172.21.15.100:0/3786287242 shutdown_connections 2024-04-23T14:32:58.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.266+0000 7f1327a00640 1 --2- 172.21.15.100:0/3786287242 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1300068380 0x7f130006a840 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.266+0000 7f1327a00640 1 --2- 172.21.15.100:0/3786287242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1320160de0 0x7f1320153170 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.266+0000 7f1327a00640 1 --2- 172.21.15.100:0/3786287242 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f132015c470 0x7f13200726a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.266+0000 7f1327a00640 1 --2- 172.21.15.100:0/3786287242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13200ad870 0x7f1320099910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.266+0000 7f1327a00640 1 -- 172.21.15.100:0/3786287242 >> 172.21.15.100:0/3786287242 conn(0x7f13200b1e80 msgr2=0x7f13200b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:58.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.266+0000 7f1327a00640 1 -- 172.21.15.100:0/3786287242 shutdown_connections 2024-04-23T14:32:58.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.266+0000 7f1327a00640 1 -- 172.21.15.100:0/3786287242 wait complete. 2024-04-23T14:32:58.287 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:32:06.835560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:32:07.930489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-23T14:32:11.636814+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-23T14:32:58.287 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:58.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.422+0000 7ff10a400640 1 Processor -- start 2024-04-23T14:32:58.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.422+0000 7ff10a400640 1 -- start start 2024-04-23T14:32:58.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff10a400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff10415c340 0x7ff10406dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:58.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff10a400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff10406bcc0 con 0x7ff10406d190 2024-04-23T14:32:58.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff10a400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff10406bfa0 con 0x7ff10406e110 2024-04-23T14:32:58.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff10a400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff10406c5f0 con 0x7ff10415c340 2024-04-23T14:32:58.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff109a00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff10406d190 0x7ff1040acfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42736/0 (socket says 172.21.15.100:42736) 2024-04-23T14:32:58.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff109a00640 1 -- 172.21.15.100:0/1333073372 learned_addr learned my addr 172.21.15.100:0/1333073372 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:58.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff109000640 1 --2- 172.21.15.100:0/1333073372 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff10415c340 0x7ff10406dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:58.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff103400640 1 -- 172.21.15.100:0/1333073372 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3591741203 0 0) 0x7ff10406bcc0 con 0x7ff10406d190 2024-04-23T14:32:58.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff103400640 1 -- 172.21.15.100:0/1333073372 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0e8003530 con 0x7ff10406d190 2024-04-23T14:32:58.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff103400640 1 -- 172.21.15.100:0/1333073372 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1967451475 0 0) 0x7ff0e8003530 con 0x7ff10406d190 2024-04-23T14:32:58.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff103400640 1 -- 172.21.15.100:0/1333073372 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff10415c340 msgr2=0x7ff10406dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:58.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff103400640 1 --2- 172.21.15.100:0/1333073372 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff10415c340 0x7ff10406dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff103400640 1 -- 172.21.15.100:0/1333073372 >> v1:172.21.15.151:6789/0 conn(0x7ff10406e110 legacy=0x7ff10406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:58.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff103400640 1 -- 172.21.15.100:0/1333073372 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10406c280 con 0x7ff10406d190 2024-04-23T14:32:58.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff103400640 1 -- 172.21.15.100:0/1333073372 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7ff0fc001680 con 0x7ff10406d190 2024-04-23T14:32:58.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff103400640 1 -- 172.21.15.100:0/1333073372 >> v1:172.21.15.100:6789/0 conn(0x7ff10406d190 legacy=0x7ff1040acfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:58.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff103400640 1 --2- 172.21.15.100:0/1333073372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0e80039f0 0x7ff0e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:58.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff103400640 1 --2- 172.21.15.100:0/1333073372 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff10415c340 0x7ff0e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:58.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff103400640 1 --2- 172.21.15.100:0/1333073372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0e8004d50 0x7ff0e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:58.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff103400640 1 -- 172.21.15.100:0/1333073372 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff10406c5f0 con 0x7ff0e80039f0 2024-04-23T14:32:58.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff103400640 1 -- 172.21.15.100:0/1333073372 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e800b9d0 con 0x7ff0e8004d50 2024-04-23T14:32:58.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff103400640 1 -- 172.21.15.100:0/1333073372 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff0e800bbd0 con 0x7ff10415c340 2024-04-23T14:32:58.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff109000640 1 --2- 172.21.15.100:0/1333073372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0e80039f0 0x7ff0e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:58.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff103e00640 1 --2- 172.21.15.100:0/1333073372 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff10415c340 0x7ff0e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:58.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff109a00640 1 --2- 172.21.15.100:0/1333073372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0e8004d50 0x7ff0e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:58.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff109000640 1 -- 172.21.15.100:0/1333073372 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff10415c340 msgr2=0x7ff0e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:58.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff109000640 1 --2- 172.21.15.100:0/1333073372 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff10415c340 0x7ff0e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff109000640 1 -- 172.21.15.100:0/1333073372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0e8004d50 msgr2=0x7ff0e80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:58.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff109000640 1 --2- 172.21.15.100:0/1333073372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0e8004d50 0x7ff0e80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff109000640 1 -- 172.21.15.100:0/1333073372 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e8003530 con 0x7ff0e80039f0 2024-04-23T14:32:58.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.426+0000 7ff103e00640 1 --2- 172.21.15.100:0/1333073372 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff10415c340 0x7ff0e8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:58.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.430+0000 7ff109000640 1 --2- 172.21.15.100:0/1333073372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0e80039f0 0x7ff0e8003dd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009fc0 tx=0x7ff0f401cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d544f799f0cd431 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:58.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.430+0000 7ff103400640 1 -- 172.21.15.100:0/1333073372 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f40b2070 con 0x7ff0e80039f0 2024-04-23T14:32:58.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.430+0000 7ff103400640 1 -- 172.21.15.100:0/1333073372 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f40a5bd0 con 0x7ff0e80039f0 2024-04-23T14:32:58.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.430+0000 7ff103400640 1 -- 172.21.15.100:0/1333073372 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4009c10 con 0x7ff0e80039f0 2024-04-23T14:32:58.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.430+0000 7ff10a400640 1 -- 172.21.15.100:0/1333073372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0e80039f0 msgr2=0x7ff0e8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:58.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.430+0000 7ff10a400640 1 --2- 172.21.15.100:0/1333073372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0e80039f0 0x7ff0e8003dd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009fc0 tx=0x7ff0f401cbc0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.430+0000 7ff109a00640 1 -- 172.21.15.100:0/1333073372 reap_dead start 2024-04-23T14:32:58.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.430+0000 7ff10a400640 1 -- 172.21.15.100:0/1333073372 shutdown_connections 2024-04-23T14:32:58.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.430+0000 7ff10a400640 1 -- 172.21.15.100:0/1333073372 >> 172.21.15.100:0/1333073372 conn(0x7ff1040b1d00 msgr2=0x7ff1040b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:58.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.430+0000 7ff10a400640 1 -- 172.21.15.100:0/1333073372 shutdown_connections 2024-04-23T14:32:58.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.430+0000 7ff10a400640 1 -- 172.21.15.100:0/1333073372 wait complete. 2024-04-23T14:32:58.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.430+0000 7ff10a400640 1 Processor -- start 2024-04-23T14:32:58.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.430+0000 7ff10a400640 1 -- start start 2024-04-23T14:32:58.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff10a400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0e80039f0 0x7ff104157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:58.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff10a400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1041534a0 0x7ff1041457f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:58.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff10a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1041f3940 0x7ff10406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:58.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff10a400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff1040ad870 con 0x7ff1041f3940 2024-04-23T14:32:58.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff10a400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff1040ad6f0 con 0x7ff1041534a0 2024-04-23T14:32:58.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff10a400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff10406dbd0 con 0x7ff0e80039f0 2024-04-23T14:32:58.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff109a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1041f3940 0x7ff10406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:58.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff103e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1041534a0 0x7ff1041457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:58.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff109a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1041f3940 0x7ff10406f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50948/0 (socket says 172.21.15.100:50948) 2024-04-23T14:32:58.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff109a00640 1 -- 172.21.15.100:0/533582574 learned_addr learned my addr 172.21.15.100:0/533582574 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:58.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff109000640 1 --2- 172.21.15.100:0/533582574 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0e80039f0 0x7ff104157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff109a00640 1 -- 172.21.15.100:0/533582574 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0e80039f0 msgr2=0x7ff104157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff109a00640 1 --2- 172.21.15.100:0/533582574 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0e80039f0 0x7ff104157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff109a00640 1 -- 172.21.15.100:0/533582574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1041534a0 msgr2=0x7ff1041457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff109a00640 1 --2- 172.21.15.100:0/533582574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1041534a0 0x7ff1041457f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff109a00640 1 -- 172.21.15.100:0/533582574 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff104153b30 con 0x7ff1041f3940 2024-04-23T14:32:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff103e00640 1 --2- 172.21.15.100:0/533582574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1041534a0 0x7ff1041457f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff109a00640 1 --2- 172.21.15.100:0/533582574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1041f3940 0x7ff10406f080 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff10406fca0 tx=0x7ff0fc00e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff100c00640 1 -- 172.21.15.100:0/533582574 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc019070 con 0x7ff1041f3940 2024-04-23T14:32:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff100c00640 1 -- 172.21.15.100:0/533582574 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0fc010510 con 0x7ff1041f3940 2024-04-23T14:32:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff100c00640 1 -- 172.21.15.100:0/533582574 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc010900 con 0x7ff1041f3940 2024-04-23T14:32:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff10a400640 1 -- 172.21.15.100:0/533582574 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff104153e10 con 0x7ff1041f3940 2024-04-23T14:32:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff10a400640 1 -- 172.21.15.100:0/533582574 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1041543d0 con 0x7ff1041f3940 2024-04-23T14:32:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff10a400640 1 -- 172.21.15.100:0/533582574 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff104068ad0 con 0x7ff1041f3940 2024-04-23T14:32:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.434+0000 7ff100c00640 1 -- 172.21.15.100:0/533582574 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff0fc01c070 con 0x7ff1041f3940 2024-04-23T14:32:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.438+0000 7ff100c00640 1 --2- 172.21.15.100:0/533582574 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff0e40683b0 0x7ff0e406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.438+0000 7ff100c00640 1 -- 172.21.15.100:0/533582574 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff0fc086340 con 0x7ff1041f3940 2024-04-23T14:32:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.438+0000 7ff109000640 1 --2- 172.21.15.100:0/533582574 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff0e40683b0 0x7ff0e406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.438+0000 7ff109000640 1 --2- 172.21.15.100:0/533582574 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff0e40683b0 0x7ff0e406a870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff0f40af000 tx=0x7ff0f4004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.442+0000 7ff100c00640 1 -- 172.21.15.100:0/533582574 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff0fc055e10 con 0x7ff1041f3940 2024-04-23T14:32:58.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.598+0000 7ff10a400640 1 -- 172.21.15.100:0/533582574 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff104153880 con 0x7ff1041f3940 2024-04-23T14:32:58.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.598+0000 7ff100c00640 1 -- 172.21.15.100:0/533582574 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7ff0fc05a110 con 0x7ff1041f3940 2024-04-23T14:32:58.600 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:32:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.598+0000 7ff10a400640 1 -- 172.21.15.100:0/533582574 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff0e40683b0 msgr2=0x7ff0e406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.598+0000 7ff10a400640 1 --2- 172.21.15.100:0/533582574 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff0e40683b0 0x7ff0e406a870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff0f40af000 tx=0x7ff0f4004ac0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.598+0000 7ff10a400640 1 -- 172.21.15.100:0/533582574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1041f3940 msgr2=0x7ff10406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.598+0000 7ff10a400640 1 --2- 172.21.15.100:0/533582574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1041f3940 0x7ff10406f080 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff10406fca0 tx=0x7ff0fc00e8b0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.598+0000 7ff10a400640 1 -- 172.21.15.100:0/533582574 shutdown_connections 2024-04-23T14:32:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.598+0000 7ff10a400640 1 --2- 172.21.15.100:0/533582574 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff0e40683b0 0x7ff0e406a870 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.598+0000 7ff10a400640 1 --2- 172.21.15.100:0/533582574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1041f3940 0x7ff10406f080 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.598+0000 7ff10a400640 1 --2- 172.21.15.100:0/533582574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1041534a0 0x7ff1041457f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.598+0000 7ff10a400640 1 --2- 172.21.15.100:0/533582574 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0e80039f0 0x7ff104157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.598+0000 7ff10a400640 1 -- 172.21.15.100:0/533582574 >> 172.21.15.100:0/533582574 conn(0x7ff1040b1d00 msgr2=0x7ff10415e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.598+0000 7ff10a400640 1 -- 172.21.15.100:0/533582574 shutdown_connections 2024-04-23T14:32:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.598+0000 7ff10a400640 1 -- 172.21.15.100:0/533582574 wait complete. 2024-04-23T14:32:58.620 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:32:58.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.758+0000 7f8cfee00640 1 Processor -- start 2024-04-23T14:32:58.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.758+0000 7f8cfee00640 1 -- start start 2024-04-23T14:32:58.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.758+0000 7f8cfee00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8cf806f8c0 0x7f8cf806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:58.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.758+0000 7f8cfee00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cf806dd60 con 0x7f8cf80ce7f0 2024-04-23T14:32:58.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.758+0000 7f8cfee00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cf80d09c0 con 0x7f8cf80d0170 2024-04-23T14:32:58.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.758+0000 7f8cfee00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf80ad050 con 0x7f8cf806f8c0 2024-04-23T14:32:58.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.758+0000 7f8cfce00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f8cf80ce7f0 0x7f8cf80add80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:59596/0 (socket says 172.21.15.100:59596) 2024-04-23T14:32:58.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.758+0000 7f8cfce00640 1 -- 172.21.15.100:0/3790741027 learned_addr learned my addr 172.21.15.100:0/3790741027 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:58.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.758+0000 7f8cf7e00640 1 --2- 172.21.15.100:0/3790741027 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8cf806f8c0 0x7f8cf806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:58.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.758+0000 7f8cf7400640 1 -- 172.21.15.100:0/3790741027 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2858273065 0 0) 0x7f8cf0001120 con 0x7f8cf80ce7f0 2024-04-23T14:32:58.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.758+0000 7f8cf7400640 1 -- 172.21.15.100:0/3790741027 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cdc005740 con 0x7f8cf80ce7f0 2024-04-23T14:32:58.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.758+0000 7f8cf7400640 1 -- 172.21.15.100:0/3790741027 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 360991465 0 0) 0x7f8cf80d09c0 con 0x7f8cf80d0170 2024-04-23T14:32:58.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.758+0000 7f8cf7400640 1 -- 172.21.15.100:0/3790741027 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cf0001120 con 0x7f8cf80d0170 2024-04-23T14:32:58.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.758+0000 7f8cf7e00640 1 -- 172.21.15.100:0/3790741027 >> v1:172.21.15.151:6789/0 conn(0x7f8cf80d0170 legacy=0x7f8cf80ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:58.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.758+0000 7f8cf7e00640 1 -- 172.21.15.100:0/3790741027 >> v1:172.21.15.100:6789/0 conn(0x7f8cf80ce7f0 legacy=0x7f8cf80add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:58.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.758+0000 7f8cf7e00640 1 -- 172.21.15.100:0/3790741027 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf80ad2a0 con 0x7f8cf806f8c0 2024-04-23T14:32:58.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.762+0000 7f8cf7e00640 1 --2- 172.21.15.100:0/3790741027 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8cf806f8c0 0x7f8cf806c6c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009dc0 tx=0x7f8ce801c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed109d07072bbe54 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:58.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.762+0000 7f8cf7400640 1 -- 172.21.15.100:0/3790741027 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce80b0600 con 0x7f8cf806f8c0 2024-04-23T14:32:58.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.762+0000 7f8cf7400640 1 -- 172.21.15.100:0/3790741027 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ce80b07a0 con 0x7f8cf806f8c0 2024-04-23T14:32:58.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.762+0000 7f8cf7400640 1 -- 172.21.15.100:0/3790741027 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce80b0b70 con 0x7f8cf806f8c0 2024-04-23T14:32:58.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.762+0000 7f8cfee00640 1 -- 172.21.15.100:0/3790741027 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8cf806f8c0 msgr2=0x7f8cf806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:58.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.762+0000 7f8cfee00640 1 --2- 172.21.15.100:0/3790741027 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8cf806f8c0 0x7f8cf806c6c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009dc0 tx=0x7f8ce801c990 comp rx=0 tx=0).stop 2024-04-23T14:32:58.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.762+0000 7f8cfee00640 1 -- 172.21.15.100:0/3790741027 shutdown_connections 2024-04-23T14:32:58.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.762+0000 7f8cfee00640 1 --2- 172.21.15.100:0/3790741027 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8cf806f8c0 0x7f8cf806c6c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.762+0000 7f8cfee00640 1 -- 172.21.15.100:0/3790741027 >> 172.21.15.100:0/3790741027 conn(0x7f8cf80b5200 msgr2=0x7f8cf80b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:58.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.762+0000 7f8cfee00640 1 -- 172.21.15.100:0/3790741027 shutdown_connections 2024-04-23T14:32:58.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.762+0000 7f8cfee00640 1 -- 172.21.15.100:0/3790741027 wait complete. 2024-04-23T14:32:58.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cfee00640 1 Processor -- start 2024-04-23T14:32:58.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cfee00640 1 -- start start 2024-04-23T14:32:58.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cfee00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8cf806f8c0 0x7f8cf80c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:58.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cfee00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cf80ce7f0 0x7f8cf815a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:58.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cfee00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cf80d0170 0x7f8cf806bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:58.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cfee00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf80ccff0 con 0x7f8cf80ce7f0 2024-04-23T14:32:58.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cfee00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf80cce70 con 0x7f8cf80d0170 2024-04-23T14:32:58.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cfee00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf80cd170 con 0x7f8cf806f8c0 2024-04-23T14:32:58.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cef400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cf80ce7f0 0x7f8cf815a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:58.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cef400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cf80ce7f0 0x7f8cf815a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49260/0 (socket says 172.21.15.100:49260) 2024-04-23T14:32:58.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cef400640 1 -- 172.21.15.100:0/1712511234 learned_addr learned my addr 172.21.15.100:0/1712511234 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:58.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cfce00640 1 --2- 172.21.15.100:0/1712511234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cf80d0170 0x7f8cf806bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:58.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cf7e00640 1 --2- 172.21.15.100:0/1712511234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8cf806f8c0 0x7f8cf80c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:58.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cef400640 1 -- 172.21.15.100:0/1712511234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8cf806f8c0 msgr2=0x7f8cf80c2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:58.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cef400640 1 --2- 172.21.15.100:0/1712511234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8cf806f8c0 0x7f8cf80c2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cef400640 1 -- 172.21.15.100:0/1712511234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cf80d0170 msgr2=0x7f8cf806bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:58.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cef400640 1 --2- 172.21.15.100:0/1712511234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cf80d0170 0x7f8cf806bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cef400640 1 -- 172.21.15.100:0/1712511234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf80c92c0 con 0x7f8cf80ce7f0 2024-04-23T14:32:58.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cef400640 1 --2- 172.21.15.100:0/1712511234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cf80ce7f0 0x7f8cf815a0d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000c430 tx=0x7f8ce000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:58.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cf4c00640 1 -- 172.21.15.100:0/1712511234 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0017070 con 0x7f8cf80ce7f0 2024-04-23T14:32:58.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cf4c00640 1 -- 172.21.15.100:0/1712511234 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ce00052f0 con 0x7f8cf80ce7f0 2024-04-23T14:32:58.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cf4c00640 1 -- 172.21.15.100:0/1712511234 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0003840 con 0x7f8cf80ce7f0 2024-04-23T14:32:58.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cfee00640 1 -- 172.21.15.100:0/1712511234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf80c95a0 con 0x7f8cf80ce7f0 2024-04-23T14:32:58.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.766+0000 7f8cfee00640 1 -- 172.21.15.100:0/1712511234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf80c9b60 con 0x7f8cf80ce7f0 2024-04-23T14:32:58.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.770+0000 7f8cfee00640 1 -- 172.21.15.100:0/1712511234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf8068ad0 con 0x7f8cf80ce7f0 2024-04-23T14:32:58.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.770+0000 7f8cf4c00640 1 -- 172.21.15.100:0/1712511234 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8ce001a070 con 0x7f8cf80ce7f0 2024-04-23T14:32:58.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.770+0000 7f8cf4c00640 1 --2- 172.21.15.100:0/1712511234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8cd4068380 0x7f8cd406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:58.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.770+0000 7f8cf4c00640 1 -- 172.21.15.100:0/1712511234 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f8ce0086100 con 0x7f8cf80ce7f0 2024-04-23T14:32:58.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.770+0000 7f8cf7e00640 1 --2- 172.21.15.100:0/1712511234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8cd4068380 0x7f8cd406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:58.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.770+0000 7f8cf7e00640 1 --2- 172.21.15.100:0/1712511234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8cd4068380 0x7f8cd406a840 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80af000 tx=0x7f8ce80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:58.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.774+0000 7f8cf4c00640 1 -- 172.21.15.100:0/1712511234 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8ce0055c60 con 0x7f8cf80ce7f0 2024-04-23T14:32:58.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.934+0000 7f8cfee00640 1 -- 172.21.15.100:0/1712511234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8cf806cc00 con 0x7f8cf80ce7f0 2024-04-23T14:32:58.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.934+0000 7f8cf4c00640 1 -- 172.21.15.100:0/1712511234 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f8ce0059f60 con 0x7f8cf80ce7f0 2024-04-23T14:32:58.935 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:32:58.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.934+0000 7f8cfee00640 1 -- 172.21.15.100:0/1712511234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8cd4068380 msgr2=0x7f8cd406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:58.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.934+0000 7f8cfee00640 1 --2- 172.21.15.100:0/1712511234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8cd4068380 0x7f8cd406a840 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80af000 tx=0x7f8ce80a7000 comp rx=0 tx=0).stop 2024-04-23T14:32:58.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.934+0000 7f8cfee00640 1 -- 172.21.15.100:0/1712511234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cf80ce7f0 msgr2=0x7f8cf815a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:58.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.934+0000 7f8cfee00640 1 --2- 172.21.15.100:0/1712511234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cf80ce7f0 0x7f8cf815a0d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000c430 tx=0x7f8ce000c8f0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.934+0000 7f8cfee00640 1 -- 172.21.15.100:0/1712511234 shutdown_connections 2024-04-23T14:32:58.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.934+0000 7f8cfee00640 1 --2- 172.21.15.100:0/1712511234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8cd4068380 0x7f8cd406a840 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.934+0000 7f8cfee00640 1 --2- 172.21.15.100:0/1712511234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cf80d0170 0x7f8cf806bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.934+0000 7f8cfee00640 1 --2- 172.21.15.100:0/1712511234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cf80ce7f0 0x7f8cf815a0d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.934+0000 7f8cfee00640 1 --2- 172.21.15.100:0/1712511234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8cf806f8c0 0x7f8cf80c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:58.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.934+0000 7f8cfee00640 1 -- 172.21.15.100:0/1712511234 >> 172.21.15.100:0/1712511234 conn(0x7f8cf80b5200 msgr2=0x7f8cf806df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:58.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.934+0000 7f8cfee00640 1 -- 172.21.15.100:0/1712511234 shutdown_connections 2024-04-23T14:32:58.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:58.934+0000 7f8cfee00640 1 -- 172.21.15.100:0/1712511234 wait complete. 2024-04-23T14:32:58.957 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:32:59.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe878c00640 1 Processor -- start 2024-04-23T14:32:59.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe878c00640 1 -- start start 2024-04-23T14:32:59.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe878c00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fe8740cca50 0x7fe8740cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe878c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe87415c7e0 0x7fe874160170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe878c00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8740cd370 con 0x7fe87415d600 2024-04-23T14:32:59.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe878c00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fe8740d2270 con 0x7fe8740cca50 2024-04-23T14:32:59.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe878c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe87415db00 con 0x7fe87415c7e0 2024-04-23T14:32:59.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe872800640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe87415d600 0x7fe8741606b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:59598/0 (socket says 172.21.15.100:59598) 2024-04-23T14:32:59.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe872800640 1 -- 172.21.15.100:0/257975176 learned_addr learned my addr 172.21.15.100:0/257975176 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:59.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe871e00640 1 --2- 172.21.15.100:0/257975176 >> v2:172.21.15.151:3300/0 conn(0x7fe8740cca50 0x7fe8740cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe869400640 1 --2- 172.21.15.100:0/257975176 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe87415c7e0 0x7fe874160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe871400640 1 -- 172.21.15.100:0/257975176 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1229527836 0 0) 0x7fe8740cd370 con 0x7fe87415d600 2024-04-23T14:32:59.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe871400640 1 -- 172.21.15.100:0/257975176 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe858003530 con 0x7fe87415d600 2024-04-23T14:32:59.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe871400640 1 -- 172.21.15.100:0/257975176 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1722502196 0 0) 0x7fe858003530 con 0x7fe87415d600 2024-04-23T14:32:59.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe871400640 1 -- 172.21.15.100:0/257975176 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe87415c7e0 msgr2=0x7fe874160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:59.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe871400640 1 --2- 172.21.15.100:0/257975176 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe87415c7e0 0x7fe874160170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe871400640 1 -- 172.21.15.100:0/257975176 >> v2:172.21.15.151:3300/0 conn(0x7fe8740cca50 msgr2=0x7fe8740cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:59.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe871400640 1 --2- 172.21.15.100:0/257975176 >> v2:172.21.15.151:3300/0 conn(0x7fe8740cca50 0x7fe8740cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe871400640 1 -- 172.21.15.100:0/257975176 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8740d04a0 con 0x7fe87415d600 2024-04-23T14:32:59.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe871400640 1 -- 172.21.15.100:0/257975176 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7fe86c001a00 con 0x7fe87415d600 2024-04-23T14:32:59.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe869400640 1 --2- 172.21.15.100:0/257975176 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe87415c7e0 0x7fe874160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:59.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe871400640 1 -- 172.21.15.100:0/257975176 >> v1:172.21.15.100:6789/0 conn(0x7fe87415d600 legacy=0x7fe8741606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe871400640 1 --2- 172.21.15.100:0/257975176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8580039c0 0x7fe858003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe871400640 1 --2- 172.21.15.100:0/257975176 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe87415c7e0 0x7fe858004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe871400640 1 --2- 172.21.15.100:0/257975176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe858004d50 0x7fe8580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe871400640 1 -- 172.21.15.100:0/257975176 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe87415db00 con 0x7fe858004d50 2024-04-23T14:32:59.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe871400640 1 -- 172.21.15.100:0/257975176 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe85800b9d0 con 0x7fe8580039c0 2024-04-23T14:32:59.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe871400640 1 -- 172.21.15.100:0/257975176 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe85800bb50 con 0x7fe87415c7e0 2024-04-23T14:32:59.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.094+0000 7fe872800640 1 --2- 172.21.15.100:0/257975176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe858004d50 0x7fe8580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.098+0000 7fe871e00640 1 --2- 172.21.15.100:0/257975176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8580039c0 0x7fe858003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.098+0000 7fe869400640 1 --2- 172.21.15.100:0/257975176 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe87415c7e0 0x7fe858004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.098+0000 7fe872800640 1 -- 172.21.15.100:0/257975176 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe87415c7e0 msgr2=0x7fe858004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:59.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.098+0000 7fe872800640 1 --2- 172.21.15.100:0/257975176 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe87415c7e0 0x7fe858004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.098+0000 7fe872800640 1 -- 172.21.15.100:0/257975176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8580039c0 msgr2=0x7fe858003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:59.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.098+0000 7fe872800640 1 --2- 172.21.15.100:0/257975176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8580039c0 0x7fe858003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.098+0000 7fe872800640 1 -- 172.21.15.100:0/257975176 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe858003530 con 0x7fe858004d50 2024-04-23T14:32:59.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.098+0000 7fe872800640 1 --2- 172.21.15.100:0/257975176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe858004d50 0x7fe8580051d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00a870 tx=0x7fe86c01fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0576fbc17773899 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:59.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.098+0000 7fe871400640 1 -- 172.21.15.100:0/257975176 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c00a4c0 con 0x7fe858004d50 2024-04-23T14:32:59.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.098+0000 7fe871400640 1 -- 172.21.15.100:0/257975176 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe86c0ad020 con 0x7fe858004d50 2024-04-23T14:32:59.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.098+0000 7fe871400640 1 -- 172.21.15.100:0/257975176 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0a9720 con 0x7fe858004d50 2024-04-23T14:32:59.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.098+0000 7fe878c00640 1 -- 172.21.15.100:0/257975176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe858004d50 msgr2=0x7fe8580051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.098+0000 7fe878c00640 1 --2- 172.21.15.100:0/257975176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe858004d50 0x7fe8580051d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00a870 tx=0x7fe86c01fb00 comp rx=0 tx=0).stop 2024-04-23T14:32:59.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.098+0000 7fe872800640 1 -- 172.21.15.100:0/257975176 reap_dead start 2024-04-23T14:32:59.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.098+0000 7fe878c00640 1 -- 172.21.15.100:0/257975176 shutdown_connections 2024-04-23T14:32:59.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.098+0000 7fe878c00640 1 -- 172.21.15.100:0/257975176 >> 172.21.15.100:0/257975176 conn(0x7fe8740cee80 msgr2=0x7fe8740cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:59.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.102+0000 7fe878c00640 1 -- 172.21.15.100:0/257975176 shutdown_connections 2024-04-23T14:32:59.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.102+0000 7fe878c00640 1 -- 172.21.15.100:0/257975176 wait complete. 2024-04-23T14:32:59.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.102+0000 7fe878c00640 1 Processor -- start 2024-04-23T14:32:59.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.102+0000 7fe878c00640 1 -- start start 2024-04-23T14:32:59.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.102+0000 7fe878c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe858004d50 0x7fe8740cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.102+0000 7fe878c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe874150be0 0x7fe8740ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.102+0000 7fe878c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe874151060 0x7fe8740ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.102+0000 7fe878c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe8740d0d40 con 0x7fe874151060 2024-04-23T14:32:59.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.102+0000 7fe878c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe8740d0a60 con 0x7fe874150be0 2024-04-23T14:32:59.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.102+0000 7fe878c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe8740b52b0 con 0x7fe858004d50 2024-04-23T14:32:59.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.102+0000 7fe872800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe874151060 0x7fe8740ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.102+0000 7fe872800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe874151060 0x7fe8740ca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49278/0 (socket says 172.21.15.100:49278) 2024-04-23T14:32:59.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.102+0000 7fe872800640 1 -- 172.21.15.100:0/861840793 learned_addr learned my addr 172.21.15.100:0/861840793 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:59.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.102+0000 7fe871e00640 1 --2- 172.21.15.100:0/861840793 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe858004d50 0x7fe8740cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.102+0000 7fe869400640 1 --2- 172.21.15.100:0/861840793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe874150be0 0x7fe8740ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.102+0000 7fe872800640 1 -- 172.21.15.100:0/861840793 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe858004d50 msgr2=0x7fe8740cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.102+0000 7fe872800640 1 --2- 172.21.15.100:0/861840793 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe858004d50 0x7fe8740cbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.102+0000 7fe872800640 1 -- 172.21.15.100:0/861840793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe874150be0 msgr2=0x7fe8740ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.102+0000 7fe872800640 1 --2- 172.21.15.100:0/861840793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe874150be0 0x7fe8740ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.102+0000 7fe872800640 1 -- 172.21.15.100:0/861840793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8740c9670 con 0x7fe874151060 2024-04-23T14:32:59.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.102+0000 7fe872800640 1 --2- 172.21.15.100:0/861840793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe874151060 0x7fe8740ca930 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe86c007040 tx=0x7fe858005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:59.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.106+0000 7fe86aa00640 1 -- 172.21.15.100:0/861840793 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c002020 con 0x7fe874151060 2024-04-23T14:32:59.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.106+0000 7fe86aa00640 1 -- 172.21.15.100:0/861840793 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe86c004d70 con 0x7fe874151060 2024-04-23T14:32:59.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.106+0000 7fe878c00640 1 -- 172.21.15.100:0/861840793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe874155b90 con 0x7fe874151060 2024-04-23T14:32:59.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.106+0000 7fe86aa00640 1 -- 172.21.15.100:0/861840793 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0a9740 con 0x7fe874151060 2024-04-23T14:32:59.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.106+0000 7fe878c00640 1 -- 172.21.15.100:0/861840793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8740c9bf0 con 0x7fe874151060 2024-04-23T14:32:59.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.106+0000 7fe878c00640 1 -- 172.21.15.100:0/861840793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe874068ad0 con 0x7fe874151060 2024-04-23T14:32:59.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.106+0000 7fe86aa00640 1 -- 172.21.15.100:0/861840793 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe86c0a98e0 con 0x7fe874151060 2024-04-23T14:32:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.106+0000 7fe86aa00640 1 --2- 172.21.15.100:0/861840793 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe850068380 0x7fe85006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.106+0000 7fe86aa00640 1 -- 172.21.15.100:0/861840793 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe86c1240c0 con 0x7fe874151060 2024-04-23T14:32:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.106+0000 7fe871e00640 1 --2- 172.21.15.100:0/861840793 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe850068380 0x7fe85006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.106+0000 7fe871e00640 1 --2- 172.21.15.100:0/861840793 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe850068380 0x7fe85006a840 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe864009be0 tx=0x7fe864009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:59.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.110+0000 7fe86aa00640 1 -- 172.21.15.100:0/861840793 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe86c019810 con 0x7fe874151060 2024-04-23T14:32:59.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.266+0000 7fe878c00640 1 -- 172.21.15.100:0/861840793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe8740c8310 con 0x7fe874151060 2024-04-23T14:32:59.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.266+0000 7fe86aa00640 1 -- 172.21.15.100:0/861840793 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fe86c0f4280 con 0x7fe874151060 2024-04-23T14:32:59.270 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:32:59.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.270+0000 7fe878c00640 1 -- 172.21.15.100:0/861840793 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe850068380 msgr2=0x7fe85006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.270+0000 7fe878c00640 1 --2- 172.21.15.100:0/861840793 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe850068380 0x7fe85006a840 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe864009be0 tx=0x7fe864009190 comp rx=0 tx=0).stop 2024-04-23T14:32:59.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.270+0000 7fe878c00640 1 -- 172.21.15.100:0/861840793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe874151060 msgr2=0x7fe8740ca930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.270+0000 7fe878c00640 1 --2- 172.21.15.100:0/861840793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe874151060 0x7fe8740ca930 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe86c007040 tx=0x7fe858005840 comp rx=0 tx=0).stop 2024-04-23T14:32:59.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.270+0000 7fe878c00640 1 -- 172.21.15.100:0/861840793 shutdown_connections 2024-04-23T14:32:59.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.270+0000 7fe878c00640 1 --2- 172.21.15.100:0/861840793 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe850068380 0x7fe85006a840 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.270+0000 7fe878c00640 1 --2- 172.21.15.100:0/861840793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe874151060 0x7fe8740ca930 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.270+0000 7fe878c00640 1 --2- 172.21.15.100:0/861840793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe874150be0 0x7fe8740ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.270+0000 7fe878c00640 1 --2- 172.21.15.100:0/861840793 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe858004d50 0x7fe8740cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.270+0000 7fe878c00640 1 -- 172.21.15.100:0/861840793 >> 172.21.15.100:0/861840793 conn(0x7fe8740cee80 msgr2=0x7fe8740c7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:59.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.270+0000 7fe878c00640 1 -- 172.21.15.100:0/861840793 shutdown_connections 2024-04-23T14:32:59.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.270+0000 7fe878c00640 1 -- 172.21.15.100:0/861840793 wait complete. 2024-04-23T14:32:59.292 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-23T14:32:59.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.430+0000 7f114b400640 1 Processor -- start 2024-04-23T14:32:59.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.430+0000 7f114b400640 1 -- start start 2024-04-23T14:32:59.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.430+0000 7f114b400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f114406bb20 0x7f11440ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.430+0000 7f114b400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f11440cd130 0x7f11440ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.430+0000 7f114b400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f114406cbb0 con 0x7f114415c7a0 2024-04-23T14:32:59.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.430+0000 7f114b400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f11440cd5a0 con 0x7f11440cd130 2024-04-23T14:32:59.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.430+0000 7f114b400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f114406d450 con 0x7f114406bb20 2024-04-23T14:32:59.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.430+0000 7f1149400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f11440cd130 0x7f11440ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.430+0000 7f1149400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f11440cd130 0x7f11440ad430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49280/0 (socket says 172.21.15.100:49280) 2024-04-23T14:32:59.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.430+0000 7f1149400640 1 -- 172.21.15.100:0/3139346284 learned_addr learned my addr 172.21.15.100:0/3139346284 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:59.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.430+0000 7f113be00640 1 --2- 172.21.15.100:0/3139346284 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f114406bb20 0x7f11440ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.430+0000 7f1149400640 1 -- 172.21.15.100:0/3139346284 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f114406bb20 msgr2=0x7f11440ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:59.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.430+0000 7f1149400640 1 --2- 172.21.15.100:0/3139346284 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f114406bb20 0x7f11440ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.430+0000 7f1149400640 1 -- 172.21.15.100:0/3139346284 >> v1:172.21.15.151:6789/0 conn(0x7f114415c7a0 legacy=0x7f11440af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:59.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.430+0000 7f1149400640 1 -- 172.21.15.100:0/3139346284 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114406ce90 con 0x7f11440cd130 2024-04-23T14:32:59.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.430+0000 7f113be00640 1 --2- 172.21.15.100:0/3139346284 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f114406bb20 0x7f11440ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:59.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.434+0000 7f1149400640 1 --2- 172.21.15.100:0/3139346284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11440cd130 0x7f11440ad430 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f113c009e50 tx=0x7f113c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a48827e33bdd746 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:59.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.434+0000 7f1143e00640 1 -- 172.21.15.100:0/3139346284 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c0a55e0 con 0x7f11440cd130 2024-04-23T14:32:59.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.434+0000 7f1143e00640 1 -- 172.21.15.100:0/3139346284 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f113c0a5780 con 0x7f11440cd130 2024-04-23T14:32:59.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.434+0000 7f1143e00640 1 -- 172.21.15.100:0/3139346284 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c0a5b50 con 0x7f11440cd130 2024-04-23T14:32:59.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.434+0000 7f114b400640 1 -- 172.21.15.100:0/3139346284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11440cd130 msgr2=0x7f11440ad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.434+0000 7f114b400640 1 --2- 172.21.15.100:0/3139346284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11440cd130 0x7f11440ad430 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f113c009e50 tx=0x7f113c01c960 comp rx=0 tx=0).stop 2024-04-23T14:32:59.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.434+0000 7f114b400640 1 -- 172.21.15.100:0/3139346284 shutdown_connections 2024-04-23T14:32:59.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.434+0000 7f114b400640 1 --2- 172.21.15.100:0/3139346284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11440cd130 0x7f11440ad430 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.434+0000 7f114b400640 1 --2- 172.21.15.100:0/3139346284 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f114406bb20 0x7f11440ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.434+0000 7f114b400640 1 -- 172.21.15.100:0/3139346284 >> 172.21.15.100:0/3139346284 conn(0x7f11440b5200 msgr2=0x7f11440b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:59.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.434+0000 7f114b400640 1 -- 172.21.15.100:0/3139346284 shutdown_connections 2024-04-23T14:32:59.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.434+0000 7f114b400640 1 -- 172.21.15.100:0/3139346284 wait complete. 2024-04-23T14:32:59.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f114b400640 1 Processor -- start 2024-04-23T14:32:59.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f114b400640 1 -- start start 2024-04-23T14:32:59.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f114b400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f114406bb20 0x7f11440c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f114b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11440cd130 0x7f11440c3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f114b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f114415c7a0 0x7f11440b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f114b400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f11440ae510 con 0x7f11440cd130 2024-04-23T14:32:59.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f114b400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f11440ae390 con 0x7f114415c7a0 2024-04-23T14:32:59.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f114b400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f11440ae690 con 0x7f114406bb20 2024-04-23T14:32:59.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f113be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11440cd130 0x7f11440c3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f113be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11440cd130 0x7f11440c3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49296/0 (socket says 172.21.15.100:49296) 2024-04-23T14:32:59.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f113be00640 1 -- 172.21.15.100:0/2441225741 learned_addr learned my addr 172.21.15.100:0/2441225741 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:59.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f1149400640 1 --2- 172.21.15.100:0/2441225741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f114415c7a0 0x7f11440b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f1148a00640 1 --2- 172.21.15.100:0/2441225741 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f114406bb20 0x7f11440c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f113be00640 1 -- 172.21.15.100:0/2441225741 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f114406bb20 msgr2=0x7f11440c2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f113be00640 1 --2- 172.21.15.100:0/2441225741 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f114406bb20 0x7f11440c2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f113be00640 1 -- 172.21.15.100:0/2441225741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f114415c7a0 msgr2=0x7f11440b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f113be00640 1 --2- 172.21.15.100:0/2441225741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f114415c7a0 0x7f11440b0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f113be00640 1 -- 172.21.15.100:0/2441225741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11440c4d50 con 0x7f11440cd130 2024-04-23T14:32:59.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f113be00640 1 --2- 172.21.15.100:0/2441225741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11440cd130 0x7f11440c3d10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f112c00c820 tx=0x7f112c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:59.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f1141600640 1 -- 172.21.15.100:0/2441225741 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c015070 con 0x7f11440cd130 2024-04-23T14:32:59.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f1141600640 1 -- 172.21.15.100:0/2441225741 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f112c004220 con 0x7f11440cd130 2024-04-23T14:32:59.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f1141600640 1 -- 172.21.15.100:0/2441225741 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c00ce10 con 0x7f11440cd130 2024-04-23T14:32:59.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f114b400640 1 -- 172.21.15.100:0/2441225741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11440c4790 con 0x7f11440cd130 2024-04-23T14:32:59.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f114b400640 1 -- 172.21.15.100:0/2441225741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1144072fc0 con 0x7f11440cd130 2024-04-23T14:32:59.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.438+0000 7f114b400640 1 -- 172.21.15.100:0/2441225741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1144068ad0 con 0x7f11440cd130 2024-04-23T14:32:59.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.442+0000 7f1141600640 1 -- 172.21.15.100:0/2441225741 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f112c007830 con 0x7f11440cd130 2024-04-23T14:32:59.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.442+0000 7f1141600640 1 --2- 172.21.15.100:0/2441225741 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1120068380 0x7f112006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.442+0000 7f1141600640 1 -- 172.21.15.100:0/2441225741 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f112c086ef0 con 0x7f11440cd130 2024-04-23T14:32:59.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.442+0000 7f1148a00640 1 --2- 172.21.15.100:0/2441225741 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1120068380 0x7f112006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.442+0000 7f1148a00640 1 --2- 172.21.15.100:0/2441225741 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1120068380 0x7f112006a840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f11340098e0 tx=0x7f1134009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:59.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.446+0000 7f1141600640 1 -- 172.21.15.100:0/2441225741 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f112c056a50 con 0x7f11440cd130 2024-04-23T14:32:59.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.606+0000 7f114b400640 1 -- 172.21.15.100:0/2441225741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1144099ad0 con 0x7f11440cd130 2024-04-23T14:32:59.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.606+0000 7f1141600640 1 -- 172.21.15.100:0/2441225741 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f112c05ad50 con 0x7f11440cd130 2024-04-23T14:32:59.608 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:32:59.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.610+0000 7f114b400640 1 -- 172.21.15.100:0/2441225741 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1120068380 msgr2=0x7f112006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.610+0000 7f114b400640 1 --2- 172.21.15.100:0/2441225741 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1120068380 0x7f112006a840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f11340098e0 tx=0x7f1134009060 comp rx=0 tx=0).stop 2024-04-23T14:32:59.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.610+0000 7f114b400640 1 -- 172.21.15.100:0/2441225741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11440cd130 msgr2=0x7f11440c3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.610+0000 7f114b400640 1 --2- 172.21.15.100:0/2441225741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11440cd130 0x7f11440c3d10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f112c00c820 tx=0x7f112c00cce0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.610+0000 7f114b400640 1 -- 172.21.15.100:0/2441225741 shutdown_connections 2024-04-23T14:32:59.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.610+0000 7f114b400640 1 --2- 172.21.15.100:0/2441225741 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1120068380 0x7f112006a840 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.610+0000 7f114b400640 1 --2- 172.21.15.100:0/2441225741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f114415c7a0 0x7f11440b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.610+0000 7f114b400640 1 --2- 172.21.15.100:0/2441225741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11440cd130 0x7f11440c3d10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.610+0000 7f114b400640 1 --2- 172.21.15.100:0/2441225741 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f114406bb20 0x7f11440c2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.610+0000 7f114b400640 1 -- 172.21.15.100:0/2441225741 >> 172.21.15.100:0/2441225741 conn(0x7f11440b5200 msgr2=0x7f11440cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:59.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.610+0000 7f114b400640 1 -- 172.21.15.100:0/2441225741 shutdown_connections 2024-04-23T14:32:59.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.610+0000 7f114b400640 1 -- 172.21.15.100:0/2441225741 wait complete. 2024-04-23T14:32:59.631 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:59.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.766+0000 7fce0fe00640 1 Processor -- start 2024-04-23T14:32:59.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.766+0000 7fce0fe00640 1 -- start start 2024-04-23T14:32:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.766+0000 7fce0fe00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fce10162d50 0x7fce1006d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.766+0000 7fce0fe00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce10162290 con 0x7fce1006d990 2024-04-23T14:32:59.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.766+0000 7fce0fe00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce1015f7d0 con 0x7fce100cf330 2024-04-23T14:32:59.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.766+0000 7fce0fe00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fce1006dde0 con 0x7fce10162d50 2024-04-23T14:32:59.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.766+0000 7fce0f400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fce1006d990 0x7fce10154e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:59600/0 (socket says 172.21.15.100:59600) 2024-04-23T14:32:59.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.766+0000 7fce0f400640 1 -- 172.21.15.100:0/3763802129 learned_addr learned my addr 172.21.15.100:0/3763802129 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:32:59.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.766+0000 7fce0e000640 1 --2- 172.21.15.100:0/3763802129 >> v2:172.21.15.151:3300/0 conn(0x7fce10162d50 0x7fce1006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.766+0000 7fce0d600640 1 -- 172.21.15.100:0/3763802129 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2477682518 0 0) 0x7fce10162290 con 0x7fce1006d990 2024-04-23T14:32:59.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.766+0000 7fce0d600640 1 -- 172.21.15.100:0/3763802129 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcdf4003530 con 0x7fce1006d990 2024-04-23T14:32:59.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.766+0000 7fce0e000640 1 -- 172.21.15.100:0/3763802129 >> v1:172.21.15.151:6789/0 conn(0x7fce100cf330 legacy=0x7fce1015eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.766+0000 7fce0e000640 1 -- 172.21.15.100:0/3763802129 >> v1:172.21.15.100:6789/0 conn(0x7fce1006d990 legacy=0x7fce10154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.766+0000 7fce0e000640 1 -- 172.21.15.100:0/3763802129 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1015fab0 con 0x7fce10162d50 2024-04-23T14:32:59.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.770+0000 7fce0e000640 1 --2- 172.21.15.100:0/3763802129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce10162d50 0x7fce1006d450 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fce000097c0 tx=0x7fce0001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5b4376c1e770061 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:59.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.770+0000 7fce0d600640 1 -- 172.21.15.100:0/3763802129 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce000b0070 con 0x7fce10162d50 2024-04-23T14:32:59.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.770+0000 7fce0d600640 1 -- 172.21.15.100:0/3763802129 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce000ab600 con 0x7fce10162d50 2024-04-23T14:32:59.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.770+0000 7fce0d600640 1 -- 172.21.15.100:0/3763802129 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce000ab9d0 con 0x7fce10162d50 2024-04-23T14:32:59.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.770+0000 7fce0fe00640 1 -- 172.21.15.100:0/3763802129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce10162d50 msgr2=0x7fce1006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.770+0000 7fce0fe00640 1 --2- 172.21.15.100:0/3763802129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce10162d50 0x7fce1006d450 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fce000097c0 tx=0x7fce0001c4a0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.770+0000 7fce0fe00640 1 -- 172.21.15.100:0/3763802129 shutdown_connections 2024-04-23T14:32:59.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.770+0000 7fce0fe00640 1 --2- 172.21.15.100:0/3763802129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce10162d50 0x7fce1006d450 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.770+0000 7fce0fe00640 1 -- 172.21.15.100:0/3763802129 >> 172.21.15.100:0/3763802129 conn(0x7fce10072120 msgr2=0x7fce100b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:59.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.770+0000 7fce0fe00640 1 -- 172.21.15.100:0/3763802129 shutdown_connections 2024-04-23T14:32:59.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.770+0000 7fce0fe00640 1 -- 172.21.15.100:0/3763802129 wait complete. 2024-04-23T14:32:59.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.770+0000 7fce0fe00640 1 Processor -- start 2024-04-23T14:32:59.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.770+0000 7fce0fe00640 1 -- start start 2024-04-23T14:32:59.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce0fe00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce1006d990 0x7fce100c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce0fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce100cf330 0x7fce100cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce0fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce10162d50 0x7fce1014d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce0fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fce10155970 con 0x7fce10162d50 2024-04-23T14:32:59.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce0fe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fce101628c0 con 0x7fce100cf330 2024-04-23T14:32:59.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce0fe00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fce10155af0 con 0x7fce1006d990 2024-04-23T14:32:59.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce0f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce10162d50 0x7fce1014d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce0f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce10162d50 0x7fce1014d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49302/0 (socket says 172.21.15.100:49302) 2024-04-23T14:32:59.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce0f400640 1 -- 172.21.15.100:0/2350594913 learned_addr learned my addr 172.21.15.100:0/2350594913 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:32:59.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce0ea00640 1 --2- 172.21.15.100:0/2350594913 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce1006d990 0x7fce100c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce0e000640 1 --2- 172.21.15.100:0/2350594913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce100cf330 0x7fce100cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce0f400640 1 -- 172.21.15.100:0/2350594913 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce1006d990 msgr2=0x7fce100c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce0f400640 1 --2- 172.21.15.100:0/2350594913 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce1006d990 0x7fce100c7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce0f400640 1 -- 172.21.15.100:0/2350594913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce100cf330 msgr2=0x7fce100cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce0f400640 1 --2- 172.21.15.100:0/2350594913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce100cf330 0x7fce100cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce0f400640 1 -- 172.21.15.100:0/2350594913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce100c8670 con 0x7fce10162d50 2024-04-23T14:32:59.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce0e000640 1 --2- 172.21.15.100:0/2350594913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce100cf330 0x7fce100cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:59.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce0f400640 1 --2- 172.21.15.100:0/2350594913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce10162d50 0x7fce1014d010 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fce0800caa0 tx=0x7fce0800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:59.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce06a00640 1 -- 172.21.15.100:0/2350594913 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08017070 con 0x7fce10162d50 2024-04-23T14:32:59.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce06a00640 1 -- 172.21.15.100:0/2350594913 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce0800ac00 con 0x7fce10162d50 2024-04-23T14:32:59.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce0fe00640 1 -- 172.21.15.100:0/2350594913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce100c6aa0 con 0x7fce10162d50 2024-04-23T14:32:59.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce06a00640 1 -- 172.21.15.100:0/2350594913 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08012400 con 0x7fce10162d50 2024-04-23T14:32:59.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce0fe00640 1 -- 172.21.15.100:0/2350594913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce100c7020 con 0x7fce10162d50 2024-04-23T14:32:59.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.774+0000 7fce0fe00640 1 -- 172.21.15.100:0/2350594913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce10075c20 con 0x7fce10162d50 2024-04-23T14:32:59.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.778+0000 7fce06a00640 1 -- 172.21.15.100:0/2350594913 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fce0801a070 con 0x7fce10162d50 2024-04-23T14:32:59.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.778+0000 7fce06a00640 1 --2- 172.21.15.100:0/2350594913 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcdec068380 0x7fcdec06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.778+0000 7fce06a00640 1 -- 172.21.15.100:0/2350594913 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fce08086aa0 con 0x7fce10162d50 2024-04-23T14:32:59.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.778+0000 7fce0ea00640 1 --2- 172.21.15.100:0/2350594913 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcdec068380 0x7fcdec06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.778+0000 7fce0ea00640 1 --2- 172.21.15.100:0/2350594913 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcdec068380 0x7fcdec06a840 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80015b0 tx=0x7fcdf8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:59.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.782+0000 7fce06a00640 1 -- 172.21.15.100:0/2350594913 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fce08056cf0 con 0x7fce10162d50 2024-04-23T14:32:59.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.942+0000 7fce0fe00640 1 -- 172.21.15.100:0/2350594913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce100b4d00 con 0x7fce10162d50 2024-04-23T14:32:59.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.942+0000 7fce06a00640 1 -- 172.21.15.100:0/2350594913 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18859 (secure 0 0 0) 0x7fce080565f0 con 0x7fce10162d50 2024-04-23T14:32:59.944 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:32:59.944 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:49.784608+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:59.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.942+0000 7fce0fe00640 1 -- 172.21.15.100:0/2350594913 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcdec068380 msgr2=0x7fcdec06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.942+0000 7fce0fe00640 1 --2- 172.21.15.100:0/2350594913 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcdec068380 0x7fcdec06a840 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80015b0 tx=0x7fcdf8009040 comp rx=0 tx=0).stop 2024-04-23T14:32:59.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.942+0000 7fce0fe00640 1 -- 172.21.15.100:0/2350594913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce10162d50 msgr2=0x7fce1014d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.942+0000 7fce0fe00640 1 --2- 172.21.15.100:0/2350594913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce10162d50 0x7fce1014d010 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fce0800caa0 tx=0x7fce0800cf60 comp rx=0 tx=0).stop 2024-04-23T14:32:59.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.942+0000 7fce0fe00640 1 -- 172.21.15.100:0/2350594913 shutdown_connections 2024-04-23T14:32:59.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.942+0000 7fce0fe00640 1 --2- 172.21.15.100:0/2350594913 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcdec068380 0x7fcdec06a840 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.942+0000 7fce0fe00640 1 --2- 172.21.15.100:0/2350594913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce10162d50 0x7fce1014d010 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.942+0000 7fce0fe00640 1 --2- 172.21.15.100:0/2350594913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce100cf330 0x7fce100cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.942+0000 7fce0fe00640 1 --2- 172.21.15.100:0/2350594913 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce1006d990 0x7fce100c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.942+0000 7fce0fe00640 1 -- 172.21.15.100:0/2350594913 >> 172.21.15.100:0/2350594913 conn(0x7fce10072120 msgr2=0x7fce100d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:59.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.942+0000 7fce0fe00640 1 -- 172.21.15.100:0/2350594913 shutdown_connections 2024-04-23T14:32:59.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:32:59.942+0000 7fce0fe00640 1 -- 172.21.15.100:0/2350594913 wait complete. 2024-04-23T14:32:59.965 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:32:06.835560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:32:07.930489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-23T14:32:11.636814+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-23T14:32:59.965 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:33:00.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.102+0000 7f20a0400640 1 Processor -- start 2024-04-23T14:33:00.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.102+0000 7f20a0400640 1 -- start start 2024-04-23T14:33:00.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.106+0000 7f20a0400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f209815c7e0 0x7f20980d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.106+0000 7f20a0400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f209815d600 0x7f20980d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.106+0000 7f20a0400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20980d04a0 con 0x7f20980cca50 2024-04-23T14:33:00.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.106+0000 7f20a0400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f209815db00 con 0x7f209815d600 2024-04-23T14:33:00.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.106+0000 7f20a0400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f20980d0780 con 0x7f209815c7e0 2024-04-23T14:33:00.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.106+0000 7f209e400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f209815d600 0x7f20980d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.106+0000 7f209e400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f209815d600 0x7f20980d17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49314/0 (socket says 172.21.15.100:49314) 2024-04-23T14:33:00.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.106+0000 7f209e400640 1 -- 172.21.15.100:0/293452004 learned_addr learned my addr 172.21.15.100:0/293452004 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:00.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.106+0000 7f209e400640 1 -- 172.21.15.100:0/293452004 >> v2:172.21.15.151:3300/0 conn(0x7f209815c7e0 msgr2=0x7f20980d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:00.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.106+0000 7f209d000640 1 --2- 172.21.15.100:0/293452004 >> v2:172.21.15.151:3300/0 conn(0x7f209815c7e0 0x7f20980d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.106+0000 7f209e400640 1 --2- 172.21.15.100:0/293452004 >> v2:172.21.15.151:3300/0 conn(0x7f209815c7e0 0x7f20980d1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.106+0000 7f209e400640 1 -- 172.21.15.100:0/293452004 >> v1:172.21.15.151:6789/0 conn(0x7f20980cca50 legacy=0x7f20980cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:00.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.106+0000 7f209e400640 1 -- 172.21.15.100:0/293452004 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20980d0a60 con 0x7f209815d600 2024-04-23T14:33:00.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.106+0000 7f209d000640 1 --2- 172.21.15.100:0/293452004 >> v2:172.21.15.151:3300/0 conn(0x7f209815c7e0 0x7f20980d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:00.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.110+0000 7f209e400640 1 --2- 172.21.15.100:0/293452004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f209815d600 0x7f20980d17a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2098074250 tx=0x7f209001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab8c480a5f9ef13 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:00.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.110+0000 7f2097e00640 1 -- 172.21.15.100:0/293452004 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20900a5810 con 0x7f209815d600 2024-04-23T14:33:00.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.110+0000 7f2097e00640 1 -- 172.21.15.100:0/293452004 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20900a59b0 con 0x7f209815d600 2024-04-23T14:33:00.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.110+0000 7f2097e00640 1 -- 172.21.15.100:0/293452004 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20900a5d80 con 0x7f209815d600 2024-04-23T14:33:00.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.110+0000 7f20a0400640 1 -- 172.21.15.100:0/293452004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f209815d600 msgr2=0x7f20980d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.110+0000 7f20a0400640 1 --2- 172.21.15.100:0/293452004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f209815d600 0x7f20980d17a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2098074250 tx=0x7f209001c930 comp rx=0 tx=0).stop 2024-04-23T14:33:00.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.110+0000 7f20a0400640 1 -- 172.21.15.100:0/293452004 shutdown_connections 2024-04-23T14:33:00.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.110+0000 7f20a0400640 1 --2- 172.21.15.100:0/293452004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f209815d600 0x7f20980d17a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.110+0000 7f20a0400640 1 --2- 172.21.15.100:0/293452004 >> v2:172.21.15.151:3300/0 conn(0x7f209815c7e0 0x7f20980d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.110+0000 7f20a0400640 1 -- 172.21.15.100:0/293452004 >> 172.21.15.100:0/293452004 conn(0x7f20980cee80 msgr2=0x7f20980cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:00.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.110+0000 7f20a0400640 1 -- 172.21.15.100:0/293452004 shutdown_connections 2024-04-23T14:33:00.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.110+0000 7f20a0400640 1 -- 172.21.15.100:0/293452004 wait complete. 2024-04-23T14:33:00.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.110+0000 7f20a0400640 1 Processor -- start 2024-04-23T14:33:00.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f20a0400640 1 -- start start 2024-04-23T14:33:00.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f20a0400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20980cca50 0x7f209814acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f20a0400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f209815c7e0 0x7f209806de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f20a0400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f209815d600 0x7f2098165690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f20a0400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f20980b3d70 con 0x7f209815c7e0 2024-04-23T14:33:00.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f20a0400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f20980b3bf0 con 0x7f20980cca50 2024-04-23T14:33:00.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f20a0400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f2098169ad0 con 0x7f209815d600 2024-04-23T14:33:00.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f209d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f209815c7e0 0x7f209806de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f209d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f209815c7e0 0x7f209806de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49322/0 (socket says 172.21.15.100:49322) 2024-04-23T14:33:00.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f209d000640 1 -- 172.21.15.100:0/3890086355 learned_addr learned my addr 172.21.15.100:0/3890086355 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:00.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f209da00640 1 --2- 172.21.15.100:0/3890086355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20980cca50 0x7f209814acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f209e400640 1 --2- 172.21.15.100:0/3890086355 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f209815d600 0x7f2098165690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f209d000640 1 -- 172.21.15.100:0/3890086355 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f209815d600 msgr2=0x7f2098165690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f209d000640 1 --2- 172.21.15.100:0/3890086355 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f209815d600 0x7f2098165690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f209d000640 1 -- 172.21.15.100:0/3890086355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20980cca50 msgr2=0x7f209814acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f209d000640 1 --2- 172.21.15.100:0/3890086355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20980cca50 0x7f209814acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f209d000640 1 -- 172.21.15.100:0/3890086355 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209806e740 con 0x7f209815c7e0 2024-04-23T14:33:00.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f209d000640 1 --2- 172.21.15.100:0/3890086355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f209815c7e0 0x7f209806de30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f208400c7c0 tx=0x7f208400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:00.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f2095600640 1 -- 172.21.15.100:0/3890086355 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084007a30 con 0x7f209815c7e0 2024-04-23T14:33:00.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f2095600640 1 -- 172.21.15.100:0/3890086355 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2084007bd0 con 0x7f209815c7e0 2024-04-23T14:33:00.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f2095600640 1 -- 172.21.15.100:0/3890086355 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20840113f0 con 0x7f209815c7e0 2024-04-23T14:33:00.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f20a0400640 1 -- 172.21.15.100:0/3890086355 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f209806ea80 con 0x7f209815c7e0 2024-04-23T14:33:00.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.114+0000 7f20a0400640 1 -- 172.21.15.100:0/3890086355 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f209814b710 con 0x7f209815c7e0 2024-04-23T14:33:00.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.118+0000 7f20a0400640 1 -- 172.21.15.100:0/3890086355 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f209814b910 con 0x7f209815c7e0 2024-04-23T14:33:00.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.118+0000 7f2095600640 1 -- 172.21.15.100:0/3890086355 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2084011590 con 0x7f209815c7e0 2024-04-23T14:33:00.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.118+0000 7f2095600640 1 --2- 172.21.15.100:0/3890086355 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f20780683b0 0x7f207806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.118+0000 7f2095600640 1 -- 172.21.15.100:0/3890086355 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2084086fe0 con 0x7f209815c7e0 2024-04-23T14:33:00.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.118+0000 7f209da00640 1 --2- 172.21.15.100:0/3890086355 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f20780683b0 0x7f207806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.118+0000 7f209da00640 1 --2- 172.21.15.100:0/3890086355 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f20780683b0 0x7f207806a870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f208c007fd0 tx=0x7f208c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:00.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.122+0000 7f2095600640 1 -- 172.21.15.100:0/3890086355 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f209806ea80 con 0x7f209815c7e0 2024-04-23T14:33:00.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.282+0000 7f20a0400640 1 -- 172.21.15.100:0/3890086355 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f20980b3850 con 0x7f209815c7e0 2024-04-23T14:33:00.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.282+0000 7f2095600640 1 -- 172.21.15.100:0/3890086355 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f2084015020 con 0x7f209815c7e0 2024-04-23T14:33:00.285 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:33:00.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.286+0000 7f20a0400640 1 -- 172.21.15.100:0/3890086355 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f20780683b0 msgr2=0x7f207806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.286+0000 7f20a0400640 1 --2- 172.21.15.100:0/3890086355 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f20780683b0 0x7f207806a870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f208c007fd0 tx=0x7f208c009040 comp rx=0 tx=0).stop 2024-04-23T14:33:00.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.286+0000 7f20a0400640 1 -- 172.21.15.100:0/3890086355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f209815c7e0 msgr2=0x7f209806de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.286+0000 7f20a0400640 1 --2- 172.21.15.100:0/3890086355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f209815c7e0 0x7f209806de30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f208400c7c0 tx=0x7f208400cc80 comp rx=0 tx=0).stop 2024-04-23T14:33:00.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.286+0000 7f20a0400640 1 -- 172.21.15.100:0/3890086355 shutdown_connections 2024-04-23T14:33:00.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.286+0000 7f20a0400640 1 --2- 172.21.15.100:0/3890086355 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f20780683b0 0x7f207806a870 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.286+0000 7f20a0400640 1 --2- 172.21.15.100:0/3890086355 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f209815d600 0x7f2098165690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.286+0000 7f20a0400640 1 --2- 172.21.15.100:0/3890086355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f209815c7e0 0x7f209806de30 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.286+0000 7f20a0400640 1 --2- 172.21.15.100:0/3890086355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20980cca50 0x7f209814acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.286+0000 7f20a0400640 1 -- 172.21.15.100:0/3890086355 >> 172.21.15.100:0/3890086355 conn(0x7f20980cee80 msgr2=0x7f20980b6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:00.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.286+0000 7f20a0400640 1 -- 172.21.15.100:0/3890086355 shutdown_connections 2024-04-23T14:33:00.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.286+0000 7f20a0400640 1 -- 172.21.15.100:0/3890086355 wait complete. 2024-04-23T14:33:00.306 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:33:00.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.442+0000 7f5cfcc00640 1 Processor -- start 2024-04-23T14:33:00.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.442+0000 7f5cfcc00640 1 -- start start 2024-04-23T14:33:00.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.446+0000 7f5cfcc00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5cf815b060 0x7f5cf815b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.446+0000 7f5cfcc00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5cf80734b0 0x7f5cf80b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.446+0000 7f5cfcc00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cf80cf910 con 0x7f5cf80d09b0 2024-04-23T14:33:00.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.446+0000 7f5cfcc00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f5cf80cfbf0 con 0x7f5cf815b060 2024-04-23T14:33:00.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.446+0000 7f5cfcc00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5cf80cfed0 con 0x7f5cf80734b0 2024-04-23T14:33:00.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.446+0000 7f5cefe00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f5cf80d09b0 0x7f5cf8072da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:59602/0 (socket says 172.21.15.100:59602) 2024-04-23T14:33:00.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.446+0000 7f5cefe00640 1 -- 172.21.15.100:0/3015931398 learned_addr learned my addr 172.21.15.100:0/3015931398 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.446+0000 7f5cf7400640 1 --2- 172.21.15.100:0/3015931398 >> v2:172.21.15.100:3300/0 conn(0x7f5cf815b060 0x7f5cf815b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.446+0000 7f5cf6a00640 1 -- 172.21.15.100:0/3015931398 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 171695035 0 0) 0x7f5cf80cf910 con 0x7f5cf80d09b0 2024-04-23T14:33:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.446+0000 7f5cf7e00640 1 --2- 172.21.15.100:0/3015931398 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5cf80734b0 0x7f5cf80b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.446+0000 7f5cf6a00640 1 -- 172.21.15.100:0/3015931398 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5cdc003530 con 0x7f5cf80d09b0 2024-04-23T14:33:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.446+0000 7f5cf7400640 1 -- 172.21.15.100:0/3015931398 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5cf80734b0 msgr2=0x7f5cf80b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.446+0000 7f5cf7400640 1 --2- 172.21.15.100:0/3015931398 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5cf80734b0 0x7f5cf80b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.446+0000 7f5cf7400640 1 -- 172.21.15.100:0/3015931398 >> v1:172.21.15.100:6789/0 conn(0x7f5cf80d09b0 legacy=0x7f5cf8072da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.446+0000 7f5cf7400640 1 -- 172.21.15.100:0/3015931398 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf80d01b0 con 0x7f5cf815b060 2024-04-23T14:33:00.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cf7400640 1 --2- 172.21.15.100:0/3015931398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cf815b060 0x7f5cf815b440 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80097c0 tx=0x7f5ce801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5355aa08ed73f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:00.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cf6a00640 1 -- 172.21.15.100:0/3015931398 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80ab600 con 0x7f5cf815b060 2024-04-23T14:33:00.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cf6a00640 1 -- 172.21.15.100:0/3015931398 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ce80b0070 con 0x7f5cf815b060 2024-04-23T14:33:00.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cf6a00640 1 -- 172.21.15.100:0/3015931398 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80ab9c0 con 0x7f5cf815b060 2024-04-23T14:33:00.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cfcc00640 1 -- 172.21.15.100:0/3015931398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cf815b060 msgr2=0x7f5cf815b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cfcc00640 1 --2- 172.21.15.100:0/3015931398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cf815b060 0x7f5cf815b440 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80097c0 tx=0x7f5ce801c3e0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cfcc00640 1 -- 172.21.15.100:0/3015931398 shutdown_connections 2024-04-23T14:33:00.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cfcc00640 1 --2- 172.21.15.100:0/3015931398 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5cf80734b0 0x7f5cf80b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cfcc00640 1 --2- 172.21.15.100:0/3015931398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cf815b060 0x7f5cf815b440 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cfcc00640 1 -- 172.21.15.100:0/3015931398 >> 172.21.15.100:0/3015931398 conn(0x7f5cf80d1290 msgr2=0x7f5cf80d16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:00.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cfcc00640 1 -- 172.21.15.100:0/3015931398 shutdown_connections 2024-04-23T14:33:00.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cfcc00640 1 -- 172.21.15.100:0/3015931398 wait complete. 2024-04-23T14:33:00.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cfcc00640 1 Processor -- start 2024-04-23T14:33:00.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cfcc00640 1 -- start start 2024-04-23T14:33:00.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cfcc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cf80734b0 0x7f5cf8140620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cfcc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cf80d09b0 0x7f5cf8161710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cfcc00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5cf815b060 0x7f5cf8161c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cfcc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf806dec0 con 0x7f5cf80734b0 2024-04-23T14:33:00.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cfcc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf806dd40 con 0x7f5cf80d09b0 2024-04-23T14:33:00.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cfcc00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5cf806e040 con 0x7f5cf815b060 2024-04-23T14:33:00.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cf7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cf80734b0 0x7f5cf8140620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cf7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cf80734b0 0x7f5cf8140620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49332/0 (socket says 172.21.15.100:49332) 2024-04-23T14:33:00.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.450+0000 7f5cf7400640 1 -- 172.21.15.100:0/413812264 learned_addr learned my addr 172.21.15.100:0/413812264 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:00.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.454+0000 7f5cefe00640 1 --2- 172.21.15.100:0/413812264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cf80d09b0 0x7f5cf8161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.454+0000 7f5cf7e00640 1 --2- 172.21.15.100:0/413812264 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5cf815b060 0x7f5cf8161c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.454+0000 7f5cf7400640 1 -- 172.21.15.100:0/413812264 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5cf815b060 msgr2=0x7f5cf8161c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.454+0000 7f5cf7400640 1 --2- 172.21.15.100:0/413812264 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5cf815b060 0x7f5cf8161c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.454+0000 7f5cf7400640 1 -- 172.21.15.100:0/413812264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cf80d09b0 msgr2=0x7f5cf8161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.454+0000 7f5cf7400640 1 --2- 172.21.15.100:0/413812264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cf80d09b0 0x7f5cf8161710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.454+0000 7f5cf7400640 1 -- 172.21.15.100:0/413812264 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf8140b60 con 0x7f5cf80734b0 2024-04-23T14:33:00.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.454+0000 7f5cefe00640 1 --2- 172.21.15.100:0/413812264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cf80d09b0 0x7f5cf8161710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:00.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.454+0000 7f5cf7400640 1 --2- 172.21.15.100:0/413812264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cf80734b0 0x7f5cf8140620 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80ad040 tx=0x7f5ce801ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:00.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.454+0000 7f5cef400640 1 -- 172.21.15.100:0/413812264 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80bf610 con 0x7f5cf80734b0 2024-04-23T14:33:00.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.454+0000 7f5cef400640 1 -- 172.21.15.100:0/413812264 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ce80bf7d0 con 0x7f5cf80734b0 2024-04-23T14:33:00.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.454+0000 7f5cfcc00640 1 -- 172.21.15.100:0/413812264 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf8162280 con 0x7f5cf80734b0 2024-04-23T14:33:00.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.454+0000 7f5cef400640 1 -- 172.21.15.100:0/413812264 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80bfbc0 con 0x7f5cf80734b0 2024-04-23T14:33:00.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.454+0000 7f5cfcc00640 1 -- 172.21.15.100:0/413812264 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf81410d0 con 0x7f5cf80734b0 2024-04-23T14:33:00.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.454+0000 7f5cfcc00640 1 -- 172.21.15.100:0/413812264 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf80691e0 con 0x7f5cf80734b0 2024-04-23T14:33:00.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.454+0000 7f5cef400640 1 -- 172.21.15.100:0/413812264 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5ce80bd020 con 0x7f5cf80734b0 2024-04-23T14:33:00.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.454+0000 7f5cef400640 1 --2- 172.21.15.100:0/413812264 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5cd4068380 0x7f5cd406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.458+0000 7f5cef400640 1 -- 172.21.15.100:0/413812264 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5ce8121350 con 0x7f5cf80734b0 2024-04-23T14:33:00.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.458+0000 7f5cefe00640 1 --2- 172.21.15.100:0/413812264 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5cd4068380 0x7f5cd406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.458+0000 7f5cefe00640 1 --2- 172.21.15.100:0/413812264 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5cd4068380 0x7f5cd406a840 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8162aa0 tx=0x7f5ce0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:00.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.462+0000 7f5cef400640 1 -- 172.21.15.100:0/413812264 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5ce8016090 con 0x7f5cf80734b0 2024-04-23T14:33:00.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.618+0000 7f5cfcc00640 1 -- 172.21.15.100:0/413812264 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5cf815da40 con 0x7f5cf80734b0 2024-04-23T14:33:00.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.618+0000 7f5cef400640 1 -- 172.21.15.100:0/413812264 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f5ce80f13e0 con 0x7f5cf80734b0 2024-04-23T14:33:00.621 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:00.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.622+0000 7f5cfcc00640 1 -- 172.21.15.100:0/413812264 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5cd4068380 msgr2=0x7f5cd406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.622+0000 7f5cfcc00640 1 --2- 172.21.15.100:0/413812264 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5cd4068380 0x7f5cd406a840 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8162aa0 tx=0x7f5ce0009040 comp rx=0 tx=0).stop 2024-04-23T14:33:00.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.622+0000 7f5cfcc00640 1 -- 172.21.15.100:0/413812264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cf80734b0 msgr2=0x7f5cf8140620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.622+0000 7f5cfcc00640 1 --2- 172.21.15.100:0/413812264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cf80734b0 0x7f5cf8140620 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80ad040 tx=0x7f5ce801ac20 comp rx=0 tx=0).stop 2024-04-23T14:33:00.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.622+0000 7f5cfcc00640 1 -- 172.21.15.100:0/413812264 shutdown_connections 2024-04-23T14:33:00.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.622+0000 7f5cfcc00640 1 --2- 172.21.15.100:0/413812264 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5cd4068380 0x7f5cd406a840 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.622+0000 7f5cfcc00640 1 --2- 172.21.15.100:0/413812264 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5cf815b060 0x7f5cf8161c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.622+0000 7f5cfcc00640 1 --2- 172.21.15.100:0/413812264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cf80d09b0 0x7f5cf8161710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.622+0000 7f5cfcc00640 1 --2- 172.21.15.100:0/413812264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cf80734b0 0x7f5cf8140620 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.622+0000 7f5cfcc00640 1 -- 172.21.15.100:0/413812264 >> 172.21.15.100:0/413812264 conn(0x7f5cf80d1290 msgr2=0x7f5cf815e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:00.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.622+0000 7f5cfcc00640 1 -- 172.21.15.100:0/413812264 shutdown_connections 2024-04-23T14:33:00.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.622+0000 7f5cfcc00640 1 -- 172.21.15.100:0/413812264 wait complete. 2024-04-23T14:33:00.641 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:33:00.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.778+0000 7f630be00640 1 Processor -- start 2024-04-23T14:33:00.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.778+0000 7f630be00640 1 -- start start 2024-04-23T14:33:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.778+0000 7f630be00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f630406bb20 0x7f63040ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.778+0000 7f630be00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f63040cd130 0x7f63040ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.778+0000 7f630be00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f630406cbb0 con 0x7f630415c7a0 2024-04-23T14:33:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.778+0000 7f630be00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f63040cd5a0 con 0x7f63040cd130 2024-04-23T14:33:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.778+0000 7f630be00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f630406d450 con 0x7f630406bb20 2024-04-23T14:33:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.778+0000 7f6309400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f630415c7a0 0x7f63040af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:59610/0 (socket says 172.21.15.100:59610) 2024-04-23T14:33:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.778+0000 7f6309400640 1 -- 172.21.15.100:0/3180283914 learned_addr learned my addr 172.21.15.100:0/3180283914 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.778+0000 7f6309e00640 1 --2- 172.21.15.100:0/3180283914 >> v2:172.21.15.100:3300/0 conn(0x7f63040cd130 0x7f63040ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.782+0000 7f6308a00640 1 --2- 172.21.15.100:0/3180283914 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f630406bb20 0x7f63040ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.782+0000 7f6309e00640 1 -- 172.21.15.100:0/3180283914 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f630406bb20 msgr2=0x7f63040ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.782+0000 7f6309e00640 1 --2- 172.21.15.100:0/3180283914 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f630406bb20 0x7f63040ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.782+0000 7f6309e00640 1 -- 172.21.15.100:0/3180283914 >> v1:172.21.15.100:6789/0 conn(0x7f630415c7a0 legacy=0x7f63040af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.782+0000 7f6309e00640 1 -- 172.21.15.100:0/3180283914 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630406ce90 con 0x7f63040cd130 2024-04-23T14:33:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.782+0000 7f6303e00640 1 -- 172.21.15.100:0/3180283914 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2025369168 0 0) 0x7f630406cbb0 con 0x7f630415c7a0 2024-04-23T14:33:00.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.782+0000 7f6309e00640 1 --2- 172.21.15.100:0/3180283914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63040cd130 0x7f63040ad430 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009d90 tx=0x7f62fc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14096f141e91adfc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:00.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.782+0000 7f6303e00640 1 -- 172.21.15.100:0/3180283914 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc0b2070 con 0x7f63040cd130 2024-04-23T14:33:00.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.782+0000 7f6303e00640 1 -- 172.21.15.100:0/3180283914 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62fc0a57a0 con 0x7f63040cd130 2024-04-23T14:33:00.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.782+0000 7f6303e00640 1 -- 172.21.15.100:0/3180283914 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc0a5b70 con 0x7f63040cd130 2024-04-23T14:33:00.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.782+0000 7f630be00640 1 -- 172.21.15.100:0/3180283914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63040cd130 msgr2=0x7f63040ad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.782+0000 7f630be00640 1 --2- 172.21.15.100:0/3180283914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63040cd130 0x7f63040ad430 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009d90 tx=0x7f62fc01c8e0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f630be00640 1 -- 172.21.15.100:0/3180283914 shutdown_connections 2024-04-23T14:33:00.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f630be00640 1 --2- 172.21.15.100:0/3180283914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63040cd130 0x7f63040ad430 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f630be00640 1 --2- 172.21.15.100:0/3180283914 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f630406bb20 0x7f63040ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f630be00640 1 -- 172.21.15.100:0/3180283914 >> 172.21.15.100:0/3180283914 conn(0x7f63040b5200 msgr2=0x7f63040b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:00.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f630be00640 1 -- 172.21.15.100:0/3180283914 shutdown_connections 2024-04-23T14:33:00.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f630be00640 1 -- 172.21.15.100:0/3180283914 wait complete. 2024-04-23T14:33:00.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f630be00640 1 Processor -- start 2024-04-23T14:33:00.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f630be00640 1 -- start start 2024-04-23T14:33:00.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f630be00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f630406bb20 0x7f630409dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f630be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63040cd130 0x7f6304158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f630be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f630415c7a0 0x7f6304158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f630be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f63040ae510 con 0x7f63040cd130 2024-04-23T14:33:00.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f630be00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f63040ae390 con 0x7f630415c7a0 2024-04-23T14:33:00.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f630be00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f63040ae690 con 0x7f630406bb20 2024-04-23T14:33:00.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f6308a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63040cd130 0x7f6304158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f6308a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63040cd130 0x7f6304158330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49340/0 (socket says 172.21.15.100:49340) 2024-04-23T14:33:00.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f6308a00640 1 -- 172.21.15.100:0/901870524 learned_addr learned my addr 172.21.15.100:0/901870524 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:00.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f6309400640 1 --2- 172.21.15.100:0/901870524 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f630406bb20 0x7f630409dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f6309e00640 1 --2- 172.21.15.100:0/901870524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f630415c7a0 0x7f6304158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f6308a00640 1 -- 172.21.15.100:0/901870524 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f630406bb20 msgr2=0x7f630409dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f6308a00640 1 --2- 172.21.15.100:0/901870524 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f630406bb20 0x7f630409dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f6308a00640 1 -- 172.21.15.100:0/901870524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f630415c7a0 msgr2=0x7f6304158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f6308a00640 1 --2- 172.21.15.100:0/901870524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f630415c7a0 0x7f6304158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f6308a00640 1 -- 172.21.15.100:0/901870524 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63040740e0 con 0x7f63040cd130 2024-04-23T14:33:00.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f6308a00640 1 --2- 172.21.15.100:0/901870524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63040cd130 0x7f6304158330 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f62f000d800 tx=0x7f62f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:00.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f6301600640 1 -- 172.21.15.100:0/901870524 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f000b6c0 con 0x7f63040cd130 2024-04-23T14:33:00.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.786+0000 7f6301600640 1 -- 172.21.15.100:0/901870524 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62f0019070 con 0x7f63040cd130 2024-04-23T14:33:00.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.790+0000 7f6301600640 1 -- 172.21.15.100:0/901870524 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f000ba80 con 0x7f63040cd130 2024-04-23T14:33:00.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.790+0000 7f630be00640 1 -- 172.21.15.100:0/901870524 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63041645f0 con 0x7f63040cd130 2024-04-23T14:33:00.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.790+0000 7f630be00640 1 -- 172.21.15.100:0/901870524 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63040c3d10 con 0x7f63040cd130 2024-04-23T14:33:00.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.790+0000 7f630be00640 1 -- 172.21.15.100:0/901870524 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6304068ad0 con 0x7f63040cd130 2024-04-23T14:33:00.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.790+0000 7f6301600640 1 -- 172.21.15.100:0/901870524 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f62f000bc20 con 0x7f63040cd130 2024-04-23T14:33:00.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.790+0000 7f6301600640 1 --2- 172.21.15.100:0/901870524 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f62e4068380 0x7f62e406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.790+0000 7f6301600640 1 -- 172.21.15.100:0/901870524 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f62f00871c0 con 0x7f63040cd130 2024-04-23T14:33:00.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.790+0000 7f6309400640 1 --2- 172.21.15.100:0/901870524 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f62e4068380 0x7f62e406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.794+0000 7f6309400640 1 --2- 172.21.15.100:0/901870524 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f62e4068380 0x7f62e406a840 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f62f8000f70 tx=0x7f62f8007b60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:00.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.794+0000 7f6301600640 1 -- 172.21.15.100:0/901870524 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f62f0057380 con 0x7f63040cd130 2024-04-23T14:33:00.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.950+0000 7f630be00640 1 -- 172.21.15.100:0/901870524 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f63040cf050 con 0x7f63040cd130 2024-04-23T14:33:00.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.954+0000 7f6301600640 1 -- 172.21.15.100:0/901870524 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f62f0056c80 con 0x7f63040cd130 2024-04-23T14:33:00.954 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:33:00.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.954+0000 7f630be00640 1 -- 172.21.15.100:0/901870524 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f62e4068380 msgr2=0x7f62e406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.954+0000 7f630be00640 1 --2- 172.21.15.100:0/901870524 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f62e4068380 0x7f62e406a840 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f62f8000f70 tx=0x7f62f8007b60 comp rx=0 tx=0).stop 2024-04-23T14:33:00.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.954+0000 7f630be00640 1 -- 172.21.15.100:0/901870524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63040cd130 msgr2=0x7f6304158330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.954+0000 7f630be00640 1 --2- 172.21.15.100:0/901870524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63040cd130 0x7f6304158330 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f62f000d800 tx=0x7f62f000dcc0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.954+0000 7f630be00640 1 -- 172.21.15.100:0/901870524 shutdown_connections 2024-04-23T14:33:00.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.954+0000 7f630be00640 1 --2- 172.21.15.100:0/901870524 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f62e4068380 0x7f62e406a840 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.954+0000 7f630be00640 1 --2- 172.21.15.100:0/901870524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f630415c7a0 0x7f6304158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.954+0000 7f630be00640 1 --2- 172.21.15.100:0/901870524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63040cd130 0x7f6304158330 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.954+0000 7f630be00640 1 --2- 172.21.15.100:0/901870524 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f630406bb20 0x7f630409dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.954+0000 7f630be00640 1 -- 172.21.15.100:0/901870524 >> 172.21.15.100:0/901870524 conn(0x7f63040b5200 msgr2=0x7f6304155fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:00.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.954+0000 7f630be00640 1 -- 172.21.15.100:0/901870524 shutdown_connections 2024-04-23T14:33:00.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:00.954+0000 7f630be00640 1 -- 172.21.15.100:0/901870524 wait complete. 2024-04-23T14:33:00.976 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-23T14:33:01.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.110+0000 7fb8d7c00640 1 Processor -- start 2024-04-23T14:33:01.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.110+0000 7fb8d7c00640 1 -- start start 2024-04-23T14:33:01.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8d7c00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fb8d006f8c0 0x7fb8d006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8d7c00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8d006dd60 con 0x7fb8d00d0170 2024-04-23T14:33:01.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8d7c00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8d00d09c0 con 0x7fb8d00ce7f0 2024-04-23T14:33:01.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8d7c00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fb8d00ad050 con 0x7fb8d006f8c0 2024-04-23T14:33:01.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8cfe00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fb8d00d0170 0x7fb8d00ce0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:59614/0 (socket says 172.21.15.100:59614) 2024-04-23T14:33:01.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8cfe00640 1 -- 172.21.15.100:0/2277319063 learned_addr learned my addr 172.21.15.100:0/2277319063 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8d5200640 1 --2- 172.21.15.100:0/2277319063 >> v2:172.21.15.151:3300/0 conn(0x7fb8d006f8c0 0x7fb8d006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8cf400640 1 -- 172.21.15.100:0/2277319063 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 245977494 0 0) 0x7fb8b8001120 con 0x7fb8d00d0170 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8cf400640 1 -- 172.21.15.100:0/2277319063 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8b4005740 con 0x7fb8d00d0170 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8cf400640 1 -- 172.21.15.100:0/2277319063 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 866996977 0 0) 0x7fb8d00d09c0 con 0x7fb8d00ce7f0 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8cf400640 1 -- 172.21.15.100:0/2277319063 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8b8001120 con 0x7fb8d00ce7f0 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8cf400640 1 -- 172.21.15.100:0/2277319063 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3896944835 0 0) 0x7fb8b4005740 con 0x7fb8d00d0170 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8cf400640 1 -- 172.21.15.100:0/2277319063 >> v2:172.21.15.151:3300/0 conn(0x7fb8d006f8c0 msgr2=0x7fb8d006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8cf400640 1 --2- 172.21.15.100:0/2277319063 >> v2:172.21.15.151:3300/0 conn(0x7fb8d006f8c0 0x7fb8d006c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8cf400640 1 -- 172.21.15.100:0/2277319063 >> v1:172.21.15.151:6789/0 conn(0x7fb8d00ce7f0 legacy=0x7fb8d00add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8cf400640 1 -- 172.21.15.100:0/2277319063 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d00ad2a0 con 0x7fb8d00d0170 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8cf400640 1 -- 172.21.15.100:0/2277319063 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7fb8b8001850 con 0x7fb8d00d0170 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8cf400640 1 -- 172.21.15.100:0/2277319063 >> v1:172.21.15.100:6789/0 conn(0x7fb8d00d0170 legacy=0x7fb8d00ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8cf400640 1 --2- 172.21.15.100:0/2277319063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b4005d60 0x7fb8b4006160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8cf400640 1 --2- 172.21.15.100:0/2277319063 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb8b4006a20 0x7fb8b4006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8cf400640 1 --2- 172.21.15.100:0/2277319063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8b40073e0 0x7fb8b400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8cf400640 1 -- 172.21.15.100:0/2277319063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d00ad050 con 0x7fb8b4005d60 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8cf400640 1 -- 172.21.15.100:0/2277319063 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b400e260 con 0x7fb8b40073e0 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8cf400640 1 -- 172.21.15.100:0/2277319063 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb8b400e460 con 0x7fb8b4006a20 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8d5200640 1 --2- 172.21.15.100:0/2277319063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b4005d60 0x7fb8b4006160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8d5c00640 1 --2- 172.21.15.100:0/2277319063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8b40073e0 0x7fb8b400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8cfe00640 1 --2- 172.21.15.100:0/2277319063 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb8b4006a20 0x7fb8b4006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8d5200640 1 -- 172.21.15.100:0/2277319063 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb8b4006a20 msgr2=0x7fb8b4006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:01.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8d5200640 1 --2- 172.21.15.100:0/2277319063 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb8b4006a20 0x7fb8b4006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8d5200640 1 -- 172.21.15.100:0/2277319063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8b40073e0 msgr2=0x7fb8b400dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:01.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8d5200640 1 --2- 172.21.15.100:0/2277319063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8b40073e0 0x7fb8b400dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8d5200640 1 -- 172.21.15.100:0/2277319063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b4005740 con 0x7fb8b4005d60 2024-04-23T14:33:01.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.114+0000 7fb8d5c00640 1 -- 172.21.15.100:0/2277319063 reap_dead start 2024-04-23T14:33:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.118+0000 7fb8d5200640 1 --2- 172.21.15.100:0/2277319063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b4005d60 0x7fb8b4006160 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009fc0 tx=0x7fb8c001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19ce7b221edd2d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.118+0000 7fb8cf400640 1 -- 172.21.15.100:0/2277319063 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c00b2070 con 0x7fb8b4005d60 2024-04-23T14:33:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.118+0000 7fb8cf400640 1 -- 172.21.15.100:0/2277319063 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c00a5b80 con 0x7fb8b4005d60 2024-04-23T14:33:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.118+0000 7fb8cf400640 1 -- 172.21.15.100:0/2277319063 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0009c10 con 0x7fb8b4005d60 2024-04-23T14:33:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.118+0000 7fb8d7c00640 1 -- 172.21.15.100:0/2277319063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b4005d60 msgr2=0x7fb8b4006160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.118+0000 7fb8d7c00640 1 --2- 172.21.15.100:0/2277319063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b4005d60 0x7fb8b4006160 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009fc0 tx=0x7fb8c001cc80 comp rx=0 tx=0).stop 2024-04-23T14:33:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.118+0000 7fb8d7c00640 1 -- 172.21.15.100:0/2277319063 shutdown_connections 2024-04-23T14:33:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.118+0000 7fb8d7c00640 1 --2- 172.21.15.100:0/2277319063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b4005d60 0x7fb8b4006160 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.118+0000 7fb8d7c00640 1 -- 172.21.15.100:0/2277319063 >> 172.21.15.100:0/2277319063 conn(0x7fb8d00b5200 msgr2=0x7fb8d00b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:01.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.118+0000 7fb8d7c00640 1 -- 172.21.15.100:0/2277319063 shutdown_connections 2024-04-23T14:33:01.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.118+0000 7fb8d7c00640 1 -- 172.21.15.100:0/2277319063 wait complete. 2024-04-23T14:33:01.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.118+0000 7fb8d7c00640 1 Processor -- start 2024-04-23T14:33:01.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.118+0000 7fb8d7c00640 1 -- start start 2024-04-23T14:33:01.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8d7c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b4005d60 0x7fb8d015e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8d7c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb8d01f30d0 0x7fb8d00cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8d7c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8d00ccf20 0x7fb8d01e8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8d7c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d00d0760 con 0x7fb8b4005d60 2024-04-23T14:33:01.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8d7c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d00ad8f0 con 0x7fb8d00ccf20 2024-04-23T14:33:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8d7c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb8d015f8d0 con 0x7fb8d01f30d0 2024-04-23T14:33:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8d5200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b4005d60 0x7fb8d015e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8d5200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b4005d60 0x7fb8d015e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49364/0 (socket says 172.21.15.100:49364) 2024-04-23T14:33:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8d5200640 1 -- 172.21.15.100:0/1633229316 learned_addr learned my addr 172.21.15.100:0/1633229316 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8cfe00640 1 --2- 172.21.15.100:0/1633229316 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb8d01f30d0 0x7fb8d00cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8d5c00640 1 --2- 172.21.15.100:0/1633229316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8d00ccf20 0x7fb8d01e8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8d5200640 1 -- 172.21.15.100:0/1633229316 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb8d01f30d0 msgr2=0x7fb8d00cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8d5200640 1 --2- 172.21.15.100:0/1633229316 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb8d01f30d0 0x7fb8d00cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8d5200640 1 -- 172.21.15.100:0/1633229316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8d00ccf20 msgr2=0x7fb8d01e8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8d5200640 1 --2- 172.21.15.100:0/1633229316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8d00ccf20 0x7fb8d01e8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8d5200640 1 -- 172.21.15.100:0/1633229316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d01e70e0 con 0x7fb8b4005d60 2024-04-23T14:33:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8cfe00640 1 --2- 172.21.15.100:0/1633229316 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb8d01f30d0 0x7fb8d00cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:01.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8d5200640 1 --2- 172.21.15.100:0/1633229316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b4005d60 0x7fb8d015e5b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb8c001c6f0 tx=0x7fb8c0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8ccc00640 1 -- 172.21.15.100:0/1633229316 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c00ba070 con 0x7fb8b4005d60 2024-04-23T14:33:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8ccc00640 1 -- 172.21.15.100:0/1633229316 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c00adac0 con 0x7fb8b4005d60 2024-04-23T14:33:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8d7c00640 1 -- 172.21.15.100:0/1633229316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d01e7380 con 0x7fb8b4005d60 2024-04-23T14:33:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8ccc00640 1 -- 172.21.15.100:0/1633229316 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c00b2040 con 0x7fb8b4005d60 2024-04-23T14:33:01.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8d7c00640 1 -- 172.21.15.100:0/1633229316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d01e7940 con 0x7fb8b4005d60 2024-04-23T14:33:01.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8d7c00640 1 -- 172.21.15.100:0/1633229316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d0068ad0 con 0x7fb8b4005d60 2024-04-23T14:33:01.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.122+0000 7fb8ccc00640 1 -- 172.21.15.100:0/1633229316 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb8c00b6410 con 0x7fb8b4005d60 2024-04-23T14:33:01.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.126+0000 7fb8ccc00640 1 --2- 172.21.15.100:0/1633229316 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb8ac068380 0x7fb8ac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.126+0000 7fb8ccc00640 1 -- 172.21.15.100:0/1633229316 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb8c01213c0 con 0x7fb8b4005d60 2024-04-23T14:33:01.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.126+0000 7fb8cfe00640 1 --2- 172.21.15.100:0/1633229316 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb8ac068380 0x7fb8ac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.126+0000 7fb8cfe00640 1 --2- 172.21.15.100:0/1633229316 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb8ac068380 0x7fb8ac06a840 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00b1970 tx=0x7fb8b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.130+0000 7fb8ccc00640 1 -- 172.21.15.100:0/1633229316 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb8c0016990 con 0x7fb8b4005d60 2024-04-23T14:33:01.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.282+0000 7fb8d7c00640 1 -- 172.21.15.100:0/1633229316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb8d006cc00 con 0x7fb8b4005d60 2024-04-23T14:33:01.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.286+0000 7fb8ccc00640 1 -- 172.21.15.100:0/1633229316 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fb8c00f0fc0 con 0x7fb8b4005d60 2024-04-23T14:33:01.286 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.286+0000 7fb8d7c00640 1 -- 172.21.15.100:0/1633229316 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb8ac068380 msgr2=0x7fb8ac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.286+0000 7fb8d7c00640 1 --2- 172.21.15.100:0/1633229316 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb8ac068380 0x7fb8ac06a840 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00b1970 tx=0x7fb8b8009040 comp rx=0 tx=0).stop 2024-04-23T14:33:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.286+0000 7fb8d7c00640 1 -- 172.21.15.100:0/1633229316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b4005d60 msgr2=0x7fb8d015e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.286+0000 7fb8d7c00640 1 --2- 172.21.15.100:0/1633229316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b4005d60 0x7fb8d015e5b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb8c001c6f0 tx=0x7fb8c0009750 comp rx=0 tx=0).stop 2024-04-23T14:33:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.286+0000 7fb8d7c00640 1 -- 172.21.15.100:0/1633229316 shutdown_connections 2024-04-23T14:33:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.286+0000 7fb8d7c00640 1 --2- 172.21.15.100:0/1633229316 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb8ac068380 0x7fb8ac06a840 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.286+0000 7fb8d7c00640 1 --2- 172.21.15.100:0/1633229316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8d00ccf20 0x7fb8d01e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.286+0000 7fb8d7c00640 1 --2- 172.21.15.100:0/1633229316 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb8d01f30d0 0x7fb8d00cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.286+0000 7fb8d7c00640 1 --2- 172.21.15.100:0/1633229316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b4005d60 0x7fb8d015e5b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.286+0000 7fb8d7c00640 1 -- 172.21.15.100:0/1633229316 >> 172.21.15.100:0/1633229316 conn(0x7fb8d00b5200 msgr2=0x7fb8d006df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.286+0000 7fb8d7c00640 1 -- 172.21.15.100:0/1633229316 shutdown_connections 2024-04-23T14:33:01.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.286+0000 7fb8d7c00640 1 -- 172.21.15.100:0/1633229316 wait complete. 2024-04-23T14:33:01.307 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.442+0000 7fd697000640 1 Processor -- start 2024-04-23T14:33:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.442+0000 7fd697000640 1 -- start start 2024-04-23T14:33:01.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.442+0000 7fd697000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd6900ad870 0x7fd6900b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.442+0000 7fd697000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd690160de0 0x7fd69006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.442+0000 7fd697000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd690160320 con 0x7fd69015c470 2024-04-23T14:33:01.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.442+0000 7fd697000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fd6900cc7e0 con 0x7fd6900ad870 2024-04-23T14:33:01.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.442+0000 7fd697000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd6901605e0 con 0x7fd690160de0 2024-04-23T14:33:01.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.442+0000 7fd695000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd690160de0 0x7fd69006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.442+0000 7fd695000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd690160de0 0x7fd69006bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:60830/0 (socket says 172.21.15.100:60830) 2024-04-23T14:33:01.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.442+0000 7fd695000640 1 -- 172.21.15.100:0/1284435385 learned_addr learned my addr 172.21.15.100:0/1284435385 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:01.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.442+0000 7fd68f400640 1 --2- 172.21.15.100:0/1284435385 >> v2:172.21.15.151:3300/0 conn(0x7fd6900ad870 0x7fd6900b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.442+0000 7fd695000640 1 -- 172.21.15.100:0/1284435385 >> v2:172.21.15.151:3300/0 conn(0x7fd6900ad870 msgr2=0x7fd6900b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:01.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.442+0000 7fd695000640 1 --2- 172.21.15.100:0/1284435385 >> v2:172.21.15.151:3300/0 conn(0x7fd6900ad870 0x7fd6900b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.442+0000 7fd68ea00640 1 -- 172.21.15.100:0/1284435385 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2172441654 0 0) 0x7fd690160320 con 0x7fd69015c470 2024-04-23T14:33:01.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.442+0000 7fd695000640 1 -- 172.21.15.100:0/1284435385 >> v1:172.21.15.151:6789/0 conn(0x7fd69015c470 legacy=0x7fd6900ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.442+0000 7fd695000640 1 -- 172.21.15.100:0/1284435385 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6901608c0 con 0x7fd690160de0 2024-04-23T14:33:01.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.442+0000 7fd68f400640 1 --2- 172.21.15.100:0/1284435385 >> v2:172.21.15.151:3300/0 conn(0x7fd6900ad870 0x7fd6900b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:33:01.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.446+0000 7fd695000640 1 --2- 172.21.15.100:0/1284435385 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd690160de0 0x7fd69006bab0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd688009e50 tx=0x7fd68801c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2807ba7806a7fb0c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.446+0000 7fd68ea00640 1 -- 172.21.15.100:0/1284435385 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6880a5770 con 0x7fd690160de0 2024-04-23T14:33:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.446+0000 7fd68ea00640 1 -- 172.21.15.100:0/1284435385 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6880a5910 con 0x7fd690160de0 2024-04-23T14:33:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.446+0000 7fd68ea00640 1 -- 172.21.15.100:0/1284435385 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6880a5ce0 con 0x7fd690160de0 2024-04-23T14:33:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.446+0000 7fd697000640 1 -- 172.21.15.100:0/1284435385 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd690160de0 msgr2=0x7fd69006bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.446+0000 7fd697000640 1 --2- 172.21.15.100:0/1284435385 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd690160de0 0x7fd69006bab0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd688009e50 tx=0x7fd68801c9a0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.446+0000 7fd697000640 1 -- 172.21.15.100:0/1284435385 shutdown_connections 2024-04-23T14:33:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.446+0000 7fd697000640 1 --2- 172.21.15.100:0/1284435385 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd690160de0 0x7fd69006bab0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.446+0000 7fd697000640 1 --2- 172.21.15.100:0/1284435385 >> v2:172.21.15.151:3300/0 conn(0x7fd6900ad870 0x7fd6900b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.446+0000 7fd697000640 1 -- 172.21.15.100:0/1284435385 >> 172.21.15.100:0/1284435385 conn(0x7fd6900b1e80 msgr2=0x7fd6900b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.446+0000 7fd697000640 1 -- 172.21.15.100:0/1284435385 shutdown_connections 2024-04-23T14:33:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.446+0000 7fd697000640 1 -- 172.21.15.100:0/1284435385 wait complete. 2024-04-23T14:33:01.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd697000640 1 Processor -- start 2024-04-23T14:33:01.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd697000640 1 -- start start 2024-04-23T14:33:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd697000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6900ad870 0x7fd690155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd697000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd69015c470 0x7fd6901f6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd697000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd690160de0 0x7fd6901f6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd697000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd69015ca60 con 0x7fd69015c470 2024-04-23T14:33:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd697000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6900cc580 con 0x7fd6900ad870 2024-04-23T14:33:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd697000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd690071ce0 con 0x7fd690160de0 2024-04-23T14:33:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd68f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd69015c470 0x7fd6901f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd68f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd69015c470 0x7fd6901f6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49368/0 (socket says 172.21.15.100:49368) 2024-04-23T14:33:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd68f400640 1 -- 172.21.15.100:0/3505529316 learned_addr learned my addr 172.21.15.100:0/3505529316 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd68fe00640 1 --2- 172.21.15.100:0/3505529316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6900ad870 0x7fd690155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd695000640 1 --2- 172.21.15.100:0/3505529316 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd690160de0 0x7fd6901f6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd68f400640 1 -- 172.21.15.100:0/3505529316 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd690160de0 msgr2=0x7fd6901f6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd68f400640 1 --2- 172.21.15.100:0/3505529316 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd690160de0 0x7fd6901f6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd68f400640 1 -- 172.21.15.100:0/3505529316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6900ad870 msgr2=0x7fd690155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd68f400640 1 --2- 172.21.15.100:0/3505529316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6900ad870 0x7fd690155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd68f400640 1 -- 172.21.15.100:0/3505529316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6901565c0 con 0x7fd69015c470 2024-04-23T14:33:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd695000640 1 --2- 172.21.15.100:0/3505529316 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd690160de0 0x7fd6901f6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd68f400640 1 --2- 172.21.15.100:0/3505529316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd69015c470 0x7fd6901f6440 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00e9d0 tx=0x7fd67c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd687e00640 1 -- 172.21.15.100:0/3505529316 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c017070 con 0x7fd69015c470 2024-04-23T14:33:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd687e00640 1 -- 172.21.15.100:0/3505529316 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd67c00ca60 con 0x7fd69015c470 2024-04-23T14:33:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd697000640 1 -- 172.21.15.100:0/3505529316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd690099400 con 0x7fd69015c470 2024-04-23T14:33:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd687e00640 1 -- 172.21.15.100:0/3505529316 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c00ce50 con 0x7fd69015c470 2024-04-23T14:33:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd697000640 1 -- 172.21.15.100:0/3505529316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd69015e6e0 con 0x7fd69015c470 2024-04-23T14:33:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.450+0000 7fd697000640 1 -- 172.21.15.100:0/3505529316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd690068ad0 con 0x7fd69015c470 2024-04-23T14:33:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.454+0000 7fd687e00640 1 -- 172.21.15.100:0/3505529316 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd67c01a070 con 0x7fd69015c470 2024-04-23T14:33:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.454+0000 7fd687e00640 1 --2- 172.21.15.100:0/3505529316 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd670068520 0x7fd67006a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.454+0000 7fd687e00640 1 -- 172.21.15.100:0/3505529316 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd67c0868d0 con 0x7fd69015c470 2024-04-23T14:33:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.454+0000 7fd68fe00640 1 --2- 172.21.15.100:0/3505529316 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd670068520 0x7fd67006a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.454+0000 7fd68fe00640 1 --2- 172.21.15.100:0/3505529316 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd670068520 0x7fd67006a9e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd680000f70 tx=0x7fd680009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:01.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.458+0000 7fd687e00640 1 -- 172.21.15.100:0/3505529316 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd67c056430 con 0x7fd69015c470 2024-04-23T14:33:01.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.614+0000 7fd697000640 1 -- 172.21.15.100:0/3505529316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd69006fea0 con 0x7fd69015c470 2024-04-23T14:33:01.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.614+0000 7fd687e00640 1 -- 172.21.15.100:0/3505529316 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18859 (secure 0 0 0) 0x7fd67c05a730 con 0x7fd69015c470 2024-04-23T14:33:01.618 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:01.618 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:49.784608+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:01.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.618+0000 7fd697000640 1 -- 172.21.15.100:0/3505529316 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd670068520 msgr2=0x7fd67006a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.618+0000 7fd697000640 1 --2- 172.21.15.100:0/3505529316 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd670068520 0x7fd67006a9e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd680000f70 tx=0x7fd680009040 comp rx=0 tx=0).stop 2024-04-23T14:33:01.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.618+0000 7fd697000640 1 -- 172.21.15.100:0/3505529316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd69015c470 msgr2=0x7fd6901f6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.618+0000 7fd697000640 1 --2- 172.21.15.100:0/3505529316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd69015c470 0x7fd6901f6440 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00e9d0 tx=0x7fd67c00ee90 comp rx=0 tx=0).stop 2024-04-23T14:33:01.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.618+0000 7fd697000640 1 -- 172.21.15.100:0/3505529316 shutdown_connections 2024-04-23T14:33:01.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.618+0000 7fd697000640 1 --2- 172.21.15.100:0/3505529316 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd670068520 0x7fd67006a9e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.618+0000 7fd697000640 1 --2- 172.21.15.100:0/3505529316 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd690160de0 0x7fd6901f6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.618+0000 7fd697000640 1 --2- 172.21.15.100:0/3505529316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd69015c470 0x7fd6901f6440 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.618+0000 7fd697000640 1 --2- 172.21.15.100:0/3505529316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6900ad870 0x7fd690155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.618+0000 7fd697000640 1 -- 172.21.15.100:0/3505529316 >> 172.21.15.100:0/3505529316 conn(0x7fd6900b1e80 msgr2=0x7fd6900b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:01.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.618+0000 7fd697000640 1 -- 172.21.15.100:0/3505529316 shutdown_connections 2024-04-23T14:33:01.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.618+0000 7fd697000640 1 -- 172.21.15.100:0/3505529316 wait complete. 2024-04-23T14:33:01.639 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:32:06.835560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:32:07.930489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-23T14:32:11.636814+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-23T14:33:01.640 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:33:01.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.778+0000 7fd01ac00640 1 Processor -- start 2024-04-23T14:33:01.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.782+0000 7fd01ac00640 1 -- start start 2024-04-23T14:33:01.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.782+0000 7fd01ac00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd01415c820 0x7fd01406cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.782+0000 7fd01ac00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd01406d000 0x7fd0140cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.782+0000 7fd01ac00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd01406e640 con 0x7fd0140afd90 2024-04-23T14:33:01.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.782+0000 7fd01ac00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fd01406e920 con 0x7fd01406d000 2024-04-23T14:33:01.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.782+0000 7fd01ac00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd01406ec00 con 0x7fd01415c820 2024-04-23T14:33:01.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.782+0000 7fd01a200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd01406d000 0x7fd0140cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.782+0000 7fd018e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd01415c820 0x7fd01406cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.782+0000 7fd018e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd01415c820 0x7fd01406cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:60844/0 (socket says 172.21.15.100:60844) 2024-04-23T14:33:01.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.782+0000 7fd018e00640 1 -- 172.21.15.100:0/1848175144 learned_addr learned my addr 172.21.15.100:0/1848175144 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:01.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.782+0000 7fd01a200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd01406d000 0x7fd0140cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49370/0 (socket says 172.21.15.100:49370) 2024-04-23T14:33:01.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.782+0000 7fd01a200640 1 -- 172.21.15.100:0/1848175144 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd01415c820 msgr2=0x7fd01406cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:01.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.782+0000 7fd01a200640 1 --2- 172.21.15.100:0/1848175144 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd01415c820 0x7fd01406cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.782+0000 7fd01a200640 1 -- 172.21.15.100:0/1848175144 >> v1:172.21.15.151:6789/0 conn(0x7fd0140afd90 legacy=0x7fd01406de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.782+0000 7fd01a200640 1 -- 172.21.15.100:0/1848175144 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0140cd0f0 con 0x7fd01406d000 2024-04-23T14:33:01.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.786+0000 7fd01a200640 1 --2- 172.21.15.100:0/1848175144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd01406d000 0x7fd0140cd610 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009d90 tx=0x7fd00c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c655eb554cbeec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:01.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.786+0000 7fd013e00640 1 -- 172.21.15.100:0/1848175144 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c0b2070 con 0x7fd01406d000 2024-04-23T14:33:01.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.786+0000 7fd013e00640 1 -- 172.21.15.100:0/1848175144 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd00c0a57a0 con 0x7fd01406d000 2024-04-23T14:33:01.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.786+0000 7fd013e00640 1 -- 172.21.15.100:0/1848175144 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c0a5b70 con 0x7fd01406d000 2024-04-23T14:33:01.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.786+0000 7fd01ac00640 1 -- 172.21.15.100:0/1848175144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd01406d000 msgr2=0x7fd0140cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.786+0000 7fd01ac00640 1 --2- 172.21.15.100:0/1848175144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd01406d000 0x7fd0140cd610 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009d90 tx=0x7fd00c01c8e0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.786+0000 7fd01ac00640 1 -- 172.21.15.100:0/1848175144 shutdown_connections 2024-04-23T14:33:01.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.786+0000 7fd01ac00640 1 --2- 172.21.15.100:0/1848175144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd01406d000 0x7fd0140cd610 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.786+0000 7fd01ac00640 1 --2- 172.21.15.100:0/1848175144 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd01415c820 0x7fd01406cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.786+0000 7fd01ac00640 1 -- 172.21.15.100:0/1848175144 >> 172.21.15.100:0/1848175144 conn(0x7fd0140b2900 msgr2=0x7fd0140720b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:01.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.786+0000 7fd01ac00640 1 -- 172.21.15.100:0/1848175144 shutdown_connections 2024-04-23T14:33:01.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.786+0000 7fd01ac00640 1 -- 172.21.15.100:0/1848175144 wait complete. 2024-04-23T14:33:01.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.786+0000 7fd01ac00640 1 Processor -- start 2024-04-23T14:33:01.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.786+0000 7fd01ac00640 1 -- start start 2024-04-23T14:33:01.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd01ac00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd01406d000 0x7fd014147d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd01ac00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0140afd90 0x7fd01415d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd01ac00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd01415c820 0x7fd01414bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd01ac00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd0140ce6f0 con 0x7fd0140afd90 2024-04-23T14:33:01.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd01ac00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd0140ce570 con 0x7fd01415c820 2024-04-23T14:33:01.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd01ac00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd0140ce870 con 0x7fd01406d000 2024-04-23T14:33:01.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd018e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0140afd90 0x7fd01415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd018e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0140afd90 0x7fd01415d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49382/0 (socket says 172.21.15.100:49382) 2024-04-23T14:33:01.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd018e00640 1 -- 172.21.15.100:0/1907300423 learned_addr learned my addr 172.21.15.100:0/1907300423 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:01.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd019800640 1 --2- 172.21.15.100:0/1907300423 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd01406d000 0x7fd014147d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd01a200640 1 --2- 172.21.15.100:0/1907300423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd01415c820 0x7fd01414bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd018e00640 1 -- 172.21.15.100:0/1907300423 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd01406d000 msgr2=0x7fd014147d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd018e00640 1 --2- 172.21.15.100:0/1907300423 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd01406d000 0x7fd014147d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd018e00640 1 -- 172.21.15.100:0/1907300423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd01415c820 msgr2=0x7fd01414bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd018e00640 1 --2- 172.21.15.100:0/1907300423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd01415c820 0x7fd01414bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd018e00640 1 -- 172.21.15.100:0/1907300423 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01415de90 con 0x7fd0140afd90 2024-04-23T14:33:01.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd018e00640 1 --2- 172.21.15.100:0/1907300423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0140afd90 0x7fd01415d950 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd00000e8c0 tx=0x7fd00000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:01.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd011600640 1 -- 172.21.15.100:0/1907300423 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000017070 con 0x7fd0140afd90 2024-04-23T14:33:01.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd011600640 1 -- 172.21.15.100:0/1907300423 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd00000c940 con 0x7fd0140afd90 2024-04-23T14:33:01.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd011600640 1 -- 172.21.15.100:0/1907300423 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00000cd30 con 0x7fd0140afd90 2024-04-23T14:33:01.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd01ac00640 1 -- 172.21.15.100:0/1907300423 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd014160350 con 0x7fd0140afd90 2024-04-23T14:33:01.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd01ac00640 1 -- 172.21.15.100:0/1907300423 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd014149130 con 0x7fd0140afd90 2024-04-23T14:33:01.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.790+0000 7fd01ac00640 1 -- 172.21.15.100:0/1907300423 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0140691e0 con 0x7fd0140afd90 2024-04-23T14:33:01.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.794+0000 7fd011600640 1 -- 172.21.15.100:0/1907300423 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd00001a070 con 0x7fd0140afd90 2024-04-23T14:33:01.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.794+0000 7fd011600640 1 --2- 172.21.15.100:0/1907300423 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcff4068380 0x7fcff406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.794+0000 7fd011600640 1 -- 172.21.15.100:0/1907300423 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd000086c70 con 0x7fd0140afd90 2024-04-23T14:33:01.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.794+0000 7fd019800640 1 --2- 172.21.15.100:0/1907300423 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcff4068380 0x7fcff406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.794+0000 7fd019800640 1 --2- 172.21.15.100:0/1907300423 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcff4068380 0x7fcff406a840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd008007fd0 tx=0x7fd008009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:01.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.798+0000 7fd011600640 1 -- 172.21.15.100:0/1907300423 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd0000567e0 con 0x7fd0140afd90 2024-04-23T14:33:01.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.954+0000 7fd01ac00640 1 -- 172.21.15.100:0/1907300423 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd014073180 con 0x7fd0140afd90 2024-04-23T14:33:01.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.954+0000 7fd011600640 1 -- 172.21.15.100:0/1907300423 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fd00005aae0 con 0x7fd0140afd90 2024-04-23T14:33:01.957 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:33:01.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.958+0000 7fd01ac00640 1 -- 172.21.15.100:0/1907300423 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcff4068380 msgr2=0x7fcff406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.958+0000 7fd01ac00640 1 --2- 172.21.15.100:0/1907300423 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcff4068380 0x7fcff406a840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd008007fd0 tx=0x7fd008009040 comp rx=0 tx=0).stop 2024-04-23T14:33:01.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.958+0000 7fd01ac00640 1 -- 172.21.15.100:0/1907300423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0140afd90 msgr2=0x7fd01415d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.958+0000 7fd01ac00640 1 --2- 172.21.15.100:0/1907300423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0140afd90 0x7fd01415d950 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd00000e8c0 tx=0x7fd00000ed80 comp rx=0 tx=0).stop 2024-04-23T14:33:01.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.958+0000 7fd01ac00640 1 -- 172.21.15.100:0/1907300423 shutdown_connections 2024-04-23T14:33:01.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.958+0000 7fd01ac00640 1 --2- 172.21.15.100:0/1907300423 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcff4068380 0x7fcff406a840 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.958+0000 7fd01ac00640 1 --2- 172.21.15.100:0/1907300423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd01415c820 0x7fd01414bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.958+0000 7fd01ac00640 1 --2- 172.21.15.100:0/1907300423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0140afd90 0x7fd01415d950 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.958+0000 7fd01ac00640 1 --2- 172.21.15.100:0/1907300423 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd01406d000 0x7fd014147d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.958+0000 7fd01ac00640 1 -- 172.21.15.100:0/1907300423 >> 172.21.15.100:0/1907300423 conn(0x7fd0140b2900 msgr2=0x7fd01415b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:01.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.958+0000 7fd01ac00640 1 -- 172.21.15.100:0/1907300423 shutdown_connections 2024-04-23T14:33:01.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:01.958+0000 7fd01ac00640 1 -- 172.21.15.100:0/1907300423 wait complete. 2024-04-23T14:33:01.978 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:33:02.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.114+0000 7f2db9400640 1 Processor -- start 2024-04-23T14:33:02.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.114+0000 7f2db9400640 1 -- start start 2024-04-23T14:33:02.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.114+0000 7f2db9400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2db406f8c0 0x7f2db406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:02.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.114+0000 7f2db9400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f2db40d0170 0x7f2db40ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:02.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.114+0000 7f2db9400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2db406dd60 con 0x7f2db40ce620 2024-04-23T14:33:02.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.114+0000 7f2db9400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f2db40ad050 con 0x7f2db40d0170 2024-04-23T14:33:02.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.114+0000 7f2db9400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f2db40ad8f0 con 0x7f2db406f8c0 2024-04-23T14:33:02.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.118+0000 7f2db2800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2db406f8c0 0x7f2db406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:02.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.118+0000 7f2db2800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2db406f8c0 0x7f2db406c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:60856/0 (socket says 172.21.15.100:60856) 2024-04-23T14:33:02.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.118+0000 7f2da9e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f2db40d0170 0x7f2db40ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:02.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.118+0000 7f2db2800640 1 -- 172.21.15.100:0/3580447215 learned_addr learned my addr 172.21.15.100:0/3580447215 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:02.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.118+0000 7f2db3200640 1 --1- 172.21.15.100:0/3580447215 >> v1:172.21.15.151:6789/0 conn(0x7f2db40ce620 0x7f2db415f0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:60214/0 (socket says 172.21.15.100:60214) 2024-04-23T14:33:02.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.118+0000 7f2db2800640 1 -- 172.21.15.100:0/3580447215 >> v2:172.21.15.151:3300/0 conn(0x7f2db40d0170 msgr2=0x7f2db40ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:02.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.118+0000 7f2db2800640 1 --2- 172.21.15.100:0/3580447215 >> v2:172.21.15.151:3300/0 conn(0x7f2db40d0170 0x7f2db40ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.118+0000 7f2db2800640 1 -- 172.21.15.100:0/3580447215 >> v1:172.21.15.151:6789/0 conn(0x7f2db40ce620 legacy=0x7f2db415f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:02.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.118+0000 7f2db2800640 1 -- 172.21.15.100:0/3580447215 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db40d09c0 con 0x7f2db406f8c0 2024-04-23T14:33:02.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.118+0000 7f2db2800640 1 --2- 172.21.15.100:0/3580447215 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2db406f8c0 0x7f2db406c6c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009e50 tx=0x7f2da401ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28e327a88ef55d72 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:02.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.118+0000 7f2db1e00640 1 -- 172.21.15.100:0/3580447215 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da40a7940 con 0x7f2db406f8c0 2024-04-23T14:33:02.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.118+0000 7f2db1e00640 1 -- 172.21.15.100:0/3580447215 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2da40a7ae0 con 0x7f2db406f8c0 2024-04-23T14:33:02.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.118+0000 7f2db1e00640 1 -- 172.21.15.100:0/3580447215 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da40af3f0 con 0x7f2db406f8c0 2024-04-23T14:33:02.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db9400640 1 -- 172.21.15.100:0/3580447215 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2db406f8c0 msgr2=0x7f2db406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:02.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db9400640 1 --2- 172.21.15.100:0/3580447215 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2db406f8c0 0x7f2db406c6c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009e50 tx=0x7f2da401ec40 comp rx=0 tx=0).stop 2024-04-23T14:33:02.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db9400640 1 -- 172.21.15.100:0/3580447215 shutdown_connections 2024-04-23T14:33:02.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db9400640 1 --2- 172.21.15.100:0/3580447215 >> v2:172.21.15.151:3300/0 conn(0x7f2db40d0170 0x7f2db40ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db9400640 1 --2- 172.21.15.100:0/3580447215 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2db406f8c0 0x7f2db406c6c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db9400640 1 -- 172.21.15.100:0/3580447215 >> 172.21.15.100:0/3580447215 conn(0x7f2db40b5200 msgr2=0x7f2db40b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:02.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db9400640 1 -- 172.21.15.100:0/3580447215 shutdown_connections 2024-04-23T14:33:02.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db9400640 1 -- 172.21.15.100:0/3580447215 wait complete. 2024-04-23T14:33:02.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db9400640 1 Processor -- start 2024-04-23T14:33:02.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db9400640 1 -- start start 2024-04-23T14:33:02.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2db406f8c0 0x7f2db4072880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:02.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db40ce620 0x7f2db41561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:02.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db9400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2db40d0170 0x7f2db415a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:02.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db9400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2db40d06d0 con 0x7f2db406f8c0 2024-04-23T14:33:02.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db9400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2db40d0550 con 0x7f2db40ce620 2024-04-23T14:33:02.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db9400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f2db40ccc60 con 0x7f2db40d0170 2024-04-23T14:33:02.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2db406f8c0 0x7f2db4072880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:02.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2db406f8c0 0x7f2db4072880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49392/0 (socket says 172.21.15.100:49392) 2024-04-23T14:33:02.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db2800640 1 -- 172.21.15.100:0/961088763 learned_addr learned my addr 172.21.15.100:0/961088763 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:02.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db3200640 1 --2- 172.21.15.100:0/961088763 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2db40d0170 0x7f2db415a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:02.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2da9e00640 1 --2- 172.21.15.100:0/961088763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db40ce620 0x7f2db41561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:02.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db2800640 1 -- 172.21.15.100:0/961088763 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2db40d0170 msgr2=0x7f2db415a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:02.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db2800640 1 --2- 172.21.15.100:0/961088763 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2db40d0170 0x7f2db415a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db2800640 1 -- 172.21.15.100:0/961088763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db40ce620 msgr2=0x7f2db41561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:02.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db2800640 1 --2- 172.21.15.100:0/961088763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db40ce620 0x7f2db41561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db2800640 1 -- 172.21.15.100:0/961088763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db4073ba0 con 0x7f2db406f8c0 2024-04-23T14:33:02.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.122+0000 7f2db2800640 1 --2- 172.21.15.100:0/961088763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2db406f8c0 0x7f2db4072880 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2da401c870 tx=0x7f2da401b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:02.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.126+0000 7f2dab400640 1 -- 172.21.15.100:0/961088763 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da40be070 con 0x7f2db406f8c0 2024-04-23T14:33:02.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.126+0000 7f2dab400640 1 -- 172.21.15.100:0/961088763 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2da4007c50 con 0x7f2db406f8c0 2024-04-23T14:33:02.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.126+0000 7f2dab400640 1 -- 172.21.15.100:0/961088763 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da40a7940 con 0x7f2db406f8c0 2024-04-23T14:33:02.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.126+0000 7f2db9400640 1 -- 172.21.15.100:0/961088763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db4071da0 con 0x7f2db406f8c0 2024-04-23T14:33:02.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.126+0000 7f2db9400640 1 -- 172.21.15.100:0/961088763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db4072360 con 0x7f2db406f8c0 2024-04-23T14:33:02.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.126+0000 7f2db9400640 1 -- 172.21.15.100:0/961088763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2db4068ad0 con 0x7f2db406f8c0 2024-04-23T14:33:02.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.126+0000 7f2dab400640 1 -- 172.21.15.100:0/961088763 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2da40c1070 con 0x7f2db406f8c0 2024-04-23T14:33:02.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.126+0000 7f2dab400640 1 --2- 172.21.15.100:0/961088763 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2d90068380 0x7f2d9006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:02.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.126+0000 7f2dab400640 1 -- 172.21.15.100:0/961088763 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2da4120e70 con 0x7f2db406f8c0 2024-04-23T14:33:02.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.126+0000 7f2da9e00640 1 --2- 172.21.15.100:0/961088763 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2d90068380 0x7f2d9006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:02.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.126+0000 7f2da9e00640 1 --2- 172.21.15.100:0/961088763 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2d90068380 0x7f2d9006a840 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2db40b5690 tx=0x7f2d9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:02.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.130+0000 7f2dab400640 1 -- 172.21.15.100:0/961088763 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2da4018830 con 0x7f2db406f8c0 2024-04-23T14:33:02.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.290+0000 7f2db9400640 1 -- 172.21.15.100:0/961088763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2db406cc00 con 0x7f2db406f8c0 2024-04-23T14:33:02.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.290+0000 7f2dab400640 1 -- 172.21.15.100:0/961088763 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f2da40f10e0 con 0x7f2db406f8c0 2024-04-23T14:33:02.292 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:02.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.290+0000 7f2db9400640 1 -- 172.21.15.100:0/961088763 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2d90068380 msgr2=0x7f2d9006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:02.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.290+0000 7f2db9400640 1 --2- 172.21.15.100:0/961088763 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2d90068380 0x7f2d9006a840 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2db40b5690 tx=0x7f2d9c008040 comp rx=0 tx=0).stop 2024-04-23T14:33:02.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.290+0000 7f2db9400640 1 -- 172.21.15.100:0/961088763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2db406f8c0 msgr2=0x7f2db4072880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:02.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.290+0000 7f2db9400640 1 --2- 172.21.15.100:0/961088763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2db406f8c0 0x7f2db4072880 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2da401c870 tx=0x7f2da401b9e0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.290+0000 7f2db9400640 1 -- 172.21.15.100:0/961088763 shutdown_connections 2024-04-23T14:33:02.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.290+0000 7f2db9400640 1 --2- 172.21.15.100:0/961088763 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2d90068380 0x7f2d9006a840 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.290+0000 7f2db9400640 1 --2- 172.21.15.100:0/961088763 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2db40d0170 0x7f2db415a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.290+0000 7f2db9400640 1 --2- 172.21.15.100:0/961088763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db40ce620 0x7f2db41561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.290+0000 7f2db9400640 1 --2- 172.21.15.100:0/961088763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2db406f8c0 0x7f2db4072880 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.290+0000 7f2db9400640 1 -- 172.21.15.100:0/961088763 >> 172.21.15.100:0/961088763 conn(0x7f2db40b5200 msgr2=0x7f2db406df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:02.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.290+0000 7f2db9400640 1 -- 172.21.15.100:0/961088763 shutdown_connections 2024-04-23T14:33:02.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.290+0000 7f2db9400640 1 -- 172.21.15.100:0/961088763 wait complete. 2024-04-23T14:33:02.313 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:33:02.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.450+0000 7fc065c00640 1 Processor -- start 2024-04-23T14:33:02.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.450+0000 7fc065c00640 1 -- start start 2024-04-23T14:33:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.450+0000 7fc065c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0600cca50 0x7fc0600cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.450+0000 7fc065c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc06015c7e0 0x7fc060160170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.450+0000 7fc065c00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0600cd370 con 0x7fc06015d600 2024-04-23T14:33:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.450+0000 7fc065c00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc0600d2270 con 0x7fc06015c7e0 2024-04-23T14:33:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.450+0000 7fc065c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc06015db00 con 0x7fc0600cca50 2024-04-23T14:33:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.450+0000 7fc05f400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fc06015d600 0x7fc0601606b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:59616/0 (socket says 172.21.15.100:59616) 2024-04-23T14:33:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.450+0000 7fc05f400640 1 -- 172.21.15.100:0/597295047 learned_addr learned my addr 172.21.15.100:0/597295047 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.450+0000 7fc057e00640 1 --2- 172.21.15.100:0/597295047 >> v2:172.21.15.100:3300/0 conn(0x7fc06015c7e0 0x7fc060160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.450+0000 7fc05ea00640 1 --2- 172.21.15.100:0/597295047 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0600cca50 0x7fc0600cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.450+0000 7fc057e00640 1 -- 172.21.15.100:0/597295047 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0600cca50 msgr2=0x7fc0600cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.450+0000 7fc057e00640 1 --2- 172.21.15.100:0/597295047 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0600cca50 0x7fc0600cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.450+0000 7fc057e00640 1 -- 172.21.15.100:0/597295047 >> v1:172.21.15.100:6789/0 conn(0x7fc06015d600 legacy=0x7fc0601606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.450+0000 7fc057e00640 1 -- 172.21.15.100:0/597295047 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0600d04a0 con 0x7fc06015c7e0 2024-04-23T14:33:02.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.454+0000 7fc057e00640 1 --2- 172.21.15.100:0/597295047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc06015c7e0 0x7fc060160170 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc0480097c0 tx=0x7fc04801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97da5ac2a1926cad server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:02.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.454+0000 7fc05e000640 1 -- 172.21.15.100:0/597295047 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0480b1070 con 0x7fc06015c7e0 2024-04-23T14:33:02.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.454+0000 7fc05e000640 1 -- 172.21.15.100:0/597295047 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0480a8050 con 0x7fc06015c7e0 2024-04-23T14:33:02.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.454+0000 7fc05e000640 1 -- 172.21.15.100:0/597295047 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0480ac860 con 0x7fc06015c7e0 2024-04-23T14:33:02.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.454+0000 7fc065c00640 1 -- 172.21.15.100:0/597295047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc06015c7e0 msgr2=0x7fc060160170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:02.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.454+0000 7fc065c00640 1 --2- 172.21.15.100:0/597295047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc06015c7e0 0x7fc060160170 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc0480097c0 tx=0x7fc04801c3e0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.454+0000 7fc065c00640 1 -- 172.21.15.100:0/597295047 shutdown_connections 2024-04-23T14:33:02.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.454+0000 7fc065c00640 1 --2- 172.21.15.100:0/597295047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc06015c7e0 0x7fc060160170 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.454+0000 7fc065c00640 1 --2- 172.21.15.100:0/597295047 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0600cca50 0x7fc0600cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.454+0000 7fc065c00640 1 -- 172.21.15.100:0/597295047 >> 172.21.15.100:0/597295047 conn(0x7fc0600cee80 msgr2=0x7fc0600cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:02.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.454+0000 7fc065c00640 1 -- 172.21.15.100:0/597295047 shutdown_connections 2024-04-23T14:33:02.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.454+0000 7fc065c00640 1 -- 172.21.15.100:0/597295047 wait complete. 2024-04-23T14:33:02.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc065c00640 1 Processor -- start 2024-04-23T14:33:02.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc065c00640 1 -- start start 2024-04-23T14:33:02.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc065c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0600cca50 0x7fc0600c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:02.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc065c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc06015c7e0 0x7fc060150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:02.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc065c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc06015d600 0x7fc0600c7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:02.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc065c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc0600b5430 con 0x7fc0600cca50 2024-04-23T14:33:02.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc065c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc0600b52b0 con 0x7fc06015d600 2024-04-23T14:33:02.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc065c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc0600b3bf0 con 0x7fc06015c7e0 2024-04-23T14:33:02.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc05ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0600cca50 0x7fc0600c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:02.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc05ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0600cca50 0x7fc0600c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49404/0 (socket says 172.21.15.100:49404) 2024-04-23T14:33:02.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc05ea00640 1 -- 172.21.15.100:0/3827866847 learned_addr learned my addr 172.21.15.100:0/3827866847 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:02.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc057e00640 1 --2- 172.21.15.100:0/3827866847 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc06015c7e0 0x7fc060150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:02.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc05f400640 1 --2- 172.21.15.100:0/3827866847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc06015d600 0x7fc0600c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:02.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc05ea00640 1 -- 172.21.15.100:0/3827866847 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc06015c7e0 msgr2=0x7fc060150500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:02.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc05ea00640 1 --2- 172.21.15.100:0/3827866847 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc06015c7e0 0x7fc060150500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc05ea00640 1 -- 172.21.15.100:0/3827866847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc06015d600 msgr2=0x7fc0600c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:02.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc05ea00640 1 --2- 172.21.15.100:0/3827866847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc06015d600 0x7fc0600c7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc05ea00640 1 -- 172.21.15.100:0/3827866847 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0600c7650 con 0x7fc0600cca50 2024-04-23T14:33:02.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc05ea00640 1 --2- 172.21.15.100:0/3827866847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0600cca50 0x7fc0600c8990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc05000d5f0 tx=0x7fc05000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:02.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc056a00640 1 -- 172.21.15.100:0/3827866847 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0500125f0 con 0x7fc0600cca50 2024-04-23T14:33:02.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc056a00640 1 -- 172.21.15.100:0/3827866847 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc050019070 con 0x7fc0600cca50 2024-04-23T14:33:02.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc065c00640 1 -- 172.21.15.100:0/3827866847 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0600c78f0 con 0x7fc0600cca50 2024-04-23T14:33:02.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc056a00640 1 -- 172.21.15.100:0/3827866847 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0500129b0 con 0x7fc0600cca50 2024-04-23T14:33:02.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc065c00640 1 -- 172.21.15.100:0/3827866847 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0600c7eb0 con 0x7fc0600cca50 2024-04-23T14:33:02.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.458+0000 7fc065c00640 1 -- 172.21.15.100:0/3827866847 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc060068ad0 con 0x7fc0600cca50 2024-04-23T14:33:02.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.462+0000 7fc056a00640 1 -- 172.21.15.100:0/3827866847 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc050012b50 con 0x7fc0600cca50 2024-04-23T14:33:02.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.462+0000 7fc056a00640 1 --2- 172.21.15.100:0/3827866847 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc03c068380 0x7fc03c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:02.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.462+0000 7fc056a00640 1 -- 172.21.15.100:0/3827866847 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc050087280 con 0x7fc0600cca50 2024-04-23T14:33:02.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.462+0000 7fc057e00640 1 --2- 172.21.15.100:0/3827866847 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc03c068380 0x7fc03c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:02.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.462+0000 7fc057e00640 1 --2- 172.21.15.100:0/3827866847 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc03c068380 0x7fc03c06a840 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc0480ae000 tx=0x7fc048008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:02.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.466+0000 7fc056a00640 1 -- 172.21.15.100:0/3827866847 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc050011340 con 0x7fc0600cca50 2024-04-23T14:33:02.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.622+0000 7fc065c00640 1 -- 172.21.15.100:0/3827866847 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc060074f20 con 0x7fc0600cca50 2024-04-23T14:33:02.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.622+0000 7fc056a00640 1 -- 172.21.15.100:0/3827866847 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fc0500573a0 con 0x7fc0600cca50 2024-04-23T14:33:02.624 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:33:02.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.622+0000 7fc065c00640 1 -- 172.21.15.100:0/3827866847 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc03c068380 msgr2=0x7fc03c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:02.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.622+0000 7fc065c00640 1 --2- 172.21.15.100:0/3827866847 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc03c068380 0x7fc03c06a840 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc0480ae000 tx=0x7fc048008040 comp rx=0 tx=0).stop 2024-04-23T14:33:02.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.622+0000 7fc065c00640 1 -- 172.21.15.100:0/3827866847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0600cca50 msgr2=0x7fc0600c8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:02.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.622+0000 7fc065c00640 1 --2- 172.21.15.100:0/3827866847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0600cca50 0x7fc0600c8990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc05000d5f0 tx=0x7fc05000dab0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.622+0000 7fc065c00640 1 -- 172.21.15.100:0/3827866847 shutdown_connections 2024-04-23T14:33:02.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.622+0000 7fc065c00640 1 --2- 172.21.15.100:0/3827866847 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc03c068380 0x7fc03c06a840 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.622+0000 7fc065c00640 1 --2- 172.21.15.100:0/3827866847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc06015d600 0x7fc0600c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.622+0000 7fc065c00640 1 --2- 172.21.15.100:0/3827866847 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc06015c7e0 0x7fc060150500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.622+0000 7fc065c00640 1 --2- 172.21.15.100:0/3827866847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0600cca50 0x7fc0600c8990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.622+0000 7fc065c00640 1 -- 172.21.15.100:0/3827866847 >> 172.21.15.100:0/3827866847 conn(0x7fc0600cee80 msgr2=0x7fc0601668b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:02.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.622+0000 7fc065c00640 1 -- 172.21.15.100:0/3827866847 shutdown_connections 2024-04-23T14:33:02.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.622+0000 7fc065c00640 1 -- 172.21.15.100:0/3827866847 wait complete. 2024-04-23T14:33:02.644 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-23T14:33:02.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.782+0000 7fc573e00640 1 Processor -- start 2024-04-23T14:33:02.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.782+0000 7fc573e00640 1 -- start start 2024-04-23T14:33:02.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.782+0000 7fc573e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc57406ca80 0x7fc57406cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:02.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.782+0000 7fc573e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc5740acfe0 0x7fc5740ad470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:02.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.782+0000 7fc573e00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc57406bcc0 con 0x7fc57415c340 2024-04-23T14:33:02.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.782+0000 7fc573e00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc57406d400 con 0x7fc5740acfe0 2024-04-23T14:33:02.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.782+0000 7fc573e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc57406c5f0 con 0x7fc57406ca80 2024-04-23T14:33:02.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.782+0000 7fc573400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc5740acfe0 0x7fc5740ad470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:02.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.782+0000 7fc572a00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fc57415c340 0x7fc57406dbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:59628/0 (socket says 172.21.15.100:59628) 2024-04-23T14:33:02.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.782+0000 7fc572a00640 1 -- 172.21.15.100:0/2032829362 learned_addr learned my addr 172.21.15.100:0/2032829362 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:02.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.782+0000 7fc56a000640 1 --2- 172.21.15.100:0/2032829362 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc57406ca80 0x7fc57406cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:02.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.782+0000 7fc573400640 1 -- 172.21.15.100:0/2032829362 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc57406ca80 msgr2=0x7fc57406cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:02.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.782+0000 7fc573400640 1 --2- 172.21.15.100:0/2032829362 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc57406ca80 0x7fc57406cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.782+0000 7fc573400640 1 -- 172.21.15.100:0/2032829362 >> v1:172.21.15.100:6789/0 conn(0x7fc57415c340 legacy=0x7fc57406dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:02.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.782+0000 7fc573400640 1 -- 172.21.15.100:0/2032829362 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc57406bfa0 con 0x7fc5740acfe0 2024-04-23T14:33:02.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.782+0000 7fc56a000640 1 --2- 172.21.15.100:0/2032829362 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc57406ca80 0x7fc57406cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:02.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.782+0000 7fc572000640 1 -- 172.21.15.100:0/2032829362 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3923470475 0 0) 0x7fc57406bcc0 con 0x7fc57415c340 2024-04-23T14:33:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.786+0000 7fc573400640 1 --2- 172.21.15.100:0/2032829362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5740acfe0 0x7fc5740ad470 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009d90 tx=0x7fc56c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580ed0d6092740c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.786+0000 7fc572000640 1 -- 172.21.15.100:0/2032829362 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c0b2070 con 0x7fc5740acfe0 2024-04-23T14:33:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.786+0000 7fc572000640 1 -- 172.21.15.100:0/2032829362 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc56c0a5830 con 0x7fc5740acfe0 2024-04-23T14:33:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.786+0000 7fc572000640 1 -- 172.21.15.100:0/2032829362 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c0a5c00 con 0x7fc5740acfe0 2024-04-23T14:33:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.786+0000 7fc573e00640 1 -- 172.21.15.100:0/2032829362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5740acfe0 msgr2=0x7fc5740ad470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.786+0000 7fc573e00640 1 --2- 172.21.15.100:0/2032829362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5740acfe0 0x7fc5740ad470 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009d90 tx=0x7fc56c01c970 comp rx=0 tx=0).stop 2024-04-23T14:33:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.786+0000 7fc573e00640 1 -- 172.21.15.100:0/2032829362 shutdown_connections 2024-04-23T14:33:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.786+0000 7fc573e00640 1 --2- 172.21.15.100:0/2032829362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5740acfe0 0x7fc5740ad470 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.786+0000 7fc573e00640 1 --2- 172.21.15.100:0/2032829362 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc57406ca80 0x7fc57406cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.786+0000 7fc573e00640 1 -- 172.21.15.100:0/2032829362 >> 172.21.15.100:0/2032829362 conn(0x7fc5740b1d00 msgr2=0x7fc5740b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.786+0000 7fc573e00640 1 -- 172.21.15.100:0/2032829362 shutdown_connections 2024-04-23T14:33:02.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.786+0000 7fc573e00640 1 -- 172.21.15.100:0/2032829362 wait complete. 2024-04-23T14:33:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.786+0000 7fc573e00640 1 Processor -- start 2024-04-23T14:33:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc573e00640 1 -- start start 2024-04-23T14:33:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc573e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc57406ca80 0x7fc574073690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc573e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5740acfe0 0x7fc5741450f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc573e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc57415c340 0x7fc574145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc573e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc574160b70 con 0x7fc5740acfe0 2024-04-23T14:33:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc573e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741609f0 con 0x7fc57415c340 2024-04-23T14:33:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc573e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc574160cf0 con 0x7fc57406ca80 2024-04-23T14:33:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc56a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5740acfe0 0x7fc5741450f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc56a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5740acfe0 0x7fc5741450f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49426/0 (socket says 172.21.15.100:49426) 2024-04-23T14:33:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc56a000640 1 -- 172.21.15.100:0/3190045331 learned_addr learned my addr 172.21.15.100:0/3190045331 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc572a00640 1 --2- 172.21.15.100:0/3190045331 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc57406ca80 0x7fc574073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc573400640 1 --2- 172.21.15.100:0/3190045331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc57415c340 0x7fc574145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc56a000640 1 -- 172.21.15.100:0/3190045331 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc57406ca80 msgr2=0x7fc574073690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:02.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc56a000640 1 --2- 172.21.15.100:0/3190045331 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc57406ca80 0x7fc574073690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc56a000640 1 -- 172.21.15.100:0/3190045331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc57415c340 msgr2=0x7fc574145630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:02.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc56a000640 1 --2- 172.21.15.100:0/3190045331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc57415c340 0x7fc574145630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc56a000640 1 -- 172.21.15.100:0/3190045331 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5741543a0 con 0x7fc5740acfe0 2024-04-23T14:33:02.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc573400640 1 --2- 172.21.15.100:0/3190045331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc57415c340 0x7fc574145630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:02.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc56a000640 1 --2- 172.21.15.100:0/3190045331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5740acfe0 0x7fc5741450f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc55c00d830 tx=0x7fc55c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:02.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc56b400640 1 -- 172.21.15.100:0/3190045331 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c018020 con 0x7fc5740acfe0 2024-04-23T14:33:02.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc56b400640 1 -- 172.21.15.100:0/3190045331 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc55c0096c0 con 0x7fc5740acfe0 2024-04-23T14:33:02.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc573e00640 1 -- 172.21.15.100:0/3190045331 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc574142f50 con 0x7fc5740acfe0 2024-04-23T14:33:02.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc56b400640 1 -- 172.21.15.100:0/3190045331 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c009ab0 con 0x7fc5740acfe0 2024-04-23T14:33:02.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc573e00640 1 -- 172.21.15.100:0/3190045331 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc574143510 con 0x7fc5740acfe0 2024-04-23T14:33:02.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.790+0000 7fc573e00640 1 -- 172.21.15.100:0/3190045331 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc574068ad0 con 0x7fc5740acfe0 2024-04-23T14:33:02.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.794+0000 7fc56b400640 1 -- 172.21.15.100:0/3190045331 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc55c01b020 con 0x7fc5740acfe0 2024-04-23T14:33:02.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.794+0000 7fc56b400640 1 --2- 172.21.15.100:0/3190045331 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc5500683b0 0x7fc55006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:02.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.794+0000 7fc56b400640 1 -- 172.21.15.100:0/3190045331 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc55c087550 con 0x7fc5740acfe0 2024-04-23T14:33:02.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.794+0000 7fc572a00640 1 --2- 172.21.15.100:0/3190045331 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc5500683b0 0x7fc55006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:02.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.794+0000 7fc572a00640 1 --2- 172.21.15.100:0/3190045331 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc5500683b0 0x7fc55006a870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc564000f70 tx=0x7fc564007d10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:02.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.798+0000 7fc56b400640 1 -- 172.21.15.100:0/3190045331 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc55c022020 con 0x7fc5740acfe0 2024-04-23T14:33:02.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.954+0000 7fc573e00640 1 -- 172.21.15.100:0/3190045331 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc574073c20 con 0x7fc5740acfe0 2024-04-23T14:33:02.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.954+0000 7fc56b400640 1 -- 172.21.15.100:0/3190045331 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fc55c057680 con 0x7fc5740acfe0 2024-04-23T14:33:02.957 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:02.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.958+0000 7fc573e00640 1 -- 172.21.15.100:0/3190045331 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc5500683b0 msgr2=0x7fc55006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:02.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.958+0000 7fc573e00640 1 --2- 172.21.15.100:0/3190045331 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc5500683b0 0x7fc55006a870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc564000f70 tx=0x7fc564007d10 comp rx=0 tx=0).stop 2024-04-23T14:33:02.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.958+0000 7fc573e00640 1 -- 172.21.15.100:0/3190045331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5740acfe0 msgr2=0x7fc5741450f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:02.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.958+0000 7fc573e00640 1 --2- 172.21.15.100:0/3190045331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5740acfe0 0x7fc5741450f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc55c00d830 tx=0x7fc55c00dcf0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.958+0000 7fc573e00640 1 -- 172.21.15.100:0/3190045331 shutdown_connections 2024-04-23T14:33:02.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.958+0000 7fc573e00640 1 --2- 172.21.15.100:0/3190045331 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc5500683b0 0x7fc55006a870 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.958+0000 7fc573e00640 1 --2- 172.21.15.100:0/3190045331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc57415c340 0x7fc574145630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.958+0000 7fc573e00640 1 --2- 172.21.15.100:0/3190045331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5740acfe0 0x7fc5741450f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.958+0000 7fc573e00640 1 --2- 172.21.15.100:0/3190045331 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc57406ca80 0x7fc574073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:02.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.958+0000 7fc573e00640 1 -- 172.21.15.100:0/3190045331 >> 172.21.15.100:0/3190045331 conn(0x7fc5740b1d00 msgr2=0x7fc57406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:02.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.958+0000 7fc573e00640 1 -- 172.21.15.100:0/3190045331 shutdown_connections 2024-04-23T14:33:02.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:02.958+0000 7fc573e00640 1 -- 172.21.15.100:0/3190045331 wait complete. 2024-04-23T14:33:02.983 INFO:tasks.cephfs_test_runner:Starting test: test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) That a client who does not have the capability for setting 2024-04-23T14:33:02.983 INFO:tasks.cephfs_test_runner:test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-04-23T14:33:02.983 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:03.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.118+0000 7efeb5e00640 1 Processor -- start 2024-04-23T14:33:03.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.118+0000 7efeb5e00640 1 -- start start 2024-04-23T14:33:03.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.118+0000 7efeb5e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7efeb00cca50 0x7efeb00cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:03.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.118+0000 7efeb5e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efeb015c7e0 0x7efeb0160170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:03.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.118+0000 7efeb5e00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeb00cd370 con 0x7efeb015d600 2024-04-23T14:33:03.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.118+0000 7efeb5e00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7efeb00d2270 con 0x7efeb015c7e0 2024-04-23T14:33:03.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.118+0000 7efeb5e00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7efeb015db00 con 0x7efeb00cca50 2024-04-23T14:33:03.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.118+0000 7efea7e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efeb015c7e0 0x7efeb0160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:03.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.118+0000 7efea7e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efeb015c7e0 0x7efeb0160170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49436/0 (socket says 172.21.15.100:49436) 2024-04-23T14:33:03.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.118+0000 7efea7e00640 1 -- 172.21.15.100:0/2435490446 learned_addr learned my addr 172.21.15.100:0/2435490446 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:03.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.118+0000 7efeaea00640 1 --2- 172.21.15.100:0/2435490446 >> v2:172.21.15.151:3300/0 conn(0x7efeb00cca50 0x7efeb00cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:03.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.118+0000 7efea7e00640 1 -- 172.21.15.100:0/2435490446 >> v2:172.21.15.151:3300/0 conn(0x7efeb00cca50 msgr2=0x7efeb00cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:03.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.118+0000 7efea7e00640 1 --2- 172.21.15.100:0/2435490446 >> v2:172.21.15.151:3300/0 conn(0x7efeb00cca50 0x7efeb00cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.118+0000 7efea7e00640 1 -- 172.21.15.100:0/2435490446 >> v1:172.21.15.151:6789/0 conn(0x7efeb015d600 legacy=0x7efeb01606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:03.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.118+0000 7efea7e00640 1 -- 172.21.15.100:0/2435490446 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb00d04a0 con 0x7efeb015c7e0 2024-04-23T14:33:03.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.118+0000 7efeaea00640 1 --2- 172.21.15.100:0/2435490446 >> v2:172.21.15.151:3300/0 conn(0x7efeb00cca50 0x7efeb00cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:03.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.122+0000 7efea7e00640 1 --2- 172.21.15.100:0/2435490446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb015c7e0 0x7efeb0160170 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efeb00741f0 tx=0x7efe9801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d991232228fdc95 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:03.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.122+0000 7efeae000640 1 -- 172.21.15.100:0/2435490446 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe980ab660 con 0x7efeb015c7e0 2024-04-23T14:33:03.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.122+0000 7efeae000640 1 -- 172.21.15.100:0/2435490446 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe980b0070 con 0x7efeb015c7e0 2024-04-23T14:33:03.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.122+0000 7efeae000640 1 -- 172.21.15.100:0/2435490446 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe980aba20 con 0x7efeb015c7e0 2024-04-23T14:33:03.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.122+0000 7efeb5e00640 1 -- 172.21.15.100:0/2435490446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb015c7e0 msgr2=0x7efeb0160170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:03.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.122+0000 7efeb5e00640 1 --2- 172.21.15.100:0/2435490446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb015c7e0 0x7efeb0160170 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efeb00741f0 tx=0x7efe9801c460 comp rx=0 tx=0).stop 2024-04-23T14:33:03.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.122+0000 7efeb5e00640 1 -- 172.21.15.100:0/2435490446 shutdown_connections 2024-04-23T14:33:03.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.122+0000 7efeb5e00640 1 --2- 172.21.15.100:0/2435490446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb015c7e0 0x7efeb0160170 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.122+0000 7efeb5e00640 1 --2- 172.21.15.100:0/2435490446 >> v2:172.21.15.151:3300/0 conn(0x7efeb00cca50 0x7efeb00cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.122+0000 7efeb5e00640 1 -- 172.21.15.100:0/2435490446 >> 172.21.15.100:0/2435490446 conn(0x7efeb00cee80 msgr2=0x7efeb00cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:03.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.122+0000 7efeb5e00640 1 -- 172.21.15.100:0/2435490446 shutdown_connections 2024-04-23T14:33:03.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.122+0000 7efeb5e00640 1 -- 172.21.15.100:0/2435490446 wait complete. 2024-04-23T14:33:03.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.122+0000 7efeb5e00640 1 Processor -- start 2024-04-23T14:33:03.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.122+0000 7efeb5e00640 1 -- start start 2024-04-23T14:33:03.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efeb5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeb00cca50 0x7efeb014e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:03.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efeb5e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efeb015c7e0 0x7efeb015f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:03.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efeb5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb015d600 0x7efeb014cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:03.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efeb5e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efeb00b5430 con 0x7efeb015d600 2024-04-23T14:33:03.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efeb5e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efeb00b52b0 con 0x7efeb00cca50 2024-04-23T14:33:03.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efeb5e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7efeb00b3bf0 con 0x7efeb015c7e0 2024-04-23T14:33:03.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efeaf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb015d600 0x7efeb014cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:03.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efeaea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeb00cca50 0x7efeb014e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:03.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efeaea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeb00cca50 0x7efeb014e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:49190/0 (socket says 172.21.15.100:49190) 2024-04-23T14:33:03.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efeaea00640 1 -- 172.21.15.100:0/1806241263 learned_addr learned my addr 172.21.15.100:0/1806241263 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:03.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efeaf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb015d600 0x7efeb014cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49440/0 (socket says 172.21.15.100:49440) 2024-04-23T14:33:03.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efea7e00640 1 --2- 172.21.15.100:0/1806241263 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efeb015c7e0 0x7efeb015f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:03.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efeaf400640 1 -- 172.21.15.100:0/1806241263 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efeb015c7e0 msgr2=0x7efeb015f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:03.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efeaf400640 1 --2- 172.21.15.100:0/1806241263 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efeb015c7e0 0x7efeb015f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efeaf400640 1 -- 172.21.15.100:0/1806241263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeb00cca50 msgr2=0x7efeb014e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:03.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efeaf400640 1 --2- 172.21.15.100:0/1806241263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeb00cca50 0x7efeb014e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efeaf400640 1 -- 172.21.15.100:0/1806241263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb015e730 con 0x7efeb015d600 2024-04-23T14:33:03.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efeaf400640 1 --2- 172.21.15.100:0/1806241263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb015d600 0x7efeb014cc50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efea800c760 tx=0x7efea800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:03.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efea7400640 1 -- 172.21.15.100:0/1806241263 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea8016020 con 0x7efeb015d600 2024-04-23T14:33:03.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efea7400640 1 -- 172.21.15.100:0/1806241263 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea8006d60 con 0x7efeb015d600 2024-04-23T14:33:03.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efeb5e00640 1 -- 172.21.15.100:0/1806241263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb015ea10 con 0x7efeb015d600 2024-04-23T14:33:03.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efea7400640 1 -- 172.21.15.100:0/1806241263 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea8003bc0 con 0x7efeb015d600 2024-04-23T14:33:03.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efeb5e00640 1 -- 172.21.15.100:0/1806241263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb015efd0 con 0x7efeb015d600 2024-04-23T14:33:03.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.126+0000 7efeb5e00640 1 -- 172.21.15.100:0/1806241263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb0068ad0 con 0x7efeb015d600 2024-04-23T14:33:03.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.130+0000 7efea7400640 1 -- 172.21.15.100:0/1806241263 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7efea80050d0 con 0x7efeb015d600 2024-04-23T14:33:03.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.130+0000 7efea7400640 1 --2- 172.21.15.100:0/1806241263 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7efe8c068380 0x7efe8c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:03.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.130+0000 7efea7400640 1 -- 172.21.15.100:0/1806241263 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7efea80866f0 con 0x7efeb015d600 2024-04-23T14:33:03.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.130+0000 7efeaea00640 1 --2- 172.21.15.100:0/1806241263 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7efe8c068380 0x7efe8c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:03.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.130+0000 7efeaea00640 1 --2- 172.21.15.100:0/1806241263 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7efe8c068380 0x7efe8c06a840 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efeb00c8f50 tx=0x7efea0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:03.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.134+0000 7efea7400640 1 -- 172.21.15.100:0/1806241263 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efea8020020 con 0x7efeb015d600 2024-04-23T14:33:03.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.290+0000 7efeb5e00640 1 -- 172.21.15.100:0/1806241263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efeb0067a80 con 0x7efeb015d600 2024-04-23T14:33:03.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.294+0000 7efea7400640 1 -- 172.21.15.100:0/1806241263 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18859 (secure 0 0 0) 0x7efea80562f0 con 0x7efeb015d600 2024-04-23T14:33:03.295 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:03.295 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:49.784608+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:03.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.294+0000 7efeb5e00640 1 -- 172.21.15.100:0/1806241263 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7efe8c068380 msgr2=0x7efe8c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:03.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.294+0000 7efeb5e00640 1 --2- 172.21.15.100:0/1806241263 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7efe8c068380 0x7efe8c06a840 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efeb00c8f50 tx=0x7efea0008040 comp rx=0 tx=0).stop 2024-04-23T14:33:03.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.294+0000 7efeb5e00640 1 -- 172.21.15.100:0/1806241263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb015d600 msgr2=0x7efeb014cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:03.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.294+0000 7efeb5e00640 1 --2- 172.21.15.100:0/1806241263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb015d600 0x7efeb014cc50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efea800c760 tx=0x7efea800cc20 comp rx=0 tx=0).stop 2024-04-23T14:33:03.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.294+0000 7efeb5e00640 1 -- 172.21.15.100:0/1806241263 shutdown_connections 2024-04-23T14:33:03.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.294+0000 7efeb5e00640 1 --2- 172.21.15.100:0/1806241263 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7efe8c068380 0x7efe8c06a840 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.294+0000 7efeb5e00640 1 --2- 172.21.15.100:0/1806241263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeb015d600 0x7efeb014cc50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.294+0000 7efeb5e00640 1 --2- 172.21.15.100:0/1806241263 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efeb015c7e0 0x7efeb015f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.294+0000 7efeb5e00640 1 --2- 172.21.15.100:0/1806241263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeb00cca50 0x7efeb014e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.294+0000 7efeb5e00640 1 -- 172.21.15.100:0/1806241263 >> 172.21.15.100:0/1806241263 conn(0x7efeb00cee80 msgr2=0x7efeb0158c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:03.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.294+0000 7efeb5e00640 1 -- 172.21.15.100:0/1806241263 shutdown_connections 2024-04-23T14:33:03.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.294+0000 7efeb5e00640 1 -- 172.21.15.100:0/1806241263 wait complete. 2024-04-23T14:33:03.315 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:32:06.835560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:32:07.930489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-23T14:32:11.636814+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-23T14:33:03.316 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:33:03.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.450+0000 7f8705200640 1 Processor -- start 2024-04-23T14:33:03.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.450+0000 7f8705200640 1 -- start start 2024-04-23T14:33:03.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.454+0000 7f8705200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f87000cd300 0x7f87001671f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:03.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.454+0000 7f8705200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f870006cbb0 con 0x7f870006bb20 2024-04-23T14:33:03.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.454+0000 7f8705200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f870006ce90 con 0x7f870015c7a0 2024-04-23T14:33:03.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.454+0000 7f8705200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f870006d450 con 0x7f87000cd300 2024-04-23T14:33:03.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.454+0000 7f86f5400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f870006bb20 0x7f87000ccbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:59634/0 (socket says 172.21.15.100:59634) 2024-04-23T14:33:03.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.454+0000 7f86f5400640 1 -- 172.21.15.100:0/811598118 learned_addr learned my addr 172.21.15.100:0/811598118 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:03.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.454+0000 7f86fe800640 1 --2- 172.21.15.100:0/811598118 >> v2:172.21.15.151:3300/0 conn(0x7f87000cd300 0x7f87001671f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:03.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.454+0000 7f86fd400640 1 -- 172.21.15.100:0/811598118 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3946585705 0 0) 0x7f870006cbb0 con 0x7f870006bb20 2024-04-23T14:33:03.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.454+0000 7f86fd400640 1 -- 172.21.15.100:0/811598118 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86e4003530 con 0x7f870006bb20 2024-04-23T14:33:03.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.454+0000 7f86fd400640 1 -- 172.21.15.100:0/811598118 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3500538049 0 0) 0x7f870006ce90 con 0x7f870015c7a0 2024-04-23T14:33:03.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.454+0000 7f86fd400640 1 -- 172.21.15.100:0/811598118 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f870006cbb0 con 0x7f870015c7a0 2024-04-23T14:33:03.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.454+0000 7f86fe800640 1 -- 172.21.15.100:0/811598118 >> v1:172.21.15.151:6789/0 conn(0x7f870015c7a0 legacy=0x7f87000af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:03.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.454+0000 7f86fe800640 1 -- 172.21.15.100:0/811598118 >> v1:172.21.15.100:6789/0 conn(0x7f870006bb20 legacy=0x7f87000ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:03.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.454+0000 7f86fe800640 1 -- 172.21.15.100:0/811598118 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870006d170 con 0x7f87000cd300 2024-04-23T14:33:03.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.458+0000 7f86fe800640 1 --2- 172.21.15.100:0/811598118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87000cd300 0x7f87001671f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009d90 tx=0x7f86f801c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ee3351431d83d5b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:03.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.458+0000 7f86fd400640 1 -- 172.21.15.100:0/811598118 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f80b2070 con 0x7f87000cd300 2024-04-23T14:33:03.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.458+0000 7f86fd400640 1 -- 172.21.15.100:0/811598118 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86f80a57a0 con 0x7f87000cd300 2024-04-23T14:33:03.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.458+0000 7f86fd400640 1 -- 172.21.15.100:0/811598118 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f80a5b50 con 0x7f87000cd300 2024-04-23T14:33:03.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.458+0000 7f8705200640 1 -- 172.21.15.100:0/811598118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87000cd300 msgr2=0x7f87001671f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:03.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.458+0000 7f8705200640 1 --2- 172.21.15.100:0/811598118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87000cd300 0x7f87001671f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009d90 tx=0x7f86f801c8e0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.458+0000 7f8705200640 1 -- 172.21.15.100:0/811598118 shutdown_connections 2024-04-23T14:33:03.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.458+0000 7f8705200640 1 --2- 172.21.15.100:0/811598118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87000cd300 0x7f87001671f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.458+0000 7f8705200640 1 -- 172.21.15.100:0/811598118 >> 172.21.15.100:0/811598118 conn(0x7f87000b5200 msgr2=0x7f87000b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:03.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.458+0000 7f8705200640 1 -- 172.21.15.100:0/811598118 shutdown_connections 2024-04-23T14:33:03.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.458+0000 7f8705200640 1 -- 172.21.15.100:0/811598118 wait complete. 2024-04-23T14:33:03.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.458+0000 7f8705200640 1 Processor -- start 2024-04-23T14:33:03.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.458+0000 7f8705200640 1 -- start start 2024-04-23T14:33:03.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.458+0000 7f8705200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f870006bb20 0x7f87000c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:03.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.458+0000 7f8705200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87000cd300 0x7f87000cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:03.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.458+0000 7f8705200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f870015c7a0 0x7f8700154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:03.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.458+0000 7f8705200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f870016a2e0 con 0x7f87000cd300 2024-04-23T14:33:03.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.458+0000 7f8705200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f870016a160 con 0x7f870006bb20 2024-04-23T14:33:03.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.458+0000 7f8705200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f870016a460 con 0x7f870015c7a0 2024-04-23T14:33:03.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.462+0000 7f86f5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87000cd300 0x7f87000cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:03.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.462+0000 7f86f5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87000cd300 0x7f87000cbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49448/0 (socket says 172.21.15.100:49448) 2024-04-23T14:33:03.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.462+0000 7f86f5400640 1 -- 172.21.15.100:0/1478901711 learned_addr learned my addr 172.21.15.100:0/1478901711 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:03.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.462+0000 7f86fde00640 1 --2- 172.21.15.100:0/1478901711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f870006bb20 0x7f87000c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:03.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.462+0000 7f86fe800640 1 --2- 172.21.15.100:0/1478901711 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f870015c7a0 0x7f8700154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:03.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.462+0000 7f86f5400640 1 -- 172.21.15.100:0/1478901711 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f870015c7a0 msgr2=0x7f8700154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:03.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.462+0000 7f86f5400640 1 --2- 172.21.15.100:0/1478901711 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f870015c7a0 0x7f8700154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.462+0000 7f86f5400640 1 -- 172.21.15.100:0/1478901711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f870006bb20 msgr2=0x7f87000c2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:03.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.462+0000 7f86f5400640 1 --2- 172.21.15.100:0/1478901711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f870006bb20 0x7f87000c2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.462+0000 7f86f5400640 1 -- 172.21.15.100:0/1478901711 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87000cf900 con 0x7f87000cd300 2024-04-23T14:33:03.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.462+0000 7f86f5400640 1 --2- 172.21.15.100:0/1478901711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87000cd300 0x7f87000cbd40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f86e8005890 tx=0x7f86e800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:03.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.462+0000 7f86f6a00640 1 -- 172.21.15.100:0/1478901711 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8017070 con 0x7f87000cd300 2024-04-23T14:33:03.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.462+0000 7f86f6a00640 1 -- 172.21.15.100:0/1478901711 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86e8005cf0 con 0x7f87000cd300 2024-04-23T14:33:03.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.462+0000 7f8705200640 1 -- 172.21.15.100:0/1478901711 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87000caca0 con 0x7f87000cd300 2024-04-23T14:33:03.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.462+0000 7f86f6a00640 1 -- 172.21.15.100:0/1478901711 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e80072d0 con 0x7f87000cd300 2024-04-23T14:33:03.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.462+0000 7f8705200640 1 -- 172.21.15.100:0/1478901711 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87000cb260 con 0x7f87000cd300 2024-04-23T14:33:03.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.462+0000 7f8705200640 1 -- 172.21.15.100:0/1478901711 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8700068ad0 con 0x7f87000cd300 2024-04-23T14:33:03.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.462+0000 7f86f6a00640 1 -- 172.21.15.100:0/1478901711 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f86e801a070 con 0x7f87000cd300 2024-04-23T14:33:03.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.466+0000 7f86f6a00640 1 --2- 172.21.15.100:0/1478901711 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f86dc068380 0x7f86dc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:03.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.466+0000 7f86f6a00640 1 -- 172.21.15.100:0/1478901711 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f86e80860a0 con 0x7f87000cd300 2024-04-23T14:33:03.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.466+0000 7f86fde00640 1 --2- 172.21.15.100:0/1478901711 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f86dc068380 0x7f86dc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:03.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.466+0000 7f86fde00640 1 --2- 172.21.15.100:0/1478901711 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f86dc068380 0x7f86dc06a840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f87001673b0 tx=0x7f86f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:03.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.470+0000 7f86f6a00640 1 -- 172.21.15.100:0/1478901711 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f86e8055c10 con 0x7f87000cd300 2024-04-23T14:33:03.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.626+0000 7f8705200640 1 -- 172.21.15.100:0/1478901711 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f870015bcd0 con 0x7f87000cd300 2024-04-23T14:33:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.626+0000 7f86f6a00640 1 -- 172.21.15.100:0/1478901711 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f86e8059f10 con 0x7f87000cd300 2024-04-23T14:33:03.628 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:33:03.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.626+0000 7f8705200640 1 -- 172.21.15.100:0/1478901711 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f86dc068380 msgr2=0x7f86dc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:03.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.626+0000 7f8705200640 1 --2- 172.21.15.100:0/1478901711 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f86dc068380 0x7f86dc06a840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f87001673b0 tx=0x7f86f0009040 comp rx=0 tx=0).stop 2024-04-23T14:33:03.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.626+0000 7f8705200640 1 -- 172.21.15.100:0/1478901711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87000cd300 msgr2=0x7f87000cbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:03.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.626+0000 7f8705200640 1 --2- 172.21.15.100:0/1478901711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87000cd300 0x7f87000cbd40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f86e8005890 tx=0x7f86e800c8b0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.630+0000 7f8705200640 1 -- 172.21.15.100:0/1478901711 shutdown_connections 2024-04-23T14:33:03.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.630+0000 7f8705200640 1 --2- 172.21.15.100:0/1478901711 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f86dc068380 0x7f86dc06a840 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.630+0000 7f8705200640 1 --2- 172.21.15.100:0/1478901711 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f870015c7a0 0x7f8700154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.630+0000 7f8705200640 1 --2- 172.21.15.100:0/1478901711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87000cd300 0x7f87000cbd40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.630+0000 7f8705200640 1 --2- 172.21.15.100:0/1478901711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f870006bb20 0x7f87000c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.630+0000 7f8705200640 1 -- 172.21.15.100:0/1478901711 >> 172.21.15.100:0/1478901711 conn(0x7f87000b5200 msgr2=0x7f87000cfdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:03.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.630+0000 7f8705200640 1 -- 172.21.15.100:0/1478901711 shutdown_connections 2024-04-23T14:33:03.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.630+0000 7f8705200640 1 -- 172.21.15.100:0/1478901711 wait complete. 2024-04-23T14:33:03.649 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:33:03.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.786+0000 7f1c7fa00640 1 Processor -- start 2024-04-23T14:33:03.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.786+0000 7f1c7fa00640 1 -- start start 2024-04-23T14:33:03.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.786+0000 7f1c7fa00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1c780b42e0 0x7f1c780b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:03.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.786+0000 7f1c7fa00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f1c780cfa20 0x7f1c780cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:03.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.786+0000 7f1c7fa00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c780cef80 con 0x7f1c7806f020 2024-04-23T14:33:03.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.786+0000 7f1c7fa00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f1c780b5740 con 0x7f1c780cfa20 2024-04-23T14:33:03.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.786+0000 7f1c7fa00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1c7815b800 con 0x7f1c780b42e0 2024-04-23T14:33:03.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.790+0000 7f1c7da00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f1c780cfa20 0x7f1c780cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:03.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.790+0000 7f1c77e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1c780b42e0 0x7f1c780b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:03.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.790+0000 7f1c7da00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f1c780cfa20 0x7f1c780cfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:49210/0 (socket says 172.21.15.100:49210) 2024-04-23T14:33:03.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.790+0000 7f1c7da00640 1 -- 172.21.15.100:0/393895022 learned_addr learned my addr 172.21.15.100:0/393895022 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:03.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.790+0000 7f1c77e00640 1 -- 172.21.15.100:0/393895022 >> v2:172.21.15.151:3300/0 conn(0x7f1c780cfa20 msgr2=0x7f1c780cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:03.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.790+0000 7f1c77e00640 1 --2- 172.21.15.100:0/393895022 >> v2:172.21.15.151:3300/0 conn(0x7f1c780cfa20 0x7f1c780cfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.790+0000 7f1c77e00640 1 -- 172.21.15.100:0/393895022 >> v1:172.21.15.151:6789/0 conn(0x7f1c7806f020 legacy=0x7f1c7806f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:03.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.790+0000 7f1c77e00640 1 -- 172.21.15.100:0/393895022 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c780b5a10 con 0x7f1c780b42e0 2024-04-23T14:33:03.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.790+0000 7f1c7da00640 1 --2- 172.21.15.100:0/393895022 >> v2:172.21.15.151:3300/0 conn(0x7f1c780cfa20 0x7f1c780cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:33:03.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.790+0000 7f1c77e00640 1 --2- 172.21.15.100:0/393895022 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1c780b42e0 0x7f1c780b5200 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1c640097c0 tx=0x7f1c6401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56c00e265ac32a36 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:03.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.794+0000 7f1c77400640 1 -- 172.21.15.100:0/393895022 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c640b2070 con 0x7f1c780b42e0 2024-04-23T14:33:03.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.794+0000 7f1c77400640 1 -- 172.21.15.100:0/393895022 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c640ad640 con 0x7f1c780b42e0 2024-04-23T14:33:03.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.794+0000 7f1c77400640 1 -- 172.21.15.100:0/393895022 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c640ada10 con 0x7f1c780b42e0 2024-04-23T14:33:03.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.794+0000 7f1c7fa00640 1 -- 172.21.15.100:0/393895022 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1c780b42e0 msgr2=0x7f1c780b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:03.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.794+0000 7f1c7fa00640 1 --2- 172.21.15.100:0/393895022 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1c780b42e0 0x7f1c780b5200 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1c640097c0 tx=0x7f1c6401c4a0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.794+0000 7f1c7fa00640 1 -- 172.21.15.100:0/393895022 shutdown_connections 2024-04-23T14:33:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.794+0000 7f1c7fa00640 1 --2- 172.21.15.100:0/393895022 >> v2:172.21.15.151:3300/0 conn(0x7f1c780cfa20 0x7f1c780cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.794+0000 7f1c7fa00640 1 --2- 172.21.15.100:0/393895022 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1c780b42e0 0x7f1c780b5200 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.794+0000 7f1c7fa00640 1 -- 172.21.15.100:0/393895022 >> 172.21.15.100:0/393895022 conn(0x7f1c780704d0 msgr2=0x7f1c780708e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.794+0000 7f1c7fa00640 1 -- 172.21.15.100:0/393895022 shutdown_connections 2024-04-23T14:33:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.794+0000 7f1c7fa00640 1 -- 172.21.15.100:0/393895022 wait complete. 2024-04-23T14:33:03.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.794+0000 7f1c7fa00640 1 Processor -- start 2024-04-23T14:33:03.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.794+0000 7f1c7fa00640 1 -- start start 2024-04-23T14:33:03.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.794+0000 7f1c7fa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c7806f020 0x7f1c78141e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:03.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.794+0000 7f1c7fa00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1c780b42e0 0x7f1c780ae390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:03.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.794+0000 7f1c7fa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c780cfa20 0x7f1c780ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:03.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.794+0000 7f1c7fa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1c78168110 con 0x7f1c7806f020 2024-04-23T14:33:03.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.794+0000 7f1c7fa00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c78167f90 con 0x7f1c780cfa20 2024-04-23T14:33:03.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.794+0000 7f1c7fa00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1c78168290 con 0x7f1c780b42e0 2024-04-23T14:33:03.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.798+0000 7f1c7d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c7806f020 0x7f1c78141e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:03.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.798+0000 7f1c7d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c7806f020 0x7f1c78141e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49464/0 (socket says 172.21.15.100:49464) 2024-04-23T14:33:03.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.798+0000 7f1c7d000640 1 -- 172.21.15.100:0/925254482 learned_addr learned my addr 172.21.15.100:0/925254482 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:03.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.798+0000 7f1c7da00640 1 --2- 172.21.15.100:0/925254482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c780cfa20 0x7f1c780ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:03.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.798+0000 7f1c77e00640 1 --2- 172.21.15.100:0/925254482 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1c780b42e0 0x7f1c780ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:03.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.798+0000 7f1c7d000640 1 -- 172.21.15.100:0/925254482 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1c780b42e0 msgr2=0x7f1c780ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:03.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.798+0000 7f1c7d000640 1 --2- 172.21.15.100:0/925254482 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1c780b42e0 0x7f1c780ae390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.798+0000 7f1c7d000640 1 -- 172.21.15.100:0/925254482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c780cfa20 msgr2=0x7f1c780ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:03.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.798+0000 7f1c7d000640 1 --2- 172.21.15.100:0/925254482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c780cfa20 0x7f1c780ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.798+0000 7f1c7d000640 1 -- 172.21.15.100:0/925254482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c780b5570 con 0x7f1c7806f020 2024-04-23T14:33:03.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.798+0000 7f1c7da00640 1 --2- 172.21.15.100:0/925254482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c780cfa20 0x7f1c780ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:03.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.798+0000 7f1c77e00640 1 --2- 172.21.15.100:0/925254482 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1c780b42e0 0x7f1c780ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:03.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.798+0000 7f1c7d000640 1 --2- 172.21.15.100:0/925254482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c7806f020 0x7f1c78141e50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00c500 tx=0x7f1c6c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:03.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.798+0000 7f1c74c00640 1 -- 172.21.15.100:0/925254482 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c016020 con 0x7f1c7806f020 2024-04-23T14:33:03.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.798+0000 7f1c74c00640 1 -- 172.21.15.100:0/925254482 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c6c005220 con 0x7f1c7806f020 2024-04-23T14:33:03.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.798+0000 7f1c7fa00640 1 -- 172.21.15.100:0/925254482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c7815de60 con 0x7f1c7806f020 2024-04-23T14:33:03.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.798+0000 7f1c74c00640 1 -- 172.21.15.100:0/925254482 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c007030 con 0x7f1c7806f020 2024-04-23T14:33:03.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.798+0000 7f1c7fa00640 1 -- 172.21.15.100:0/925254482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c7815e420 con 0x7f1c7806f020 2024-04-23T14:33:03.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.798+0000 7f1c7fa00640 1 -- 172.21.15.100:0/925254482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c78068ad0 con 0x7f1c7806f020 2024-04-23T14:33:03.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.798+0000 7f1c74c00640 1 -- 172.21.15.100:0/925254482 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1c6c0071d0 con 0x7f1c7806f020 2024-04-23T14:33:03.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.798+0000 7f1c74c00640 1 --2- 172.21.15.100:0/925254482 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1c58068380 0x7f1c5806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:03.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.802+0000 7f1c74c00640 1 -- 172.21.15.100:0/925254482 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1c6c0867f0 con 0x7f1c7806f020 2024-04-23T14:33:03.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.802+0000 7f1c77e00640 1 --2- 172.21.15.100:0/925254482 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1c58068380 0x7f1c5806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:03.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.802+0000 7f1c77e00640 1 --2- 172.21.15.100:0/925254482 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1c58068380 0x7f1c5806a840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1c640af000 tx=0x7f1c640a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:03.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.806+0000 7f1c74c00640 1 -- 172.21.15.100:0/925254482 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1c6c020020 con 0x7f1c7806f020 2024-04-23T14:33:03.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.962+0000 7f1c7fa00640 1 -- 172.21.15.100:0/925254482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1c7806ee00 con 0x7f1c7806f020 2024-04-23T14:33:03.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.962+0000 7f1c74c00640 1 -- 172.21.15.100:0/925254482 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f1c6c0074c0 con 0x7f1c7806f020 2024-04-23T14:33:03.964 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:03.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.966+0000 7f1c7fa00640 1 -- 172.21.15.100:0/925254482 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1c58068380 msgr2=0x7f1c5806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:03.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.966+0000 7f1c7fa00640 1 --2- 172.21.15.100:0/925254482 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1c58068380 0x7f1c5806a840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1c640af000 tx=0x7f1c640a7070 comp rx=0 tx=0).stop 2024-04-23T14:33:03.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.966+0000 7f1c7fa00640 1 -- 172.21.15.100:0/925254482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c7806f020 msgr2=0x7f1c78141e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:03.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.966+0000 7f1c7fa00640 1 --2- 172.21.15.100:0/925254482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c7806f020 0x7f1c78141e50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00c500 tx=0x7f1c6c00c9c0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.966+0000 7f1c7fa00640 1 -- 172.21.15.100:0/925254482 shutdown_connections 2024-04-23T14:33:03.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.966+0000 7f1c7fa00640 1 --2- 172.21.15.100:0/925254482 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1c58068380 0x7f1c5806a840 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.966+0000 7f1c7fa00640 1 --2- 172.21.15.100:0/925254482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c780cfa20 0x7f1c780ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.966+0000 7f1c7fa00640 1 --2- 172.21.15.100:0/925254482 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1c780b42e0 0x7f1c780ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.966+0000 7f1c7fa00640 1 --2- 172.21.15.100:0/925254482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c7806f020 0x7f1c78141e50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:03.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.966+0000 7f1c7fa00640 1 -- 172.21.15.100:0/925254482 >> 172.21.15.100:0/925254482 conn(0x7f1c780704d0 msgr2=0x7f1c780b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:03.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.966+0000 7f1c7fa00640 1 -- 172.21.15.100:0/925254482 shutdown_connections 2024-04-23T14:33:03.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:03.966+0000 7f1c7fa00640 1 -- 172.21.15.100:0/925254482 wait complete. 2024-04-23T14:33:03.985 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:33:04.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.122+0000 7f73e5400640 1 Processor -- start 2024-04-23T14:33:04.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.122+0000 7f73e5400640 1 -- start start 2024-04-23T14:33:04.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.122+0000 7f73e5400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f73e015c7a0 0x7f73e00af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:04.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.122+0000 7f73e5400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f73e006bb20 0x7f73e00ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:04.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.122+0000 7f73e5400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73e00cd540 con 0x7f73e00cd130 2024-04-23T14:33:04.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.122+0000 7f73e5400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f73e006d450 con 0x7f73e006bb20 2024-04-23T14:33:04.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.122+0000 7f73e5400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f73e006cbb0 con 0x7f73e015c7a0 2024-04-23T14:33:04.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.122+0000 7f73d7e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f73e006bb20 0x7f73e00ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:04.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.122+0000 7f73d7e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f73e006bb20 0x7f73e00ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49480/0 (socket says 172.21.15.100:49480) 2024-04-23T14:33:04.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.122+0000 7f73d7e00640 1 -- 172.21.15.100:0/457349204 learned_addr learned my addr 172.21.15.100:0/457349204 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:04.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.122+0000 7f73de800640 1 --2- 172.21.15.100:0/457349204 >> v2:172.21.15.151:3300/0 conn(0x7f73e015c7a0 0x7f73e00af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:04.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.122+0000 7f73d7e00640 1 -- 172.21.15.100:0/457349204 >> v2:172.21.15.151:3300/0 conn(0x7f73e015c7a0 msgr2=0x7f73e00af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:04.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.122+0000 7f73d7e00640 1 --2- 172.21.15.100:0/457349204 >> v2:172.21.15.151:3300/0 conn(0x7f73e015c7a0 0x7f73e00af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:04.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.122+0000 7f73d7e00640 1 -- 172.21.15.100:0/457349204 >> v1:172.21.15.151:6789/0 conn(0x7f73e00cd130 legacy=0x7f73e006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:04.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.122+0000 7f73d7e00640 1 -- 172.21.15.100:0/457349204 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e006ce90 con 0x7f73e006bb20 2024-04-23T14:33:04.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.122+0000 7f73de800640 1 --2- 172.21.15.100:0/457349204 >> v2:172.21.15.151:3300/0 conn(0x7f73e015c7a0 0x7f73e00af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:04.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.126+0000 7f73d7e00640 1 --2- 172.21.15.100:0/457349204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73e006bb20 0x7f73e00ccbf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f73e00716b0 tx=0x7f73c801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d13497bc62e1152 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:04.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.126+0000 7f73dde00640 1 -- 172.21.15.100:0/457349204 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c80b0070 con 0x7f73e006bb20 2024-04-23T14:33:04.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.126+0000 7f73dde00640 1 -- 172.21.15.100:0/457349204 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73c80ab600 con 0x7f73e006bb20 2024-04-23T14:33:04.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.126+0000 7f73dde00640 1 -- 172.21.15.100:0/457349204 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c80ab9d0 con 0x7f73e006bb20 2024-04-23T14:33:04.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.126+0000 7f73e5400640 1 -- 172.21.15.100:0/457349204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73e006bb20 msgr2=0x7f73e00ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:04.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.126+0000 7f73e5400640 1 --2- 172.21.15.100:0/457349204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73e006bb20 0x7f73e00ccbf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f73e00716b0 tx=0x7f73c801c490 comp rx=0 tx=0).stop 2024-04-23T14:33:04.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.126+0000 7f73e5400640 1 -- 172.21.15.100:0/457349204 shutdown_connections 2024-04-23T14:33:04.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.126+0000 7f73e5400640 1 --2- 172.21.15.100:0/457349204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73e006bb20 0x7f73e00ccbf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:04.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.126+0000 7f73e5400640 1 --2- 172.21.15.100:0/457349204 >> v2:172.21.15.151:3300/0 conn(0x7f73e015c7a0 0x7f73e00af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:04.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.126+0000 7f73e5400640 1 -- 172.21.15.100:0/457349204 >> 172.21.15.100:0/457349204 conn(0x7f73e00b5200 msgr2=0x7f73e00b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:04.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.126+0000 7f73e5400640 1 -- 172.21.15.100:0/457349204 shutdown_connections 2024-04-23T14:33:04.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.126+0000 7f73e5400640 1 -- 172.21.15.100:0/457349204 wait complete. 2024-04-23T14:33:04.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73e5400640 1 Processor -- start 2024-04-23T14:33:04.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73e5400640 1 -- start start 2024-04-23T14:33:04.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73e5400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73e006bb20 0x7f73e0099810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:04.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73e5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73e00cd130 0x7f73e00cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:04.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73e5400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f73e015c7a0 0x7f73e00b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:04.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73e5400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f73e0169590 con 0x7f73e00cd130 2024-04-23T14:33:04.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73e5400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f73e0169410 con 0x7f73e006bb20 2024-04-23T14:33:04.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73e5400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f73e0169710 con 0x7f73e015c7a0 2024-04-23T14:33:04.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73d7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73e00cd130 0x7f73e00cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:04.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73d7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73e00cd130 0x7f73e00cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49484/0 (socket says 172.21.15.100:49484) 2024-04-23T14:33:04.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73d7e00640 1 -- 172.21.15.100:0/2194214036 learned_addr learned my addr 172.21.15.100:0/2194214036 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:04.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73de800640 1 --2- 172.21.15.100:0/2194214036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73e006bb20 0x7f73e0099810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:04.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73df200640 1 --2- 172.21.15.100:0/2194214036 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f73e015c7a0 0x7f73e00b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:04.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73d7e00640 1 -- 172.21.15.100:0/2194214036 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f73e015c7a0 msgr2=0x7f73e00b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:04.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73d7e00640 1 --2- 172.21.15.100:0/2194214036 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f73e015c7a0 0x7f73e00b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:04.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73d7e00640 1 -- 172.21.15.100:0/2194214036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73e006bb20 msgr2=0x7f73e0099810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:04.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73d7e00640 1 --2- 172.21.15.100:0/2194214036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73e006bb20 0x7f73e0099810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:04.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73d7e00640 1 -- 172.21.15.100:0/2194214036 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e0099d50 con 0x7f73e00cd130 2024-04-23T14:33:04.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73df200640 1 --2- 172.21.15.100:0/2194214036 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f73e015c7a0 0x7f73e00b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:04.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73d7e00640 1 --2- 172.21.15.100:0/2194214036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73e00cd130 0x7f73e00cc1a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f73c80190a0 tx=0x7f73c8019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:04.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73d6a00640 1 -- 172.21.15.100:0/2194214036 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c80b9070 con 0x7f73e00cd130 2024-04-23T14:33:04.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73d6a00640 1 -- 172.21.15.100:0/2194214036 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73c80b4400 con 0x7f73e00cd130 2024-04-23T14:33:04.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73d6a00640 1 -- 172.21.15.100:0/2194214036 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c80b0040 con 0x7f73e00cd130 2024-04-23T14:33:04.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73e5400640 1 -- 172.21.15.100:0/2194214036 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e0099290 con 0x7f73e00cd130 2024-04-23T14:33:04.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73e5400640 1 -- 172.21.15.100:0/2194214036 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e00b0300 con 0x7f73e00cd130 2024-04-23T14:33:04.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.130+0000 7f73e5400640 1 -- 172.21.15.100:0/2194214036 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73e0068ad0 con 0x7f73e00cd130 2024-04-23T14:33:04.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.134+0000 7f73d6a00640 1 -- 172.21.15.100:0/2194214036 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f73c80b33a0 con 0x7f73e00cd130 2024-04-23T14:33:04.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.134+0000 7f73d6a00640 1 --2- 172.21.15.100:0/2194214036 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f73bc0683b0 0x7f73bc06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:04.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.134+0000 7f73d6a00640 1 -- 172.21.15.100:0/2194214036 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f73c811ff50 con 0x7f73e00cd130 2024-04-23T14:33:04.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.134+0000 7f73de800640 1 --2- 172.21.15.100:0/2194214036 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f73bc0683b0 0x7f73bc06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:04.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.134+0000 7f73de800640 1 --2- 172.21.15.100:0/2194214036 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f73bc0683b0 0x7f73bc06a870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f73e0153e10 tx=0x7f73d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:04.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.138+0000 7f73d6a00640 1 -- 172.21.15.100:0/2194214036 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f73c8016110 con 0x7f73e00cd130 2024-04-23T14:33:04.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.294+0000 7f73e5400640 1 -- 172.21.15.100:0/2194214036 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f73e01534c0 con 0x7f73e00cd130 2024-04-23T14:33:04.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.294+0000 7f73d6a00640 1 -- 172.21.15.100:0/2194214036 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f73c80efab0 con 0x7f73e00cd130 2024-04-23T14:33:04.296 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:33:04.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.298+0000 7f73e5400640 1 -- 172.21.15.100:0/2194214036 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f73bc0683b0 msgr2=0x7f73bc06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:04.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.298+0000 7f73e5400640 1 --2- 172.21.15.100:0/2194214036 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f73bc0683b0 0x7f73bc06a870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f73e0153e10 tx=0x7f73d0008040 comp rx=0 tx=0).stop 2024-04-23T14:33:04.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.298+0000 7f73e5400640 1 -- 172.21.15.100:0/2194214036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73e00cd130 msgr2=0x7f73e00cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:04.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.298+0000 7f73e5400640 1 --2- 172.21.15.100:0/2194214036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73e00cd130 0x7f73e00cc1a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f73c80190a0 tx=0x7f73c8019230 comp rx=0 tx=0).stop 2024-04-23T14:33:04.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.298+0000 7f73e5400640 1 -- 172.21.15.100:0/2194214036 shutdown_connections 2024-04-23T14:33:04.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.298+0000 7f73e5400640 1 --2- 172.21.15.100:0/2194214036 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f73bc0683b0 0x7f73bc06a870 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:04.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.298+0000 7f73e5400640 1 --2- 172.21.15.100:0/2194214036 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f73e015c7a0 0x7f73e00b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:04.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.298+0000 7f73e5400640 1 --2- 172.21.15.100:0/2194214036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73e00cd130 0x7f73e00cc1a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:04.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.298+0000 7f73e5400640 1 --2- 172.21.15.100:0/2194214036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73e006bb20 0x7f73e0099810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:04.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.298+0000 7f73e5400640 1 -- 172.21.15.100:0/2194214036 >> 172.21.15.100:0/2194214036 conn(0x7f73e00b5200 msgr2=0x7f73e015b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:04.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.298+0000 7f73e5400640 1 -- 172.21.15.100:0/2194214036 shutdown_connections 2024-04-23T14:33:04.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.298+0000 7f73e5400640 1 -- 172.21.15.100:0/2194214036 wait complete. 2024-04-23T14:33:04.317 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-23T14:33:04.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.454+0000 7f95c2000640 1 Processor -- start 2024-04-23T14:33:04.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.454+0000 7f95c2000640 1 -- start start 2024-04-23T14:33:04.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.454+0000 7f95c2000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f95bc0adfe0 0x7f95bc06d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:04.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.454+0000 7f95c2000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95bc06e5f0 con 0x7f95bc0adb90 2024-04-23T14:33:04.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.454+0000 7f95c2000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95bc06e8b0 con 0x7f95bc15e860 2024-04-23T14:33:04.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.454+0000 7f95c2000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f95bc0ae410 con 0x7f95bc0adfe0 2024-04-23T14:33:04.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.454+0000 7f95b2a00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f95bc0adb90 0x7f95bc06f950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:59642/0 (socket says 172.21.15.100:59642) 2024-04-23T14:33:04.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.454+0000 7f95b2a00640 1 -- 172.21.15.100:0/777100397 learned_addr learned my addr 172.21.15.100:0/777100397 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:04.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.454+0000 7f95bbe00640 1 --2- 172.21.15.100:0/777100397 >> v2:172.21.15.151:3300/0 conn(0x7f95bc0adfe0 0x7f95bc06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:04.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.458+0000 7f95baa00640 1 -- 172.21.15.100:0/777100397 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1232298280 0 0) 0x7f95bc06e5f0 con 0x7f95bc0adb90 2024-04-23T14:33:04.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.458+0000 7f95baa00640 1 -- 172.21.15.100:0/777100397 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95a0003530 con 0x7f95bc0adb90 2024-04-23T14:33:04.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.458+0000 7f95bbe00640 1 -- 172.21.15.100:0/777100397 >> v1:172.21.15.151:6789/0 conn(0x7f95bc15e860 legacy=0x7f95bc0b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:04.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.458+0000 7f95bbe00640 1 -- 172.21.15.100:0/777100397 >> v1:172.21.15.100:6789/0 conn(0x7f95bc0adb90 legacy=0x7f95bc06f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:04.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.458+0000 7f95bbe00640 1 -- 172.21.15.100:0/777100397 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95bc06eb90 con 0x7f95bc0adfe0 2024-04-23T14:33:04.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.458+0000 7f95bbe00640 1 --2- 172.21.15.100:0/777100397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95bc0adfe0 0x7f95bc06d350 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009d90 tx=0x7f95b401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0a76d9d5d805c7c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:04.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.458+0000 7f95baa00640 1 -- 172.21.15.100:0/777100397 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b40b2070 con 0x7f95bc0adfe0 2024-04-23T14:33:04.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.458+0000 7f95baa00640 1 -- 172.21.15.100:0/777100397 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95b40a5910 con 0x7f95bc0adfe0 2024-04-23T14:33:04.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.458+0000 7f95baa00640 1 -- 172.21.15.100:0/777100397 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b40a5ce0 con 0x7f95bc0adfe0 2024-04-23T14:33:04.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95c2000640 1 -- 172.21.15.100:0/777100397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95bc0adfe0 msgr2=0x7f95bc06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:04.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95c2000640 1 --2- 172.21.15.100:0/777100397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95bc0adfe0 0x7f95bc06d350 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009d90 tx=0x7f95b401c940 comp rx=0 tx=0).stop 2024-04-23T14:33:04.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95c2000640 1 -- 172.21.15.100:0/777100397 shutdown_connections 2024-04-23T14:33:04.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95c2000640 1 --2- 172.21.15.100:0/777100397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95bc0adfe0 0x7f95bc06d350 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:04.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95c2000640 1 -- 172.21.15.100:0/777100397 >> 172.21.15.100:0/777100397 conn(0x7f95bc072590 msgr2=0x7f95bc0729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:04.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95c2000640 1 -- 172.21.15.100:0/777100397 shutdown_connections 2024-04-23T14:33:04.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95c2000640 1 -- 172.21.15.100:0/777100397 wait complete. 2024-04-23T14:33:04.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95c2000640 1 Processor -- start 2024-04-23T14:33:04.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95c2000640 1 -- start start 2024-04-23T14:33:04.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95c2000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95bc0adb90 0x7f95bc161960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:04.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95c2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95bc0adfe0 0x7f95bc14c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:04.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95c2000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f95bc15e860 0x7f95bc160dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:04.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95c2000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc15ec40 con 0x7f95bc0adfe0 2024-04-23T14:33:04.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95c2000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc073790 con 0x7f95bc0adb90 2024-04-23T14:33:04.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95c2000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f95bc070060 con 0x7f95bc15e860 2024-04-23T14:33:04.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95b2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95bc0adfe0 0x7f95bc14c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:04.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95b2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95bc0adfe0 0x7f95bc14c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49488/0 (socket says 172.21.15.100:49488) 2024-04-23T14:33:04.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95b2a00640 1 -- 172.21.15.100:0/1246670110 learned_addr learned my addr 172.21.15.100:0/1246670110 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:04.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95bbe00640 1 --2- 172.21.15.100:0/1246670110 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f95bc15e860 0x7f95bc160dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:04.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95bb400640 1 --2- 172.21.15.100:0/1246670110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95bc0adb90 0x7f95bc161960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:04.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95b2a00640 1 -- 172.21.15.100:0/1246670110 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f95bc15e860 msgr2=0x7f95bc160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:04.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95b2a00640 1 --2- 172.21.15.100:0/1246670110 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f95bc15e860 0x7f95bc160dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:04.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95b2a00640 1 -- 172.21.15.100:0/1246670110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95bc0adb90 msgr2=0x7f95bc161960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:04.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95b2a00640 1 --2- 172.21.15.100:0/1246670110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95bc0adb90 0x7f95bc161960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:04.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95b2a00640 1 -- 172.21.15.100:0/1246670110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95bc152920 con 0x7f95bc0adfe0 2024-04-23T14:33:04.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.462+0000 7f95bbe00640 1 --2- 172.21.15.100:0/1246670110 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f95bc15e860 0x7f95bc160dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:04.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.466+0000 7f95b2a00640 1 --2- 172.21.15.100:0/1246670110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95bc0adfe0 0x7f95bc14c960 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f95a400c420 tx=0x7f95a400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:04.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.466+0000 7f95b3e00640 1 -- 172.21.15.100:0/1246670110 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4017070 con 0x7f95bc0adfe0 2024-04-23T14:33:04.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.466+0000 7f95b3e00640 1 -- 172.21.15.100:0/1246670110 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95a4005cf0 con 0x7f95bc0adfe0 2024-04-23T14:33:04.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.466+0000 7f95c2000640 1 -- 172.21.15.100:0/1246670110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95bc152c00 con 0x7f95bc0adfe0 2024-04-23T14:33:04.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.466+0000 7f95b3e00640 1 -- 172.21.15.100:0/1246670110 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a40072d0 con 0x7f95bc0adfe0 2024-04-23T14:33:04.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.466+0000 7f95c2000640 1 -- 172.21.15.100:0/1246670110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95bc1536e0 con 0x7f95bc0adfe0 2024-04-23T14:33:04.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.466+0000 7f95c2000640 1 -- 172.21.15.100:0/1246670110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95bc068ad0 con 0x7f95bc0adfe0 2024-04-23T14:33:04.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.466+0000 7f95b3e00640 1 -- 172.21.15.100:0/1246670110 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f95a401a070 con 0x7f95bc0adfe0 2024-04-23T14:33:04.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.466+0000 7f95b3e00640 1 --2- 172.21.15.100:0/1246670110 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9598068380 0x7f959806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:04.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.466+0000 7f95b3e00640 1 -- 172.21.15.100:0/1246670110 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f95a40860c0 con 0x7f95bc0adfe0 2024-04-23T14:33:04.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.466+0000 7f95bb400640 1 --2- 172.21.15.100:0/1246670110 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9598068380 0x7f959806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:04.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.470+0000 7f95bb400640 1 --2- 172.21.15.100:0/1246670110 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9598068380 0x7f959806a840 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f95ac001650 tx=0x7f95ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:04.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.470+0000 7f95b3e00640 1 -- 172.21.15.100:0/1246670110 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f95a4055c20 con 0x7f95bc0adfe0 2024-04-23T14:33:04.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.630+0000 7f95c2000640 1 -- 172.21.15.100:0/1246670110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f95bc1621e0 con 0x7f95bc0adfe0 2024-04-23T14:33:04.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.634+0000 7f95b3e00640 1 -- 172.21.15.100:0/1246670110 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f95a4059f20 con 0x7f95bc0adfe0 2024-04-23T14:33:04.634 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:04.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.634+0000 7f95c2000640 1 -- 172.21.15.100:0/1246670110 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9598068380 msgr2=0x7f959806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:04.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.634+0000 7f95c2000640 1 --2- 172.21.15.100:0/1246670110 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9598068380 0x7f959806a840 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f95ac001650 tx=0x7f95ac009040 comp rx=0 tx=0).stop 2024-04-23T14:33:04.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.634+0000 7f95c2000640 1 -- 172.21.15.100:0/1246670110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95bc0adfe0 msgr2=0x7f95bc14c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:04.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.634+0000 7f95c2000640 1 --2- 172.21.15.100:0/1246670110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95bc0adfe0 0x7f95bc14c960 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f95a400c420 tx=0x7f95a400c8e0 comp rx=0 tx=0).stop 2024-04-23T14:33:04.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.634+0000 7f95c2000640 1 -- 172.21.15.100:0/1246670110 shutdown_connections 2024-04-23T14:33:04.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.634+0000 7f95c2000640 1 --2- 172.21.15.100:0/1246670110 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9598068380 0x7f959806a840 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:04.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.634+0000 7f95c2000640 1 --2- 172.21.15.100:0/1246670110 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f95bc15e860 0x7f95bc160dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:04.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.634+0000 7f95c2000640 1 --2- 172.21.15.100:0/1246670110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95bc0adfe0 0x7f95bc14c960 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:04.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.634+0000 7f95c2000640 1 --2- 172.21.15.100:0/1246670110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95bc0adb90 0x7f95bc161960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:04.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.634+0000 7f95c2000640 1 -- 172.21.15.100:0/1246670110 >> 172.21.15.100:0/1246670110 conn(0x7f95bc072590 msgr2=0x7f95bc1588e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:04.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.634+0000 7f95c2000640 1 -- 172.21.15.100:0/1246670110 shutdown_connections 2024-04-23T14:33:04.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.634+0000 7f95c2000640 1 -- 172.21.15.100:0/1246670110 wait complete. 2024-04-23T14:33:04.655 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2024-04-23T14:33:04.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.790+0000 7f30aac00640 1 Processor -- start 2024-04-23T14:33:04.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.790+0000 7f30aac00640 1 -- start start 2024-04-23T14:33:04.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.790+0000 7f30aac00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f30a415c470 0x7f30a40b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:04.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.790+0000 7f30aac00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f30a4160de0 0x7f30a40ad160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:04.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.790+0000 7f30aac00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30a40adab0 con 0x7f30a40ad6a0 2024-04-23T14:33:04.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.790+0000 7f30aac00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f30a4160320 con 0x7f30a4160de0 2024-04-23T14:33:04.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.790+0000 7f30aac00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f30a40cc7e0 con 0x7f30a415c470 2024-04-23T14:33:04.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.790+0000 7f30a3400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f30a4160de0 0x7f30a40ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:04.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.790+0000 7f30a3400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f30a4160de0 0x7f30a40ad160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49504/0 (socket says 172.21.15.100:49504) 2024-04-23T14:33:04.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.790+0000 7f30a3400640 1 -- 172.21.15.100:0/4223857988 learned_addr learned my addr 172.21.15.100:0/4223857988 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:04.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.790+0000 7f30a3e00640 1 --2- 172.21.15.100:0/4223857988 >> v2:172.21.15.151:3300/0 conn(0x7f30a415c470 0x7f30a40b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:04.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.790+0000 7f30a3400640 1 -- 172.21.15.100:0/4223857988 >> v2:172.21.15.151:3300/0 conn(0x7f30a415c470 msgr2=0x7f30a40b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:04.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.790+0000 7f30a3400640 1 --2- 172.21.15.100:0/4223857988 >> v2:172.21.15.151:3300/0 conn(0x7f30a415c470 0x7f30a40b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:04.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.790+0000 7f30a3400640 1 -- 172.21.15.100:0/4223857988 >> v1:172.21.15.151:6789/0 conn(0x7f30a40ad6a0 legacy=0x7f30a406bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:04.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.794+0000 7f30a3400640 1 -- 172.21.15.100:0/4223857988 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a41605e0 con 0x7f30a4160de0 2024-04-23T14:33:04.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.794+0000 7f30a3e00640 1 --2- 172.21.15.100:0/4223857988 >> v2:172.21.15.151:3300/0 conn(0x7f30a415c470 0x7f30a40b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:04.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.794+0000 7f30a3400640 1 --2- 172.21.15.100:0/4223857988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30a4160de0 0x7f30a40ad160 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f30a4074300 tx=0x7f309401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b00b2bb29ad0de server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:04.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.794+0000 7f30a2a00640 1 -- 172.21.15.100:0/4223857988 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30940b1070 con 0x7f30a4160de0 2024-04-23T14:33:04.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.794+0000 7f30a2a00640 1 -- 172.21.15.100:0/4223857988 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30940ac600 con 0x7f30a4160de0 2024-04-23T14:33:04.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.794+0000 7f30a2a00640 1 -- 172.21.15.100:0/4223857988 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30940ac9d0 con 0x7f30a4160de0 2024-04-23T14:33:04.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.794+0000 7f30aac00640 1 -- 172.21.15.100:0/4223857988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30a4160de0 msgr2=0x7f30a40ad160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:04.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.794+0000 7f30aac00640 1 --2- 172.21.15.100:0/4223857988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30a4160de0 0x7f30a40ad160 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f30a4074300 tx=0x7f309401c490 comp rx=0 tx=0).stop 2024-04-23T14:33:04.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.794+0000 7f30aac00640 1 -- 172.21.15.100:0/4223857988 shutdown_connections 2024-04-23T14:33:04.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.794+0000 7f30aac00640 1 --2- 172.21.15.100:0/4223857988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30a4160de0 0x7f30a40ad160 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:04.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.794+0000 7f30aac00640 1 --2- 172.21.15.100:0/4223857988 >> v2:172.21.15.151:3300/0 conn(0x7f30a415c470 0x7f30a40b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:04.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.794+0000 7f30aac00640 1 -- 172.21.15.100:0/4223857988 >> 172.21.15.100:0/4223857988 conn(0x7f30a40b1e80 msgr2=0x7f30a40b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:04.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.794+0000 7f30aac00640 1 -- 172.21.15.100:0/4223857988 shutdown_connections 2024-04-23T14:33:04.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.794+0000 7f30aac00640 1 -- 172.21.15.100:0/4223857988 wait complete. 2024-04-23T14:33:04.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f30aac00640 1 Processor -- start 2024-04-23T14:33:04.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f30aac00640 1 -- start start 2024-04-23T14:33:04.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f30aac00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30a40ad6a0 0x7f30a4099800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:04.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f30aac00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30a415c470 0x7f30a4152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:04.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f30aac00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30a4160de0 0x7f30a4166b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:04.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f30aac00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f30a40cc370 con 0x7f30a40ad6a0 2024-04-23T14:33:04.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f30aac00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f30a415ca60 con 0x7f30a4160de0 2024-04-23T14:33:04.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f30aac00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f30a40cc4f0 con 0x7f30a415c470 2024-04-23T14:33:04.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f30a3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30a40ad6a0 0x7f30a4099800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:04.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f30a3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30a40ad6a0 0x7f30a4099800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49514/0 (socket says 172.21.15.100:49514) 2024-04-23T14:33:04.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f30a3e00640 1 -- 172.21.15.100:0/1325311473 learned_addr learned my addr 172.21.15.100:0/1325311473 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:04.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f30a3400640 1 --2- 172.21.15.100:0/1325311473 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30a415c470 0x7f30a4152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:04.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f30a8c00640 1 --2- 172.21.15.100:0/1325311473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30a4160de0 0x7f30a4166b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:04.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f30a3e00640 1 -- 172.21.15.100:0/1325311473 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30a415c470 msgr2=0x7f30a4152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:04.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f30a3e00640 1 --2- 172.21.15.100:0/1325311473 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30a415c470 0x7f30a4152180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:04.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f30a3e00640 1 -- 172.21.15.100:0/1325311473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30a4160de0 msgr2=0x7f30a4166b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:04.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f30a3e00640 1 --2- 172.21.15.100:0/1325311473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30a4160de0 0x7f30a4166b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:04.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f30a3e00640 1 -- 172.21.15.100:0/1325311473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a415afd0 con 0x7f30a40ad6a0 2024-04-23T14:33:04.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f30a8c00640 1 --2- 172.21.15.100:0/1325311473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30a4160de0 0x7f30a4166b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:04.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f30a3e00640 1 --2- 172.21.15.100:0/1325311473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30a40ad6a0 0x7f30a4099800 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f308c00c7f0 tx=0x7f308c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:04.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f309be00640 1 -- 172.21.15.100:0/1325311473 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c015070 con 0x7f30a40ad6a0 2024-04-23T14:33:04.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f309be00640 1 -- 172.21.15.100:0/1325311473 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f308c007820 con 0x7f30a40ad6a0 2024-04-23T14:33:04.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f309be00640 1 -- 172.21.15.100:0/1325311473 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c007c10 con 0x7f30a40ad6a0 2024-04-23T14:33:04.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f30aac00640 1 -- 172.21.15.100:0/1325311473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a40b0710 con 0x7f30a40ad6a0 2024-04-23T14:33:04.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.798+0000 7f30aac00640 1 -- 172.21.15.100:0/1325311473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a4165da0 con 0x7f30a40ad6a0 2024-04-23T14:33:04.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.802+0000 7f30aac00640 1 -- 172.21.15.100:0/1325311473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30a4068ad0 con 0x7f30a40ad6a0 2024-04-23T14:33:04.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.802+0000 7f309be00640 1 -- 172.21.15.100:0/1325311473 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f308c018070 con 0x7f30a40ad6a0 2024-04-23T14:33:04.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.802+0000 7f309be00640 1 --2- 172.21.15.100:0/1325311473 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3080068380 0x7f308006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:04.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.802+0000 7f309be00640 1 -- 172.21.15.100:0/1325311473 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f308c086f50 con 0x7f30a40ad6a0 2024-04-23T14:33:04.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.802+0000 7f30a3400640 1 --2- 172.21.15.100:0/1325311473 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3080068380 0x7f308006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:04.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.802+0000 7f30a3400640 1 --2- 172.21.15.100:0/1325311473 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3080068380 0x7f308006a840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f30940ae000 tx=0x7f3094008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:04.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:04.806+0000 7f309be00640 1 -- 172.21.15.100:0/1325311473 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f308c057070 con 0x7f30a40ad6a0 2024-04-23T14:33:05.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:05.022+0000 7f30aac00640 1 -- 172.21.15.100:0/1325311473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) -- 0x7f30a406fea0 con 0x7f30a40ad6a0 2024-04-23T14:33:05.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:05.854+0000 7f309be00640 1 -- 172.21.15.100:0/1325311473 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v75) ==== 150+0+0 (secure 0 0 0) 0x7f308c056970 con 0x7f30a40ad6a0 2024-04-23T14:33:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:05.858+0000 7f30aac00640 1 -- 172.21.15.100:0/1325311473 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3080068380 msgr2=0x7f308006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:05.858+0000 7f30aac00640 1 --2- 172.21.15.100:0/1325311473 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3080068380 0x7f308006a840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f30940ae000 tx=0x7f3094008040 comp rx=0 tx=0).stop 2024-04-23T14:33:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:05.858+0000 7f30aac00640 1 -- 172.21.15.100:0/1325311473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30a40ad6a0 msgr2=0x7f30a4099800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:05.858+0000 7f30aac00640 1 --2- 172.21.15.100:0/1325311473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30a40ad6a0 0x7f30a4099800 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f308c00c7f0 tx=0x7f308c00ccb0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:05.858+0000 7f30aac00640 1 -- 172.21.15.100:0/1325311473 shutdown_connections 2024-04-23T14:33:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:05.858+0000 7f30aac00640 1 --2- 172.21.15.100:0/1325311473 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3080068380 0x7f308006a840 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:05.858+0000 7f30aac00640 1 --2- 172.21.15.100:0/1325311473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30a4160de0 0x7f30a4166b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:05.858+0000 7f30aac00640 1 --2- 172.21.15.100:0/1325311473 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30a415c470 0x7f30a4152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:05.858+0000 7f30aac00640 1 --2- 172.21.15.100:0/1325311473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30a40ad6a0 0x7f30a4099800 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:05.858+0000 7f30aac00640 1 -- 172.21.15.100:0/1325311473 >> 172.21.15.100:0/1325311473 conn(0x7f30a40b1e80 msgr2=0x7f30a40b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:05.858+0000 7f30aac00640 1 -- 172.21.15.100:0/1325311473 shutdown_connections 2024-04-23T14:33:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:05.858+0000 7f30aac00640 1 -- 172.21.15.100:0/1325311473 wait complete. 2024-04-23T14:33:05.882 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-23T14:33:06.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.018+0000 7fa2e6a00640 1 Processor -- start 2024-04-23T14:33:06.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.018+0000 7fa2e6a00640 1 -- start start 2024-04-23T14:33:06.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.022+0000 7fa2e6a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa2e006f8c0 0x7fa2e006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.022+0000 7fa2e6a00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fa2e00d0170 0x7fa2e00ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.022+0000 7fa2e6a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa2e00ce620 0x7fa2e015f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.022+0000 7fa2e6a00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fa2e00ad050 con 0x7fa2e006f8c0 2024-04-23T14:33:06.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.022+0000 7fa2e6a00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fa2e00ad8f0 con 0x7fa2e00d0170 2024-04-23T14:33:06.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.022+0000 7fa2e6a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e006dd60 con 0x7fa2e00ce620 2024-04-23T14:33:06.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.022+0000 7fa2dfe00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa2e006f8c0 0x7fa2e006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.022+0000 7fa2dfe00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa2e006f8c0 0x7fa2e006c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49520/0 (socket says 172.21.15.100:49520) 2024-04-23T14:33:06.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.022+0000 7fa2dfe00640 1 -- 172.21.15.100:0/757872582 learned_addr learned my addr 172.21.15.100:0/757872582 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:06.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.022+0000 7fa2e4a00640 1 --2- 172.21.15.100:0/757872582 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa2e00ce620 0x7fa2e015f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.022+0000 7fa2d7400640 1 --2- 172.21.15.100:0/757872582 >> v2:172.21.15.151:3300/0 conn(0x7fa2e00d0170 0x7fa2e00ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.022+0000 7fa2dfe00640 1 -- 172.21.15.100:0/757872582 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa2e00ce620 msgr2=0x7fa2e015f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:06.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.022+0000 7fa2dfe00640 1 --2- 172.21.15.100:0/757872582 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa2e00ce620 0x7fa2e015f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.022+0000 7fa2dfe00640 1 -- 172.21.15.100:0/757872582 >> v2:172.21.15.151:3300/0 conn(0x7fa2e00d0170 msgr2=0x7fa2e00ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:06.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.022+0000 7fa2dfe00640 1 --2- 172.21.15.100:0/757872582 >> v2:172.21.15.151:3300/0 conn(0x7fa2e00d0170 0x7fa2e00ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.022+0000 7fa2dfe00640 1 -- 172.21.15.100:0/757872582 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e00d09c0 con 0x7fa2e006f8c0 2024-04-23T14:33:06.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.022+0000 7fa2e4a00640 1 --2- 172.21.15.100:0/757872582 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa2e00ce620 0x7fa2e015f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:06.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.022+0000 7fa2d7400640 1 --2- 172.21.15.100:0/757872582 >> v2:172.21.15.151:3300/0 conn(0x7fa2e00d0170 0x7fa2e00ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:06.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.026+0000 7fa2dfe00640 1 --2- 172.21.15.100:0/757872582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e006f8c0 0x7fa2e006c6c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00d00c0 tx=0x7fa2d001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2d5b3be6ccc1e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:06.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.026+0000 7fa2df400640 1 -- 172.21.15.100:0/757872582 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00b0070 con 0x7fa2e006f8c0 2024-04-23T14:33:06.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.026+0000 7fa2df400640 1 -- 172.21.15.100:0/757872582 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d00ab600 con 0x7fa2e006f8c0 2024-04-23T14:33:06.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.026+0000 7fa2df400640 1 -- 172.21.15.100:0/757872582 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00ab9d0 con 0x7fa2e006f8c0 2024-04-23T14:33:06.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.026+0000 7fa2e6a00640 1 -- 172.21.15.100:0/757872582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e006f8c0 msgr2=0x7fa2e006c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:06.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.026+0000 7fa2e6a00640 1 --2- 172.21.15.100:0/757872582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e006f8c0 0x7fa2e006c6c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00d00c0 tx=0x7fa2d001c260 comp rx=0 tx=0).stop 2024-04-23T14:33:06.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.026+0000 7fa2e6a00640 1 -- 172.21.15.100:0/757872582 shutdown_connections 2024-04-23T14:33:06.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.026+0000 7fa2e6a00640 1 --2- 172.21.15.100:0/757872582 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa2e00ce620 0x7fa2e015f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.026+0000 7fa2e6a00640 1 --2- 172.21.15.100:0/757872582 >> v2:172.21.15.151:3300/0 conn(0x7fa2e00d0170 0x7fa2e00ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.026+0000 7fa2e6a00640 1 --2- 172.21.15.100:0/757872582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e006f8c0 0x7fa2e006c6c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.026+0000 7fa2e6a00640 1 -- 172.21.15.100:0/757872582 >> 172.21.15.100:0/757872582 conn(0x7fa2e00b5200 msgr2=0x7fa2e00b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:06.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.026+0000 7fa2e6a00640 1 -- 172.21.15.100:0/757872582 shutdown_connections 2024-04-23T14:33:06.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.026+0000 7fa2e6a00640 1 -- 172.21.15.100:0/757872582 wait complete. 2024-04-23T14:33:06.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.026+0000 7fa2e6a00640 1 Processor -- start 2024-04-23T14:33:06.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.026+0000 7fa2e6a00640 1 -- start start 2024-04-23T14:33:06.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.026+0000 7fa2e6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e006f8c0 0x7fa2e00c5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.026+0000 7fa2e6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2e00ce620 0x7fa2e00c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2e6a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa2e00d0170 0x7fa2e00c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2e6a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e00add80 con 0x7fa2e006f8c0 2024-04-23T14:33:06.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2e6a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e00d0760 con 0x7fa2e00ce620 2024-04-23T14:33:06.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2e6a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e00adf00 con 0x7fa2e00d0170 2024-04-23T14:33:06.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2dfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e006f8c0 0x7fa2e00c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2dfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e006f8c0 0x7fa2e00c5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49528/0 (socket says 172.21.15.100:49528) 2024-04-23T14:33:06.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2dfe00640 1 -- 172.21.15.100:0/4185098859 learned_addr learned my addr 172.21.15.100:0/4185098859 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:06.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2e4a00640 1 --2- 172.21.15.100:0/4185098859 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa2e00d0170 0x7fa2e00c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2d7400640 1 --2- 172.21.15.100:0/4185098859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2e00ce620 0x7fa2e00c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2dfe00640 1 -- 172.21.15.100:0/4185098859 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa2e00d0170 msgr2=0x7fa2e00c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:06.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2dfe00640 1 --2- 172.21.15.100:0/4185098859 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa2e00d0170 0x7fa2e00c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2dfe00640 1 -- 172.21.15.100:0/4185098859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2e00ce620 msgr2=0x7fa2e00c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:06.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2dfe00640 1 --2- 172.21.15.100:0/4185098859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2e00ce620 0x7fa2e00c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2dfe00640 1 -- 172.21.15.100:0/4185098859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e00cdb40 con 0x7fa2e006f8c0 2024-04-23T14:33:06.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2e4a00640 1 --2- 172.21.15.100:0/4185098859 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa2e00d0170 0x7fa2e00c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:06.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2d7400640 1 --2- 172.21.15.100:0/4185098859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2e00ce620 0x7fa2e00c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:06.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2dfe00640 1 --2- 172.21.15.100:0/4185098859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e006f8c0 0x7fa2e00c5170 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00ae040 tx=0x7fa2d001aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:06.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2dcc00640 1 -- 172.21.15.100:0/4185098859 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00b0040 con 0x7fa2e006f8c0 2024-04-23T14:33:06.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2dcc00640 1 -- 172.21.15.100:0/4185098859 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d00bc070 con 0x7fa2e006f8c0 2024-04-23T14:33:06.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2e6a00640 1 -- 172.21.15.100:0/4185098859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e00c40d0 con 0x7fa2e006f8c0 2024-04-23T14:33:06.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2dcc00640 1 -- 172.21.15.100:0/4185098859 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00b6880 con 0x7fa2e006f8c0 2024-04-23T14:33:06.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2e6a00640 1 -- 172.21.15.100:0/4185098859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e00c4690 con 0x7fa2e006f8c0 2024-04-23T14:33:06.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2e6a00640 1 -- 172.21.15.100:0/4185098859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2e0068ad0 con 0x7fa2e006f8c0 2024-04-23T14:33:06.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.030+0000 7fa2dcc00640 1 -- 172.21.15.100:0/4185098859 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa2d00b6a20 con 0x7fa2e006f8c0 2024-04-23T14:33:06.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.034+0000 7fa2dcc00640 1 --2- 172.21.15.100:0/4185098859 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa2bc068380 0x7fa2bc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.034+0000 7fa2dcc00640 1 -- 172.21.15.100:0/4185098859 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa2d0121220 con 0x7fa2e006f8c0 2024-04-23T14:33:06.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.034+0000 7fa2d7400640 1 --2- 172.21.15.100:0/4185098859 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa2bc068380 0x7fa2bc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.034+0000 7fa2d7400640 1 --2- 172.21.15.100:0/4185098859 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa2bc068380 0x7fa2bc06a840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00b5f90 tx=0x7fa2c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:06.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.038+0000 7fa2dcc00640 1 -- 172.21.15.100:0/4185098859 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa2d0016110 con 0x7fa2e006f8c0 2024-04-23T14:33:06.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.194+0000 7fa2e6a00640 1 -- 172.21.15.100:0/4185098859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa2e00cce00 con 0x7fa2e006f8c0 2024-04-23T14:33:06.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.198+0000 7fa2dcc00640 1 -- 172.21.15.100:0/4185098859 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fa2d00f14f0 con 0x7fa2e006f8c0 2024-04-23T14:33:06.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.198+0000 7fa2e6a00640 1 -- 172.21.15.100:0/4185098859 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa2bc068380 msgr2=0x7fa2bc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:06.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.198+0000 7fa2e6a00640 1 --2- 172.21.15.100:0/4185098859 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa2bc068380 0x7fa2bc06a840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00b5f90 tx=0x7fa2c8008040 comp rx=0 tx=0).stop 2024-04-23T14:33:06.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.198+0000 7fa2e6a00640 1 -- 172.21.15.100:0/4185098859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e006f8c0 msgr2=0x7fa2e00c5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:06.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.198+0000 7fa2e6a00640 1 --2- 172.21.15.100:0/4185098859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e006f8c0 0x7fa2e00c5170 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00ae040 tx=0x7fa2d001aaa0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.198+0000 7fa2e6a00640 1 -- 172.21.15.100:0/4185098859 shutdown_connections 2024-04-23T14:33:06.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.198+0000 7fa2e6a00640 1 --2- 172.21.15.100:0/4185098859 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa2bc068380 0x7fa2bc06a840 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.198+0000 7fa2e6a00640 1 --2- 172.21.15.100:0/4185098859 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa2e00d0170 0x7fa2e00c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.198+0000 7fa2e6a00640 1 --2- 172.21.15.100:0/4185098859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2e00ce620 0x7fa2e00c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.198+0000 7fa2e6a00640 1 --2- 172.21.15.100:0/4185098859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e006f8c0 0x7fa2e00c5170 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.198+0000 7fa2e6a00640 1 -- 172.21.15.100:0/4185098859 >> 172.21.15.100:0/4185098859 conn(0x7fa2e00b5200 msgr2=0x7fa2e00cb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:06.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.198+0000 7fa2e6a00640 1 -- 172.21.15.100:0/4185098859 shutdown_connections 2024-04-23T14:33:06.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.198+0000 7fa2e6a00640 1 -- 172.21.15.100:0/4185098859 wait complete. 2024-04-23T14:33:06.220 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-23T14:33:06.220 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:33:06.340 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:33:06.426 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-23T14:33:06.427 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:33:06.427 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:06.435 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:33:06.435 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:06.485 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:06.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.658+0000 7fd5bec00640 1 Processor -- start 2024-04-23T14:33:06.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.658+0000 7fd5bec00640 1 -- start start 2024-04-23T14:33:06.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5bec00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd5b8158a80 0x7fd5b815f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5bec00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd5b8158e60 0x7fd5b8072f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5bec00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5b80b4200 con 0x7fd5b8073480 2024-04-23T14:33:06.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5bec00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fd5b80d04d0 con 0x7fd5b8158e60 2024-04-23T14:33:06.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5bec00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b815fb20 con 0x7fd5b8158a80 2024-04-23T14:33:06.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5be200640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fd5b8073480 0x7fd5b8162180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:59646/0 (socket says 172.21.15.100:59646) 2024-04-23T14:33:06.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5be200640 1 -- 172.21.15.100:0/3604581888 learned_addr learned my addr 172.21.15.100:0/3604581888 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:06.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5bce00640 1 --2- 172.21.15.100:0/3604581888 >> v2:172.21.15.151:3300/0 conn(0x7fd5b8158e60 0x7fd5b8072f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5bd800640 1 --2- 172.21.15.100:0/3604581888 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd5b8158a80 0x7fd5b815f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5b7e00640 1 -- 172.21.15.100:0/3604581888 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 424948973 0 0) 0x7fd5b00014a0 con 0x7fd5b8073480 2024-04-23T14:33:06.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5b7e00640 1 -- 172.21.15.100:0/3604581888 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5a0003530 con 0x7fd5b8073480 2024-04-23T14:33:06.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5b7e00640 1 -- 172.21.15.100:0/3604581888 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1340356897 0 0) 0x7fd5a0003530 con 0x7fd5b8073480 2024-04-23T14:33:06.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5b7e00640 1 -- 172.21.15.100:0/3604581888 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd5b8158a80 msgr2=0x7fd5b815f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:06.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5b7e00640 1 --2- 172.21.15.100:0/3604581888 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd5b8158a80 0x7fd5b815f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5b7e00640 1 -- 172.21.15.100:0/3604581888 >> v2:172.21.15.151:3300/0 conn(0x7fd5b8158e60 msgr2=0x7fd5b8072f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:06.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5b7e00640 1 --2- 172.21.15.100:0/3604581888 >> v2:172.21.15.151:3300/0 conn(0x7fd5b8158e60 0x7fd5b8072f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5b7e00640 1 -- 172.21.15.100:0/3604581888 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b80b4480 con 0x7fd5b8073480 2024-04-23T14:33:06.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5bce00640 1 --2- 172.21.15.100:0/3604581888 >> v2:172.21.15.151:3300/0 conn(0x7fd5b8158e60 0x7fd5b8072f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:33:06.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5b7e00640 1 -- 172.21.15.100:0/3604581888 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7fd5b0001bd0 con 0x7fd5b8073480 2024-04-23T14:33:06.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5b7e00640 1 -- 172.21.15.100:0/3604581888 >> v1:172.21.15.100:6789/0 conn(0x7fd5b8073480 legacy=0x7fd5b8162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:06.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5b7e00640 1 --2- 172.21.15.100:0/3604581888 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd5b8158a80 0x7fd5a00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5b7e00640 1 --2- 172.21.15.100:0/3604581888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5a0003f40 0x7fd5a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5b7e00640 1 --2- 172.21.15.100:0/3604581888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5a0004d50 0x7fd5a00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5b7e00640 1 -- 172.21.15.100:0/3604581888 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b80d04d0 con 0x7fd5a0004d50 2024-04-23T14:33:06.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5b7e00640 1 -- 172.21.15.100:0/3604581888 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b815fb20 con 0x7fd5a0003f40 2024-04-23T14:33:06.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5b7e00640 1 -- 172.21.15.100:0/3604581888 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd5a000bab0 con 0x7fd5b8158a80 2024-04-23T14:33:06.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5be200640 1 --2- 172.21.15.100:0/3604581888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5a0004d50 0x7fd5a00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5bd800640 1 --2- 172.21.15.100:0/3604581888 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd5b8158a80 0x7fd5a00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5bce00640 1 --2- 172.21.15.100:0/3604581888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5a0003f40 0x7fd5a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5be200640 1 -- 172.21.15.100:0/3604581888 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd5b8158a80 msgr2=0x7fd5a00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:06.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5be200640 1 --2- 172.21.15.100:0/3604581888 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd5b8158a80 0x7fd5a00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5be200640 1 -- 172.21.15.100:0/3604581888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5a0003f40 msgr2=0x7fd5a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:06.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5be200640 1 --2- 172.21.15.100:0/3604581888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5a0003f40 0x7fd5a0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.662+0000 7fd5be200640 1 -- 172.21.15.100:0/3604581888 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a0003530 con 0x7fd5a0004d50 2024-04-23T14:33:06.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.666+0000 7fd5be200640 1 --2- 172.21.15.100:0/3604581888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5a0004d50 0x7fd5a00051d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd5b000aa70 tx=0x7fd5b001d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d6c0550280d38f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:06.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.666+0000 7fd5b7e00640 1 -- 172.21.15.100:0/3604581888 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b000a690 con 0x7fd5a0004d50 2024-04-23T14:33:06.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.666+0000 7fd5b7e00640 1 -- 172.21.15.100:0/3604581888 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5b00072d0 con 0x7fd5a0004d50 2024-04-23T14:33:06.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.666+0000 7fd5b7e00640 1 -- 172.21.15.100:0/3604581888 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0007740 con 0x7fd5a0004d50 2024-04-23T14:33:06.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.666+0000 7fd5bec00640 1 -- 172.21.15.100:0/3604581888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5a0004d50 msgr2=0x7fd5a00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:06.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.666+0000 7fd5bec00640 1 --2- 172.21.15.100:0/3604581888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5a0004d50 0x7fd5a00051d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd5b000aa70 tx=0x7fd5b001d3c0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.666+0000 7fd5be200640 1 -- 172.21.15.100:0/3604581888 reap_dead start 2024-04-23T14:33:06.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.666+0000 7fd5bec00640 1 -- 172.21.15.100:0/3604581888 shutdown_connections 2024-04-23T14:33:06.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.666+0000 7fd5bec00640 1 -- 172.21.15.100:0/3604581888 >> 172.21.15.100:0/3604581888 conn(0x7fd5b80b3100 msgr2=0x7fd5b80b3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:06.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.666+0000 7fd5bec00640 1 -- 172.21.15.100:0/3604581888 shutdown_connections 2024-04-23T14:33:06.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.666+0000 7fd5bec00640 1 -- 172.21.15.100:0/3604581888 wait complete. 2024-04-23T14:33:06.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.666+0000 7fd5bec00640 1 Processor -- start 2024-04-23T14:33:06.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.666+0000 7fd5bec00640 1 -- start start 2024-04-23T14:33:06.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5bec00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd5a0004d50 0x7fd5b81f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5bec00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5b8161020 0x7fd5b8161480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5bec00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5b81595f0 0x7fd5b8159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5bec00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b806f6d0 con 0x7fd5b8161020 2024-04-23T14:33:06.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5bec00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b80738f0 con 0x7fd5b81595f0 2024-04-23T14:33:06.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5bec00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b806f850 con 0x7fd5a0004d50 2024-04-23T14:33:06.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5bce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5b8161020 0x7fd5b8161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5bce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5b8161020 0x7fd5b8161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49548/0 (socket says 172.21.15.100:49548) 2024-04-23T14:33:06.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5bce00640 1 -- 172.21.15.100:0/2753389004 learned_addr learned my addr 172.21.15.100:0/2753389004 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:06.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5be200640 1 --2- 172.21.15.100:0/2753389004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5b81595f0 0x7fd5b8159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5bd800640 1 --2- 172.21.15.100:0/2753389004 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd5a0004d50 0x7fd5b81f6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5bce00640 1 -- 172.21.15.100:0/2753389004 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd5a0004d50 msgr2=0x7fd5b81f6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:06.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5bce00640 1 --2- 172.21.15.100:0/2753389004 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd5a0004d50 0x7fd5b81f6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5bce00640 1 -- 172.21.15.100:0/2753389004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5b81595f0 msgr2=0x7fd5b8159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:06.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5bce00640 1 --2- 172.21.15.100:0/2753389004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5b81595f0 0x7fd5b8159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5bce00640 1 -- 172.21.15.100:0/2753389004 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b815a300 con 0x7fd5b8161020 2024-04-23T14:33:06.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5bce00640 1 --2- 172.21.15.100:0/2753389004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5b8161020 0x7fd5b8161480 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4011cd0 tx=0x7fd5a400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:06.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5b5600640 1 -- 172.21.15.100:0/2753389004 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4011d00 con 0x7fd5b8161020 2024-04-23T14:33:06.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5b5600640 1 -- 172.21.15.100:0/2753389004 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a4004030 con 0x7fd5b8161020 2024-04-23T14:33:06.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5b5600640 1 -- 172.21.15.100:0/2753389004 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4011ea0 con 0x7fd5b8161020 2024-04-23T14:33:06.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5bec00640 1 -- 172.21.15.100:0/2753389004 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b815a5e0 con 0x7fd5b8161020 2024-04-23T14:33:06.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5bec00640 1 -- 172.21.15.100:0/2753389004 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b81f8740 con 0x7fd5b8161020 2024-04-23T14:33:06.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.670+0000 7fd5bec00640 1 -- 172.21.15.100:0/2753389004 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b8075c20 con 0x7fd5b8161020 2024-04-23T14:33:06.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.674+0000 7fd5b5600640 1 -- 172.21.15.100:0/2753389004 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd5a4012040 con 0x7fd5b8161020 2024-04-23T14:33:06.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.674+0000 7fd5b5600640 1 --2- 172.21.15.100:0/2753389004 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd5980683b0 0x7fd59806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.674+0000 7fd5b5600640 1 -- 172.21.15.100:0/2753389004 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd5a40872c0 con 0x7fd5b8161020 2024-04-23T14:33:06.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.674+0000 7fd5bd800640 1 --2- 172.21.15.100:0/2753389004 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd5980683b0 0x7fd59806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.674+0000 7fd5bd800640 1 --2- 172.21.15.100:0/2753389004 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd5980683b0 0x7fd59806a870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009be0 tx=0x7fd5ac009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:06.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.678+0000 7fd5b5600640 1 -- 172.21.15.100:0/2753389004 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd5a4019070 con 0x7fd5b8161020 2024-04-23T14:33:06.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.838+0000 7fd5bec00640 1 -- 172.21.15.100:0/2753389004 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5b806ff10 con 0x7fd5b8161020 2024-04-23T14:33:06.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.838+0000 7fd5b5600640 1 -- 172.21.15.100:0/2753389004 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18859 (secure 0 0 0) 0x7fd5a4056d90 con 0x7fd5b8161020 2024-04-23T14:33:06.840 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:06.841 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:49.784608+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:06.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.842+0000 7fd5bec00640 1 -- 172.21.15.100:0/2753389004 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd5980683b0 msgr2=0x7fd59806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:06.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.842+0000 7fd5bec00640 1 --2- 172.21.15.100:0/2753389004 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd5980683b0 0x7fd59806a870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009be0 tx=0x7fd5ac009190 comp rx=0 tx=0).stop 2024-04-23T14:33:06.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.842+0000 7fd5bec00640 1 -- 172.21.15.100:0/2753389004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5b8161020 msgr2=0x7fd5b8161480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:06.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.842+0000 7fd5bec00640 1 --2- 172.21.15.100:0/2753389004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5b8161020 0x7fd5b8161480 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4011cd0 tx=0x7fd5a400f610 comp rx=0 tx=0).stop 2024-04-23T14:33:06.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.842+0000 7fd5bec00640 1 -- 172.21.15.100:0/2753389004 shutdown_connections 2024-04-23T14:33:06.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.842+0000 7fd5bec00640 1 --2- 172.21.15.100:0/2753389004 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd5980683b0 0x7fd59806a870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.842+0000 7fd5bec00640 1 --2- 172.21.15.100:0/2753389004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5b81595f0 0x7fd5b8159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.842+0000 7fd5bec00640 1 --2- 172.21.15.100:0/2753389004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5b8161020 0x7fd5b8161480 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.842+0000 7fd5bec00640 1 --2- 172.21.15.100:0/2753389004 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd5a0004d50 0x7fd5b81f6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.842+0000 7fd5bec00640 1 -- 172.21.15.100:0/2753389004 >> 172.21.15.100:0/2753389004 conn(0x7fd5b80b3100 msgr2=0x7fd5b80ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:06.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.842+0000 7fd5bec00640 1 -- 172.21.15.100:0/2753389004 shutdown_connections 2024-04-23T14:33:06.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.842+0000 7fd5bec00640 1 -- 172.21.15.100:0/2753389004 wait complete. 2024-04-23T14:33:06.861 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:32:06.835560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:32:07.930489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-23T14:32:11.636814+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-23T14:33:06.862 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:33:06.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d9200640 1 Processor -- start 2024-04-23T14:33:06.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d9200640 1 -- start start 2024-04-23T14:33:07.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d9200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc4d415c7a0 0x7fc4d40af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d9200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4d406cbb0 con 0x7fc4d40cd300 2024-04-23T14:33:07.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d9200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4d406ce90 con 0x7fc4d406bb20 2024-04-23T14:33:07.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d9200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fc4d406d450 con 0x7fc4d415c7a0 2024-04-23T14:33:07.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d2800640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fc4d40cd300 0x7fc4d40ad430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:59662/0 (socket says 172.21.15.100:59662) 2024-04-23T14:33:07.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d2800640 1 -- 172.21.15.100:0/3599290418 learned_addr learned my addr 172.21.15.100:0/3599290418 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:07.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d0a00640 1 -- 172.21.15.100:0/3599290418 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1937387800 0 0) 0x7fc4d406cbb0 con 0x7fc4d40cd300 2024-04-23T14:33:07.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d1e00640 1 --2- 172.21.15.100:0/3599290418 >> v2:172.21.15.151:3300/0 conn(0x7fc4d415c7a0 0x7fc4d40af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d0a00640 1 -- 172.21.15.100:0/3599290418 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4b8003530 con 0x7fc4d40cd300 2024-04-23T14:33:07.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d0a00640 1 -- 172.21.15.100:0/3599290418 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3465842080 0 0) 0x7fc4b8003530 con 0x7fc4d40cd300 2024-04-23T14:33:07.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d0a00640 1 -- 172.21.15.100:0/3599290418 >> v2:172.21.15.151:3300/0 conn(0x7fc4d415c7a0 msgr2=0x7fc4d40af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:07.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d0a00640 1 --2- 172.21.15.100:0/3599290418 >> v2:172.21.15.151:3300/0 conn(0x7fc4d415c7a0 0x7fc4d40af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d0a00640 1 -- 172.21.15.100:0/3599290418 >> v1:172.21.15.151:6789/0 conn(0x7fc4d406bb20 legacy=0x7fc4d40ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d0a00640 1 -- 172.21.15.100:0/3599290418 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d406d170 con 0x7fc4d40cd300 2024-04-23T14:33:07.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d0a00640 1 -- 172.21.15.100:0/3599290418 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7fc4cc001a00 con 0x7fc4d40cd300 2024-04-23T14:33:07.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d0a00640 1 -- 172.21.15.100:0/3599290418 >> v1:172.21.15.100:6789/0 conn(0x7fc4d40cd300 legacy=0x7fc4d40ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d0a00640 1 --2- 172.21.15.100:0/3599290418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4b80039c0 0x7fc4b8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d0a00640 1 --2- 172.21.15.100:0/3599290418 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc4b8004810 0x7fc4b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d0a00640 1 --2- 172.21.15.100:0/3599290418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4b80051d0 0x7fc4b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d0a00640 1 -- 172.21.15.100:0/3599290418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d406d450 con 0x7fc4b80051d0 2024-04-23T14:33:07.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d0a00640 1 -- 172.21.15.100:0/3599290418 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b800c170 con 0x7fc4b80039c0 2024-04-23T14:33:07.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d0a00640 1 -- 172.21.15.100:0/3599290418 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b800c370 con 0x7fc4b8004810 2024-04-23T14:33:07.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d2800640 1 --2- 172.21.15.100:0/3599290418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4b80051d0 0x7fc4b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d1e00640 1 --2- 172.21.15.100:0/3599290418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4b80039c0 0x7fc4b8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:06.998+0000 7fc4d1400640 1 --2- 172.21.15.100:0/3599290418 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc4b8004810 0x7fc4b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.002+0000 7fc4d2800640 1 -- 172.21.15.100:0/3599290418 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc4b8004810 msgr2=0x7fc4b8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:07.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.002+0000 7fc4d2800640 1 --2- 172.21.15.100:0/3599290418 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc4b8004810 0x7fc4b8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.002+0000 7fc4d2800640 1 -- 172.21.15.100:0/3599290418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4b80039c0 msgr2=0x7fc4b8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:07.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.002+0000 7fc4d2800640 1 --2- 172.21.15.100:0/3599290418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4b80039c0 0x7fc4b8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.002+0000 7fc4d2800640 1 -- 172.21.15.100:0/3599290418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b8003530 con 0x7fc4b80051d0 2024-04-23T14:33:07.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.002+0000 7fc4d2800640 1 -- 172.21.15.100:0/3599290418 reap_dead start 2024-04-23T14:33:07.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.002+0000 7fc4d2800640 1 --2- 172.21.15.100:0/3599290418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4b80051d0 0x7fc4b800b9d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc00a850 tx=0x7fc4cc01fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2f0a6e57f24d40 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:07.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.002+0000 7fc4d0a00640 1 -- 172.21.15.100:0/3599290418 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc00a4a0 con 0x7fc4b80051d0 2024-04-23T14:33:07.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.002+0000 7fc4d0a00640 1 -- 172.21.15.100:0/3599290418 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4cc0ad070 con 0x7fc4b80051d0 2024-04-23T14:33:07.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.002+0000 7fc4d0a00640 1 -- 172.21.15.100:0/3599290418 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc0a9980 con 0x7fc4b80051d0 2024-04-23T14:33:07.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.002+0000 7fc4d9200640 1 -- 172.21.15.100:0/3599290418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4b80051d0 msgr2=0x7fc4b800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.002+0000 7fc4d9200640 1 --2- 172.21.15.100:0/3599290418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4b80051d0 0x7fc4b800b9d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc00a850 tx=0x7fc4cc01fb00 comp rx=0 tx=0).stop 2024-04-23T14:33:07.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.002+0000 7fc4d9200640 1 -- 172.21.15.100:0/3599290418 shutdown_connections 2024-04-23T14:33:07.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.002+0000 7fc4d9200640 1 --2- 172.21.15.100:0/3599290418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4b80051d0 0x7fc4b800b9d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.002+0000 7fc4d9200640 1 -- 172.21.15.100:0/3599290418 >> 172.21.15.100:0/3599290418 conn(0x7fc4d40b5200 msgr2=0x7fc4d40b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:07.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.002+0000 7fc4d9200640 1 -- 172.21.15.100:0/3599290418 shutdown_connections 2024-04-23T14:33:07.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.006+0000 7fc4d9200640 1 -- 172.21.15.100:0/3599290418 wait complete. 2024-04-23T14:33:07.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.006+0000 7fc4d9200640 1 Processor -- start 2024-04-23T14:33:07.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.006+0000 7fc4d9200640 1 -- start start 2024-04-23T14:33:07.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.006+0000 7fc4d9200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc4b80051d0 0x7fc4d409de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.006+0000 7fc4d9200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4d40cafa0 0x7fc4d415e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.006+0000 7fc4d9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4d415e300 0x7fc4d415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.006+0000 7fc4d9200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d40ccf80 con 0x7fc4d415e300 2024-04-23T14:33:07.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.006+0000 7fc4d9200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d40cce00 con 0x7fc4d40cafa0 2024-04-23T14:33:07.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.006+0000 7fc4d9200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d40cd100 con 0x7fc4b80051d0 2024-04-23T14:33:07.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.006+0000 7fc4d2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4d415e300 0x7fc4d415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.006+0000 7fc4d2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4d415e300 0x7fc4d415ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49556/0 (socket says 172.21.15.100:49556) 2024-04-23T14:33:07.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.006+0000 7fc4d2800640 1 -- 172.21.15.100:0/1357866774 learned_addr learned my addr 172.21.15.100:0/1357866774 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:07.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.006+0000 7fc4d1400640 1 --2- 172.21.15.100:0/1357866774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4d40cafa0 0x7fc4d415e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.006+0000 7fc4d1e00640 1 --2- 172.21.15.100:0/1357866774 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc4b80051d0 0x7fc4d409de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.006+0000 7fc4d2800640 1 -- 172.21.15.100:0/1357866774 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc4b80051d0 msgr2=0x7fc4d409de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.006+0000 7fc4d2800640 1 --2- 172.21.15.100:0/1357866774 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc4b80051d0 0x7fc4d409de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.006+0000 7fc4d2800640 1 -- 172.21.15.100:0/1357866774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4d40cafa0 msgr2=0x7fc4d415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.006+0000 7fc4d2800640 1 --2- 172.21.15.100:0/1357866774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4d40cafa0 0x7fc4d415e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.006+0000 7fc4d2800640 1 -- 172.21.15.100:0/1357866774 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d415de50 con 0x7fc4d415e300 2024-04-23T14:33:07.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.006+0000 7fc4d2800640 1 --2- 172.21.15.100:0/1357866774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4d415e300 0x7fc4d415ed90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0203a0 tx=0x7fc4b8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:07.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.006+0000 7fc4ca000640 1 -- 172.21.15.100:0/1357866774 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc001260 con 0x7fc4d415e300 2024-04-23T14:33:07.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.010+0000 7fc4ca000640 1 -- 172.21.15.100:0/1357866774 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4cc00a480 con 0x7fc4d415e300 2024-04-23T14:33:07.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.010+0000 7fc4ca000640 1 -- 172.21.15.100:0/1357866774 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc0a9720 con 0x7fc4d415e300 2024-04-23T14:33:07.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.010+0000 7fc4d9200640 1 -- 172.21.15.100:0/1357866774 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d40d07b0 con 0x7fc4d415e300 2024-04-23T14:33:07.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.010+0000 7fc4d9200640 1 -- 172.21.15.100:0/1357866774 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d40d0d70 con 0x7fc4d415e300 2024-04-23T14:33:07.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.010+0000 7fc4d9200640 1 -- 172.21.15.100:0/1357866774 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d4068ad0 con 0x7fc4d415e300 2024-04-23T14:33:07.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.010+0000 7fc4ca000640 1 -- 172.21.15.100:0/1357866774 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc4cc0af030 con 0x7fc4d415e300 2024-04-23T14:33:07.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.010+0000 7fc4ca000640 1 --2- 172.21.15.100:0/1357866774 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc4b4068550 0x7fc4b406aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.010+0000 7fc4ca000640 1 -- 172.21.15.100:0/1357866774 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc4cc1236e0 con 0x7fc4d415e300 2024-04-23T14:33:07.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.010+0000 7fc4d1e00640 1 --2- 172.21.15.100:0/1357866774 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc4b4068550 0x7fc4b406aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.010+0000 7fc4d1e00640 1 --2- 172.21.15.100:0/1357866774 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc4b4068550 0x7fc4b406aa10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009b60 tx=0x7fc4c4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:07.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.014+0000 7fc4ca000640 1 -- 172.21.15.100:0/1357866774 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc4cc0c1070 con 0x7fc4d415e300 2024-04-23T14:33:07.078 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:33:07.081 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:33:07.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.174+0000 7fc4d9200640 1 -- 172.21.15.100:0/1357866774 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc4d4074f20 con 0x7fc4d415e300 2024-04-23T14:33:07.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.174+0000 7fc4ca000640 1 -- 172.21.15.100:0/1357866774 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fc4cc019810 con 0x7fc4d415e300 2024-04-23T14:33:07.177 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:33:07.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.178+0000 7fc4d9200640 1 -- 172.21.15.100:0/1357866774 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc4b4068550 msgr2=0x7fc4b406aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.178+0000 7fc4d9200640 1 --2- 172.21.15.100:0/1357866774 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc4b4068550 0x7fc4b406aa10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009b60 tx=0x7fc4c4009190 comp rx=0 tx=0).stop 2024-04-23T14:33:07.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.178+0000 7fc4d9200640 1 -- 172.21.15.100:0/1357866774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4d415e300 msgr2=0x7fc4d415ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.178+0000 7fc4d9200640 1 --2- 172.21.15.100:0/1357866774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4d415e300 0x7fc4d415ed90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0203a0 tx=0x7fc4b8005840 comp rx=0 tx=0).stop 2024-04-23T14:33:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.178+0000 7fc4d9200640 1 -- 172.21.15.100:0/1357866774 shutdown_connections 2024-04-23T14:33:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.178+0000 7fc4d9200640 1 --2- 172.21.15.100:0/1357866774 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc4b4068550 0x7fc4b406aa10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.178+0000 7fc4d9200640 1 --2- 172.21.15.100:0/1357866774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4d415e300 0x7fc4d415ed90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.178+0000 7fc4d9200640 1 --2- 172.21.15.100:0/1357866774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4d40cafa0 0x7fc4d415e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.178+0000 7fc4d9200640 1 --2- 172.21.15.100:0/1357866774 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc4b80051d0 0x7fc4d409de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.178+0000 7fc4d9200640 1 -- 172.21.15.100:0/1357866774 >> 172.21.15.100:0/1357866774 conn(0x7fc4d40b5200 msgr2=0x7fc4d406bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.178+0000 7fc4d9200640 1 -- 172.21.15.100:0/1357866774 shutdown_connections 2024-04-23T14:33:07.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.178+0000 7fc4d9200640 1 -- 172.21.15.100:0/1357866774 wait complete. 2024-04-23T14:33:07.198 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:33:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.334+0000 7fc42a200640 1 Processor -- start 2024-04-23T14:33:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.334+0000 7fc42a200640 1 -- start start 2024-04-23T14:33:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.334+0000 7fc42a200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc42406bb20 0x7fc4240ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.334+0000 7fc42a200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc4240cd130 0x7fc4240ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.334+0000 7fc42a200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4240cd540 con 0x7fc42415c7a0 2024-04-23T14:33:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.334+0000 7fc42a200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc42406d450 con 0x7fc4240cd130 2024-04-23T14:33:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.334+0000 7fc42a200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fc42406cbb0 con 0x7fc42406bb20 2024-04-23T14:33:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.334+0000 7fc423e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc4240cd130 0x7fc4240ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.334+0000 7fc423e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc4240cd130 0x7fc4240ad430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49570/0 (socket says 172.21.15.100:49570) 2024-04-23T14:33:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.334+0000 7fc423e00640 1 -- 172.21.15.100:0/448772788 learned_addr learned my addr 172.21.15.100:0/448772788 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.334+0000 7fc423e00640 1 -- 172.21.15.100:0/448772788 >> v2:172.21.15.151:3300/0 conn(0x7fc42406bb20 msgr2=0x7fc4240ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.334+0000 7fc423e00640 1 --2- 172.21.15.100:0/448772788 >> v2:172.21.15.151:3300/0 conn(0x7fc42406bb20 0x7fc4240ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.334+0000 7fc423e00640 1 -- 172.21.15.100:0/448772788 >> v1:172.21.15.151:6789/0 conn(0x7fc42415c7a0 legacy=0x7fc4240af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.334+0000 7fc423e00640 1 -- 172.21.15.100:0/448772788 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42406ce90 con 0x7fc4240cd130 2024-04-23T14:33:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.338+0000 7fc423e00640 1 --2- 172.21.15.100:0/448772788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4240cd130 0x7fc4240ad430 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc4240d1280 tx=0x7fc41c01c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3a997ffb80a26c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.338+0000 7fc422a00640 1 -- 172.21.15.100:0/448772788 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c0ac020 con 0x7fc4240cd130 2024-04-23T14:33:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.338+0000 7fc422a00640 1 -- 172.21.15.100:0/448772788 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc41c0a6410 con 0x7fc4240cd130 2024-04-23T14:33:07.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.338+0000 7fc422a00640 1 -- 172.21.15.100:0/448772788 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c0a67e0 con 0x7fc4240cd130 2024-04-23T14:33:07.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.338+0000 7fc42a200640 1 -- 172.21.15.100:0/448772788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4240cd130 msgr2=0x7fc4240ad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.338+0000 7fc42a200640 1 --2- 172.21.15.100:0/448772788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4240cd130 0x7fc4240ad430 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc4240d1280 tx=0x7fc41c01c740 comp rx=0 tx=0).stop 2024-04-23T14:33:07.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.338+0000 7fc42a200640 1 -- 172.21.15.100:0/448772788 shutdown_connections 2024-04-23T14:33:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.338+0000 7fc42a200640 1 --2- 172.21.15.100:0/448772788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4240cd130 0x7fc4240ad430 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.338+0000 7fc42a200640 1 --2- 172.21.15.100:0/448772788 >> v2:172.21.15.151:3300/0 conn(0x7fc42406bb20 0x7fc4240ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.338+0000 7fc42a200640 1 -- 172.21.15.100:0/448772788 >> 172.21.15.100:0/448772788 conn(0x7fc4240b5200 msgr2=0x7fc4240b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.338+0000 7fc42a200640 1 -- 172.21.15.100:0/448772788 shutdown_connections 2024-04-23T14:33:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.338+0000 7fc42a200640 1 -- 172.21.15.100:0/448772788 wait complete. 2024-04-23T14:33:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc42a200640 1 Processor -- start 2024-04-23T14:33:07.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc42a200640 1 -- start start 2024-04-23T14:33:07.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc42a200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc42406bb20 0x7fc424099050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc42a200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc4240cd130 0x7fc4240cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc42a200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc42415c7a0 0x7fc424158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc42a200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc4240ae550 con 0x7fc42406bb20 2024-04-23T14:33:07.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc42a200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc4240ae3d0 con 0x7fc42415c7a0 2024-04-23T14:33:07.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc42a200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc4240ae6d0 con 0x7fc4240cd130 2024-04-23T14:33:07.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc423400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc42406bb20 0x7fc424099050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc423400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc42406bb20 0x7fc424099050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49574/0 (socket says 172.21.15.100:49574) 2024-04-23T14:33:07.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc423400640 1 -- 172.21.15.100:0/1198065446 learned_addr learned my addr 172.21.15.100:0/1198065446 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:07.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc423400640 1 -- 172.21.15.100:0/1198065446 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc4240cd130 msgr2=0x7fc4240cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc423400640 1 --2- 172.21.15.100:0/1198065446 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc4240cd130 0x7fc4240cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc423400640 1 -- 172.21.15.100:0/1198065446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc42415c7a0 msgr2=0x7fc424158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc423400640 1 --2- 172.21.15.100:0/1198065446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc42415c7a0 0x7fc424158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc423400640 1 -- 172.21.15.100:0/1198065446 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4241593d0 con 0x7fc42406bb20 2024-04-23T14:33:07.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc423400640 1 --2- 172.21.15.100:0/1198065446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc42406bb20 0x7fc424099050 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc41400c490 tx=0x7fc414001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:07.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc41be00640 1 -- 172.21.15.100:0/1198065446 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414017070 con 0x7fc42406bb20 2024-04-23T14:33:07.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc41be00640 1 -- 172.21.15.100:0/1198065446 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc414003600 con 0x7fc42406bb20 2024-04-23T14:33:07.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc42a200640 1 -- 172.21.15.100:0/1198065446 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4241596b0 con 0x7fc42406bb20 2024-04-23T14:33:07.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc41be00640 1 -- 172.21.15.100:0/1198065446 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414003a30 con 0x7fc42406bb20 2024-04-23T14:33:07.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc42a200640 1 -- 172.21.15.100:0/1198065446 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4240cacb0 con 0x7fc42406bb20 2024-04-23T14:33:07.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.342+0000 7fc42a200640 1 -- 172.21.15.100:0/1198065446 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4240caeb0 con 0x7fc42406bb20 2024-04-23T14:33:07.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.346+0000 7fc41be00640 1 -- 172.21.15.100:0/1198065446 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc41401a070 con 0x7fc42406bb20 2024-04-23T14:33:07.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.346+0000 7fc41be00640 1 --2- 172.21.15.100:0/1198065446 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc4000683b0 0x7fc40006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.346+0000 7fc41be00640 1 -- 172.21.15.100:0/1198065446 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc4140861a0 con 0x7fc42406bb20 2024-04-23T14:33:07.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.346+0000 7fc41aa00640 1 --2- 172.21.15.100:0/1198065446 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc4000683b0 0x7fc40006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.346+0000 7fc41aa00640 1 --2- 172.21.15.100:0/1198065446 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc4000683b0 0x7fc40006a870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc424152af0 tx=0x7fc40c006c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:07.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.350+0000 7fc41be00640 1 -- 172.21.15.100:0/1198065446 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc4240caeb0 con 0x7fc42406bb20 2024-04-23T14:33:07.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.506+0000 7fc42a200640 1 -- 172.21.15.100:0/1198065446 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc424068ad0 con 0x7fc42406bb20 2024-04-23T14:33:07.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.506+0000 7fc41be00640 1 -- 172.21.15.100:0/1198065446 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fc414056360 con 0x7fc42406bb20 2024-04-23T14:33:07.508 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:07.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.506+0000 7fc42a200640 1 -- 172.21.15.100:0/1198065446 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc4000683b0 msgr2=0x7fc40006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.506+0000 7fc42a200640 1 --2- 172.21.15.100:0/1198065446 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc4000683b0 0x7fc40006a870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc424152af0 tx=0x7fc40c006c00 comp rx=0 tx=0).stop 2024-04-23T14:33:07.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.506+0000 7fc42a200640 1 -- 172.21.15.100:0/1198065446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc42406bb20 msgr2=0x7fc424099050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.506+0000 7fc42a200640 1 --2- 172.21.15.100:0/1198065446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc42406bb20 0x7fc424099050 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc41400c490 tx=0x7fc414001e80 comp rx=0 tx=0).stop 2024-04-23T14:33:07.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.506+0000 7fc42a200640 1 -- 172.21.15.100:0/1198065446 shutdown_connections 2024-04-23T14:33:07.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.506+0000 7fc42a200640 1 --2- 172.21.15.100:0/1198065446 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc4000683b0 0x7fc40006a870 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.506+0000 7fc42a200640 1 --2- 172.21.15.100:0/1198065446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc42415c7a0 0x7fc424158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.506+0000 7fc42a200640 1 --2- 172.21.15.100:0/1198065446 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc4240cd130 0x7fc4240cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.506+0000 7fc42a200640 1 --2- 172.21.15.100:0/1198065446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc42406bb20 0x7fc424099050 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.506+0000 7fc42a200640 1 -- 172.21.15.100:0/1198065446 >> 172.21.15.100:0/1198065446 conn(0x7fc4240b5200 msgr2=0x7fc4241532e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:07.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.506+0000 7fc42a200640 1 -- 172.21.15.100:0/1198065446 shutdown_connections 2024-04-23T14:33:07.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.510+0000 7fc42a200640 1 -- 172.21.15.100:0/1198065446 wait complete. 2024-04-23T14:33:07.529 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:33:07.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe592a00640 1 Processor -- start 2024-04-23T14:33:07.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe592a00640 1 -- start start 2024-04-23T14:33:07.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe592a00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fe58c06bb20 0x7fe58c0ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe592a00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe58c0cd540 con 0x7fe58c15c7a0 2024-04-23T14:33:07.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe592a00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe58c06cbb0 con 0x7fe58c0cd130 2024-04-23T14:33:07.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe592a00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fe58c06d450 con 0x7fe58c06bb20 2024-04-23T14:33:07.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe58be00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe58c15c7a0 0x7fe58c0af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:59676/0 (socket says 172.21.15.100:59676) 2024-04-23T14:33:07.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe58be00640 1 -- 172.21.15.100:0/3964451391 learned_addr learned my addr 172.21.15.100:0/3964451391 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:07.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe58b400640 1 --2- 172.21.15.100:0/3964451391 >> v2:172.21.15.151:3300/0 conn(0x7fe58c06bb20 0x7fe58c0ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe58aa00640 1 -- 172.21.15.100:0/3964451391 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2688229073 0 0) 0x7fe57c001120 con 0x7fe58c15c7a0 2024-04-23T14:33:07.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe58aa00640 1 -- 172.21.15.100:0/3964451391 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe570003530 con 0x7fe58c15c7a0 2024-04-23T14:33:07.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe58aa00640 1 -- 172.21.15.100:0/3964451391 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2279616828 0 0) 0x7fe570003530 con 0x7fe58c15c7a0 2024-04-23T14:33:07.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe58aa00640 1 -- 172.21.15.100:0/3964451391 >> v2:172.21.15.151:3300/0 conn(0x7fe58c06bb20 msgr2=0x7fe58c0ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:07.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe58aa00640 1 --2- 172.21.15.100:0/3964451391 >> v2:172.21.15.151:3300/0 conn(0x7fe58c06bb20 0x7fe58c0ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe58aa00640 1 -- 172.21.15.100:0/3964451391 >> v1:172.21.15.151:6789/0 conn(0x7fe58c0cd130 legacy=0x7fe58c0ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe58aa00640 1 -- 172.21.15.100:0/3964451391 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58c06ce90 con 0x7fe58c15c7a0 2024-04-23T14:33:07.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe58aa00640 1 -- 172.21.15.100:0/3964451391 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7fe57c001850 con 0x7fe58c15c7a0 2024-04-23T14:33:07.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe58aa00640 1 -- 172.21.15.100:0/3964451391 >> v1:172.21.15.100:6789/0 conn(0x7fe58c15c7a0 legacy=0x7fe58c0af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe58aa00640 1 --2- 172.21.15.100:0/3964451391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5700039c0 0x7fe570003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe58aa00640 1 --2- 172.21.15.100:0/3964451391 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe5700043c0 0x7fe570004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe58aa00640 1 --2- 172.21.15.100:0/3964451391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe570005130 0x7fe57000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe58aa00640 1 -- 172.21.15.100:0/3964451391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c06d450 con 0x7fe570005130 2024-04-23T14:33:07.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe58aa00640 1 -- 172.21.15.100:0/3964451391 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe57000c070 con 0x7fe5700039c0 2024-04-23T14:33:07.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe58aa00640 1 -- 172.21.15.100:0/3964451391 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe57000c270 con 0x7fe5700043c0 2024-04-23T14:33:07.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe58b400640 1 --2- 172.21.15.100:0/3964451391 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe5700043c0 0x7fe570004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe590a00640 1 --2- 172.21.15.100:0/3964451391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe570005130 0x7fe57000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe58be00640 1 --2- 172.21.15.100:0/3964451391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5700039c0 0x7fe570003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe590a00640 1 -- 172.21.15.100:0/3964451391 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe5700043c0 msgr2=0x7fe570004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:07.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe590a00640 1 --2- 172.21.15.100:0/3964451391 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe5700043c0 0x7fe570004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe590a00640 1 -- 172.21.15.100:0/3964451391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5700039c0 msgr2=0x7fe570003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:07.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe590a00640 1 --2- 172.21.15.100:0/3964451391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5700039c0 0x7fe570003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe590a00640 1 -- 172.21.15.100:0/3964451391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe570003530 con 0x7fe570005130 2024-04-23T14:33:07.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.666+0000 7fe58b400640 1 --2- 172.21.15.100:0/3964451391 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe5700043c0 0x7fe570004890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:33:07.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.670+0000 7fe590a00640 1 -- 172.21.15.100:0/3964451391 reap_dead start 2024-04-23T14:33:07.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.670+0000 7fe590a00640 1 --2- 172.21.15.100:0/3964451391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe570005130 0x7fe57000b9d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe584001f60 tx=0x7fe58401c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2bae19f2b8f84c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:07.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.670+0000 7fe58aa00640 1 -- 172.21.15.100:0/3964451391 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5780096b0 con 0x7fe570005130 2024-04-23T14:33:07.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.670+0000 7fe58aa00640 1 -- 172.21.15.100:0/3964451391 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe584003b80 con 0x7fe570005130 2024-04-23T14:33:07.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.670+0000 7fe58aa00640 1 -- 172.21.15.100:0/3964451391 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584001b60 con 0x7fe570005130 2024-04-23T14:33:07.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.670+0000 7fe592a00640 1 -- 172.21.15.100:0/3964451391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe570005130 msgr2=0x7fe57000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.670+0000 7fe592a00640 1 --2- 172.21.15.100:0/3964451391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe570005130 0x7fe57000b9d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe584001f60 tx=0x7fe58401c880 comp rx=0 tx=0).stop 2024-04-23T14:33:07.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.670+0000 7fe592a00640 1 -- 172.21.15.100:0/3964451391 shutdown_connections 2024-04-23T14:33:07.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.670+0000 7fe592a00640 1 --2- 172.21.15.100:0/3964451391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe570005130 0x7fe57000b9d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.670+0000 7fe592a00640 1 -- 172.21.15.100:0/3964451391 >> 172.21.15.100:0/3964451391 conn(0x7fe58c0b5200 msgr2=0x7fe58c0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:07.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.670+0000 7fe592a00640 1 -- 172.21.15.100:0/3964451391 shutdown_connections 2024-04-23T14:33:07.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.670+0000 7fe592a00640 1 -- 172.21.15.100:0/3964451391 wait complete. 2024-04-23T14:33:07.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe592a00640 1 Processor -- start 2024-04-23T14:33:07.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe592a00640 1 -- start start 2024-04-23T14:33:07.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe592a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe570005130 0x7fe58c099ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe592a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe58c0bd220 0x7fe58c150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe592a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe58c0bc680 0x7fe58c0bcb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe592a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c0ccf80 con 0x7fe58c0bd220 2024-04-23T14:33:07.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe592a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c0cce00 con 0x7fe570005130 2024-04-23T14:33:07.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe592a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe58c0af9b0 con 0x7fe58c0bc680 2024-04-23T14:33:07.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe58b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe58c0bd220 0x7fe58c150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe58b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe58c0bd220 0x7fe58c150c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49588/0 (socket says 172.21.15.100:49588) 2024-04-23T14:33:07.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe58b400640 1 -- 172.21.15.100:0/192136904 learned_addr learned my addr 172.21.15.100:0/192136904 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:07.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe58be00640 1 --2- 172.21.15.100:0/192136904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe570005130 0x7fe58c099ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe590a00640 1 --2- 172.21.15.100:0/192136904 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe58c0bc680 0x7fe58c0bcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe58b400640 1 -- 172.21.15.100:0/192136904 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe58c0bc680 msgr2=0x7fe58c0bcb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe58b400640 1 --2- 172.21.15.100:0/192136904 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe58c0bc680 0x7fe58c0bcb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe58b400640 1 -- 172.21.15.100:0/192136904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe570005130 msgr2=0x7fe58c099ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe58b400640 1 --2- 172.21.15.100:0/192136904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe570005130 0x7fe58c099ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe58b400640 1 -- 172.21.15.100:0/192136904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58c0c4520 con 0x7fe58c0bd220 2024-04-23T14:33:07.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe58b400640 1 --2- 172.21.15.100:0/192136904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe58c0bd220 0x7fe58c150c80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe578011990 tx=0x7fe578011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:07.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe583e00640 1 -- 172.21.15.100:0/192136904 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57800fa30 con 0x7fe58c0bd220 2024-04-23T14:33:07.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe583e00640 1 -- 172.21.15.100:0/192136904 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe57800fbd0 con 0x7fe58c0bd220 2024-04-23T14:33:07.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe583e00640 1 -- 172.21.15.100:0/192136904 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe578004020 con 0x7fe58c0bd220 2024-04-23T14:33:07.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe592a00640 1 -- 172.21.15.100:0/192136904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe58c0c3f60 con 0x7fe58c0bd220 2024-04-23T14:33:07.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.674+0000 7fe592a00640 1 -- 172.21.15.100:0/192136904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe58c0c10e0 con 0x7fe58c0bd220 2024-04-23T14:33:07.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.678+0000 7fe592a00640 1 -- 172.21.15.100:0/192136904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe58c068ad0 con 0x7fe58c0bd220 2024-04-23T14:33:07.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.678+0000 7fe583e00640 1 -- 172.21.15.100:0/192136904 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe578012040 con 0x7fe58c0bd220 2024-04-23T14:33:07.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.678+0000 7fe583e00640 1 --2- 172.21.15.100:0/192136904 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe56c0683b0 0x7fe56c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.678+0000 7fe583e00640 1 -- 172.21.15.100:0/192136904 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe578086f60 con 0x7fe58c0bd220 2024-04-23T14:33:07.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.678+0000 7fe58be00640 1 --2- 172.21.15.100:0/192136904 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe56c0683b0 0x7fe56c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.678+0000 7fe58be00640 1 --2- 172.21.15.100:0/192136904 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe56c0683b0 0x7fe56c06a870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe58c1f84c0 tx=0x7fe57c007df0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:07.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.682+0000 7fe583e00640 1 -- 172.21.15.100:0/192136904 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe578019030 con 0x7fe58c0bd220 2024-04-23T14:33:07.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.838+0000 7fe592a00640 1 -- 172.21.15.100:0/192136904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe58c0ca590 con 0x7fe58c0bd220 2024-04-23T14:33:07.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.842+0000 7fe583e00640 1 -- 172.21.15.100:0/192136904 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fe578057100 con 0x7fe58c0bd220 2024-04-23T14:33:07.842 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:33:07.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.842+0000 7fe592a00640 1 -- 172.21.15.100:0/192136904 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe56c0683b0 msgr2=0x7fe56c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.842+0000 7fe592a00640 1 --2- 172.21.15.100:0/192136904 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe56c0683b0 0x7fe56c06a870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe58c1f84c0 tx=0x7fe57c007df0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.842+0000 7fe592a00640 1 -- 172.21.15.100:0/192136904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe58c0bd220 msgr2=0x7fe58c150c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.842+0000 7fe592a00640 1 --2- 172.21.15.100:0/192136904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe58c0bd220 0x7fe58c150c80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe578011990 tx=0x7fe578011d90 comp rx=0 tx=0).stop 2024-04-23T14:33:07.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.842+0000 7fe592a00640 1 -- 172.21.15.100:0/192136904 shutdown_connections 2024-04-23T14:33:07.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.842+0000 7fe592a00640 1 --2- 172.21.15.100:0/192136904 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe56c0683b0 0x7fe56c06a870 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.842+0000 7fe592a00640 1 --2- 172.21.15.100:0/192136904 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe58c0bc680 0x7fe58c0bcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.842+0000 7fe592a00640 1 --2- 172.21.15.100:0/192136904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe58c0bd220 0x7fe58c150c80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.842+0000 7fe592a00640 1 --2- 172.21.15.100:0/192136904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe570005130 0x7fe58c099ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.842+0000 7fe592a00640 1 -- 172.21.15.100:0/192136904 >> 172.21.15.100:0/192136904 conn(0x7fe58c0b5200 msgr2=0x7fe58c0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:07.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.842+0000 7fe592a00640 1 -- 172.21.15.100:0/192136904 shutdown_connections 2024-04-23T14:33:07.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:07.842+0000 7fe592a00640 1 -- 172.21.15.100:0/192136904 wait complete. 2024-04-23T14:33:07.865 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-23T14:33:08.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.002+0000 7f5b63800640 1 Processor -- start 2024-04-23T14:33:08.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.002+0000 7f5b63800640 1 -- start start 2024-04-23T14:33:08.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.002+0000 7f5b63800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5b5c0adb90 0x7f5b5c0adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.002+0000 7f5b63800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b5c06e5f0 con 0x7f5b5c06f950 2024-04-23T14:33:08.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.002+0000 7f5b63800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b5c06e8b0 con 0x7f5b5c15e860 2024-04-23T14:33:08.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.002+0000 7f5b63800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f5b5c06eb90 con 0x7f5b5c0adb90 2024-04-23T14:33:08.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.002+0000 7f5b5be00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5b5c0adb90 0x7f5b5c0adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.002+0000 7f5b5be00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5b5c0adb90 0x7f5b5c0adfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49598/0 (socket says 172.21.15.100:49598) 2024-04-23T14:33:08.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.002+0000 7f5b5be00640 1 -- 172.21.15.100:0/3262551788 learned_addr learned my addr 172.21.15.100:0/3262551788 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:08.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.002+0000 7f5b5be00640 1 -- 172.21.15.100:0/3262551788 >> v1:172.21.15.151:6789/0 conn(0x7f5b5c15e860 legacy=0x7f5b5c0b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.002+0000 7f5b5be00640 1 -- 172.21.15.100:0/3262551788 >> v1:172.21.15.100:6789/0 conn(0x7f5b5c06f950 legacy=0x7f5b5c06d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.002+0000 7f5b5be00640 1 -- 172.21.15.100:0/3262551788 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5c06ee70 con 0x7f5b5c0adb90 2024-04-23T14:33:08.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.006+0000 7f5b5be00640 1 --2- 172.21.15.100:0/3262551788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b5c0adb90 0x7f5b5c0adfe0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009870 tx=0x7f5b4801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae599862bee33400 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:08.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.006+0000 7f5b5b400640 1 -- 172.21.15.100:0/3262551788 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b480a95f0 con 0x7f5b5c0adb90 2024-04-23T14:33:08.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.006+0000 7f5b5b400640 1 -- 172.21.15.100:0/3262551788 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b480a9790 con 0x7f5b5c0adb90 2024-04-23T14:33:08.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.006+0000 7f5b5b400640 1 -- 172.21.15.100:0/3262551788 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b480a9b60 con 0x7f5b5c0adb90 2024-04-23T14:33:08.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.006+0000 7f5b63800640 1 -- 172.21.15.100:0/3262551788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b5c0adb90 msgr2=0x7f5b5c0adfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.006+0000 7f5b63800640 1 --2- 172.21.15.100:0/3262551788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b5c0adb90 0x7f5b5c0adfe0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009870 tx=0x7f5b4801c530 comp rx=0 tx=0).stop 2024-04-23T14:33:08.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.006+0000 7f5b63800640 1 -- 172.21.15.100:0/3262551788 shutdown_connections 2024-04-23T14:33:08.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.006+0000 7f5b63800640 1 --2- 172.21.15.100:0/3262551788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b5c0adb90 0x7f5b5c0adfe0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.006+0000 7f5b63800640 1 -- 172.21.15.100:0/3262551788 >> 172.21.15.100:0/3262551788 conn(0x7f5b5c072590 msgr2=0x7f5b5c0729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:08.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.006+0000 7f5b63800640 1 -- 172.21.15.100:0/3262551788 shutdown_connections 2024-04-23T14:33:08.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.006+0000 7f5b63800640 1 -- 172.21.15.100:0/3262551788 wait complete. 2024-04-23T14:33:08.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.006+0000 7f5b63800640 1 Processor -- start 2024-04-23T14:33:08.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b63800640 1 -- start start 2024-04-23T14:33:08.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b63800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5b5c06f950 0x7f5b5c156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b63800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b5c0adb90 0x7f5b5c13d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b63800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b5c15e860 0x7f5b5c0b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b63800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c06da60 con 0x7f5b5c15e860 2024-04-23T14:33:08.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b63800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c15ec40 con 0x7f5b5c0adb90 2024-04-23T14:33:08.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b63800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5b5c06dbe0 con 0x7f5b5c06f950 2024-04-23T14:33:08.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b61800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b5c15e860 0x7f5b5c0b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b61800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b5c15e860 0x7f5b5c0b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49614/0 (socket says 172.21.15.100:49614) 2024-04-23T14:33:08.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b61800640 1 -- 172.21.15.100:0/4090696858 learned_addr learned my addr 172.21.15.100:0/4090696858 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:08.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b5be00640 1 --2- 172.21.15.100:0/4090696858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b5c0adb90 0x7f5b5c13d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b60e00640 1 --2- 172.21.15.100:0/4090696858 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5b5c06f950 0x7f5b5c156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b61800640 1 -- 172.21.15.100:0/4090696858 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5b5c06f950 msgr2=0x7f5b5c156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b61800640 1 --2- 172.21.15.100:0/4090696858 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5b5c06f950 0x7f5b5c156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b61800640 1 -- 172.21.15.100:0/4090696858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b5c0adb90 msgr2=0x7f5b5c13d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b61800640 1 --2- 172.21.15.100:0/4090696858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b5c0adb90 0x7f5b5c13d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b61800640 1 -- 172.21.15.100:0/4090696858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5c13d9b0 con 0x7f5b5c15e860 2024-04-23T14:33:08.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b5be00640 1 --2- 172.21.15.100:0/4090696858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b5c0adb90 0x7f5b5c13d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:08.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b61800640 1 --2- 172.21.15.100:0/4090696858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b5c15e860 0x7f5b5c0b4410 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400c460 tx=0x7f5b5400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:08.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b58c00640 1 -- 172.21.15.100:0/4090696858 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b54005220 con 0x7f5b5c15e860 2024-04-23T14:33:08.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b58c00640 1 -- 172.21.15.100:0/4090696858 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b540053c0 con 0x7f5b5c15e860 2024-04-23T14:33:08.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b63800640 1 -- 172.21.15.100:0/4090696858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b5c1531a0 con 0x7f5b5c15e860 2024-04-23T14:33:08.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b58c00640 1 -- 172.21.15.100:0/4090696858 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b54007270 con 0x7f5b5c15e860 2024-04-23T14:33:08.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b63800640 1 -- 172.21.15.100:0/4090696858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b5c153760 con 0x7f5b5c15e860 2024-04-23T14:33:08.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.010+0000 7f5b63800640 1 -- 172.21.15.100:0/4090696858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b5c068ad0 con 0x7f5b5c15e860 2024-04-23T14:33:08.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.014+0000 7f5b58c00640 1 -- 172.21.15.100:0/4090696858 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5b54019020 con 0x7f5b5c15e860 2024-04-23T14:33:08.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.014+0000 7f5b58c00640 1 --2- 172.21.15.100:0/4090696858 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5b3c068380 0x7f5b3c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.014+0000 7f5b58c00640 1 -- 172.21.15.100:0/4090696858 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5b540861f0 con 0x7f5b5c15e860 2024-04-23T14:33:08.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.014+0000 7f5b60e00640 1 --2- 172.21.15.100:0/4090696858 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5b3c068380 0x7f5b3c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.014+0000 7f5b60e00640 1 --2- 172.21.15.100:0/4090696858 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5b3c068380 0x7f5b3c06a840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c14c730 tx=0x7f5b4c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:08.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.018+0000 7f5b58c00640 1 -- 172.21.15.100:0/4090696858 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5b54056310 con 0x7f5b5c15e860 2024-04-23T14:33:08.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.174+0000 7f5b63800640 1 -- 172.21.15.100:0/4090696858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5b5c1621e0 con 0x7f5b5c15e860 2024-04-23T14:33:08.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.174+0000 7f5b58c00640 1 -- 172.21.15.100:0/4090696858 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f5b54055c10 con 0x7f5b5c15e860 2024-04-23T14:33:08.178 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:08.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.178+0000 7f5b63800640 1 -- 172.21.15.100:0/4090696858 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5b3c068380 msgr2=0x7f5b3c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.178+0000 7f5b63800640 1 --2- 172.21.15.100:0/4090696858 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5b3c068380 0x7f5b3c06a840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c14c730 tx=0x7f5b4c009040 comp rx=0 tx=0).stop 2024-04-23T14:33:08.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.178+0000 7f5b63800640 1 -- 172.21.15.100:0/4090696858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b5c15e860 msgr2=0x7f5b5c0b4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.178+0000 7f5b63800640 1 --2- 172.21.15.100:0/4090696858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b5c15e860 0x7f5b5c0b4410 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400c460 tx=0x7f5b5400c920 comp rx=0 tx=0).stop 2024-04-23T14:33:08.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.178+0000 7f5b63800640 1 -- 172.21.15.100:0/4090696858 shutdown_connections 2024-04-23T14:33:08.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.178+0000 7f5b63800640 1 --2- 172.21.15.100:0/4090696858 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5b3c068380 0x7f5b3c06a840 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.178+0000 7f5b63800640 1 --2- 172.21.15.100:0/4090696858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b5c15e860 0x7f5b5c0b4410 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.178+0000 7f5b63800640 1 --2- 172.21.15.100:0/4090696858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b5c0adb90 0x7f5b5c13d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.178+0000 7f5b63800640 1 --2- 172.21.15.100:0/4090696858 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5b5c06f950 0x7f5b5c156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.178+0000 7f5b63800640 1 -- 172.21.15.100:0/4090696858 >> 172.21.15.100:0/4090696858 conn(0x7f5b5c072590 msgr2=0x7f5b5c15cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:08.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.178+0000 7f5b63800640 1 -- 172.21.15.100:0/4090696858 shutdown_connections 2024-04-23T14:33:08.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.178+0000 7f5b63800640 1 -- 172.21.15.100:0/4090696858 wait complete. 2024-04-23T14:33:08.200 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:33:08.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.334+0000 7f3a04600640 1 Processor -- start 2024-04-23T14:33:08.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.334+0000 7f3a04600640 1 -- start start 2024-04-23T14:33:08.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.334+0000 7f3a04600640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f39fc15b660 0x7f39fc0c6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.334+0000 7f3a04600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f39fc0c6d60 0x7f39fc0d0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.334+0000 7f3a04600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39fc0cee00 con 0x7f39fc0d06d0 2024-04-23T14:33:08.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.334+0000 7f3a04600640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f39fc0d0b60 con 0x7f39fc0c6d60 2024-04-23T14:33:08.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.334+0000 7f3a04600640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f39fc15ba40 con 0x7f39fc15b660 2024-04-23T14:33:08.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.334+0000 7f3a01200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f39fc0c6d60 0x7f39fc0d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.334+0000 7f3a01200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f39fc0c6d60 0x7f39fc0d0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49620/0 (socket says 172.21.15.100:49620) 2024-04-23T14:33:08.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.334+0000 7f3a01200640 1 -- 172.21.15.100:0/85365105 learned_addr learned my addr 172.21.15.100:0/85365105 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:08.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.334+0000 7f3a01c00640 1 --2- 172.21.15.100:0/85365105 >> v2:172.21.15.151:3300/0 conn(0x7f39fc15b660 0x7f39fc0c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.334+0000 7f3a01200640 1 -- 172.21.15.100:0/85365105 >> v2:172.21.15.151:3300/0 conn(0x7f39fc15b660 msgr2=0x7f39fc0c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:08.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.334+0000 7f3a01200640 1 --2- 172.21.15.100:0/85365105 >> v2:172.21.15.151:3300/0 conn(0x7f39fc15b660 0x7f39fc0c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.334+0000 7f3a01200640 1 -- 172.21.15.100:0/85365105 >> v1:172.21.15.151:6789/0 conn(0x7f39fc0d06d0 legacy=0x7f39fc15f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.334+0000 7f3a01200640 1 -- 172.21.15.100:0/85365105 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39fc06bc50 con 0x7f39fc0c6d60 2024-04-23T14:33:08.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.334+0000 7f3a01c00640 1 --2- 172.21.15.100:0/85365105 >> v2:172.21.15.151:3300/0 conn(0x7f39fc15b660 0x7f39fc0c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:08.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.338+0000 7f3a01200640 1 --2- 172.21.15.100:0/85365105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39fc0c6d60 0x7f39fc0d0190 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0b6080 tx=0x7f39e801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540fb6bd570f281a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:08.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.338+0000 7f39fbe00640 1 -- 172.21.15.100:0/85365105 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e80aa650 con 0x7f39fc0c6d60 2024-04-23T14:33:08.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.338+0000 7f39fbe00640 1 -- 172.21.15.100:0/85365105 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39e80af070 con 0x7f39fc0c6d60 2024-04-23T14:33:08.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.338+0000 7f39fbe00640 1 -- 172.21.15.100:0/85365105 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e80aaa10 con 0x7f39fc0c6d60 2024-04-23T14:33:08.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.338+0000 7f3a04600640 1 -- 172.21.15.100:0/85365105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39fc0c6d60 msgr2=0x7f39fc0d0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.338+0000 7f3a04600640 1 --2- 172.21.15.100:0/85365105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39fc0c6d60 0x7f39fc0d0190 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0b6080 tx=0x7f39e801c4f0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.338+0000 7f3a04600640 1 -- 172.21.15.100:0/85365105 shutdown_connections 2024-04-23T14:33:08.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.338+0000 7f3a04600640 1 --2- 172.21.15.100:0/85365105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39fc0c6d60 0x7f39fc0d0190 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.338+0000 7f3a04600640 1 --2- 172.21.15.100:0/85365105 >> v2:172.21.15.151:3300/0 conn(0x7f39fc15b660 0x7f39fc0c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.338+0000 7f3a04600640 1 -- 172.21.15.100:0/85365105 >> 172.21.15.100:0/85365105 conn(0x7f39fc0b5310 msgr2=0x7f39fc0b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:08.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.338+0000 7f3a04600640 1 -- 172.21.15.100:0/85365105 shutdown_connections 2024-04-23T14:33:08.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.338+0000 7f3a04600640 1 -- 172.21.15.100:0/85365105 wait complete. 2024-04-23T14:33:08.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f3a04600640 1 Processor -- start 2024-04-23T14:33:08.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f3a04600640 1 -- start start 2024-04-23T14:33:08.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f3a04600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39fc0c6d60 0x7f39fc06e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f3a04600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f39fc0d06d0 0x7f39fc1657a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f3a04600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39fc15b660 0x7f39fc159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f3a04600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc168410 con 0x7f39fc15b660 2024-04-23T14:33:08.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f3a04600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc06c7d0 con 0x7f39fc0c6d60 2024-04-23T14:33:08.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f3a04600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f39fc168590 con 0x7f39fc0d06d0 2024-04-23T14:33:08.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f3a02600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39fc15b660 0x7f39fc159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f3a02600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39fc15b660 0x7f39fc159930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49630/0 (socket says 172.21.15.100:49630) 2024-04-23T14:33:08.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f3a02600640 1 -- 172.21.15.100:0/191691553 learned_addr learned my addr 172.21.15.100:0/191691553 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:08.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f3a01200640 1 --2- 172.21.15.100:0/191691553 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f39fc0d06d0 0x7f39fc1657a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f3a01c00640 1 --2- 172.21.15.100:0/191691553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39fc0c6d60 0x7f39fc06e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f3a02600640 1 -- 172.21.15.100:0/191691553 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f39fc0d06d0 msgr2=0x7f39fc1657a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f3a02600640 1 --2- 172.21.15.100:0/191691553 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f39fc0d06d0 0x7f39fc1657a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f3a02600640 1 -- 172.21.15.100:0/191691553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39fc0c6d60 msgr2=0x7f39fc06e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f3a02600640 1 --2- 172.21.15.100:0/191691553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39fc0c6d60 0x7f39fc06e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f3a02600640 1 -- 172.21.15.100:0/191691553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39fc06e710 con 0x7f39fc15b660 2024-04-23T14:33:08.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f3a02600640 1 --2- 172.21.15.100:0/191691553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39fc15b660 0x7f39fc159930 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f39f400c460 tx=0x7f39f400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:08.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f39f9600640 1 -- 172.21.15.100:0/191691553 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4016020 con 0x7f39fc15b660 2024-04-23T14:33:08.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f39f9600640 1 -- 172.21.15.100:0/191691553 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39f4005220 con 0x7f39fc15b660 2024-04-23T14:33:08.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f39f9600640 1 -- 172.21.15.100:0/191691553 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4007030 con 0x7f39fc15b660 2024-04-23T14:33:08.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f3a04600640 1 -- 172.21.15.100:0/191691553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39fc1448b0 con 0x7f39fc15b660 2024-04-23T14:33:08.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.342+0000 7f3a04600640 1 -- 172.21.15.100:0/191691553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39fc144e30 con 0x7f39fc15b660 2024-04-23T14:33:08.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.346+0000 7f3a04600640 1 -- 172.21.15.100:0/191691553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39fc1457d0 con 0x7f39fc15b660 2024-04-23T14:33:08.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.346+0000 7f39f9600640 1 -- 172.21.15.100:0/191691553 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f39f40071d0 con 0x7f39fc15b660 2024-04-23T14:33:08.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.346+0000 7f39f9600640 1 --2- 172.21.15.100:0/191691553 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f39dc068380 0x7f39dc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.346+0000 7f39f9600640 1 -- 172.21.15.100:0/191691553 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f39f40866e0 con 0x7f39fc15b660 2024-04-23T14:33:08.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.346+0000 7f3a01c00640 1 --2- 172.21.15.100:0/191691553 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f39dc068380 0x7f39dc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.346+0000 7f3a01c00640 1 --2- 172.21.15.100:0/191691553 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f39dc068380 0x7f39dc06a840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f39f0004430 tx=0x7f39f0003a00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:08.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.350+0000 7f39f9600640 1 -- 172.21.15.100:0/191691553 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f39fc1457d0 con 0x7f39fc15b660 2024-04-23T14:33:08.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.522+0000 7f3a04600640 1 -- 172.21.15.100:0/191691553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f39fc06d3a0 con 0x7f39fc15b660 2024-04-23T14:33:08.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.522+0000 7f39f9600640 1 -- 172.21.15.100:0/191691553 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f39f4020020 con 0x7f39fc15b660 2024-04-23T14:33:08.523 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-04-23T14:33:08.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.522+0000 7f3a04600640 1 -- 172.21.15.100:0/191691553 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f39dc068380 msgr2=0x7f39dc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.522+0000 7f3a04600640 1 --2- 172.21.15.100:0/191691553 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f39dc068380 0x7f39dc06a840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f39f0004430 tx=0x7f39f0003a00 comp rx=0 tx=0).stop 2024-04-23T14:33:08.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.522+0000 7f3a04600640 1 -- 172.21.15.100:0/191691553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39fc15b660 msgr2=0x7f39fc159930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.522+0000 7f3a04600640 1 --2- 172.21.15.100:0/191691553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39fc15b660 0x7f39fc159930 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f39f400c460 tx=0x7f39f400c920 comp rx=0 tx=0).stop 2024-04-23T14:33:08.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.522+0000 7f3a04600640 1 -- 172.21.15.100:0/191691553 shutdown_connections 2024-04-23T14:33:08.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.522+0000 7f3a04600640 1 --2- 172.21.15.100:0/191691553 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f39dc068380 0x7f39dc06a840 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.522+0000 7f3a04600640 1 --2- 172.21.15.100:0/191691553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39fc15b660 0x7f39fc159930 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.522+0000 7f3a04600640 1 --2- 172.21.15.100:0/191691553 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f39fc0d06d0 0x7f39fc1657a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.522+0000 7f3a04600640 1 --2- 172.21.15.100:0/191691553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39fc0c6d60 0x7f39fc06e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.522+0000 7f3a04600640 1 -- 172.21.15.100:0/191691553 >> 172.21.15.100:0/191691553 conn(0x7f39fc0b5310 msgr2=0x7f39fc15f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:08.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.522+0000 7f3a04600640 1 -- 172.21.15.100:0/191691553 shutdown_connections 2024-04-23T14:33:08.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.522+0000 7f3a04600640 1 -- 172.21.15.100:0/191691553 wait complete. 2024-04-23T14:33:08.543 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:33:08.543 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:08.552 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:08.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.726+0000 7fc702000640 1 Processor -- start 2024-04-23T14:33:08.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.726+0000 7fc702000640 1 -- start start 2024-04-23T14:33:08.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.726+0000 7fc702000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc6fc0ad870 0x7fc6fc0b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.726+0000 7fc702000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6fc160320 con 0x7fc6fc15c470 2024-04-23T14:33:08.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.726+0000 7fc702000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6fc0cc7e0 con 0x7fc6fc160de0 2024-04-23T14:33:08.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.726+0000 7fc702000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc6fc1605e0 con 0x7fc6fc0ad870 2024-04-23T14:33:08.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.726+0000 7fc6faa00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc6fc0ad870 0x7fc6fc0b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.726+0000 7fc6fb400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fc6fc15c470 0x7fc6fc0ad160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:36240/0 (socket says 172.21.15.100:36240) 2024-04-23T14:33:08.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.726+0000 7fc6fb400640 1 -- 172.21.15.100:0/793840694 learned_addr learned my addr 172.21.15.100:0/793840694 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:08.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.726+0000 7fc6faa00640 1 -- 172.21.15.100:0/793840694 >> v1:172.21.15.151:6789/0 conn(0x7fc6fc160de0 legacy=0x7fc6fc06bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.726+0000 7fc6faa00640 1 -- 172.21.15.100:0/793840694 >> v1:172.21.15.100:6789/0 conn(0x7fc6fc15c470 legacy=0x7fc6fc0ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.726+0000 7fc6faa00640 1 -- 172.21.15.100:0/793840694 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6fc1608c0 con 0x7fc6fc0ad870 2024-04-23T14:33:08.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.726+0000 7fc6fa000640 1 -- 172.21.15.100:0/793840694 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2780084082 0 0) 0x7fc6fc160320 con 0x7fc6fc15c470 2024-04-23T14:33:08.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.730+0000 7fc6faa00640 1 --2- 172.21.15.100:0/793840694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6fc0ad870 0x7fc6fc0b29a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80097c0 tx=0x7fc6e801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d387e85bb13324d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:08.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.730+0000 7fc6fa000640 1 -- 172.21.15.100:0/793840694 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e80b0070 con 0x7fc6fc0ad870 2024-04-23T14:33:08.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.730+0000 7fc6fa000640 1 -- 172.21.15.100:0/793840694 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e80ab600 con 0x7fc6fc0ad870 2024-04-23T14:33:08.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.730+0000 7fc6fa000640 1 -- 172.21.15.100:0/793840694 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e80ab9d0 con 0x7fc6fc0ad870 2024-04-23T14:33:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.730+0000 7fc702000640 1 -- 172.21.15.100:0/793840694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6fc0ad870 msgr2=0x7fc6fc0b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.730+0000 7fc702000640 1 --2- 172.21.15.100:0/793840694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6fc0ad870 0x7fc6fc0b29a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80097c0 tx=0x7fc6e801c410 comp rx=0 tx=0).stop 2024-04-23T14:33:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.730+0000 7fc702000640 1 -- 172.21.15.100:0/793840694 shutdown_connections 2024-04-23T14:33:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.730+0000 7fc702000640 1 --2- 172.21.15.100:0/793840694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6fc0ad870 0x7fc6fc0b29a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.730+0000 7fc702000640 1 -- 172.21.15.100:0/793840694 >> 172.21.15.100:0/793840694 conn(0x7fc6fc0b1e80 msgr2=0x7fc6fc0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.730+0000 7fc702000640 1 -- 172.21.15.100:0/793840694 shutdown_connections 2024-04-23T14:33:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.730+0000 7fc702000640 1 -- 172.21.15.100:0/793840694 wait complete. 2024-04-23T14:33:08.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc702000640 1 Processor -- start 2024-04-23T14:33:08.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc702000640 1 -- start start 2024-04-23T14:33:08.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc702000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc6fc0ad870 0x7fc6fc153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc702000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6fc15c470 0x7fc6fc0998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc702000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6fc160de0 0x7fc6fc072070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc702000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc6fc15ca60 con 0x7fc6fc15c470 2024-04-23T14:33:08.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc702000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc6fc0cc5c0 con 0x7fc6fc160de0 2024-04-23T14:33:08.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc702000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc6fc16a550 con 0x7fc6fc0ad870 2024-04-23T14:33:08.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc6faa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6fc15c470 0x7fc6fc0998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc6faa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6fc15c470 0x7fc6fc0998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60668/0 (socket says 172.21.15.100:60668) 2024-04-23T14:33:08.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc6faa00640 1 -- 172.21.15.100:0/1729466430 learned_addr learned my addr 172.21.15.100:0/1729466430 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:08.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc6fbe00640 1 --2- 172.21.15.100:0/1729466430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6fc160de0 0x7fc6fc072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc6fb400640 1 --2- 172.21.15.100:0/1729466430 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc6fc0ad870 0x7fc6fc153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc6faa00640 1 -- 172.21.15.100:0/1729466430 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc6fc0ad870 msgr2=0x7fc6fc153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc6faa00640 1 --2- 172.21.15.100:0/1729466430 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc6fc0ad870 0x7fc6fc153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc6faa00640 1 -- 172.21.15.100:0/1729466430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6fc160de0 msgr2=0x7fc6fc072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc6faa00640 1 --2- 172.21.15.100:0/1729466430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6fc160de0 0x7fc6fc072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc6faa00640 1 -- 172.21.15.100:0/1729466430 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6fc145340 con 0x7fc6fc15c470 2024-04-23T14:33:08.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc6faa00640 1 --2- 172.21.15.100:0/1729466430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6fc15c470 0x7fc6fc0998c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80ae040 tx=0x7fc6e801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:08.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc6f3400640 1 -- 172.21.15.100:0/1729466430 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e80b0050 con 0x7fc6fc15c470 2024-04-23T14:33:08.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc6f3400640 1 -- 172.21.15.100:0/1729466430 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e80bc070 con 0x7fc6fc15c470 2024-04-23T14:33:08.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc702000640 1 -- 172.21.15.100:0/1729466430 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6fc153fc0 con 0x7fc6fc15c470 2024-04-23T14:33:08.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc6f3400640 1 -- 172.21.15.100:0/1729466430 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e80b5830 con 0x7fc6fc15c470 2024-04-23T14:33:08.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc702000640 1 -- 172.21.15.100:0/1729466430 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6fc14bcb0 con 0x7fc6fc15c470 2024-04-23T14:33:08.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.734+0000 7fc702000640 1 -- 172.21.15.100:0/1729466430 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6fc068ad0 con 0x7fc6fc15c470 2024-04-23T14:33:08.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.738+0000 7fc6f3400640 1 -- 172.21.15.100:0/1729466430 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc6e80b59d0 con 0x7fc6fc15c470 2024-04-23T14:33:08.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.738+0000 7fc6f3400640 1 --2- 172.21.15.100:0/1729466430 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc6dc0683b0 0x7fc6dc06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.738+0000 7fc6f3400640 1 -- 172.21.15.100:0/1729466430 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc6e8121410 con 0x7fc6fc15c470 2024-04-23T14:33:08.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.738+0000 7fc6fb400640 1 --2- 172.21.15.100:0/1729466430 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc6dc0683b0 0x7fc6dc06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.738+0000 7fc6fb400640 1 --2- 172.21.15.100:0/1729466430 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc6dc0683b0 0x7fc6dc06a870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec001950 tx=0x7fc6ec0018e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:08.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.742+0000 7fc6f3400640 1 -- 172.21.15.100:0/1729466430 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc6e8016090 con 0x7fc6fc15c470 2024-04-23T14:33:08.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.902+0000 7fc702000640 1 -- 172.21.15.100:0/1729466430 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6fc0b5460 con 0x7fc6fc15c470 2024-04-23T14:33:08.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.902+0000 7fc6f3400640 1 -- 172.21.15.100:0/1729466430 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18859 (secure 0 0 0) 0x7fc6e80f1530 con 0x7fc6fc15c470 2024-04-23T14:33:08.905 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:08.905 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:49.784608+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:08.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.906+0000 7fc702000640 1 -- 172.21.15.100:0/1729466430 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc6dc0683b0 msgr2=0x7fc6dc06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.906+0000 7fc702000640 1 --2- 172.21.15.100:0/1729466430 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc6dc0683b0 0x7fc6dc06a870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec001950 tx=0x7fc6ec0018e0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.906+0000 7fc702000640 1 -- 172.21.15.100:0/1729466430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6fc15c470 msgr2=0x7fc6fc0998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.906+0000 7fc702000640 1 --2- 172.21.15.100:0/1729466430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6fc15c470 0x7fc6fc0998c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80ae040 tx=0x7fc6e801ac50 comp rx=0 tx=0).stop 2024-04-23T14:33:08.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.906+0000 7fc702000640 1 -- 172.21.15.100:0/1729466430 shutdown_connections 2024-04-23T14:33:08.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.906+0000 7fc702000640 1 --2- 172.21.15.100:0/1729466430 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc6dc0683b0 0x7fc6dc06a870 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.906+0000 7fc702000640 1 --2- 172.21.15.100:0/1729466430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6fc160de0 0x7fc6fc072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.906+0000 7fc702000640 1 --2- 172.21.15.100:0/1729466430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6fc15c470 0x7fc6fc0998c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.906+0000 7fc702000640 1 --2- 172.21.15.100:0/1729466430 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc6fc0ad870 0x7fc6fc153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.906+0000 7fc702000640 1 -- 172.21.15.100:0/1729466430 >> 172.21.15.100:0/1729466430 conn(0x7fc6fc0b1e80 msgr2=0x7fc6fc155750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:08.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.906+0000 7fc702000640 1 -- 172.21.15.100:0/1729466430 shutdown_connections 2024-04-23T14:33:08.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:08.906+0000 7fc702000640 1 -- 172.21.15.100:0/1729466430 wait complete. 2024-04-23T14:33:08.926 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:32:06.835560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:32:07.930489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-23T14:32:11.636814+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-23T14:33:08.926 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:33:09.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.062+0000 7f9b53200640 1 Processor -- start 2024-04-23T14:33:09.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.062+0000 7f9b53200640 1 -- start start 2024-04-23T14:33:09.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.062+0000 7f9b53200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f9b4c0730b0 0x7f9b4c0ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.062+0000 7f9b53200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9b4c06d990 0x7f9b4c0cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.062+0000 7f9b53200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b4c06fe90 con 0x7f9b4c06d5b0 2024-04-23T14:33:09.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.062+0000 7f9b53200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f9b4c0703e0 con 0x7f9b4c0730b0 2024-04-23T14:33:09.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.062+0000 7f9b53200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9b4c06f610 con 0x7f9b4c06d990 2024-04-23T14:33:09.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.062+0000 7f9b51200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9b4c06d990 0x7f9b4c0cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.066+0000 7f9b4b400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f9b4c0730b0 0x7f9b4c0ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.066+0000 7f9b51200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9b4c06d990 0x7f9b4c0cd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:58856/0 (socket says 172.21.15.100:58856) 2024-04-23T14:33:09.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.066+0000 7f9b51200640 1 -- 172.21.15.100:0/2459572758 learned_addr learned my addr 172.21.15.100:0/2459572758 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:09.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.066+0000 7f9b4be00640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f9b4c06d5b0 0x7f9b4c0cdf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:58262/0 (socket says 172.21.15.100:58262) 2024-04-23T14:33:09.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.066+0000 7f9b4b400640 1 -- 172.21.15.100:0/2459572758 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9b4c06d990 msgr2=0x7f9b4c0cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:09.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.066+0000 7f9b4b400640 1 --2- 172.21.15.100:0/2459572758 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9b4c06d990 0x7f9b4c0cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.066+0000 7f9b4b400640 1 -- 172.21.15.100:0/2459572758 >> v1:172.21.15.151:6789/0 conn(0x7f9b4c06d5b0 legacy=0x7f9b4c0cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.066+0000 7f9b4b400640 1 -- 172.21.15.100:0/2459572758 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4c0afff0 con 0x7f9b4c0730b0 2024-04-23T14:33:09.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.066+0000 7f9b51200640 1 --2- 172.21.15.100:0/2459572758 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9b4c06d990 0x7f9b4c0cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:33:09.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.066+0000 7f9b4b400640 1 --2- 172.21.15.100:0/2459572758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b4c0730b0 0x7f9b4c0ccca0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9b380097c0 tx=0x7f9b3801cd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=288eecf95c9497e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:09.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.066+0000 7f9b4aa00640 1 -- 172.21.15.100:0/2459572758 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b380b0070 con 0x7f9b4c0730b0 2024-04-23T14:33:09.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.066+0000 7f9b4aa00640 1 -- 172.21.15.100:0/2459572758 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b380ab600 con 0x7f9b4c0730b0 2024-04-23T14:33:09.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.066+0000 7f9b4aa00640 1 -- 172.21.15.100:0/2459572758 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b380ab9d0 con 0x7f9b4c0730b0 2024-04-23T14:33:09.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.070+0000 7f9b53200640 1 -- 172.21.15.100:0/2459572758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b4c0730b0 msgr2=0x7f9b4c0ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.070+0000 7f9b53200640 1 --2- 172.21.15.100:0/2459572758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b4c0730b0 0x7f9b4c0ccca0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9b380097c0 tx=0x7f9b3801cd90 comp rx=0 tx=0).stop 2024-04-23T14:33:09.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.070+0000 7f9b53200640 1 -- 172.21.15.100:0/2459572758 shutdown_connections 2024-04-23T14:33:09.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.070+0000 7f9b53200640 1 --2- 172.21.15.100:0/2459572758 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9b4c06d990 0x7f9b4c0cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.070+0000 7f9b53200640 1 --2- 172.21.15.100:0/2459572758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b4c0730b0 0x7f9b4c0ccca0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.070+0000 7f9b53200640 1 -- 172.21.15.100:0/2459572758 >> 172.21.15.100:0/2459572758 conn(0x7f9b4c072590 msgr2=0x7f9b4c0729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:09.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.070+0000 7f9b53200640 1 -- 172.21.15.100:0/2459572758 shutdown_connections 2024-04-23T14:33:09.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.070+0000 7f9b53200640 1 -- 172.21.15.100:0/2459572758 wait complete. 2024-04-23T14:33:09.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.070+0000 7f9b53200640 1 Processor -- start 2024-04-23T14:33:09.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.070+0000 7f9b53200640 1 -- start start 2024-04-23T14:33:09.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.070+0000 7f9b53200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9b4c06d5b0 0x7f9b4c0c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.070+0000 7f9b53200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b4c06d990 0x7f9b4c0c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.070+0000 7f9b53200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b4c0730b0 0x7f9b4c1571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.070+0000 7f9b53200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c0cdcb0 con 0x7f9b4c06d990 2024-04-23T14:33:09.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.070+0000 7f9b53200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c0cdb30 con 0x7f9b4c0730b0 2024-04-23T14:33:09.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.070+0000 7f9b53200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9b4c169640 con 0x7f9b4c06d5b0 2024-04-23T14:33:09.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.070+0000 7f9b4b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b4c06d990 0x7f9b4c0c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.070+0000 7f9b4b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b4c06d990 0x7f9b4c0c64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60670/0 (socket says 172.21.15.100:60670) 2024-04-23T14:33:09.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.070+0000 7f9b4b400640 1 -- 172.21.15.100:0/3789276071 learned_addr learned my addr 172.21.15.100:0/3789276071 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:09.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.070+0000 7f9b51200640 1 --2- 172.21.15.100:0/3789276071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b4c0730b0 0x7f9b4c1571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.070+0000 7f9b4be00640 1 --2- 172.21.15.100:0/3789276071 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9b4c06d5b0 0x7f9b4c0c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.074+0000 7f9b4b400640 1 -- 172.21.15.100:0/3789276071 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9b4c06d5b0 msgr2=0x7f9b4c0c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.074+0000 7f9b4b400640 1 --2- 172.21.15.100:0/3789276071 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9b4c06d5b0 0x7f9b4c0c5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.074+0000 7f9b4b400640 1 -- 172.21.15.100:0/3789276071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b4c0730b0 msgr2=0x7f9b4c1571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.074+0000 7f9b4b400640 1 --2- 172.21.15.100:0/3789276071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b4c0730b0 0x7f9b4c1571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.074+0000 7f9b4b400640 1 -- 172.21.15.100:0/3789276071 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4c0b15a0 con 0x7f9b4c06d990 2024-04-23T14:33:09.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.074+0000 7f9b4b400640 1 --2- 172.21.15.100:0/3789276071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b4c06d990 0x7f9b4c0c64b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9b3801a9e0 tx=0x7f9b380190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:09.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.074+0000 7f9b43e00640 1 -- 172.21.15.100:0/3789276071 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b380b0040 con 0x7f9b4c06d990 2024-04-23T14:33:09.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.074+0000 7f9b43e00640 1 -- 172.21.15.100:0/3789276071 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b380bd070 con 0x7f9b4c06d990 2024-04-23T14:33:09.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.074+0000 7f9b43e00640 1 -- 172.21.15.100:0/3789276071 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b380b56c0 con 0x7f9b4c06d990 2024-04-23T14:33:09.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.074+0000 7f9b53200640 1 -- 172.21.15.100:0/3789276071 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4c1413e0 con 0x7f9b4c06d990 2024-04-23T14:33:09.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.074+0000 7f9b53200640 1 -- 172.21.15.100:0/3789276071 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b4c1419a0 con 0x7f9b4c06d990 2024-04-23T14:33:09.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.074+0000 7f9b53200640 1 -- 172.21.15.100:0/3789276071 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b4c068ad0 con 0x7f9b4c06d990 2024-04-23T14:33:09.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.074+0000 7f9b43e00640 1 -- 172.21.15.100:0/3789276071 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9b380b3070 con 0x7f9b4c06d990 2024-04-23T14:33:09.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.074+0000 7f9b43e00640 1 --2- 172.21.15.100:0/3789276071 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9b2c068380 0x7f9b2c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.074+0000 7f9b43e00640 1 -- 172.21.15.100:0/3789276071 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f9b38120ff0 con 0x7f9b4c06d990 2024-04-23T14:33:09.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.074+0000 7f9b4be00640 1 --2- 172.21.15.100:0/3789276071 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9b2c068380 0x7f9b2c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.078+0000 7f9b4be00640 1 --2- 172.21.15.100:0/3789276071 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9b2c068380 0x7f9b2c06a840 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c0012f0 tx=0x7f9b3c005b80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:09.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.078+0000 7f9b43e00640 1 -- 172.21.15.100:0/3789276071 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9b38015fb0 con 0x7f9b4c06d990 2024-04-23T14:33:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.234+0000 7f9b53200640 1 -- 172.21.15.100:0/3789276071 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9b4c073840 con 0x7f9b4c06d990 2024-04-23T14:33:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.234+0000 7f9b43e00640 1 -- 172.21.15.100:0/3789276071 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f9b380f10c0 con 0x7f9b4c06d990 2024-04-23T14:33:09.238 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:33:09.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.238+0000 7f9b53200640 1 -- 172.21.15.100:0/3789276071 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9b2c068380 msgr2=0x7f9b2c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.238+0000 7f9b53200640 1 --2- 172.21.15.100:0/3789276071 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9b2c068380 0x7f9b2c06a840 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c0012f0 tx=0x7f9b3c005b80 comp rx=0 tx=0).stop 2024-04-23T14:33:09.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.238+0000 7f9b53200640 1 -- 172.21.15.100:0/3789276071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b4c06d990 msgr2=0x7f9b4c0c64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.238+0000 7f9b53200640 1 --2- 172.21.15.100:0/3789276071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b4c06d990 0x7f9b4c0c64b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9b3801a9e0 tx=0x7f9b380190d0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.238+0000 7f9b53200640 1 -- 172.21.15.100:0/3789276071 shutdown_connections 2024-04-23T14:33:09.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.238+0000 7f9b53200640 1 --2- 172.21.15.100:0/3789276071 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9b2c068380 0x7f9b2c06a840 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.238+0000 7f9b53200640 1 --2- 172.21.15.100:0/3789276071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b4c0730b0 0x7f9b4c1571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.238+0000 7f9b53200640 1 --2- 172.21.15.100:0/3789276071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b4c06d990 0x7f9b4c0c64b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.238+0000 7f9b53200640 1 --2- 172.21.15.100:0/3789276071 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9b4c06d5b0 0x7f9b4c0c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.238+0000 7f9b53200640 1 -- 172.21.15.100:0/3789276071 >> 172.21.15.100:0/3789276071 conn(0x7f9b4c072590 msgr2=0x7f9b4c0ad860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:09.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.238+0000 7f9b53200640 1 -- 172.21.15.100:0/3789276071 shutdown_connections 2024-04-23T14:33:09.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.238+0000 7f9b53200640 1 -- 172.21.15.100:0/3789276071 wait complete. 2024-04-23T14:33:09.259 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:33:09.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.394+0000 7fa27fc00640 1 Processor -- start 2024-04-23T14:33:09.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.394+0000 7fa27fc00640 1 -- start start 2024-04-23T14:33:09.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.394+0000 7fa27fc00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa27806f8c0 0x7fa27806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.394+0000 7fa27fc00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa2780d0170 0x7fa2780ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.394+0000 7fa27fc00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa27806dd60 con 0x7fa2780ce620 2024-04-23T14:33:09.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.394+0000 7fa27fc00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fa2780ad050 con 0x7fa2780d0170 2024-04-23T14:33:09.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.394+0000 7fa27fc00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa2780ad8f0 con 0x7fa27806f8c0 2024-04-23T14:33:09.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.398+0000 7fa277e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa2780d0170 0x7fa2780ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.398+0000 7fa277e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa2780d0170 0x7fa2780ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60678/0 (socket says 172.21.15.100:60678) 2024-04-23T14:33:09.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.398+0000 7fa277e00640 1 -- 172.21.15.100:0/335326816 learned_addr learned my addr 172.21.15.100:0/335326816 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:09.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.398+0000 7fa27d200640 1 --2- 172.21.15.100:0/335326816 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa27806f8c0 0x7fa27806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.398+0000 7fa277e00640 1 -- 172.21.15.100:0/335326816 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa27806f8c0 msgr2=0x7fa27806c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:09.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.398+0000 7fa277e00640 1 --2- 172.21.15.100:0/335326816 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa27806f8c0 0x7fa27806c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.398+0000 7fa277e00640 1 -- 172.21.15.100:0/335326816 >> v1:172.21.15.151:6789/0 conn(0x7fa2780ce620 legacy=0x7fa27815f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:09.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.398+0000 7fa277e00640 1 -- 172.21.15.100:0/335326816 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2780d09c0 con 0x7fa2780d0170 2024-04-23T14:33:09.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.398+0000 7fa27d200640 1 --2- 172.21.15.100:0/335326816 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa27806f8c0 0x7fa27806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:09.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.398+0000 7fa277e00640 1 --2- 172.21.15.100:0/335326816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2780d0170 0x7fa2780ce0e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa260009870 tx=0x7fa26001c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459eb7d0d7f221c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:09.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.398+0000 7fa277400640 1 -- 172.21.15.100:0/335326816 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2600b0070 con 0x7fa2780d0170 2024-04-23T14:33:09.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.398+0000 7fa277400640 1 -- 172.21.15.100:0/335326816 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2600ab600 con 0x7fa2780d0170 2024-04-23T14:33:09.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.398+0000 7fa277400640 1 -- 172.21.15.100:0/335326816 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2600ab9d0 con 0x7fa2780d0170 2024-04-23T14:33:09.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa27fc00640 1 -- 172.21.15.100:0/335326816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2780d0170 msgr2=0x7fa2780ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa27fc00640 1 --2- 172.21.15.100:0/335326816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2780d0170 0x7fa2780ce0e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa260009870 tx=0x7fa26001c340 comp rx=0 tx=0).stop 2024-04-23T14:33:09.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa27fc00640 1 -- 172.21.15.100:0/335326816 shutdown_connections 2024-04-23T14:33:09.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa27fc00640 1 --2- 172.21.15.100:0/335326816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2780d0170 0x7fa2780ce0e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa27fc00640 1 --2- 172.21.15.100:0/335326816 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa27806f8c0 0x7fa27806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa27fc00640 1 -- 172.21.15.100:0/335326816 >> 172.21.15.100:0/335326816 conn(0x7fa2780b5200 msgr2=0x7fa2780b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:09.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa27fc00640 1 -- 172.21.15.100:0/335326816 shutdown_connections 2024-04-23T14:33:09.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa27fc00640 1 -- 172.21.15.100:0/335326816 wait complete. 2024-04-23T14:33:09.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa27fc00640 1 Processor -- start 2024-04-23T14:33:09.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa27fc00640 1 -- start start 2024-04-23T14:33:09.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa27fc00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa27806f8c0 0x7fa27815c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa27fc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2780ce620 0x7fa2780c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa27fc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2780d0170 0x7fa2780c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa27fc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa2780d0730 con 0x7fa2780ce620 2024-04-23T14:33:09.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa27fc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa2780d05b0 con 0x7fa2780d0170 2024-04-23T14:33:09.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa27fc00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa278168400 con 0x7fa27806f8c0 2024-04-23T14:33:09.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa277e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2780ce620 0x7fa2780c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa277e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2780ce620 0x7fa2780c8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60684/0 (socket says 172.21.15.100:60684) 2024-04-23T14:33:09.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa277e00640 1 -- 172.21.15.100:0/2598752798 learned_addr learned my addr 172.21.15.100:0/2598752798 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:09.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa27d200640 1 --2- 172.21.15.100:0/2598752798 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa27806f8c0 0x7fa27815c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa27dc00640 1 --2- 172.21.15.100:0/2598752798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2780d0170 0x7fa2780c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa277e00640 1 -- 172.21.15.100:0/2598752798 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa27806f8c0 msgr2=0x7fa27815c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa277e00640 1 --2- 172.21.15.100:0/2598752798 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa27806f8c0 0x7fa27815c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa277e00640 1 -- 172.21.15.100:0/2598752798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2780d0170 msgr2=0x7fa2780c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa277e00640 1 --2- 172.21.15.100:0/2598752798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2780d0170 0x7fa2780c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa277e00640 1 -- 172.21.15.100:0/2598752798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2780d0cd0 con 0x7fa2780ce620 2024-04-23T14:33:09.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa27dc00640 1 --2- 172.21.15.100:0/2598752798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2780d0170 0x7fa2780c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:09.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa277e00640 1 --2- 172.21.15.100:0/2598752798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2780ce620 0x7fa2780c8140 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa2600ae040 tx=0x7fa2600190e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:09.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.402+0000 7fa274c00640 1 -- 172.21.15.100:0/2598752798 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2600b0040 con 0x7fa2780ce620 2024-04-23T14:33:09.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.406+0000 7fa274c00640 1 -- 172.21.15.100:0/2598752798 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2600b9070 con 0x7fa2780ce620 2024-04-23T14:33:09.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.406+0000 7fa274c00640 1 -- 172.21.15.100:0/2598752798 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2600b4880 con 0x7fa2780ce620 2024-04-23T14:33:09.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.406+0000 7fa27fc00640 1 -- 172.21.15.100:0/2598752798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2780c8680 con 0x7fa2780ce620 2024-04-23T14:33:09.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.406+0000 7fa27fc00640 1 -- 172.21.15.100:0/2598752798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2780c5190 con 0x7fa2780ce620 2024-04-23T14:33:09.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.406+0000 7fa27fc00640 1 -- 172.21.15.100:0/2598752798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa278068ad0 con 0x7fa2780ce620 2024-04-23T14:33:09.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.406+0000 7fa274c00640 1 -- 172.21.15.100:0/2598752798 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa2600ab4c0 con 0x7fa2780ce620 2024-04-23T14:33:09.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.406+0000 7fa274c00640 1 --2- 172.21.15.100:0/2598752798 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa2540683b0 0x7fa25406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.406+0000 7fa274c00640 1 -- 172.21.15.100:0/2598752798 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa2601209a0 con 0x7fa2780ce620 2024-04-23T14:33:09.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.406+0000 7fa27d200640 1 --2- 172.21.15.100:0/2598752798 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa2540683b0 0x7fa25406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.406+0000 7fa27d200640 1 --2- 172.21.15.100:0/2598752798 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa2540683b0 0x7fa25406a870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa278165700 tx=0x7fa268008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:09.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.410+0000 7fa274c00640 1 -- 172.21.15.100:0/2598752798 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa27806dd60 con 0x7fa2780ce620 2024-04-23T14:33:09.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.566+0000 7fa27fc00640 1 -- 172.21.15.100:0/2598752798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa27806cc00 con 0x7fa2780ce620 2024-04-23T14:33:09.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.570+0000 7fa274c00640 1 -- 172.21.15.100:0/2598752798 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fa2600f0c70 con 0x7fa2780ce620 2024-04-23T14:33:09.571 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:09.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.570+0000 7fa27fc00640 1 -- 172.21.15.100:0/2598752798 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa2540683b0 msgr2=0x7fa25406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.570+0000 7fa27fc00640 1 --2- 172.21.15.100:0/2598752798 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa2540683b0 0x7fa25406a870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa278165700 tx=0x7fa268008040 comp rx=0 tx=0).stop 2024-04-23T14:33:09.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.570+0000 7fa27fc00640 1 -- 172.21.15.100:0/2598752798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2780ce620 msgr2=0x7fa2780c8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.570+0000 7fa27fc00640 1 --2- 172.21.15.100:0/2598752798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2780ce620 0x7fa2780c8140 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa2600ae040 tx=0x7fa2600190e0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.570+0000 7fa27fc00640 1 -- 172.21.15.100:0/2598752798 shutdown_connections 2024-04-23T14:33:09.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.570+0000 7fa27fc00640 1 --2- 172.21.15.100:0/2598752798 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa2540683b0 0x7fa25406a870 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.570+0000 7fa27fc00640 1 --2- 172.21.15.100:0/2598752798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2780d0170 0x7fa2780c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.570+0000 7fa27fc00640 1 --2- 172.21.15.100:0/2598752798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2780ce620 0x7fa2780c8140 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.570+0000 7fa27fc00640 1 --2- 172.21.15.100:0/2598752798 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa27806f8c0 0x7fa27815c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.570+0000 7fa27fc00640 1 -- 172.21.15.100:0/2598752798 >> 172.21.15.100:0/2598752798 conn(0x7fa2780b5200 msgr2=0x7fa2781657a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:09.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.570+0000 7fa27fc00640 1 -- 172.21.15.100:0/2598752798 shutdown_connections 2024-04-23T14:33:09.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.570+0000 7fa27fc00640 1 -- 172.21.15.100:0/2598752798 wait complete. 2024-04-23T14:33:09.591 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:33:09.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.730+0000 7fa9db600640 1 Processor -- start 2024-04-23T14:33:09.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.730+0000 7fa9db600640 1 -- start start 2024-04-23T14:33:09.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.730+0000 7fa9db600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa9d415c7a0 0x7fa9d40af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.730+0000 7fa9db600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa9d406bb20 0x7fa9d40ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.730+0000 7fa9db600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9d406cbb0 con 0x7fa9d40cd130 2024-04-23T14:33:09.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.730+0000 7fa9db600640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fa9d40cd5a0 con 0x7fa9d406bb20 2024-04-23T14:33:09.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.730+0000 7fa9db600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d406d450 con 0x7fa9d415c7a0 2024-04-23T14:33:09.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.730+0000 7fa9d3e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa9d406bb20 0x7fa9d40ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.730+0000 7fa9d3e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa9d406bb20 0x7fa9d40ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60688/0 (socket says 172.21.15.100:60688) 2024-04-23T14:33:09.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.730+0000 7fa9d3e00640 1 -- 172.21.15.100:0/1646863578 learned_addr learned my addr 172.21.15.100:0/1646863578 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:09.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.730+0000 7fa9d8c00640 1 --2- 172.21.15.100:0/1646863578 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa9d415c7a0 0x7fa9d40af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.730+0000 7fa9d3e00640 1 -- 172.21.15.100:0/1646863578 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa9d415c7a0 msgr2=0x7fa9d40af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:09.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.730+0000 7fa9d3e00640 1 --2- 172.21.15.100:0/1646863578 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa9d415c7a0 0x7fa9d40af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.730+0000 7fa9d3e00640 1 -- 172.21.15.100:0/1646863578 >> v1:172.21.15.100:6789/0 conn(0x7fa9d40cd130 legacy=0x7fa9d406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.730+0000 7fa9d3e00640 1 -- 172.21.15.100:0/1646863578 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d406ce90 con 0x7fa9d406bb20 2024-04-23T14:33:09.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.734+0000 7fa9d3e00640 1 --2- 172.21.15.100:0/1646863578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9d406bb20 0x7fa9d40ccbf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009870 tx=0x7fa9bc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416be64ba00a7e07 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:09.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.734+0000 7fa9d3400640 1 -- 172.21.15.100:0/1646863578 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc0b0070 con 0x7fa9d406bb20 2024-04-23T14:33:09.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.734+0000 7fa9d3400640 1 -- 172.21.15.100:0/1646863578 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9bc0ab600 con 0x7fa9d406bb20 2024-04-23T14:33:09.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.734+0000 7fa9d3400640 1 -- 172.21.15.100:0/1646863578 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc0ab9d0 con 0x7fa9d406bb20 2024-04-23T14:33:09.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.734+0000 7fa9db600640 1 -- 172.21.15.100:0/1646863578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9d406bb20 msgr2=0x7fa9d40ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.734+0000 7fa9db600640 1 --2- 172.21.15.100:0/1646863578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9d406bb20 0x7fa9d40ccbf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009870 tx=0x7fa9bc01c4d0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.734+0000 7fa9db600640 1 -- 172.21.15.100:0/1646863578 shutdown_connections 2024-04-23T14:33:09.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.734+0000 7fa9db600640 1 --2- 172.21.15.100:0/1646863578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9d406bb20 0x7fa9d40ccbf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.734+0000 7fa9db600640 1 --2- 172.21.15.100:0/1646863578 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa9d415c7a0 0x7fa9d40af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.734+0000 7fa9db600640 1 -- 172.21.15.100:0/1646863578 >> 172.21.15.100:0/1646863578 conn(0x7fa9d40b5200 msgr2=0x7fa9d40b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:09.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.734+0000 7fa9db600640 1 -- 172.21.15.100:0/1646863578 shutdown_connections 2024-04-23T14:33:09.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.734+0000 7fa9db600640 1 -- 172.21.15.100:0/1646863578 wait complete. 2024-04-23T14:33:09.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.734+0000 7fa9db600640 1 Processor -- start 2024-04-23T14:33:09.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.734+0000 7fa9db600640 1 -- start start 2024-04-23T14:33:09.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9db600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa9d406bb20 0x7fa9d40c6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9db600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9d40cd130 0x7fa9d41531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9db600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9d415c7a0 0x7fa9d4099430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9db600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4169590 con 0x7fa9d415c7a0 2024-04-23T14:33:09.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9db600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4169410 con 0x7fa9d40cd130 2024-04-23T14:33:09.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9db600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d4169710 con 0x7fa9d406bb20 2024-04-23T14:33:09.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9d9600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9d415c7a0 0x7fa9d4099430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9d9600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9d415c7a0 0x7fa9d4099430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60704/0 (socket says 172.21.15.100:60704) 2024-04-23T14:33:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9d9600640 1 -- 172.21.15.100:0/283200282 learned_addr learned my addr 172.21.15.100:0/283200282 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9d8c00640 1 --2- 172.21.15.100:0/283200282 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa9d406bb20 0x7fa9d40c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9d3e00640 1 --2- 172.21.15.100:0/283200282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9d40cd130 0x7fa9d41531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9d9600640 1 -- 172.21.15.100:0/283200282 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa9d406bb20 msgr2=0x7fa9d40c6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9d9600640 1 --2- 172.21.15.100:0/283200282 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa9d406bb20 0x7fa9d40c6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9d9600640 1 -- 172.21.15.100:0/283200282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9d40cd130 msgr2=0x7fa9d41531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9d9600640 1 --2- 172.21.15.100:0/283200282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9d40cd130 0x7fa9d41531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9d9600640 1 -- 172.21.15.100:0/283200282 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d415d020 con 0x7fa9d415c7a0 2024-04-23T14:33:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9d9600640 1 --2- 172.21.15.100:0/283200282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9d415c7a0 0x7fa9d4099430 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00c430 tx=0x7fa9cc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9d0c00640 1 -- 172.21.15.100:0/283200282 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc005220 con 0x7fa9d415c7a0 2024-04-23T14:33:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9d0c00640 1 -- 172.21.15.100:0/283200282 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9cc0053c0 con 0x7fa9d415c7a0 2024-04-23T14:33:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9db600640 1 -- 172.21.15.100:0/283200282 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d415d300 con 0x7fa9d415c7a0 2024-04-23T14:33:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9d0c00640 1 -- 172.21.15.100:0/283200282 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc007270 con 0x7fa9d415c7a0 2024-04-23T14:33:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9db600640 1 -- 172.21.15.100:0/283200282 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d4098c30 con 0x7fa9d415c7a0 2024-04-23T14:33:09.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.738+0000 7fa9db600640 1 -- 172.21.15.100:0/283200282 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d4068ad0 con 0x7fa9d415c7a0 2024-04-23T14:33:09.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.742+0000 7fa9d0c00640 1 -- 172.21.15.100:0/283200282 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa9cc019020 con 0x7fa9d415c7a0 2024-04-23T14:33:09.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.742+0000 7fa9d0c00640 1 --2- 172.21.15.100:0/283200282 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa9b0068380 0x7fa9b006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.742+0000 7fa9d0c00640 1 -- 172.21.15.100:0/283200282 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa9cc0860c0 con 0x7fa9d415c7a0 2024-04-23T14:33:09.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.742+0000 7fa9d8c00640 1 --2- 172.21.15.100:0/283200282 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa9b0068380 0x7fa9b006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.742+0000 7fa9d8c00640 1 --2- 172.21.15.100:0/283200282 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa9b0068380 0x7fa9b006a840 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4003ab0 tx=0x7fa9c4009830 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:09.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.746+0000 7fa9d0c00640 1 -- 172.21.15.100:0/283200282 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa9cc055b90 con 0x7fa9d415c7a0 2024-04-23T14:33:09.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.902+0000 7fa9db600640 1 -- 172.21.15.100:0/283200282 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa9d40cf130 con 0x7fa9d415c7a0 2024-04-23T14:33:09.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.902+0000 7fa9d0c00640 1 -- 172.21.15.100:0/283200282 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fa9cc059e90 con 0x7fa9d415c7a0 2024-04-23T14:33:09.905 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:33:09.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.906+0000 7fa9db600640 1 -- 172.21.15.100:0/283200282 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa9b0068380 msgr2=0x7fa9b006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.906+0000 7fa9db600640 1 --2- 172.21.15.100:0/283200282 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa9b0068380 0x7fa9b006a840 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4003ab0 tx=0x7fa9c4009830 comp rx=0 tx=0).stop 2024-04-23T14:33:09.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.906+0000 7fa9db600640 1 -- 172.21.15.100:0/283200282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9d415c7a0 msgr2=0x7fa9d4099430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.906+0000 7fa9db600640 1 --2- 172.21.15.100:0/283200282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9d415c7a0 0x7fa9d4099430 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00c430 tx=0x7fa9cc00c8f0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.906+0000 7fa9db600640 1 -- 172.21.15.100:0/283200282 shutdown_connections 2024-04-23T14:33:09.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.906+0000 7fa9db600640 1 --2- 172.21.15.100:0/283200282 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa9b0068380 0x7fa9b006a840 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.906+0000 7fa9db600640 1 --2- 172.21.15.100:0/283200282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9d415c7a0 0x7fa9d4099430 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.906+0000 7fa9db600640 1 --2- 172.21.15.100:0/283200282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9d40cd130 0x7fa9d41531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.906+0000 7fa9db600640 1 --2- 172.21.15.100:0/283200282 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa9d406bb20 0x7fa9d40c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.906+0000 7fa9db600640 1 -- 172.21.15.100:0/283200282 >> 172.21.15.100:0/283200282 conn(0x7fa9d40b5200 msgr2=0x7fa9d40d0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:09.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.906+0000 7fa9db600640 1 -- 172.21.15.100:0/283200282 shutdown_connections 2024-04-23T14:33:09.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:09.906+0000 7fa9db600640 1 -- 172.21.15.100:0/283200282 wait complete. 2024-04-23T14:33:09.926 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-23T14:33:10.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.062+0000 7f4f15600640 1 Processor -- start 2024-04-23T14:33:10.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.062+0000 7f4f15600640 1 -- start start 2024-04-23T14:33:10.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.062+0000 7f4f15600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4f100708c0 0x7f4f10169470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:10.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.062+0000 7f4f15600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f100af6c0 con 0x7f4f10161a30 2024-04-23T14:33:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.062+0000 7f4f15600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f100af9a0 con 0x7f4f100ce2a0 2024-04-23T14:33:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.062+0000 7f4f15600640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f4f100aff60 con 0x7f4f100708c0 2024-04-23T14:33:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.062+0000 7f4f14c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4f100708c0 0x7f4f10169470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.062+0000 7f4f0fe00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f4f10161a30 0x7f4f100b4d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:36252/0 (socket says 172.21.15.100:36252) 2024-04-23T14:33:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.062+0000 7f4f0fe00640 1 -- 172.21.15.100:0/2189854964 learned_addr learned my addr 172.21.15.100:0/2189854964 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.062+0000 7f4f14c00640 1 --2- 172.21.15.100:0/2189854964 >> v2:172.21.15.100:3300/0 conn(0x7f4f100708c0 0x7f4f10169470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60718/0 (socket says 172.21.15.100:60718) 2024-04-23T14:33:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.062+0000 7f4f14c00640 1 -- 172.21.15.100:0/2189854964 >> v1:172.21.15.151:6789/0 conn(0x7f4f100ce2a0 legacy=0x7f4f100701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.062+0000 7f4f14c00640 1 -- 172.21.15.100:0/2189854964 >> v1:172.21.15.100:6789/0 conn(0x7f4f10161a30 legacy=0x7f4f100b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:10.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.062+0000 7f4f14c00640 1 -- 172.21.15.100:0/2189854964 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f100afc80 con 0x7f4f100708c0 2024-04-23T14:33:10.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.066+0000 7f4f14c00640 1 --2- 172.21.15.100:0/2189854964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f100708c0 0x7f4f10169470 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4f080097c0 tx=0x7f4f0801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e82680d6d18893 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:10.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.066+0000 7f4f0ea00640 1 -- 172.21.15.100:0/2189854964 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f080b0070 con 0x7f4f100708c0 2024-04-23T14:33:10.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.066+0000 7f4f0ea00640 1 -- 172.21.15.100:0/2189854964 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f080ab600 con 0x7f4f100708c0 2024-04-23T14:33:10.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.066+0000 7f4f0ea00640 1 -- 172.21.15.100:0/2189854964 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f080ab9d0 con 0x7f4f100708c0 2024-04-23T14:33:10.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.066+0000 7f4f15600640 1 -- 172.21.15.100:0/2189854964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f100708c0 msgr2=0x7f4f10169470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:10.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.066+0000 7f4f15600640 1 --2- 172.21.15.100:0/2189854964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f100708c0 0x7f4f10169470 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4f080097c0 tx=0x7f4f0801c410 comp rx=0 tx=0).stop 2024-04-23T14:33:10.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.066+0000 7f4f15600640 1 -- 172.21.15.100:0/2189854964 shutdown_connections 2024-04-23T14:33:10.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.066+0000 7f4f15600640 1 --2- 172.21.15.100:0/2189854964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f100708c0 0x7f4f10169470 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:10.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.066+0000 7f4f15600640 1 -- 172.21.15.100:0/2189854964 >> 172.21.15.100:0/2189854964 conn(0x7f4f100d15f0 msgr2=0x7f4f100d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:10.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.066+0000 7f4f15600640 1 -- 172.21.15.100:0/2189854964 shutdown_connections 2024-04-23T14:33:10.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.066+0000 7f4f15600640 1 -- 172.21.15.100:0/2189854964 wait complete. 2024-04-23T14:33:10.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f15600640 1 Processor -- start 2024-04-23T14:33:10.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f15600640 1 -- start start 2024-04-23T14:33:10.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f15600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f100708c0 0x7f4f1006f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:10.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f15600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f100ce2a0 0x7f4f1013f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:10.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f15600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f10161a30 0x7f4f1013fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:10.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f15600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4f100af3f0 con 0x7f4f10161a30 2024-04-23T14:33:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f15600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4f100af270 con 0x7f4f100ce2a0 2024-04-23T14:33:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f15600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4f1016c5e0 con 0x7f4f100708c0 2024-04-23T14:33:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f14c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f10161a30 0x7f4f1013fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f14c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f10161a30 0x7f4f1013fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60726/0 (socket says 172.21.15.100:60726) 2024-04-23T14:33:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f14c00640 1 -- 172.21.15.100:0/659281482 learned_addr learned my addr 172.21.15.100:0/659281482 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f0f400640 1 --2- 172.21.15.100:0/659281482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f100ce2a0 0x7f4f1013f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f0fe00640 1 --2- 172.21.15.100:0/659281482 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f100708c0 0x7f4f1006f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f14c00640 1 -- 172.21.15.100:0/659281482 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f100708c0 msgr2=0x7f4f1006f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f14c00640 1 --2- 172.21.15.100:0/659281482 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f100708c0 0x7f4f1006f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f14c00640 1 -- 172.21.15.100:0/659281482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f100ce2a0 msgr2=0x7f4f1013f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f14c00640 1 --2- 172.21.15.100:0/659281482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f100ce2a0 0x7f4f1013f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f14c00640 1 -- 172.21.15.100:0/659281482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f101610c0 con 0x7f4f10161a30 2024-04-23T14:33:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f14c00640 1 --2- 172.21.15.100:0/659281482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f10161a30 0x7f4f1013fe30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4f080ae040 tx=0x7f4f080191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f07e00640 1 -- 172.21.15.100:0/659281482 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f080b8070 con 0x7f4f10161a30 2024-04-23T14:33:10.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f07e00640 1 -- 172.21.15.100:0/659281482 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f080b4610 con 0x7f4f10161a30 2024-04-23T14:33:10.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f15600640 1 -- 172.21.15.100:0/659281482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1006e020 con 0x7f4f10161a30 2024-04-23T14:33:10.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f07e00640 1 -- 172.21.15.100:0/659281482 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f080b0040 con 0x7f4f10161a30 2024-04-23T14:33:10.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.070+0000 7f4f15600640 1 -- 172.21.15.100:0/659281482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f1006e5e0 con 0x7f4f10161a30 2024-04-23T14:33:10.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.074+0000 7f4f15600640 1 -- 172.21.15.100:0/659281482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f10068ad0 con 0x7f4f10161a30 2024-04-23T14:33:10.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.074+0000 7f4f07e00640 1 -- 172.21.15.100:0/659281482 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4f080bb070 con 0x7f4f10161a30 2024-04-23T14:33:10.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.074+0000 7f4f07e00640 1 --2- 172.21.15.100:0/659281482 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4ef0068380 0x7f4ef006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:10.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.074+0000 7f4f07e00640 1 -- 172.21.15.100:0/659281482 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f4f08120c30 con 0x7f4f10161a30 2024-04-23T14:33:10.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.074+0000 7f4f0fe00640 1 --2- 172.21.15.100:0/659281482 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4ef0068380 0x7f4ef006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:10.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.074+0000 7f4f0fe00640 1 --2- 172.21.15.100:0/659281482 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4ef0068380 0x7f4ef006a840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4f00005c10 tx=0x7f4f00009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:10.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.078+0000 7f4f07e00640 1 -- 172.21.15.100:0/659281482 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4f08016090 con 0x7f4f10161a30 2024-04-23T14:33:10.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.234+0000 7f4f15600640 1 -- 172.21.15.100:0/659281482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4f10070470 con 0x7f4f10161a30 2024-04-23T14:33:10.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.234+0000 7f4f07e00640 1 -- 172.21.15.100:0/659281482 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f4f080f0d50 con 0x7f4f10161a30 2024-04-23T14:33:10.237 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:10.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.238+0000 7f4f15600640 1 -- 172.21.15.100:0/659281482 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4ef0068380 msgr2=0x7f4ef006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:10.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.238+0000 7f4f15600640 1 --2- 172.21.15.100:0/659281482 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4ef0068380 0x7f4ef006a840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4f00005c10 tx=0x7f4f00009040 comp rx=0 tx=0).stop 2024-04-23T14:33:10.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.238+0000 7f4f15600640 1 -- 172.21.15.100:0/659281482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f10161a30 msgr2=0x7f4f1013fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:10.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.238+0000 7f4f15600640 1 --2- 172.21.15.100:0/659281482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f10161a30 0x7f4f1013fe30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4f080ae040 tx=0x7f4f080191b0 comp rx=0 tx=0).stop 2024-04-23T14:33:10.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.238+0000 7f4f15600640 1 -- 172.21.15.100:0/659281482 shutdown_connections 2024-04-23T14:33:10.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.238+0000 7f4f15600640 1 --2- 172.21.15.100:0/659281482 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4ef0068380 0x7f4ef006a840 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:10.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.238+0000 7f4f15600640 1 --2- 172.21.15.100:0/659281482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f10161a30 0x7f4f1013fe30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:10.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.238+0000 7f4f15600640 1 --2- 172.21.15.100:0/659281482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f100ce2a0 0x7f4f1013f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:10.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.238+0000 7f4f15600640 1 --2- 172.21.15.100:0/659281482 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f100708c0 0x7f4f1006f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:10.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.238+0000 7f4f15600640 1 -- 172.21.15.100:0/659281482 >> 172.21.15.100:0/659281482 conn(0x7f4f100d15f0 msgr2=0x7f4f100b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:10.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.238+0000 7f4f15600640 1 -- 172.21.15.100:0/659281482 shutdown_connections 2024-04-23T14:33:10.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.238+0000 7f4f15600640 1 -- 172.21.15.100:0/659281482 wait complete. 2024-04-23T14:33:10.258 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:33:10.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.390+0000 7f7404e00640 1 Processor -- start 2024-04-23T14:33:10.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.394+0000 7f7404e00640 1 -- start start 2024-04-23T14:33:10.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.394+0000 7f7404e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f740015b660 0x7f74000c6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:10.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.394+0000 7f7404e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f74000d08a0 0x7f74000c5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:10.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.394+0000 7f7404e00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74000cee00 con 0x7f74000c6d60 2024-04-23T14:33:10.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.394+0000 7f7404e00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f740015ba40 con 0x7f740015b660 2024-04-23T14:33:10.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.394+0000 7f7404e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f740006bc50 con 0x7f74000d08a0 2024-04-23T14:33:10.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.394+0000 7f73fe800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f74000d08a0 0x7f74000c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:10.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.394+0000 7f73fd400640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f74000c6d60 0x7f74000d0190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:58308/0 (socket says 172.21.15.100:58308) 2024-04-23T14:33:10.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.394+0000 7f73fd400640 1 -- 172.21.15.100:0/513159135 learned_addr learned my addr 172.21.15.100:0/513159135 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:10.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.394+0000 7f73fde00640 1 --2- 172.21.15.100:0/513159135 >> v2:172.21.15.151:3300/0 conn(0x7f740015b660 0x7f74000c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:10.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.394+0000 7f73fde00640 1 -- 172.21.15.100:0/513159135 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f74000d08a0 msgr2=0x7f74000c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:10.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.394+0000 7f73fde00640 1 --2- 172.21.15.100:0/513159135 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f74000d08a0 0x7f74000c5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:10.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.394+0000 7f73fde00640 1 -- 172.21.15.100:0/513159135 >> v1:172.21.15.151:6789/0 conn(0x7f74000c6d60 legacy=0x7f74000d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:10.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.394+0000 7f73fde00640 1 -- 172.21.15.100:0/513159135 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740006bf30 con 0x7f740015b660 2024-04-23T14:33:10.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.394+0000 7f73fe800640 1 --2- 172.21.15.100:0/513159135 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f74000d08a0 0x7f74000c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:33:10.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.398+0000 7f73fde00640 1 --2- 172.21.15.100:0/513159135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f740015b660 0x7f74000c6820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f73e80097c0 tx=0x7f73e801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c18ee1aa10ca902c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:10.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.398+0000 7f73fca00640 1 -- 172.21.15.100:0/513159135 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e80a7070 con 0x7f740015b660 2024-04-23T14:33:10.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.398+0000 7f73fca00640 1 -- 172.21.15.100:0/513159135 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73e80b0070 con 0x7f740015b660 2024-04-23T14:33:10.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.398+0000 7f73fca00640 1 -- 172.21.15.100:0/513159135 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e80ab890 con 0x7f740015b660 2024-04-23T14:33:10.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.398+0000 7f7404e00640 1 -- 172.21.15.100:0/513159135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f740015b660 msgr2=0x7f74000c6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:10.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.398+0000 7f7404e00640 1 --2- 172.21.15.100:0/513159135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f740015b660 0x7f74000c6820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f73e80097c0 tx=0x7f73e801c470 comp rx=0 tx=0).stop 2024-04-23T14:33:10.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.398+0000 7f7404e00640 1 -- 172.21.15.100:0/513159135 shutdown_connections 2024-04-23T14:33:10.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.398+0000 7f7404e00640 1 --2- 172.21.15.100:0/513159135 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f74000d08a0 0x7f74000c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:10.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.398+0000 7f7404e00640 1 --2- 172.21.15.100:0/513159135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f740015b660 0x7f74000c6820 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:10.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.398+0000 7f7404e00640 1 -- 172.21.15.100:0/513159135 >> 172.21.15.100:0/513159135 conn(0x7f74000b5310 msgr2=0x7f74000b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:10.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.398+0000 7f7404e00640 1 -- 172.21.15.100:0/513159135 shutdown_connections 2024-04-23T14:33:10.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.398+0000 7f7404e00640 1 -- 172.21.15.100:0/513159135 wait complete. 2024-04-23T14:33:10.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f7404e00640 1 Processor -- start 2024-04-23T14:33:10.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f7404e00640 1 -- start start 2024-04-23T14:33:10.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f7404e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74000c6d60 0x7f740013e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:10.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f7404e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f74000d08a0 0x7f7400152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:10.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f7404e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f740015b660 0x7f7400157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:10.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f7404e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f74000c6030 con 0x7f740015b660 2024-04-23T14:33:10.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f7404e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74000c5eb0 con 0x7f74000c6d60 2024-04-23T14:33:10.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f7404e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f74000c61b0 con 0x7f74000d08a0 2024-04-23T14:33:10.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f73fe800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f740015b660 0x7f7400157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:10.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f73fe800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f740015b660 0x7f7400157110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60742/0 (socket says 172.21.15.100:60742) 2024-04-23T14:33:10.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f73fe800640 1 -- 172.21.15.100:0/1586027598 learned_addr learned my addr 172.21.15.100:0/1586027598 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:10.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f73fde00640 1 --2- 172.21.15.100:0/1586027598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74000c6d60 0x7f740013e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:10.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f73fd400640 1 --2- 172.21.15.100:0/1586027598 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f74000d08a0 0x7f7400152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:10.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f73fe800640 1 -- 172.21.15.100:0/1586027598 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f74000d08a0 msgr2=0x7f7400152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:10.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f73fe800640 1 --2- 172.21.15.100:0/1586027598 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f74000d08a0 0x7f7400152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:10.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f73fe800640 1 -- 172.21.15.100:0/1586027598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74000c6d60 msgr2=0x7f740013e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:10.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f73fe800640 1 --2- 172.21.15.100:0/1586027598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74000c6d60 0x7f740013e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:10.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f73fe800640 1 -- 172.21.15.100:0/1586027598 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740013eed0 con 0x7f740015b660 2024-04-23T14:33:10.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f73fe800640 1 --2- 172.21.15.100:0/1586027598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f740015b660 0x7f7400157110 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009410 tx=0x7f73f800e550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:10.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f73f6000640 1 -- 172.21.15.100:0/1586027598 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f801a070 con 0x7f740015b660 2024-04-23T14:33:10.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f73f6000640 1 -- 172.21.15.100:0/1586027598 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f800b2d0 con 0x7f740015b660 2024-04-23T14:33:10.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f7404e00640 1 -- 172.21.15.100:0/1586027598 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f740013f1b0 con 0x7f740015b660 2024-04-23T14:33:10.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f73f6000640 1 -- 172.21.15.100:0/1586027598 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8009740 con 0x7f740015b660 2024-04-23T14:33:10.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f7404e00640 1 -- 172.21.15.100:0/1586027598 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f740013c870 con 0x7f740015b660 2024-04-23T14:33:10.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.402+0000 7f7404e00640 1 -- 172.21.15.100:0/1586027598 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7400068ad0 con 0x7f740015b660 2024-04-23T14:33:10.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.406+0000 7f73f6000640 1 -- 172.21.15.100:0/1586027598 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f73f80098e0 con 0x7f740015b660 2024-04-23T14:33:10.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.406+0000 7f73f6000640 1 --2- 172.21.15.100:0/1586027598 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f73dc0683b0 0x7f73dc06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:10.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.406+0000 7f73f6000640 1 -- 172.21.15.100:0/1586027598 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f73f8087200 con 0x7f740015b660 2024-04-23T14:33:10.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.406+0000 7f73fde00640 1 --2- 172.21.15.100:0/1586027598 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f73dc0683b0 0x7f73dc06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:10.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.406+0000 7f73fde00640 1 --2- 172.21.15.100:0/1586027598 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f73dc0683b0 0x7f73dc06a870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f73e80af000 tx=0x7f73e80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:10.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.410+0000 7f73f6000640 1 -- 172.21.15.100:0/1586027598 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f73f8056d60 con 0x7f740015b660 2024-04-23T14:33:10.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.578+0000 7f7404e00640 1 -- 172.21.15.100:0/1586027598 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f740006d3a0 con 0x7f740015b660 2024-04-23T14:33:10.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.578+0000 7f73f6000640 1 -- 172.21.15.100:0/1586027598 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f73f805b060 con 0x7f740015b660 2024-04-23T14:33:10.582 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-04-23T14:33:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.582+0000 7f7404e00640 1 -- 172.21.15.100:0/1586027598 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f73dc0683b0 msgr2=0x7f73dc06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.582+0000 7f7404e00640 1 --2- 172.21.15.100:0/1586027598 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f73dc0683b0 0x7f73dc06a870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f73e80af000 tx=0x7f73e80a7000 comp rx=0 tx=0).stop 2024-04-23T14:33:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.582+0000 7f7404e00640 1 -- 172.21.15.100:0/1586027598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f740015b660 msgr2=0x7f7400157110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.582+0000 7f7404e00640 1 --2- 172.21.15.100:0/1586027598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f740015b660 0x7f7400157110 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009410 tx=0x7f73f800e550 comp rx=0 tx=0).stop 2024-04-23T14:33:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.582+0000 7f7404e00640 1 -- 172.21.15.100:0/1586027598 shutdown_connections 2024-04-23T14:33:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.582+0000 7f7404e00640 1 --2- 172.21.15.100:0/1586027598 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f73dc0683b0 0x7f73dc06a870 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.582+0000 7f7404e00640 1 --2- 172.21.15.100:0/1586027598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f740015b660 0x7f7400157110 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.582+0000 7f7404e00640 1 --2- 172.21.15.100:0/1586027598 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f74000d08a0 0x7f7400152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.582+0000 7f7404e00640 1 --2- 172.21.15.100:0/1586027598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74000c6d60 0x7f740013e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.582+0000 7f7404e00640 1 -- 172.21.15.100:0/1586027598 >> 172.21.15.100:0/1586027598 conn(0x7f74000b5310 msgr2=0x7f74000b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.582+0000 7f7404e00640 1 -- 172.21.15.100:0/1586027598 shutdown_connections 2024-04-23T14:33:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.582+0000 7f7404e00640 1 -- 172.21.15.100:0/1586027598 wait complete. 2024-04-23T14:33:10.602 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi100.front.sepia.ceph.com... 2024-04-23T14:33:10.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:10.602 DEBUG:teuthology.orchestra.run.smithi100:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:10.622 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:33:10.622 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:33:10.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:10.623 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:33:10.674 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:33:10.674 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:10.684 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:33:10.684 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:10.740 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:10.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.874+0000 7fb7f9000640 1 Processor -- start 2024-04-23T14:33:10.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.874+0000 7fb7f9000640 1 -- start start 2024-04-23T14:33:10.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.874+0000 7fb7f9000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb7f406d450 0x7fb7f406d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:10.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.874+0000 7fb7f9000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7f406dd70 con 0x7fb7f40cea60 2024-04-23T14:33:10.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.874+0000 7fb7f9000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7f40b1f90 con 0x7fb7f40cd460 2024-04-23T14:33:10.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.874+0000 7fb7f9000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fb7f40b2270 con 0x7fb7f406d450 2024-04-23T14:33:10.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.874+0000 7fb7f1e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb7f406d450 0x7fb7f406d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:10.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.874+0000 7fb7f1e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb7f406d450 0x7fb7f406d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60752/0 (socket says 172.21.15.100:60752) 2024-04-23T14:33:10.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.874+0000 7fb7f1e00640 1 -- 172.21.15.100:0/1482699453 learned_addr learned my addr 172.21.15.100:0/1482699453 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:10.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.874+0000 7fb7ebe00640 1 --1- 172.21.15.100:0/1482699453 >> v1:172.21.15.100:6789/0 conn(0x7fb7f40cea60 0x7fb7f40d0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:36260/0 (socket says 172.21.15.100:36260) 2024-04-23T14:33:10.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.878+0000 7fb7f1e00640 1 -- 172.21.15.100:0/1482699453 >> v1:172.21.15.151:6789/0 conn(0x7fb7f40cd460 legacy=0x7fb7f40b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:10.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.878+0000 7fb7f1e00640 1 -- 172.21.15.100:0/1482699453 >> v1:172.21.15.100:6789/0 conn(0x7fb7f40cea60 legacy=0x7fb7f40d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:10.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.878+0000 7fb7f1e00640 1 -- 172.21.15.100:0/1482699453 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f40b2550 con 0x7fb7f406d450 2024-04-23T14:33:10.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.878+0000 7fb7f1e00640 1 --2- 172.21.15.100:0/1482699453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7f406d450 0x7fb7f406d830 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009870 tx=0x7fb7e401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb93e01bc4709e3a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:10.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.878+0000 7fb7f1400640 1 -- 172.21.15.100:0/1482699453 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40b0070 con 0x7fb7f406d450 2024-04-23T14:33:10.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.878+0000 7fb7f1400640 1 -- 172.21.15.100:0/1482699453 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e40ab600 con 0x7fb7f406d450 2024-04-23T14:33:10.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.878+0000 7fb7f1400640 1 -- 172.21.15.100:0/1482699453 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40ab9d0 con 0x7fb7f406d450 2024-04-23T14:33:10.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.878+0000 7fb7f9000640 1 -- 172.21.15.100:0/1482699453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7f406d450 msgr2=0x7fb7f406d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:10.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.878+0000 7fb7f9000640 1 --2- 172.21.15.100:0/1482699453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7f406d450 0x7fb7f406d830 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009870 tx=0x7fb7e401c4d0 comp rx=0 tx=0).stop 2024-04-23T14:33:10.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.878+0000 7fb7f9000640 1 -- 172.21.15.100:0/1482699453 shutdown_connections 2024-04-23T14:33:10.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.878+0000 7fb7f9000640 1 --2- 172.21.15.100:0/1482699453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7f406d450 0x7fb7f406d830 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:10.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.878+0000 7fb7f9000640 1 -- 172.21.15.100:0/1482699453 >> 172.21.15.100:0/1482699453 conn(0x7fb7f40cee80 msgr2=0x7fb7f40cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:10.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.878+0000 7fb7f9000640 1 -- 172.21.15.100:0/1482699453 shutdown_connections 2024-04-23T14:33:10.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.878+0000 7fb7f9000640 1 -- 172.21.15.100:0/1482699453 wait complete. 2024-04-23T14:33:10.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7f9000640 1 Processor -- start 2024-04-23T14:33:10.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7f9000640 1 -- start start 2024-04-23T14:33:10.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7f9000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7f406d450 0x7fb7f40ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:10.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7f9000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7f40cd460 0x7fb7f4144560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:10.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7f9000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb7f40cea60 0x7fb7f41437d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:10.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7f9000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f416cd50 con 0x7fb7f406d450 2024-04-23T14:33:10.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7f9000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f416cbd0 con 0x7fb7f40cd460 2024-04-23T14:33:10.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7f9000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb7f416ced0 con 0x7fb7f40cea60 2024-04-23T14:33:10.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7f1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7f406d450 0x7fb7f40ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:10.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7f1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7f406d450 0x7fb7f40ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60758/0 (socket says 172.21.15.100:60758) 2024-04-23T14:33:10.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7f1e00640 1 -- 172.21.15.100:0/2366562685 learned_addr learned my addr 172.21.15.100:0/2366562685 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:10.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7ebe00640 1 --2- 172.21.15.100:0/2366562685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7f40cd460 0x7fb7f4144560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:10.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7f2800640 1 --2- 172.21.15.100:0/2366562685 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb7f40cea60 0x7fb7f41437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:10.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7f1e00640 1 -- 172.21.15.100:0/2366562685 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb7f40cea60 msgr2=0x7fb7f41437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:10.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7f1e00640 1 --2- 172.21.15.100:0/2366562685 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb7f40cea60 0x7fb7f41437d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:10.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7f1e00640 1 -- 172.21.15.100:0/2366562685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7f40cd460 msgr2=0x7fb7f4144560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:10.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7f1e00640 1 --2- 172.21.15.100:0/2366562685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7f40cd460 0x7fb7f4144560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:10.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7f1e00640 1 -- 172.21.15.100:0/2366562685 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f4142730 con 0x7fb7f406d450 2024-04-23T14:33:10.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7f1e00640 1 --2- 172.21.15.100:0/2366562685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7f406d450 0x7fb7f40ad870 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40190e0 tx=0x7fb7e4019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:10.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7ea000640 1 -- 172.21.15.100:0/2366562685 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40b0040 con 0x7fb7f406d450 2024-04-23T14:33:10.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7ea000640 1 -- 172.21.15.100:0/2366562685 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e40b9070 con 0x7fb7f406d450 2024-04-23T14:33:10.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7ea000640 1 -- 172.21.15.100:0/2366562685 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40b4880 con 0x7fb7f406d450 2024-04-23T14:33:10.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7f9000640 1 -- 172.21.15.100:0/2366562685 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f4142a10 con 0x7fb7f406d450 2024-04-23T14:33:10.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.882+0000 7fb7f9000640 1 -- 172.21.15.100:0/2366562685 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f4142fd0 con 0x7fb7f406d450 2024-04-23T14:33:10.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.886+0000 7fb7f9000640 1 -- 172.21.15.100:0/2366562685 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f4068ad0 con 0x7fb7f406d450 2024-04-23T14:33:10.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.886+0000 7fb7ea000640 1 -- 172.21.15.100:0/2366562685 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb7e40ab4c0 con 0x7fb7f406d450 2024-04-23T14:33:10.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.886+0000 7fb7ea000640 1 --2- 172.21.15.100:0/2366562685 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb7d0068380 0x7fb7d006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:10.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.886+0000 7fb7ea000640 1 -- 172.21.15.100:0/2366562685 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb7e4120bc0 con 0x7fb7f406d450 2024-04-23T14:33:10.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.886+0000 7fb7ebe00640 1 --2- 172.21.15.100:0/2366562685 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb7d0068380 0x7fb7d006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:10.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.886+0000 7fb7ebe00640 1 --2- 172.21.15.100:0/2366562685 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb7d0068380 0x7fb7d006a840 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40cc840 tx=0x7fb7dc007e50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:10.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:10.890+0000 7fb7ea000640 1 -- 172.21.15.100:0/2366562685 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb7e4016150 con 0x7fb7f406d450 2024-04-23T14:33:11.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.046+0000 7fb7f9000640 1 -- 172.21.15.100:0/2366562685 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7f40cf3e0 con 0x7fb7f406d450 2024-04-23T14:33:11.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.050+0000 7fb7ea000640 1 -- 172.21.15.100:0/2366562685 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18859 (secure 0 0 0) 0x7fb7e40f0ce0 con 0x7fb7f406d450 2024-04-23T14:33:11.050 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:11.051 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:49.784608+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:11.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.050+0000 7fb7f9000640 1 -- 172.21.15.100:0/2366562685 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb7d0068380 msgr2=0x7fb7d006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:11.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.050+0000 7fb7f9000640 1 --2- 172.21.15.100:0/2366562685 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb7d0068380 0x7fb7d006a840 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40cc840 tx=0x7fb7dc007e50 comp rx=0 tx=0).stop 2024-04-23T14:33:11.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.050+0000 7fb7f9000640 1 -- 172.21.15.100:0/2366562685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7f406d450 msgr2=0x7fb7f40ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:11.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.050+0000 7fb7f9000640 1 --2- 172.21.15.100:0/2366562685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7f406d450 0x7fb7f40ad870 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40190e0 tx=0x7fb7e4019270 comp rx=0 tx=0).stop 2024-04-23T14:33:11.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.050+0000 7fb7f9000640 1 -- 172.21.15.100:0/2366562685 shutdown_connections 2024-04-23T14:33:11.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.050+0000 7fb7f9000640 1 --2- 172.21.15.100:0/2366562685 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb7d0068380 0x7fb7d006a840 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.050+0000 7fb7f9000640 1 --2- 172.21.15.100:0/2366562685 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb7f40cea60 0x7fb7f41437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.050+0000 7fb7f9000640 1 --2- 172.21.15.100:0/2366562685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7f40cd460 0x7fb7f4144560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.050+0000 7fb7f9000640 1 --2- 172.21.15.100:0/2366562685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7f406d450 0x7fb7f40ad870 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.050+0000 7fb7f9000640 1 -- 172.21.15.100:0/2366562685 >> 172.21.15.100:0/2366562685 conn(0x7fb7f40cee80 msgr2=0x7fb7f4161600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:11.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.050+0000 7fb7f9000640 1 -- 172.21.15.100:0/2366562685 shutdown_connections 2024-04-23T14:33:11.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.050+0000 7fb7f9000640 1 -- 172.21.15.100:0/2366562685 wait complete. 2024-04-23T14:33:11.071 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:32:06.835560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:32:07.930489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-23T14:32:11.636814+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-23T14:33:11.072 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:33:11.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.206+0000 7f1618400640 1 Processor -- start 2024-04-23T14:33:11.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.206+0000 7f1618400640 1 -- start start 2024-04-23T14:33:11.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.210+0000 7f1618400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f161006bb20 0x7f16100ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:11.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.210+0000 7f1618400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f16100cd130 0x7f16100ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:11.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.210+0000 7f1618400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f161006cbb0 con 0x7f161015c7a0 2024-04-23T14:33:11.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.210+0000 7f1618400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f16100cd5a0 con 0x7f16100cd130 2024-04-23T14:33:11.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.210+0000 7f1618400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f161006d450 con 0x7f161006bb20 2024-04-23T14:33:11.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.210+0000 7f1616400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f16100cd130 0x7f16100ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:11.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.210+0000 7f1616400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f16100cd130 0x7f16100ad430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60766/0 (socket says 172.21.15.100:60766) 2024-04-23T14:33:11.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.210+0000 7f1616400640 1 -- 172.21.15.100:0/1967140027 learned_addr learned my addr 172.21.15.100:0/1967140027 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:11.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.210+0000 7f1615000640 1 --2- 172.21.15.100:0/1967140027 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f161006bb20 0x7f16100ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:11.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.210+0000 7f1616400640 1 -- 172.21.15.100:0/1967140027 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f161006bb20 msgr2=0x7f16100ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:11.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.210+0000 7f1616400640 1 --2- 172.21.15.100:0/1967140027 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f161006bb20 0x7f16100ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.210+0000 7f1616400640 1 -- 172.21.15.100:0/1967140027 >> v1:172.21.15.151:6789/0 conn(0x7f161015c7a0 legacy=0x7f16100af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:11.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.210+0000 7f1616400640 1 -- 172.21.15.100:0/1967140027 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161006ce90 con 0x7f16100cd130 2024-04-23T14:33:11.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.214+0000 7f1616400640 1 --2- 172.21.15.100:0/1967140027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16100cd130 0x7f16100ad430 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1608009e50 tx=0x7f160801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f563edf9140d08 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:11.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.214+0000 7f160fe00640 1 -- 172.21.15.100:0/1967140027 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16080a5660 con 0x7f16100cd130 2024-04-23T14:33:11.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.214+0000 7f160fe00640 1 -- 172.21.15.100:0/1967140027 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16080a5800 con 0x7f16100cd130 2024-04-23T14:33:11.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.214+0000 7f160fe00640 1 -- 172.21.15.100:0/1967140027 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16080a5bd0 con 0x7f16100cd130 2024-04-23T14:33:11.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.214+0000 7f1618400640 1 -- 172.21.15.100:0/1967140027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16100cd130 msgr2=0x7f16100ad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:11.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.214+0000 7f1618400640 1 --2- 172.21.15.100:0/1967140027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16100cd130 0x7f16100ad430 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1608009e50 tx=0x7f160801c9a0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.214+0000 7f1618400640 1 -- 172.21.15.100:0/1967140027 shutdown_connections 2024-04-23T14:33:11.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.214+0000 7f1618400640 1 --2- 172.21.15.100:0/1967140027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16100cd130 0x7f16100ad430 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.214+0000 7f1618400640 1 --2- 172.21.15.100:0/1967140027 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f161006bb20 0x7f16100ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.214+0000 7f1618400640 1 -- 172.21.15.100:0/1967140027 >> 172.21.15.100:0/1967140027 conn(0x7f16100b5200 msgr2=0x7f16100b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:11.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.214+0000 7f1618400640 1 -- 172.21.15.100:0/1967140027 shutdown_connections 2024-04-23T14:33:11.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.214+0000 7f1618400640 1 -- 172.21.15.100:0/1967140027 wait complete. 2024-04-23T14:33:11.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.214+0000 7f1618400640 1 Processor -- start 2024-04-23T14:33:11.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.214+0000 7f1618400640 1 -- start start 2024-04-23T14:33:11.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f1618400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f161006bb20 0x7f16100c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:11.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f1618400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16100cd130 0x7f16100c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:11.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f1618400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f161015c7a0 0x7f16100c5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:11.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f1618400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f16100ae510 con 0x7f161015c7a0 2024-04-23T14:33:11.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f1618400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f16100ae390 con 0x7f16100cd130 2024-04-23T14:33:11.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f1618400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f16100ae690 con 0x7f161006bb20 2024-04-23T14:33:11.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f1616400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f161015c7a0 0x7f16100c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:11.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f1616400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f161015c7a0 0x7f16100c5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60778/0 (socket says 172.21.15.100:60778) 2024-04-23T14:33:11.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f1616400640 1 -- 172.21.15.100:0/3416832341 learned_addr learned my addr 172.21.15.100:0/3416832341 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:11.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f1615000640 1 --2- 172.21.15.100:0/3416832341 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16100cd130 0x7f16100c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:11.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f1615a00640 1 --2- 172.21.15.100:0/3416832341 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f161006bb20 0x7f16100c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:11.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f1616400640 1 -- 172.21.15.100:0/3416832341 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f161006bb20 msgr2=0x7f16100c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:11.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f1616400640 1 --2- 172.21.15.100:0/3416832341 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f161006bb20 0x7f16100c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f1616400640 1 -- 172.21.15.100:0/3416832341 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16100cd130 msgr2=0x7f16100c44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:11.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f1616400640 1 --2- 172.21.15.100:0/3416832341 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16100cd130 0x7f16100c44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f1616400640 1 -- 172.21.15.100:0/3416832341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1610073a30 con 0x7f161015c7a0 2024-04-23T14:33:11.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f1616400640 1 --2- 172.21.15.100:0/3416832341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f161015c7a0 0x7f16100c5150 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f160801a5d0 tx=0x7f1608019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:11.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f160d600640 1 -- 172.21.15.100:0/3416832341 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16080bc020 con 0x7f161015c7a0 2024-04-23T14:33:11.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f160d600640 1 -- 172.21.15.100:0/3416832341 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16080b8880 con 0x7f161015c7a0 2024-04-23T14:33:11.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f1618400640 1 -- 172.21.15.100:0/3416832341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1610073d10 con 0x7f161015c7a0 2024-04-23T14:33:11.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f160d600640 1 -- 172.21.15.100:0/3416832341 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16080a5660 con 0x7f161015c7a0 2024-04-23T14:33:11.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f1618400640 1 -- 172.21.15.100:0/3416832341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1610098e10 con 0x7f161015c7a0 2024-04-23T14:33:11.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f1618400640 1 -- 172.21.15.100:0/3416832341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1610068ad0 con 0x7f161015c7a0 2024-04-23T14:33:11.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.218+0000 7f160d600640 1 -- 172.21.15.100:0/3416832341 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f16080bf020 con 0x7f161015c7a0 2024-04-23T14:33:11.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.222+0000 7f160d600640 1 --2- 172.21.15.100:0/3416832341 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f15f0068380 0x7f15f006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:11.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.222+0000 7f160d600640 1 -- 172.21.15.100:0/3416832341 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1608121e00 con 0x7f161015c7a0 2024-04-23T14:33:11.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.222+0000 7f1615a00640 1 --2- 172.21.15.100:0/3416832341 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f15f0068380 0x7f15f006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:11.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.222+0000 7f1615a00640 1 --2- 172.21.15.100:0/3416832341 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f15f0068380 0x7f15f006a840 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f16101506a0 tx=0x7f1604009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:11.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.226+0000 7f160d600640 1 -- 172.21.15.100:0/3416832341 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f16080c6030 con 0x7f161015c7a0 2024-04-23T14:33:11.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.382+0000 7f1618400640 1 -- 172.21.15.100:0/3416832341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1610074f20 con 0x7f161015c7a0 2024-04-23T14:33:11.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.382+0000 7f160d600640 1 -- 172.21.15.100:0/3416832341 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f1608016620 con 0x7f161015c7a0 2024-04-23T14:33:11.383 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:33:11.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.382+0000 7f1618400640 1 -- 172.21.15.100:0/3416832341 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f15f0068380 msgr2=0x7f15f006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:11.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.382+0000 7f1618400640 1 --2- 172.21.15.100:0/3416832341 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f15f0068380 0x7f15f006a840 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f16101506a0 tx=0x7f1604009040 comp rx=0 tx=0).stop 2024-04-23T14:33:11.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.382+0000 7f1618400640 1 -- 172.21.15.100:0/3416832341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f161015c7a0 msgr2=0x7f16100c5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:11.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.382+0000 7f1618400640 1 --2- 172.21.15.100:0/3416832341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f161015c7a0 0x7f16100c5150 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f160801a5d0 tx=0x7f1608019740 comp rx=0 tx=0).stop 2024-04-23T14:33:11.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.382+0000 7f1618400640 1 -- 172.21.15.100:0/3416832341 shutdown_connections 2024-04-23T14:33:11.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.382+0000 7f1618400640 1 --2- 172.21.15.100:0/3416832341 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f15f0068380 0x7f15f006a840 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.382+0000 7f1618400640 1 --2- 172.21.15.100:0/3416832341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f161015c7a0 0x7f16100c5150 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.382+0000 7f1618400640 1 --2- 172.21.15.100:0/3416832341 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16100cd130 0x7f16100c44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.382+0000 7f1618400640 1 --2- 172.21.15.100:0/3416832341 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f161006bb20 0x7f16100c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.382+0000 7f1618400640 1 -- 172.21.15.100:0/3416832341 >> 172.21.15.100:0/3416832341 conn(0x7f16100b5200 msgr2=0x7f16100cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:11.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.382+0000 7f1618400640 1 -- 172.21.15.100:0/3416832341 shutdown_connections 2024-04-23T14:33:11.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.382+0000 7f1618400640 1 -- 172.21.15.100:0/3416832341 wait complete. 2024-04-23T14:33:11.404 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:33:11.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.538+0000 7f211f000640 1 Processor -- start 2024-04-23T14:33:11.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.538+0000 7f211f000640 1 -- start start 2024-04-23T14:33:11.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.542+0000 7f211f000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f211806bb20 0x7f21180ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:11.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.542+0000 7f211f000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f21180cd130 0x7f21180ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:11.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.542+0000 7f211f000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f211806cbb0 con 0x7f211815c7a0 2024-04-23T14:33:11.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.542+0000 7f211f000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f21180cd5a0 con 0x7f21180cd130 2024-04-23T14:33:11.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.542+0000 7f211f000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f211806d450 con 0x7f211806bb20 2024-04-23T14:33:11.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.542+0000 7f211d000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f21180cd130 0x7f21180ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:11.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.542+0000 7f211d000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f21180cd130 0x7f21180ad430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60786/0 (socket says 172.21.15.100:60786) 2024-04-23T14:33:11.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.542+0000 7f211d000640 1 -- 172.21.15.100:0/1306048818 learned_addr learned my addr 172.21.15.100:0/1306048818 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:11.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.542+0000 7f210f400640 1 --2- 172.21.15.100:0/1306048818 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f211806bb20 0x7f21180ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:11.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.542+0000 7f211d000640 1 -- 172.21.15.100:0/1306048818 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f211806bb20 msgr2=0x7f21180ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:11.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.542+0000 7f211d000640 1 --2- 172.21.15.100:0/1306048818 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f211806bb20 0x7f21180ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.542+0000 7f211d000640 1 -- 172.21.15.100:0/1306048818 >> v1:172.21.15.151:6789/0 conn(0x7f211815c7a0 legacy=0x7f21180af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:11.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.542+0000 7f211d000640 1 -- 172.21.15.100:0/1306048818 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211806ce90 con 0x7f21180cd130 2024-04-23T14:33:11.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f211d000640 1 --2- 172.21.15.100:0/1306048818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21180cd130 0x7f21180ad430 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2110009870 tx=0x7f211001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45fa77136589395 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:11.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f2117400640 1 -- 172.21.15.100:0/1306048818 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21100b0070 con 0x7f21180cd130 2024-04-23T14:33:11.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f2117400640 1 -- 172.21.15.100:0/1306048818 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21100ab600 con 0x7f21180cd130 2024-04-23T14:33:11.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f2117400640 1 -- 172.21.15.100:0/1306048818 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21100ab9d0 con 0x7f21180cd130 2024-04-23T14:33:11.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f211f000640 1 -- 172.21.15.100:0/1306048818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21180cd130 msgr2=0x7f21180ad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:11.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f211f000640 1 --2- 172.21.15.100:0/1306048818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21180cd130 0x7f21180ad430 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2110009870 tx=0x7f211001c4d0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f211f000640 1 -- 172.21.15.100:0/1306048818 shutdown_connections 2024-04-23T14:33:11.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f211f000640 1 --2- 172.21.15.100:0/1306048818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21180cd130 0x7f21180ad430 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f211f000640 1 --2- 172.21.15.100:0/1306048818 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f211806bb20 0x7f21180ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f211f000640 1 -- 172.21.15.100:0/1306048818 >> 172.21.15.100:0/1306048818 conn(0x7f21180b5200 msgr2=0x7f21180b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:11.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f211f000640 1 -- 172.21.15.100:0/1306048818 shutdown_connections 2024-04-23T14:33:11.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f211f000640 1 -- 172.21.15.100:0/1306048818 wait complete. 2024-04-23T14:33:11.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f211f000640 1 Processor -- start 2024-04-23T14:33:11.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f211f000640 1 -- start start 2024-04-23T14:33:11.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f211f000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f211806bb20 0x7f2118153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:11.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f211f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21180cd130 0x7f2118157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:11.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f211f000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f211815c7a0 0x7f2118157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:11.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f211f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f21180ae510 con 0x7f21180cd130 2024-04-23T14:33:11.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f211f000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f21180ae390 con 0x7f211815c7a0 2024-04-23T14:33:11.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f211f000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f21180ae690 con 0x7f211806bb20 2024-04-23T14:33:11.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f210f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21180cd130 0x7f2118157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:11.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f210f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21180cd130 0x7f2118157950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60792/0 (socket says 172.21.15.100:60792) 2024-04-23T14:33:11.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.546+0000 7f210f400640 1 -- 172.21.15.100:0/1418448060 learned_addr learned my addr 172.21.15.100:0/1418448060 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:11.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.550+0000 7f211d000640 1 --2- 172.21.15.100:0/1418448060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f211815c7a0 0x7f2118157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:11.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.550+0000 7f2117e00640 1 --2- 172.21.15.100:0/1418448060 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f211806bb20 0x7f2118153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:11.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.550+0000 7f210f400640 1 -- 172.21.15.100:0/1418448060 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f211806bb20 msgr2=0x7f2118153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:11.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.550+0000 7f210f400640 1 --2- 172.21.15.100:0/1418448060 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f211806bb20 0x7f2118153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.550+0000 7f210f400640 1 -- 172.21.15.100:0/1418448060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f211815c7a0 msgr2=0x7f2118157e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:11.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.550+0000 7f210f400640 1 --2- 172.21.15.100:0/1418448060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f211815c7a0 0x7f2118157e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.550+0000 7f210f400640 1 -- 172.21.15.100:0/1418448060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2118156c50 con 0x7f21180cd130 2024-04-23T14:33:11.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.550+0000 7f210f400640 1 --2- 172.21.15.100:0/1418448060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21180cd130 0x7f2118157950 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f210000e870 tx=0x7f210000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:11.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.550+0000 7f2114c00640 1 -- 172.21.15.100:0/1418448060 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100017070 con 0x7f21180cd130 2024-04-23T14:33:11.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.550+0000 7f2114c00640 1 -- 172.21.15.100:0/1418448060 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f210000c940 con 0x7f21180cd130 2024-04-23T14:33:11.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.550+0000 7f211f000640 1 -- 172.21.15.100:0/1418448060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2118156f30 con 0x7f21180cd130 2024-04-23T14:33:11.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.550+0000 7f2114c00640 1 -- 172.21.15.100:0/1418448060 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210000cd30 con 0x7f21180cd130 2024-04-23T14:33:11.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.550+0000 7f211f000640 1 -- 172.21.15.100:0/1418448060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21181574f0 con 0x7f21180cd130 2024-04-23T14:33:11.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.550+0000 7f211f000640 1 -- 172.21.15.100:0/1418448060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2118068ad0 con 0x7f21180cd130 2024-04-23T14:33:11.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.550+0000 7f2114c00640 1 -- 172.21.15.100:0/1418448060 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f210001a070 con 0x7f21180cd130 2024-04-23T14:33:11.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.550+0000 7f2114c00640 1 --2- 172.21.15.100:0/1418448060 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f20f4068380 0x7f20f406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:11.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.550+0000 7f2114c00640 1 -- 172.21.15.100:0/1418448060 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f21000870c0 con 0x7f21180cd130 2024-04-23T14:33:11.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.550+0000 7f2117e00640 1 --2- 172.21.15.100:0/1418448060 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f20f4068380 0x7f20f406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:11.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.554+0000 7f2117e00640 1 --2- 172.21.15.100:0/1418448060 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f20f4068380 0x7f20f406a840 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2108001d50 tx=0x7f2108009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:11.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.554+0000 7f2114c00640 1 -- 172.21.15.100:0/1418448060 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2100056b00 con 0x7f21180cd130 2024-04-23T14:33:11.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.710+0000 7f211f000640 1 -- 172.21.15.100:0/1418448060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f21180cc4d0 con 0x7f21180cd130 2024-04-23T14:33:11.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.710+0000 7f2114c00640 1 -- 172.21.15.100:0/1418448060 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f210005ae00 con 0x7f21180cd130 2024-04-23T14:33:11.714 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:11.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.714+0000 7f211f000640 1 -- 172.21.15.100:0/1418448060 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f20f4068380 msgr2=0x7f20f406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:11.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.714+0000 7f211f000640 1 --2- 172.21.15.100:0/1418448060 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f20f4068380 0x7f20f406a840 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2108001d50 tx=0x7f2108009040 comp rx=0 tx=0).stop 2024-04-23T14:33:11.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.714+0000 7f211f000640 1 -- 172.21.15.100:0/1418448060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21180cd130 msgr2=0x7f2118157950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:11.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.714+0000 7f211f000640 1 --2- 172.21.15.100:0/1418448060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21180cd130 0x7f2118157950 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f210000e870 tx=0x7f210000ec70 comp rx=0 tx=0).stop 2024-04-23T14:33:11.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.714+0000 7f211f000640 1 -- 172.21.15.100:0/1418448060 shutdown_connections 2024-04-23T14:33:11.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.714+0000 7f211f000640 1 --2- 172.21.15.100:0/1418448060 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f20f4068380 0x7f20f406a840 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.714+0000 7f211f000640 1 --2- 172.21.15.100:0/1418448060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f211815c7a0 0x7f2118157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.714+0000 7f211f000640 1 --2- 172.21.15.100:0/1418448060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21180cd130 0x7f2118157950 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.714+0000 7f211f000640 1 --2- 172.21.15.100:0/1418448060 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f211806bb20 0x7f2118153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.714+0000 7f211f000640 1 -- 172.21.15.100:0/1418448060 >> 172.21.15.100:0/1418448060 conn(0x7f21180b5200 msgr2=0x7f21180b55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:11.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.714+0000 7f211f000640 1 -- 172.21.15.100:0/1418448060 shutdown_connections 2024-04-23T14:33:11.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.714+0000 7f211f000640 1 -- 172.21.15.100:0/1418448060 wait complete. 2024-04-23T14:33:11.735 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:33:11.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.870+0000 7fc370200640 1 Processor -- start 2024-04-23T14:33:11.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.870+0000 7fc370200640 1 -- start start 2024-04-23T14:33:11.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.870+0000 7fc370200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc36806c6c0 0x7fc3680ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:11.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.870+0000 7fc370200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc3680ce620 0x7fc3680add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:11.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.870+0000 7fc370200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc36806dd60 con 0x7fc36806f8c0 2024-04-23T14:33:11.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.870+0000 7fc370200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc3680ad050 con 0x7fc3680ce620 2024-04-23T14:33:11.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.870+0000 7fc370200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fc3680ad8f0 con 0x7fc36806c6c0 2024-04-23T14:33:11.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.874+0000 7fc36e200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc3680ce620 0x7fc3680add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:11.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.874+0000 7fc36ce00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc36806c6c0 0x7fc3680ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:11.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.874+0000 7fc36e200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc3680ce620 0x7fc3680add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60804/0 (socket says 172.21.15.100:60804) 2024-04-23T14:33:11.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.874+0000 7fc36e200640 1 -- 172.21.15.100:0/455560843 learned_addr learned my addr 172.21.15.100:0/455560843 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:11.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.874+0000 7fc36ce00640 1 -- 172.21.15.100:0/455560843 >> v2:172.21.15.100:3300/0 conn(0x7fc3680ce620 msgr2=0x7fc3680add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:11.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.874+0000 7fc36ce00640 1 --2- 172.21.15.100:0/455560843 >> v2:172.21.15.100:3300/0 conn(0x7fc3680ce620 0x7fc3680add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.874+0000 7fc36ce00640 1 -- 172.21.15.100:0/455560843 >> v1:172.21.15.100:6789/0 conn(0x7fc36806f8c0 legacy=0x7fc3680d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:11.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.874+0000 7fc36ce00640 1 -- 172.21.15.100:0/455560843 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3680d09c0 con 0x7fc36806c6c0 2024-04-23T14:33:11.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.874+0000 7fc36e200640 1 --2- 172.21.15.100:0/455560843 >> v2:172.21.15.100:3300/0 conn(0x7fc3680ce620 0x7fc3680add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:33:11.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.874+0000 7fc36ce00640 1 --2- 172.21.15.100:0/455560843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc36806c6c0 0x7fc3680ce0e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc3680d00c0 tx=0x7fc35401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9b4e8f2e1830ea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:11.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.878+0000 7fc367e00640 1 -- 172.21.15.100:0/455560843 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3540b1070 con 0x7fc36806c6c0 2024-04-23T14:33:11.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.878+0000 7fc367e00640 1 -- 172.21.15.100:0/455560843 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3540ac5f0 con 0x7fc36806c6c0 2024-04-23T14:33:11.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.878+0000 7fc367e00640 1 -- 172.21.15.100:0/455560843 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3540ac9c0 con 0x7fc36806c6c0 2024-04-23T14:33:11.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.878+0000 7fc370200640 1 -- 172.21.15.100:0/455560843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc36806c6c0 msgr2=0x7fc3680ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:11.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.878+0000 7fc370200640 1 --2- 172.21.15.100:0/455560843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc36806c6c0 0x7fc3680ce0e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc3680d00c0 tx=0x7fc35401c480 comp rx=0 tx=0).stop 2024-04-23T14:33:11.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.878+0000 7fc370200640 1 -- 172.21.15.100:0/455560843 shutdown_connections 2024-04-23T14:33:11.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.878+0000 7fc370200640 1 --2- 172.21.15.100:0/455560843 >> v2:172.21.15.100:3300/0 conn(0x7fc3680ce620 0x7fc3680add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.878+0000 7fc370200640 1 --2- 172.21.15.100:0/455560843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc36806c6c0 0x7fc3680ce0e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.878+0000 7fc370200640 1 -- 172.21.15.100:0/455560843 >> 172.21.15.100:0/455560843 conn(0x7fc3680b5200 msgr2=0x7fc3680b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:11.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.878+0000 7fc370200640 1 -- 172.21.15.100:0/455560843 shutdown_connections 2024-04-23T14:33:11.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.878+0000 7fc370200640 1 -- 172.21.15.100:0/455560843 wait complete. 2024-04-23T14:33:11.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc370200640 1 Processor -- start 2024-04-23T14:33:11.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc370200640 1 -- start start 2024-04-23T14:33:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc370200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc36806c6c0 0x7fc3680ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc370200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc36806f8c0 0x7fc3681561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc370200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc3680ce620 0x7fc3681552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc370200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc3680aee60 con 0x7fc36806f8c0 2024-04-23T14:33:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc370200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc3680aece0 con 0x7fc36806c6c0 2024-04-23T14:33:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc370200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc368168400 con 0x7fc3680ce620 2024-04-23T14:33:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc36ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc36806f8c0 0x7fc3681561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc36ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc36806f8c0 0x7fc3681561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60808/0 (socket says 172.21.15.100:60808) 2024-04-23T14:33:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc36ce00640 1 -- 172.21.15.100:0/2144053529 learned_addr learned my addr 172.21.15.100:0/2144053529 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc36d800640 1 --2- 172.21.15.100:0/2144053529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc36806c6c0 0x7fc3680ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc36e200640 1 --2- 172.21.15.100:0/2144053529 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc3680ce620 0x7fc3681552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc36ce00640 1 -- 172.21.15.100:0/2144053529 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc3680ce620 msgr2=0x7fc3681552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:11.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc36ce00640 1 --2- 172.21.15.100:0/2144053529 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc3680ce620 0x7fc3681552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc36ce00640 1 -- 172.21.15.100:0/2144053529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc36806c6c0 msgr2=0x7fc3680ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:11.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc36ce00640 1 --2- 172.21.15.100:0/2144053529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc36806c6c0 0x7fc3680ca180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:11.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc36ce00640 1 -- 172.21.15.100:0/2144053529 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3680c3f70 con 0x7fc36806f8c0 2024-04-23T14:33:11.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc36e200640 1 --2- 172.21.15.100:0/2144053529 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc3680ce620 0x7fc3681552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:11.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc36ce00640 1 --2- 172.21.15.100:0/2144053529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc36806f8c0 0x7fc3681561c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc35401ab30 tx=0x7fc354019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:11.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc365600640 1 -- 172.21.15.100:0/2144053529 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3540b9020 con 0x7fc36806f8c0 2024-04-23T14:33:11.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc365600640 1 -- 172.21.15.100:0/2144053529 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3540b5410 con 0x7fc36806f8c0 2024-04-23T14:33:11.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc370200640 1 -- 172.21.15.100:0/2144053529 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3680c4250 con 0x7fc36806f8c0 2024-04-23T14:33:11.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc365600640 1 -- 172.21.15.100:0/2144053529 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3540b1040 con 0x7fc36806f8c0 2024-04-23T14:33:11.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.882+0000 7fc370200640 1 -- 172.21.15.100:0/2144053529 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3680c4810 con 0x7fc36806f8c0 2024-04-23T14:33:11.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.886+0000 7fc370200640 1 -- 172.21.15.100:0/2144053529 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc368068ad0 con 0x7fc36806f8c0 2024-04-23T14:33:11.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.886+0000 7fc365600640 1 -- 172.21.15.100:0/2144053529 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc3540bc020 con 0x7fc36806f8c0 2024-04-23T14:33:11.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.886+0000 7fc365600640 1 --2- 172.21.15.100:0/2144053529 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc3480683b0 0x7fc34806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:11.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.886+0000 7fc365600640 1 -- 172.21.15.100:0/2144053529 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc3541205d0 con 0x7fc36806f8c0 2024-04-23T14:33:11.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.886+0000 7fc36d800640 1 --2- 172.21.15.100:0/2144053529 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc3480683b0 0x7fc34806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:11.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.886+0000 7fc36d800640 1 --2- 172.21.15.100:0/2144053529 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc3480683b0 0x7fc34806a870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc35c001810 tx=0x7fc35c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:11.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:11.890+0000 7fc365600640 1 -- 172.21.15.100:0/2144053529 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc3540c3030 con 0x7fc36806f8c0 2024-04-23T14:33:12.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.046+0000 7fc370200640 1 -- 172.21.15.100:0/2144053529 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc36806cc00 con 0x7fc36806f8c0 2024-04-23T14:33:12.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.046+0000 7fc365600640 1 -- 172.21.15.100:0/2144053529 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fc3540a9070 con 0x7fc36806f8c0 2024-04-23T14:33:12.048 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:33:12.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.046+0000 7fc370200640 1 -- 172.21.15.100:0/2144053529 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc3480683b0 msgr2=0x7fc34806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:12.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.046+0000 7fc370200640 1 --2- 172.21.15.100:0/2144053529 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc3480683b0 0x7fc34806a870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc35c001810 tx=0x7fc35c009040 comp rx=0 tx=0).stop 2024-04-23T14:33:12.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.046+0000 7fc370200640 1 -- 172.21.15.100:0/2144053529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc36806f8c0 msgr2=0x7fc3681561c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:12.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.046+0000 7fc370200640 1 --2- 172.21.15.100:0/2144053529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc36806f8c0 0x7fc3681561c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc35401ab30 tx=0x7fc354019220 comp rx=0 tx=0).stop 2024-04-23T14:33:12.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.046+0000 7fc370200640 1 -- 172.21.15.100:0/2144053529 shutdown_connections 2024-04-23T14:33:12.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.046+0000 7fc370200640 1 --2- 172.21.15.100:0/2144053529 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc3480683b0 0x7fc34806a870 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.046+0000 7fc370200640 1 --2- 172.21.15.100:0/2144053529 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc3680ce620 0x7fc3681552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.046+0000 7fc370200640 1 --2- 172.21.15.100:0/2144053529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc36806f8c0 0x7fc3681561c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.046+0000 7fc370200640 1 --2- 172.21.15.100:0/2144053529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc36806c6c0 0x7fc3680ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.046+0000 7fc370200640 1 -- 172.21.15.100:0/2144053529 >> 172.21.15.100:0/2144053529 conn(0x7fc3680b5200 msgr2=0x7fc3680d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:12.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.050+0000 7fc370200640 1 -- 172.21.15.100:0/2144053529 shutdown_connections 2024-04-23T14:33:12.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.050+0000 7fc370200640 1 -- 172.21.15.100:0/2144053529 wait complete. 2024-04-23T14:33:12.069 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-23T14:33:12.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.206+0000 7fcd37800640 1 Processor -- start 2024-04-23T14:33:12.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.206+0000 7fcd37800640 1 -- start start 2024-04-23T14:33:12.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.206+0000 7fcd37800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcd3015e830 0x7fcd3015ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:12.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.206+0000 7fcd37800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcd300cdd60 0x7fcd300cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:12.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.206+0000 7fcd37800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd3015f320 con 0x7fcd300cfee0 2024-04-23T14:33:12.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.206+0000 7fcd37800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fcd300d0350 con 0x7fcd3015e830 2024-04-23T14:33:12.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.206+0000 7fcd37800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fcd3015f5e0 con 0x7fcd300cdd60 2024-04-23T14:33:12.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.206+0000 7fcd34e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcd3015e830 0x7fcd3015ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:12.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.206+0000 7fcd35800640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fcd300cfee0 0x7fcd30162320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:36272/0 (socket says 172.21.15.100:36272) 2024-04-23T14:33:12.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.206+0000 7fcd35800640 1 -- 172.21.15.100:0/1979808506 learned_addr learned my addr 172.21.15.100:0/1979808506 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:12.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.206+0000 7fcd27e00640 1 --2- 172.21.15.100:0/1979808506 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcd300cdd60 0x7fcd300cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:12.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.206+0000 7fcd34e00640 1 -- 172.21.15.100:0/1979808506 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcd300cdd60 msgr2=0x7fcd300cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:12.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.206+0000 7fcd34e00640 1 --2- 172.21.15.100:0/1979808506 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcd300cdd60 0x7fcd300cf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.206+0000 7fcd34e00640 1 -- 172.21.15.100:0/1979808506 >> v1:172.21.15.100:6789/0 conn(0x7fcd300cfee0 legacy=0x7fcd30162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:12.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.206+0000 7fcd34e00640 1 -- 172.21.15.100:0/1979808506 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3015f8c0 con 0x7fcd3015e830 2024-04-23T14:33:12.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.210+0000 7fcd34e00640 1 --2- 172.21.15.100:0/1979808506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd3015e830 0x7fcd3015ec10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcd200097c0 tx=0x7fcd2001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdab834797f8eae server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:12.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.210+0000 7fcd2fe00640 1 -- 172.21.15.100:0/1979808506 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd200b0070 con 0x7fcd3015e830 2024-04-23T14:33:12.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.210+0000 7fcd2fe00640 1 -- 172.21.15.100:0/1979808506 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd200ab600 con 0x7fcd3015e830 2024-04-23T14:33:12.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.210+0000 7fcd2fe00640 1 -- 172.21.15.100:0/1979808506 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd200ab9d0 con 0x7fcd3015e830 2024-04-23T14:33:12.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.210+0000 7fcd37800640 1 -- 172.21.15.100:0/1979808506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd3015e830 msgr2=0x7fcd3015ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:12.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.210+0000 7fcd37800640 1 --2- 172.21.15.100:0/1979808506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd3015e830 0x7fcd3015ec10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcd200097c0 tx=0x7fcd2001c4a0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.210+0000 7fcd37800640 1 -- 172.21.15.100:0/1979808506 shutdown_connections 2024-04-23T14:33:12.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.210+0000 7fcd37800640 1 --2- 172.21.15.100:0/1979808506 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcd300cdd60 0x7fcd300cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.210+0000 7fcd37800640 1 --2- 172.21.15.100:0/1979808506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd3015e830 0x7fcd3015ec10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.210+0000 7fcd37800640 1 -- 172.21.15.100:0/1979808506 >> 172.21.15.100:0/1979808506 conn(0x7fcd300704d0 msgr2=0x7fcd300708e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:12.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.210+0000 7fcd37800640 1 -- 172.21.15.100:0/1979808506 shutdown_connections 2024-04-23T14:33:12.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.210+0000 7fcd37800640 1 -- 172.21.15.100:0/1979808506 wait complete. 2024-04-23T14:33:12.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd37800640 1 Processor -- start 2024-04-23T14:33:12.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd37800640 1 -- start start 2024-04-23T14:33:12.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd37800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd300cdd60 0x7fcd30156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:12.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd37800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd300cfee0 0x7fcd30158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:12.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd37800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcd3015e830 0x7fcd3015c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:12.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd37800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3016b610 con 0x7fcd300cfee0 2024-04-23T14:33:12.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd37800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3016b490 con 0x7fcd300cdd60 2024-04-23T14:33:12.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd37800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fcd3016b790 con 0x7fcd3015e830 2024-04-23T14:33:12.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd27e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd300cfee0 0x7fcd30158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:12.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd27e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd300cfee0 0x7fcd30158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60816/0 (socket says 172.21.15.100:60816) 2024-04-23T14:33:12.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd27e00640 1 -- 172.21.15.100:0/1560184550 learned_addr learned my addr 172.21.15.100:0/1560184550 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:12.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd35800640 1 --2- 172.21.15.100:0/1560184550 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcd3015e830 0x7fcd3015c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:12.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd34e00640 1 --2- 172.21.15.100:0/1560184550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd300cdd60 0x7fcd30156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:12.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd27e00640 1 -- 172.21.15.100:0/1560184550 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcd3015e830 msgr2=0x7fcd3015c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:12.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd27e00640 1 --2- 172.21.15.100:0/1560184550 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcd3015e830 0x7fcd3015c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd27e00640 1 -- 172.21.15.100:0/1560184550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd300cdd60 msgr2=0x7fcd30156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:12.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd27e00640 1 --2- 172.21.15.100:0/1560184550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd300cdd60 0x7fcd30156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd27e00640 1 -- 172.21.15.100:0/1560184550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd300b2a00 con 0x7fcd300cfee0 2024-04-23T14:33:12.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd27e00640 1 --2- 172.21.15.100:0/1560184550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd300cfee0 0x7fcd30158920 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800d830 tx=0x7fcd1800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:12.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd2d600640 1 -- 172.21.15.100:0/1560184550 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1800b6c0 con 0x7fcd300cfee0 2024-04-23T14:33:12.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd2d600640 1 -- 172.21.15.100:0/1560184550 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd18019070 con 0x7fcd300cfee0 2024-04-23T14:33:12.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd2d600640 1 -- 172.21.15.100:0/1560184550 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1800ba60 con 0x7fcd300cfee0 2024-04-23T14:33:12.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd37800640 1 -- 172.21.15.100:0/1560184550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd300b2440 con 0x7fcd300cfee0 2024-04-23T14:33:12.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd37800640 1 -- 172.21.15.100:0/1560184550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd3015dd50 con 0x7fcd300cfee0 2024-04-23T14:33:12.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.214+0000 7fcd37800640 1 -- 172.21.15.100:0/1560184550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd30068ad0 con 0x7fcd300cfee0 2024-04-23T14:33:12.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.218+0000 7fcd2d600640 1 -- 172.21.15.100:0/1560184550 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcd1801e460 con 0x7fcd300cfee0 2024-04-23T14:33:12.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.218+0000 7fcd2d600640 1 --2- 172.21.15.100:0/1560184550 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcd0c068380 0x7fcd0c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:12.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.218+0000 7fcd2d600640 1 -- 172.21.15.100:0/1560184550 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fcd18087520 con 0x7fcd300cfee0 2024-04-23T14:33:12.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.218+0000 7fcd34e00640 1 --2- 172.21.15.100:0/1560184550 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcd0c068380 0x7fcd0c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:12.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.218+0000 7fcd34e00640 1 --2- 172.21.15.100:0/1560184550 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcd0c068380 0x7fcd0c06a840 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcd200ad000 tx=0x7fcd20008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:12.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.222+0000 7fcd2d600640 1 -- 172.21.15.100:0/1560184550 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcd18057640 con 0x7fcd300cfee0 2024-04-23T14:33:12.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.378+0000 7fcd37800640 1 -- 172.21.15.100:0/1560184550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fcd3015c150 con 0x7fcd300cfee0 2024-04-23T14:33:12.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.378+0000 7fcd2d600640 1 -- 172.21.15.100:0/1560184550 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fcd18056f40 con 0x7fcd300cfee0 2024-04-23T14:33:12.381 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:12.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.382+0000 7fcd37800640 1 -- 172.21.15.100:0/1560184550 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcd0c068380 msgr2=0x7fcd0c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:12.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.382+0000 7fcd37800640 1 --2- 172.21.15.100:0/1560184550 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcd0c068380 0x7fcd0c06a840 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcd200ad000 tx=0x7fcd20008040 comp rx=0 tx=0).stop 2024-04-23T14:33:12.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.382+0000 7fcd37800640 1 -- 172.21.15.100:0/1560184550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd300cfee0 msgr2=0x7fcd30158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:12.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.382+0000 7fcd37800640 1 --2- 172.21.15.100:0/1560184550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd300cfee0 0x7fcd30158920 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800d830 tx=0x7fcd1800dcf0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.382+0000 7fcd37800640 1 -- 172.21.15.100:0/1560184550 shutdown_connections 2024-04-23T14:33:12.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.382+0000 7fcd37800640 1 --2- 172.21.15.100:0/1560184550 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcd0c068380 0x7fcd0c06a840 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.382+0000 7fcd37800640 1 --2- 172.21.15.100:0/1560184550 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcd3015e830 0x7fcd3015c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.382+0000 7fcd37800640 1 --2- 172.21.15.100:0/1560184550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd300cfee0 0x7fcd30158920 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.382+0000 7fcd37800640 1 --2- 172.21.15.100:0/1560184550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd300cdd60 0x7fcd30156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.382+0000 7fcd37800640 1 -- 172.21.15.100:0/1560184550 >> 172.21.15.100:0/1560184550 conn(0x7fcd300704d0 msgr2=0x7fcd300ae430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:12.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.382+0000 7fcd37800640 1 -- 172.21.15.100:0/1560184550 shutdown_connections 2024-04-23T14:33:12.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.382+0000 7fcd37800640 1 -- 172.21.15.100:0/1560184550 wait complete. 2024-04-23T14:33:12.401 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:33:12.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.534+0000 7f3649c00640 1 Processor -- start 2024-04-23T14:33:12.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.534+0000 7f3649c00640 1 -- start start 2024-04-23T14:33:12.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.538+0000 7f3649c00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f36440b3960 0x7f36440b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:12.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.538+0000 7f3649c00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f364406e5f0 con 0x7f364406c3e0 2024-04-23T14:33:12.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.538+0000 7f3649c00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f364406e8d0 con 0x7f3644159180 2024-04-23T14:33:12.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.538+0000 7f3649c00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f364406ebb0 con 0x7f36440b3960 2024-04-23T14:33:12.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.538+0000 7f3643400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f364406c3e0 0x7f36440b4420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:36278/0 (socket says 172.21.15.100:36278) 2024-04-23T14:33:12.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.538+0000 7f3643400640 1 -- 172.21.15.100:0/3696114100 learned_addr learned my addr 172.21.15.100:0/3696114100 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:12.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.538+0000 7f3642000640 1 --2- 172.21.15.100:0/3696114100 >> v2:172.21.15.151:3300/0 conn(0x7f36440b3960 0x7f36440b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:12.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.538+0000 7f3641600640 1 -- 172.21.15.100:0/3696114100 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1427388686 0 0) 0x7f364406e5f0 con 0x7f364406c3e0 2024-04-23T14:33:12.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.538+0000 7f3641600640 1 -- 172.21.15.100:0/3696114100 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3628003530 con 0x7f364406c3e0 2024-04-23T14:33:12.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.538+0000 7f3642000640 1 -- 172.21.15.100:0/3696114100 >> v1:172.21.15.151:6789/0 conn(0x7f3644159180 legacy=0x7f364406f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:12.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.538+0000 7f3642000640 1 -- 172.21.15.100:0/3696114100 >> v1:172.21.15.100:6789/0 conn(0x7f364406c3e0 legacy=0x7f36440b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:12.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.538+0000 7f3642000640 1 -- 172.21.15.100:0/3696114100 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364406ee90 con 0x7f36440b3960 2024-04-23T14:33:12.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.542+0000 7f3642000640 1 --2- 172.21.15.100:0/3696114100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36440b3960 0x7f36440b3e10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f36300097c0 tx=0x7f363001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4f015d0e893ac53 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:12.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.542+0000 7f3641600640 1 -- 172.21.15.100:0/3696114100 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36300aa620 con 0x7f36440b3960 2024-04-23T14:33:12.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.542+0000 7f3641600640 1 -- 172.21.15.100:0/3696114100 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36300af070 con 0x7f36440b3960 2024-04-23T14:33:12.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.542+0000 7f3641600640 1 -- 172.21.15.100:0/3696114100 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36300aa9c0 con 0x7f36440b3960 2024-04-23T14:33:12.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.542+0000 7f3649c00640 1 -- 172.21.15.100:0/3696114100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36440b3960 msgr2=0x7f36440b3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:12.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.542+0000 7f3649c00640 1 --2- 172.21.15.100:0/3696114100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36440b3960 0x7f36440b3e10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f36300097c0 tx=0x7f363001c4a0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.542+0000 7f3649c00640 1 -- 172.21.15.100:0/3696114100 shutdown_connections 2024-04-23T14:33:12.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.542+0000 7f3649c00640 1 --2- 172.21.15.100:0/3696114100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36440b3960 0x7f36440b3e10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.542+0000 7f3649c00640 1 -- 172.21.15.100:0/3696114100 >> 172.21.15.100:0/3696114100 conn(0x7f36440d0a50 msgr2=0x7f36440d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:12.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.542+0000 7f3649c00640 1 -- 172.21.15.100:0/3696114100 shutdown_connections 2024-04-23T14:33:12.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.542+0000 7f3649c00640 1 -- 172.21.15.100:0/3696114100 wait complete. 2024-04-23T14:33:12.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.542+0000 7f3649c00640 1 Processor -- start 2024-04-23T14:33:12.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.542+0000 7f3649c00640 1 -- start start 2024-04-23T14:33:12.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f3649c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364406c3e0 0x7f3644154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:12.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f3649c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f36440b3960 0x7f36440ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:12.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f3649c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3644159180 0x7f36440af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:12.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f3649c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f36440b4cb0 con 0x7f364406c3e0 2024-04-23T14:33:12.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f3649c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f36440b4b30 con 0x7f3644159180 2024-04-23T14:33:12.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f3649c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f364406c810 con 0x7f36440b3960 2024-04-23T14:33:12.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f3642a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364406c3e0 0x7f3644154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:12.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f3642a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364406c3e0 0x7f3644154740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60824/0 (socket says 172.21.15.100:60824) 2024-04-23T14:33:12.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f3642a00640 1 -- 172.21.15.100:0/141788532 learned_addr learned my addr 172.21.15.100:0/141788532 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:12.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f3642000640 1 --2- 172.21.15.100:0/141788532 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f36440b3960 0x7f36440ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:12.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f3643400640 1 --2- 172.21.15.100:0/141788532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3644159180 0x7f36440af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:12.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f3642a00640 1 -- 172.21.15.100:0/141788532 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f36440b3960 msgr2=0x7f36440ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:12.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f3642a00640 1 --2- 172.21.15.100:0/141788532 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f36440b3960 0x7f36440ae6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f3642a00640 1 -- 172.21.15.100:0/141788532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3644159180 msgr2=0x7f36440af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:12.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f3642a00640 1 --2- 172.21.15.100:0/141788532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3644159180 0x7f36440af620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f3642a00640 1 -- 172.21.15.100:0/141788532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36440b03c0 con 0x7f364406c3e0 2024-04-23T14:33:12.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f3642a00640 1 --2- 172.21.15.100:0/141788532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364406c3e0 0x7f3644154740 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f363400c760 tx=0x7f363400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:12.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f363aa00640 1 -- 172.21.15.100:0/141788532 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634016020 con 0x7f364406c3e0 2024-04-23T14:33:12.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f363aa00640 1 -- 172.21.15.100:0/141788532 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3634006d60 con 0x7f364406c3e0 2024-04-23T14:33:12.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f3649c00640 1 -- 172.21.15.100:0/141788532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36440b00e0 con 0x7f364406c3e0 2024-04-23T14:33:12.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f363aa00640 1 -- 172.21.15.100:0/141788532 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634003bc0 con 0x7f364406c3e0 2024-04-23T14:33:12.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f3649c00640 1 -- 172.21.15.100:0/141788532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36440aded0 con 0x7f364406c3e0 2024-04-23T14:33:12.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f3649c00640 1 -- 172.21.15.100:0/141788532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36440ae300 con 0x7f364406c3e0 2024-04-23T14:33:12.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.546+0000 7f363aa00640 1 -- 172.21.15.100:0/141788532 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f363400a850 con 0x7f364406c3e0 2024-04-23T14:33:12.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.550+0000 7f363aa00640 1 --2- 172.21.15.100:0/141788532 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3624068380 0x7f362406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:12.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.550+0000 7f363aa00640 1 -- 172.21.15.100:0/141788532 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f3634086ac0 con 0x7f364406c3e0 2024-04-23T14:33:12.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.550+0000 7f3642000640 1 --2- 172.21.15.100:0/141788532 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3624068380 0x7f362406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:12.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.550+0000 7f3642000640 1 --2- 172.21.15.100:0/141788532 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3624068380 0x7f362406a840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f36300ae000 tx=0x7f36300a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:12.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.554+0000 7f363aa00640 1 -- 172.21.15.100:0/141788532 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3634020020 con 0x7f364406c3e0 2024-04-23T14:33:12.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.722+0000 7f3649c00640 1 -- 172.21.15.100:0/141788532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3644068ad0 con 0x7f364406c3e0 2024-04-23T14:33:12.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.722+0000 7f363aa00640 1 -- 172.21.15.100:0/141788532 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f3634056c70 con 0x7f364406c3e0 2024-04-23T14:33:12.724 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-04-23T14:33:12.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.722+0000 7f3649c00640 1 -- 172.21.15.100:0/141788532 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3624068380 msgr2=0x7f362406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:12.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.722+0000 7f3649c00640 1 --2- 172.21.15.100:0/141788532 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3624068380 0x7f362406a840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f36300ae000 tx=0x7f36300a7000 comp rx=0 tx=0).stop 2024-04-23T14:33:12.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.722+0000 7f3649c00640 1 -- 172.21.15.100:0/141788532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364406c3e0 msgr2=0x7f3644154740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:12.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.722+0000 7f3649c00640 1 --2- 172.21.15.100:0/141788532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364406c3e0 0x7f3644154740 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f363400c760 tx=0x7f363400cc20 comp rx=0 tx=0).stop 2024-04-23T14:33:12.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.722+0000 7f3649c00640 1 -- 172.21.15.100:0/141788532 shutdown_connections 2024-04-23T14:33:12.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.722+0000 7f3649c00640 1 --2- 172.21.15.100:0/141788532 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3624068380 0x7f362406a840 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.722+0000 7f3649c00640 1 --2- 172.21.15.100:0/141788532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3644159180 0x7f36440af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.722+0000 7f3649c00640 1 --2- 172.21.15.100:0/141788532 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f36440b3960 0x7f36440ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.722+0000 7f3649c00640 1 --2- 172.21.15.100:0/141788532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364406c3e0 0x7f3644154740 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.722+0000 7f3649c00640 1 -- 172.21.15.100:0/141788532 >> 172.21.15.100:0/141788532 conn(0x7f36440d0a50 msgr2=0x7f36440b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:12.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.722+0000 7f3649c00640 1 -- 172.21.15.100:0/141788532 shutdown_connections 2024-04-23T14:33:12.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.722+0000 7f3649c00640 1 -- 172.21.15.100:0/141788532 wait complete. 2024-04-23T14:33:12.745 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:33:12.746 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:12.756 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:12.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.890+0000 7f85b8000640 1 Processor -- start 2024-04-23T14:33:12.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.890+0000 7f85b8000640 1 -- start start 2024-04-23T14:33:12.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.890+0000 7f85b8000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f85b00cca50 0x7f85b00cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:12.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.890+0000 7f85b8000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85b0160f90 con 0x7f85b015c7e0 2024-04-23T14:33:12.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.890+0000 7f85b8000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85b00cd370 con 0x7f85b015d600 2024-04-23T14:33:12.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.890+0000 7f85b8000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f85b00d2270 con 0x7f85b00cca50 2024-04-23T14:33:12.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.890+0000 7f85b4c00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f85b015c7e0 0x7f85b0160170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:36292/0 (socket says 172.21.15.100:36292) 2024-04-23T14:33:12.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.890+0000 7f85b4c00640 1 -- 172.21.15.100:0/1698680518 learned_addr learned my addr 172.21.15.100:0/1698680518 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:12.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.890+0000 7f85b5600640 1 --2- 172.21.15.100:0/1698680518 >> v2:172.21.15.151:3300/0 conn(0x7f85b00cca50 0x7f85b00cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:12.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.890+0000 7f85afe00640 1 -- 172.21.15.100:0/1698680518 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3595314336 0 0) 0x7f85b0160f90 con 0x7f85b015c7e0 2024-04-23T14:33:12.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.890+0000 7f85afe00640 1 -- 172.21.15.100:0/1698680518 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8598003530 con 0x7f85b015c7e0 2024-04-23T14:33:12.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.890+0000 7f85afe00640 1 -- 172.21.15.100:0/1698680518 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3871749130 0 0) 0x7f85b00cd370 con 0x7f85b015d600 2024-04-23T14:33:12.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.890+0000 7f85afe00640 1 -- 172.21.15.100:0/1698680518 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85b0160f90 con 0x7f85b015d600 2024-04-23T14:33:12.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.890+0000 7f85b5600640 1 -- 172.21.15.100:0/1698680518 >> v1:172.21.15.151:6789/0 conn(0x7f85b015d600 legacy=0x7f85b0160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:12.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.890+0000 7f85b5600640 1 -- 172.21.15.100:0/1698680518 >> v1:172.21.15.100:6789/0 conn(0x7f85b015c7e0 legacy=0x7f85b0160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:12.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.890+0000 7f85afe00640 1 -- 172.21.15.100:0/1698680518 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2004364440 0 0) 0x7f8598003530 con 0x7f85b015c7e0 2024-04-23T14:33:12.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.890+0000 7f85b5600640 1 -- 172.21.15.100:0/1698680518 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b00d04a0 con 0x7f85b00cca50 2024-04-23T14:33:12.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.894+0000 7f85b5600640 1 --2- 172.21.15.100:0/1698680518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85b00cca50 0x7f85b00cce30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f85a40097c0 tx=0x7f85a401c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebcf50abc038d817 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:12.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.894+0000 7f85afe00640 1 -- 172.21.15.100:0/1698680518 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a40b0070 con 0x7f85b00cca50 2024-04-23T14:33:12.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.894+0000 7f85afe00640 1 -- 172.21.15.100:0/1698680518 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a40ab400 con 0x7f85b00cca50 2024-04-23T14:33:12.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.894+0000 7f85afe00640 1 -- 172.21.15.100:0/1698680518 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c001680 con 0x7f85b00cca50 2024-04-23T14:33:12.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.894+0000 7f85b8000640 1 -- 172.21.15.100:0/1698680518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85b00cca50 msgr2=0x7f85b00cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:12.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.894+0000 7f85b8000640 1 --2- 172.21.15.100:0/1698680518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85b00cca50 0x7f85b00cce30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f85a40097c0 tx=0x7f85a401c330 comp rx=0 tx=0).stop 2024-04-23T14:33:12.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.894+0000 7f85b8000640 1 -- 172.21.15.100:0/1698680518 shutdown_connections 2024-04-23T14:33:12.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.894+0000 7f85b8000640 1 --2- 172.21.15.100:0/1698680518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85b00cca50 0x7f85b00cce30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.894+0000 7f85b8000640 1 -- 172.21.15.100:0/1698680518 >> 172.21.15.100:0/1698680518 conn(0x7f85b00cee80 msgr2=0x7f85b00cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:12.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.894+0000 7f85b8000640 1 -- 172.21.15.100:0/1698680518 shutdown_connections 2024-04-23T14:33:12.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.894+0000 7f85b8000640 1 -- 172.21.15.100:0/1698680518 wait complete. 2024-04-23T14:33:12.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85b8000640 1 Processor -- start 2024-04-23T14:33:12.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85b8000640 1 -- start start 2024-04-23T14:33:12.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85b8000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b00cca50 0x7f85b014be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:12.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85b8000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85b015c7e0 0x7f85b01fa980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:12.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85b8000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85b015d600 0x7f85b01f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:12.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85b8000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f85b00b5430 con 0x7f85b00cca50 2024-04-23T14:33:12.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85b8000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f85b00b52b0 con 0x7f85b015d600 2024-04-23T14:33:12.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85b8000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f85b00b3bf0 con 0x7f85b015c7e0 2024-04-23T14:33:12.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85b5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b00cca50 0x7f85b014be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:12.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85b5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b00cca50 0x7f85b014be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60832/0 (socket says 172.21.15.100:60832) 2024-04-23T14:33:12.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85b5600640 1 -- 172.21.15.100:0/2811846455 learned_addr learned my addr 172.21.15.100:0/2811846455 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:12.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85b6000640 1 --2- 172.21.15.100:0/2811846455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85b015d600 0x7f85b01f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:12.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85b4c00640 1 --2- 172.21.15.100:0/2811846455 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85b015c7e0 0x7f85b01fa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:12.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85b5600640 1 -- 172.21.15.100:0/2811846455 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85b015c7e0 msgr2=0x7f85b01fa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:12.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85b5600640 1 --2- 172.21.15.100:0/2811846455 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85b015c7e0 0x7f85b01fa980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85b5600640 1 -- 172.21.15.100:0/2811846455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85b015d600 msgr2=0x7f85b01f8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:12.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85b5600640 1 --2- 172.21.15.100:0/2811846455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85b015d600 0x7f85b01f8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:12.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85b5600640 1 -- 172.21.15.100:0/2811846455 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b00c82a0 con 0x7f85b00cca50 2024-04-23T14:33:12.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85b5600640 1 --2- 172.21.15.100:0/2811846455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b00cca50 0x7f85b014be00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f85a40ae040 tx=0x7f85a40190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:12.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85ad600640 1 -- 172.21.15.100:0/2811846455 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a40b8070 con 0x7f85b00cca50 2024-04-23T14:33:12.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85ad600640 1 -- 172.21.15.100:0/2811846455 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a4004020 con 0x7f85b00cca50 2024-04-23T14:33:12.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85ad600640 1 -- 172.21.15.100:0/2811846455 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a40b0040 con 0x7f85b00cca50 2024-04-23T14:33:12.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85b8000640 1 -- 172.21.15.100:0/2811846455 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b01f8c50 con 0x7f85b00cca50 2024-04-23T14:33:12.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.898+0000 7f85b8000640 1 -- 172.21.15.100:0/2811846455 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b014c840 con 0x7f85b00cca50 2024-04-23T14:33:12.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.902+0000 7f85b8000640 1 -- 172.21.15.100:0/2811846455 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b0068ad0 con 0x7f85b00cca50 2024-04-23T14:33:12.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.902+0000 7f85ad600640 1 -- 172.21.15.100:0/2811846455 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f85a4003620 con 0x7f85b00cca50 2024-04-23T14:33:12.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.902+0000 7f85ad600640 1 --2- 172.21.15.100:0/2811846455 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8590068380 0x7f859006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:12.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.902+0000 7f85ad600640 1 -- 172.21.15.100:0/2811846455 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f85a4120970 con 0x7f85b00cca50 2024-04-23T14:33:12.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.902+0000 7f85b4c00640 1 --2- 172.21.15.100:0/2811846455 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8590068380 0x7f859006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:12.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.902+0000 7f85b4c00640 1 --2- 172.21.15.100:0/2811846455 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8590068380 0x7f859006a840 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f85b0151db0 tx=0x7f859c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:12.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:12.906+0000 7f85ad600640 1 -- 172.21.15.100:0/2811846455 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f85a4015fb0 con 0x7f85b00cca50 2024-04-23T14:33:13.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.066+0000 7f85b8000640 1 -- 172.21.15.100:0/2811846455 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85b0072450 con 0x7f85b00cca50 2024-04-23T14:33:13.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.066+0000 7f85ad600640 1 -- 172.21.15.100:0/2811846455 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18859 (secure 0 0 0) 0x7f85a40f0b30 con 0x7f85b00cca50 2024-04-23T14:33:13.070 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:13.070 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:49.784608+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:13.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.070+0000 7f85b8000640 1 -- 172.21.15.100:0/2811846455 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8590068380 msgr2=0x7f859006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:13.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.070+0000 7f85b8000640 1 --2- 172.21.15.100:0/2811846455 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8590068380 0x7f859006a840 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f85b0151db0 tx=0x7f859c009040 comp rx=0 tx=0).stop 2024-04-23T14:33:13.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.070+0000 7f85b8000640 1 -- 172.21.15.100:0/2811846455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b00cca50 msgr2=0x7f85b014be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:13.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.070+0000 7f85b8000640 1 --2- 172.21.15.100:0/2811846455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b00cca50 0x7f85b014be00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f85a40ae040 tx=0x7f85a40190d0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.070+0000 7f85b8000640 1 -- 172.21.15.100:0/2811846455 shutdown_connections 2024-04-23T14:33:13.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.070+0000 7f85b8000640 1 --2- 172.21.15.100:0/2811846455 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8590068380 0x7f859006a840 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.070+0000 7f85b8000640 1 --2- 172.21.15.100:0/2811846455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85b015d600 0x7f85b01f8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.070+0000 7f85b8000640 1 --2- 172.21.15.100:0/2811846455 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85b015c7e0 0x7f85b01fa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.070+0000 7f85b8000640 1 --2- 172.21.15.100:0/2811846455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b00cca50 0x7f85b014be00 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.070+0000 7f85b8000640 1 -- 172.21.15.100:0/2811846455 >> 172.21.15.100:0/2811846455 conn(0x7f85b00cee80 msgr2=0x7f85b00b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:13.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.070+0000 7f85b8000640 1 -- 172.21.15.100:0/2811846455 shutdown_connections 2024-04-23T14:33:13.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.070+0000 7f85b8000640 1 -- 172.21.15.100:0/2811846455 wait complete. 2024-04-23T14:33:13.091 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:32:06.835560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:32:07.930489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-23T14:32:11.636814+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-23T14:33:13.091 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:33:13.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.226+0000 7f5f4ce00640 1 Processor -- start 2024-04-23T14:33:13.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.226+0000 7f5f4ce00640 1 -- start start 2024-04-23T14:33:13.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.230+0000 7f5f4ce00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5f48158c70 0x7f5f480cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:13.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.230+0000 7f5f4ce00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f480d0390 0x7f5f480b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:13.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.230+0000 7f5f4ce00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f480b29d0 con 0x7f5f480b21a0 2024-04-23T14:33:13.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.230+0000 7f5f4ce00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f5f480b2cb0 con 0x7f5f48158c70 2024-04-23T14:33:13.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.230+0000 7f5f4ce00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5f4806d580 con 0x7f5f480d0390 2024-04-23T14:33:13.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.230+0000 7f5f47400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5f48158c70 0x7f5f480cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:13.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.230+0000 7f5f47400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5f48158c70 0x7f5f480cfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60844/0 (socket says 172.21.15.100:60844) 2024-04-23T14:33:13.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.230+0000 7f5f47400640 1 -- 172.21.15.100:0/1389166010 learned_addr learned my addr 172.21.15.100:0/1389166010 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:13.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.230+0000 7f5f3ea00640 1 --2- 172.21.15.100:0/1389166010 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f480d0390 0x7f5f480b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:13.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.230+0000 7f5f47400640 1 -- 172.21.15.100:0/1389166010 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f480d0390 msgr2=0x7f5f480b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:13.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.230+0000 7f5f47400640 1 --2- 172.21.15.100:0/1389166010 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f480d0390 0x7f5f480b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.230+0000 7f5f47400640 1 -- 172.21.15.100:0/1389166010 >> v1:172.21.15.100:6789/0 conn(0x7f5f480b21a0 legacy=0x7f5f480ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:13.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.230+0000 7f5f47400640 1 -- 172.21.15.100:0/1389166010 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f4806d860 con 0x7f5f48158c70 2024-04-23T14:33:13.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.230+0000 7f5f46a00640 1 -- 172.21.15.100:0/1389166010 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3725902582 0 0) 0x7f5f480b29d0 con 0x7f5f480b21a0 2024-04-23T14:33:13.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.230+0000 7f5f47400640 1 --2- 172.21.15.100:0/1389166010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f48158c70 0x7f5f480cfe50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009e50 tx=0x7f5f3801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe955f138703a40 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:13.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f46a00640 1 -- 172.21.15.100:0/1389166010 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f380a5770 con 0x7f5f48158c70 2024-04-23T14:33:13.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f46a00640 1 -- 172.21.15.100:0/1389166010 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f380a5910 con 0x7f5f48158c70 2024-04-23T14:33:13.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f46a00640 1 -- 172.21.15.100:0/1389166010 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f380a5ce0 con 0x7f5f48158c70 2024-04-23T14:33:13.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f4ce00640 1 -- 172.21.15.100:0/1389166010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f48158c70 msgr2=0x7f5f480cfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:13.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f4ce00640 1 --2- 172.21.15.100:0/1389166010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f48158c70 0x7f5f480cfe50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009e50 tx=0x7f5f3801c9a0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f4ce00640 1 -- 172.21.15.100:0/1389166010 shutdown_connections 2024-04-23T14:33:13.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f4ce00640 1 --2- 172.21.15.100:0/1389166010 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f480d0390 0x7f5f480b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f4ce00640 1 --2- 172.21.15.100:0/1389166010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f48158c70 0x7f5f480cfe50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f4ce00640 1 -- 172.21.15.100:0/1389166010 >> 172.21.15.100:0/1389166010 conn(0x7f5f480b3df0 msgr2=0x7f5f480b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:13.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f4ce00640 1 -- 172.21.15.100:0/1389166010 shutdown_connections 2024-04-23T14:33:13.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f4ce00640 1 -- 172.21.15.100:0/1389166010 wait complete. 2024-04-23T14:33:13.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f4ce00640 1 Processor -- start 2024-04-23T14:33:13.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f4ce00640 1 -- start start 2024-04-23T14:33:13.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f4ce00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f480b21a0 0x7f5f480c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:13.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f4ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f480d0390 0x7f5f481faf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:13.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f4ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f48158c70 0x7f5f4806f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:13.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f4ce00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4806e9f0 con 0x7f5f480d0390 2024-04-23T14:33:13.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f4ce00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4806e870 con 0x7f5f48158c70 2024-04-23T14:33:13.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f4ce00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5f4806eb70 con 0x7f5f480b21a0 2024-04-23T14:33:13.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f3ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f480d0390 0x7f5f481faf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:13.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f3ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f480d0390 0x7f5f481faf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60854/0 (socket says 172.21.15.100:60854) 2024-04-23T14:33:13.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f3ea00640 1 -- 172.21.15.100:0/2701496683 learned_addr learned my addr 172.21.15.100:0/2701496683 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:13.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f47400640 1 --2- 172.21.15.100:0/2701496683 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f480b21a0 0x7f5f480c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:13.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.234+0000 7f5f47e00640 1 --2- 172.21.15.100:0/2701496683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f48158c70 0x7f5f4806f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:13.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.238+0000 7f5f3ea00640 1 -- 172.21.15.100:0/2701496683 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f480b21a0 msgr2=0x7f5f480c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:13.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.238+0000 7f5f3ea00640 1 --2- 172.21.15.100:0/2701496683 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f480b21a0 0x7f5f480c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.238+0000 7f5f3ea00640 1 -- 172.21.15.100:0/2701496683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f48158c70 msgr2=0x7f5f4806f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:13.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.238+0000 7f5f3ea00640 1 --2- 172.21.15.100:0/2701496683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f48158c70 0x7f5f4806f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.238+0000 7f5f3ea00640 1 -- 172.21.15.100:0/2701496683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f481fb670 con 0x7f5f480d0390 2024-04-23T14:33:13.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.238+0000 7f5f3ea00640 1 --2- 172.21.15.100:0/2701496683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f480d0390 0x7f5f481faf70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000d830 tx=0x7f5f3000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:13.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.238+0000 7f5f3fe00640 1 -- 172.21.15.100:0/2701496683 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3000b6c0 con 0x7f5f480d0390 2024-04-23T14:33:13.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.238+0000 7f5f3fe00640 1 -- 172.21.15.100:0/2701496683 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f30019070 con 0x7f5f480d0390 2024-04-23T14:33:13.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.238+0000 7f5f3fe00640 1 -- 172.21.15.100:0/2701496683 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3000ba60 con 0x7f5f480d0390 2024-04-23T14:33:13.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.238+0000 7f5f4ce00640 1 -- 172.21.15.100:0/2701496683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f481fb950 con 0x7f5f480d0390 2024-04-23T14:33:13.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.238+0000 7f5f4ce00640 1 -- 172.21.15.100:0/2701496683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f4815bc40 con 0x7f5f480d0390 2024-04-23T14:33:13.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.238+0000 7f5f4ce00640 1 -- 172.21.15.100:0/2701496683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f4814b030 con 0x7f5f480d0390 2024-04-23T14:33:13.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.238+0000 7f5f3fe00640 1 -- 172.21.15.100:0/2701496683 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5f3001e460 con 0x7f5f480d0390 2024-04-23T14:33:13.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.238+0000 7f5f3fe00640 1 --2- 172.21.15.100:0/2701496683 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5f240683b0 0x7f5f2406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:13.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.238+0000 7f5f3fe00640 1 -- 172.21.15.100:0/2701496683 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5f30087430 con 0x7f5f480d0390 2024-04-23T14:33:13.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.238+0000 7f5f47400640 1 --2- 172.21.15.100:0/2701496683 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5f240683b0 0x7f5f2406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:13.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.242+0000 7f5f47400640 1 --2- 172.21.15.100:0/2701496683 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5f240683b0 0x7f5f2406a870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5f380ae000 tx=0x7f5f380a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:13.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.242+0000 7f5f3fe00640 1 -- 172.21.15.100:0/2701496683 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5f30056f00 con 0x7f5f480d0390 2024-04-23T14:33:13.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.402+0000 7f5f4ce00640 1 -- 172.21.15.100:0/2701496683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5f48068ad0 con 0x7f5f480d0390 2024-04-23T14:33:13.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.402+0000 7f5f3fe00640 1 -- 172.21.15.100:0/2701496683 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f5f3005b200 con 0x7f5f480d0390 2024-04-23T14:33:13.404 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:33:13.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.402+0000 7f5f4ce00640 1 -- 172.21.15.100:0/2701496683 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5f240683b0 msgr2=0x7f5f2406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:13.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.402+0000 7f5f4ce00640 1 --2- 172.21.15.100:0/2701496683 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5f240683b0 0x7f5f2406a870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5f380ae000 tx=0x7f5f380a7070 comp rx=0 tx=0).stop 2024-04-23T14:33:13.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.402+0000 7f5f4ce00640 1 -- 172.21.15.100:0/2701496683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f480d0390 msgr2=0x7f5f481faf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:13.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.402+0000 7f5f4ce00640 1 --2- 172.21.15.100:0/2701496683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f480d0390 0x7f5f481faf70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000d830 tx=0x7f5f3000dcf0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.402+0000 7f5f4ce00640 1 -- 172.21.15.100:0/2701496683 shutdown_connections 2024-04-23T14:33:13.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.402+0000 7f5f4ce00640 1 --2- 172.21.15.100:0/2701496683 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5f240683b0 0x7f5f2406a870 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.402+0000 7f5f4ce00640 1 --2- 172.21.15.100:0/2701496683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f48158c70 0x7f5f4806f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.402+0000 7f5f4ce00640 1 --2- 172.21.15.100:0/2701496683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f480d0390 0x7f5f481faf70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.402+0000 7f5f4ce00640 1 --2- 172.21.15.100:0/2701496683 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f480b21a0 0x7f5f480c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.402+0000 7f5f4ce00640 1 -- 172.21.15.100:0/2701496683 >> 172.21.15.100:0/2701496683 conn(0x7f5f480b3df0 msgr2=0x7f5f481548b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:13.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.402+0000 7f5f4ce00640 1 -- 172.21.15.100:0/2701496683 shutdown_connections 2024-04-23T14:33:13.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.402+0000 7f5f4ce00640 1 -- 172.21.15.100:0/2701496683 wait complete. 2024-04-23T14:33:13.425 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:33:13.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b4f400640 1 Processor -- start 2024-04-23T14:33:13.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b4f400640 1 -- start start 2024-04-23T14:33:13.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b4f400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f0b480cea60 0x7f0b480d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:13.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b4f400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b480b1f90 con 0x7f0b4806d450 2024-04-23T14:33:13.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b4f400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b480b2270 con 0x7f0b480cd460 2024-04-23T14:33:13.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b4f400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f0b480b2550 con 0x7f0b480cea60 2024-04-23T14:33:13.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b4ca00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f0b4806d450 0x7f0b4806d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:36304/0 (socket says 172.21.15.100:36304) 2024-04-23T14:33:13.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b4ca00640 1 -- 172.21.15.100:0/2362970088 learned_addr learned my addr 172.21.15.100:0/2362970088 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:13.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b47e00640 1 --2- 172.21.15.100:0/2362970088 >> v2:172.21.15.151:3300/0 conn(0x7f0b480cea60 0x7f0b480d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:13.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b47400640 1 -- 172.21.15.100:0/2362970088 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 153136111 0 0) 0x7f0b480b1f90 con 0x7f0b4806d450 2024-04-23T14:33:13.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b47400640 1 -- 172.21.15.100:0/2362970088 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b2c003530 con 0x7f0b4806d450 2024-04-23T14:33:13.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b47400640 1 -- 172.21.15.100:0/2362970088 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2372301622 0 0) 0x7f0b2c003530 con 0x7f0b4806d450 2024-04-23T14:33:13.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b47400640 1 -- 172.21.15.100:0/2362970088 >> v2:172.21.15.151:3300/0 conn(0x7f0b480cea60 msgr2=0x7f0b480d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:13.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b47400640 1 --2- 172.21.15.100:0/2362970088 >> v2:172.21.15.151:3300/0 conn(0x7f0b480cea60 0x7f0b480d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b47400640 1 -- 172.21.15.100:0/2362970088 >> v1:172.21.15.151:6789/0 conn(0x7f0b480cd460 legacy=0x7f0b480b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:13.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b47400640 1 -- 172.21.15.100:0/2362970088 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b480b2830 con 0x7f0b4806d450 2024-04-23T14:33:13.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b47400640 1 -- 172.21.15.100:0/2362970088 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f0b38001680 con 0x7f0b4806d450 2024-04-23T14:33:13.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b47400640 1 -- 172.21.15.100:0/2362970088 >> v1:172.21.15.100:6789/0 conn(0x7f0b4806d450 legacy=0x7f0b4806d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:13.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b47400640 1 --2- 172.21.15.100:0/2362970088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b2c0039c0 0x7f0b2c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:13.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b47400640 1 --2- 172.21.15.100:0/2362970088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2c004810 0x7f0b2c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:13.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b47400640 1 --2- 172.21.15.100:0/2362970088 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0b2c0051d0 0x7f0b2c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:13.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b47400640 1 -- 172.21.15.100:0/2362970088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0b480b2550 con 0x7f0b2c004810 2024-04-23T14:33:13.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b47400640 1 -- 172.21.15.100:0/2362970088 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c00c170 con 0x7f0b2c0039c0 2024-04-23T14:33:13.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b47400640 1 -- 172.21.15.100:0/2362970088 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0b2c00c370 con 0x7f0b2c0051d0 2024-04-23T14:33:13.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.562+0000 7f0b47e00640 1 --2- 172.21.15.100:0/2362970088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2c004810 0x7f0b2c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:13.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.566+0000 7f0b4ca00640 1 --2- 172.21.15.100:0/2362970088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b2c0039c0 0x7f0b2c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:13.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.566+0000 7f0b4d400640 1 --2- 172.21.15.100:0/2362970088 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0b2c0051d0 0x7f0b2c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:13.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.566+0000 7f0b47e00640 1 -- 172.21.15.100:0/2362970088 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0b2c0051d0 msgr2=0x7f0b2c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:13.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.566+0000 7f0b47e00640 1 --2- 172.21.15.100:0/2362970088 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0b2c0051d0 0x7f0b2c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.566+0000 7f0b47e00640 1 -- 172.21.15.100:0/2362970088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b2c0039c0 msgr2=0x7f0b2c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:13.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.566+0000 7f0b47e00640 1 --2- 172.21.15.100:0/2362970088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b2c0039c0 0x7f0b2c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.566+0000 7f0b4d400640 1 -- 172.21.15.100:0/2362970088 reap_dead start 2024-04-23T14:33:13.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.566+0000 7f0b47e00640 1 -- 172.21.15.100:0/2362970088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2c003530 con 0x7f0b2c004810 2024-04-23T14:33:13.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.566+0000 7f0b47e00640 1 --2- 172.21.15.100:0/2362970088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2c004810 0x7f0b2c004c90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009ae0 tx=0x7f0b3401c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd51f14c483ced5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:13.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.566+0000 7f0b47400640 1 -- 172.21.15.100:0/2362970088 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b340b1070 con 0x7f0b2c004810 2024-04-23T14:33:13.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.566+0000 7f0b47400640 1 -- 172.21.15.100:0/2362970088 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b340a49b0 con 0x7f0b2c004810 2024-04-23T14:33:13.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.566+0000 7f0b47400640 1 -- 172.21.15.100:0/2362970088 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b340096b0 con 0x7f0b2c004810 2024-04-23T14:33:13.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4f400640 1 -- 172.21.15.100:0/2362970088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2c004810 msgr2=0x7f0b2c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:13.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4f400640 1 --2- 172.21.15.100:0/2362970088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2c004810 0x7f0b2c004c90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009ae0 tx=0x7f0b3401c770 comp rx=0 tx=0).stop 2024-04-23T14:33:13.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4f400640 1 -- 172.21.15.100:0/2362970088 shutdown_connections 2024-04-23T14:33:13.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4f400640 1 --2- 172.21.15.100:0/2362970088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2c004810 0x7f0b2c004c90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4f400640 1 -- 172.21.15.100:0/2362970088 >> 172.21.15.100:0/2362970088 conn(0x7f0b480cee80 msgr2=0x7f0b480cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:13.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4f400640 1 -- 172.21.15.100:0/2362970088 shutdown_connections 2024-04-23T14:33:13.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4f400640 1 -- 172.21.15.100:0/2362970088 wait complete. 2024-04-23T14:33:13.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4f400640 1 Processor -- start 2024-04-23T14:33:13.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4f400640 1 -- start start 2024-04-23T14:33:13.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2c004810 0x7f0b48150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:13.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4f400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0b481fbad0 0x7f0b4815eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:13.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b480c70d0 0x7f0b480c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:13.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4f400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0b480b38f0 con 0x7f0b2c004810 2024-04-23T14:33:13.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4f400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0b480b3770 con 0x7f0b480c70d0 2024-04-23T14:33:13.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4f400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0b480d0300 con 0x7f0b481fbad0 2024-04-23T14:33:13.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2c004810 0x7f0b48150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:13.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2c004810 0x7f0b48150870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60878/0 (socket says 172.21.15.100:60878) 2024-04-23T14:33:13.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4ca00640 1 -- 172.21.15.100:0/267174458 learned_addr learned my addr 172.21.15.100:0/267174458 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:13.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b47e00640 1 --2- 172.21.15.100:0/267174458 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0b481fbad0 0x7f0b4815eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:13.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4d400640 1 --2- 172.21.15.100:0/267174458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b480c70d0 0x7f0b480c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:13.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4ca00640 1 -- 172.21.15.100:0/267174458 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0b481fbad0 msgr2=0x7f0b4815eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:13.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4ca00640 1 --2- 172.21.15.100:0/267174458 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0b481fbad0 0x7f0b4815eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4ca00640 1 -- 172.21.15.100:0/267174458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b480c70d0 msgr2=0x7f0b480c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:13.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4ca00640 1 --2- 172.21.15.100:0/267174458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b480c70d0 0x7f0b480c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4ca00640 1 -- 172.21.15.100:0/267174458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b48150db0 con 0x7f0b2c004810 2024-04-23T14:33:13.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4d400640 1 --2- 172.21.15.100:0/267174458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b480c70d0 0x7f0b480c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:13.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.570+0000 7f0b4ca00640 1 --2- 172.21.15.100:0/267174458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2c004810 0x7f0b48150870 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0b380101e0 tx=0x7f0b380106a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:13.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.574+0000 7f0b44c00640 1 -- 172.21.15.100:0/267174458 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b38013770 con 0x7f0b2c004810 2024-04-23T14:33:13.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.574+0000 7f0b44c00640 1 -- 172.21.15.100:0/267174458 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b38013910 con 0x7f0b2c004810 2024-04-23T14:33:13.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.574+0000 7f0b44c00640 1 -- 172.21.15.100:0/267174458 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b38013ce0 con 0x7f0b2c004810 2024-04-23T14:33:13.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.574+0000 7f0b4f400640 1 -- 172.21.15.100:0/267174458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b481fb310 con 0x7f0b2c004810 2024-04-23T14:33:13.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.574+0000 7f0b4f400640 1 -- 172.21.15.100:0/267174458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b4815f650 con 0x7f0b2c004810 2024-04-23T14:33:13.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.574+0000 7f0b4f400640 1 -- 172.21.15.100:0/267174458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b48068ad0 con 0x7f0b2c004810 2024-04-23T14:33:13.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.574+0000 7f0b44c00640 1 -- 172.21.15.100:0/267174458 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0b38013070 con 0x7f0b2c004810 2024-04-23T14:33:13.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.574+0000 7f0b44c00640 1 --2- 172.21.15.100:0/267174458 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0b280683b0 0x7f0b2806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:13.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.574+0000 7f0b44c00640 1 -- 172.21.15.100:0/267174458 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f0b38087bd0 con 0x7f0b2c004810 2024-04-23T14:33:13.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.574+0000 7f0b47e00640 1 --2- 172.21.15.100:0/267174458 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0b280683b0 0x7f0b2806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:13.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.574+0000 7f0b47e00640 1 --2- 172.21.15.100:0/267174458 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0b280683b0 0x7f0b2806a870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0b340ae000 tx=0x7f0b34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:13.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.578+0000 7f0b44c00640 1 -- 172.21.15.100:0/267174458 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0b38057cf0 con 0x7f0b2c004810 2024-04-23T14:33:13.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.742+0000 7f0b4f400640 1 -- 172.21.15.100:0/267174458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0b48070ba0 con 0x7f0b2c004810 2024-04-23T14:33:13.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.742+0000 7f0b44c00640 1 -- 172.21.15.100:0/267174458 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f0b380575f0 con 0x7f0b2c004810 2024-04-23T14:33:13.746 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:13.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.746+0000 7f0b4f400640 1 -- 172.21.15.100:0/267174458 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0b280683b0 msgr2=0x7f0b2806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:13.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.746+0000 7f0b4f400640 1 --2- 172.21.15.100:0/267174458 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0b280683b0 0x7f0b2806a870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0b340ae000 tx=0x7f0b34008040 comp rx=0 tx=0).stop 2024-04-23T14:33:13.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.746+0000 7f0b4f400640 1 -- 172.21.15.100:0/267174458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2c004810 msgr2=0x7f0b48150870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:13.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.746+0000 7f0b4f400640 1 --2- 172.21.15.100:0/267174458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2c004810 0x7f0b48150870 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0b380101e0 tx=0x7f0b380106a0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.746+0000 7f0b4f400640 1 -- 172.21.15.100:0/267174458 shutdown_connections 2024-04-23T14:33:13.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.746+0000 7f0b4f400640 1 --2- 172.21.15.100:0/267174458 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0b280683b0 0x7f0b2806a870 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.746+0000 7f0b4f400640 1 --2- 172.21.15.100:0/267174458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b480c70d0 0x7f0b480c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.746+0000 7f0b4f400640 1 --2- 172.21.15.100:0/267174458 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0b481fbad0 0x7f0b4815eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.746+0000 7f0b4f400640 1 --2- 172.21.15.100:0/267174458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2c004810 0x7f0b48150870 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.746+0000 7f0b4f400640 1 -- 172.21.15.100:0/267174458 >> 172.21.15.100:0/267174458 conn(0x7f0b480cee80 msgr2=0x7f0b480c6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:13.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.746+0000 7f0b4f400640 1 -- 172.21.15.100:0/267174458 shutdown_connections 2024-04-23T14:33:13.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.746+0000 7f0b4f400640 1 -- 172.21.15.100:0/267174458 wait complete. 2024-04-23T14:33:13.766 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:33:13.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.902+0000 7fc7dca00640 1 Processor -- start 2024-04-23T14:33:13.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.902+0000 7fc7dca00640 1 -- start start 2024-04-23T14:33:13.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.902+0000 7fc7dca00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc7d815b370 0x7fc7d815c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:13.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.906+0000 7fc7dca00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc7d80cc3a0 0x7fc7d8072e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:13.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.906+0000 7fc7dca00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc7d8073390 0x7fc7d8071c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:13.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.906+0000 7fc7dca00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc7d815b750 con 0x7fc7d815b370 2024-04-23T14:33:13.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.906+0000 7fc7dca00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fc7d815efa0 con 0x7fc7d8073390 2024-04-23T14:33:13.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.906+0000 7fc7dca00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc7d815f280 con 0x7fc7d80cc3a0 2024-04-23T14:33:13.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.906+0000 7fc7d7400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc7d815b370 0x7fc7d815c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:13.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.906+0000 7fc7d7400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc7d815b370 0x7fc7d815c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60894/0 (socket says 172.21.15.100:60894) 2024-04-23T14:33:13.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.906+0000 7fc7d7400640 1 -- 172.21.15.100:0/3693847509 learned_addr learned my addr 172.21.15.100:0/3693847509 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:13.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.906+0000 7fc7d7e00640 1 --2- 172.21.15.100:0/3693847509 >> v2:172.21.15.151:3300/0 conn(0x7fc7d8073390 0x7fc7d8071c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:13.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.906+0000 7fc7d6a00640 1 --2- 172.21.15.100:0/3693847509 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc7d80cc3a0 0x7fc7d8072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:13.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.906+0000 7fc7d7400640 1 -- 172.21.15.100:0/3693847509 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc7d80cc3a0 msgr2=0x7fc7d8072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:13.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.906+0000 7fc7d7400640 1 --2- 172.21.15.100:0/3693847509 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc7d80cc3a0 0x7fc7d8072e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.906+0000 7fc7d7400640 1 -- 172.21.15.100:0/3693847509 >> v2:172.21.15.151:3300/0 conn(0x7fc7d8073390 msgr2=0x7fc7d8071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:13.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.906+0000 7fc7d7400640 1 --2- 172.21.15.100:0/3693847509 >> v2:172.21.15.151:3300/0 conn(0x7fc7d8073390 0x7fc7d8071c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.906+0000 7fc7d7400640 1 -- 172.21.15.100:0/3693847509 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d815f560 con 0x7fc7d815b370 2024-04-23T14:33:13.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.906+0000 7fc7d7400640 1 --2- 172.21.15.100:0/3693847509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7d815b370 0x7fc7d815c300 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80b5490 tx=0x7fc7c801c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c935319061d255f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:13.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.906+0000 7fc7d6000640 1 -- 172.21.15.100:0/3693847509 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c80ac020 con 0x7fc7d815b370 2024-04-23T14:33:13.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.906+0000 7fc7d6000640 1 -- 172.21.15.100:0/3693847509 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7c80a6440 con 0x7fc7d815b370 2024-04-23T14:33:13.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.906+0000 7fc7d6000640 1 -- 172.21.15.100:0/3693847509 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c80a6810 con 0x7fc7d815b370 2024-04-23T14:33:13.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7dca00640 1 -- 172.21.15.100:0/3693847509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7d815b370 msgr2=0x7fc7d815c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:13.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7dca00640 1 --2- 172.21.15.100:0/3693847509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7d815b370 0x7fc7d815c300 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80b5490 tx=0x7fc7c801c6a0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7dca00640 1 -- 172.21.15.100:0/3693847509 shutdown_connections 2024-04-23T14:33:13.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7dca00640 1 --2- 172.21.15.100:0/3693847509 >> v2:172.21.15.151:3300/0 conn(0x7fc7d8073390 0x7fc7d8071c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7dca00640 1 --2- 172.21.15.100:0/3693847509 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc7d80cc3a0 0x7fc7d8072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7dca00640 1 --2- 172.21.15.100:0/3693847509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7d815b370 0x7fc7d815c300 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7dca00640 1 -- 172.21.15.100:0/3693847509 >> 172.21.15.100:0/3693847509 conn(0x7fc7d80b1d00 msgr2=0x7fc7d80b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:13.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7dca00640 1 -- 172.21.15.100:0/3693847509 shutdown_connections 2024-04-23T14:33:13.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7dca00640 1 -- 172.21.15.100:0/3693847509 wait complete. 2024-04-23T14:33:13.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7dca00640 1 Processor -- start 2024-04-23T14:33:13.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7dca00640 1 -- start start 2024-04-23T14:33:13.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7dca00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc7d8073390 0x7fc7d8157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:13.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7dca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7d80cc3a0 0x7fc7d81571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:13.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7dca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7d815b370 0x7fc7d80ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:13.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7dca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d80ae3b0 con 0x7fc7d80cc3a0 2024-04-23T14:33:13.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7dca00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d80ae230 con 0x7fc7d815b370 2024-04-23T14:33:13.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7dca00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc7d80ae530 con 0x7fc7d8073390 2024-04-23T14:33:13.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7d6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7d80cc3a0 0x7fc7d81571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:13.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7d6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7d80cc3a0 0x7fc7d81571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60908/0 (socket says 172.21.15.100:60908) 2024-04-23T14:33:13.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7d6a00640 1 -- 172.21.15.100:0/3144088952 learned_addr learned my addr 172.21.15.100:0/3144088952 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:13.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7d7400640 1 --2- 172.21.15.100:0/3144088952 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc7d8073390 0x7fc7d8157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:13.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7d7e00640 1 --2- 172.21.15.100:0/3144088952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7d815b370 0x7fc7d80ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:13.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7d6a00640 1 -- 172.21.15.100:0/3144088952 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc7d8073390 msgr2=0x7fc7d8157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:13.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7d6a00640 1 --2- 172.21.15.100:0/3144088952 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc7d8073390 0x7fc7d8157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7d6a00640 1 -- 172.21.15.100:0/3144088952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7d815b370 msgr2=0x7fc7d80ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:13.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7d6a00640 1 --2- 172.21.15.100:0/3144088952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7d815b370 0x7fc7d80ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:13.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7d6a00640 1 -- 172.21.15.100:0/3144088952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d81563f0 con 0x7fc7d80cc3a0 2024-04-23T14:33:13.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.910+0000 7fc7d7e00640 1 --2- 172.21.15.100:0/3144088952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7d815b370 0x7fc7d80ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:13.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.914+0000 7fc7d6a00640 1 --2- 172.21.15.100:0/3144088952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7d80cc3a0 0x7fc7d81571b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400d800 tx=0x7fc7c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:13.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.914+0000 7fc7cf400640 1 -- 172.21.15.100:0/3144088952 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4004020 con 0x7fc7d80cc3a0 2024-04-23T14:33:13.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.914+0000 7fc7cf400640 1 -- 172.21.15.100:0/3144088952 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7c40041c0 con 0x7fc7d80cc3a0 2024-04-23T14:33:13.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.914+0000 7fc7dca00640 1 -- 172.21.15.100:0/3144088952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7d81566d0 con 0x7fc7d80cc3a0 2024-04-23T14:33:13.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.914+0000 7fc7cf400640 1 -- 172.21.15.100:0/3144088952 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40043b0 con 0x7fc7d80cc3a0 2024-04-23T14:33:13.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.914+0000 7fc7dca00640 1 -- 172.21.15.100:0/3144088952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7d80d0350 con 0x7fc7d80cc3a0 2024-04-23T14:33:13.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.914+0000 7fc7dca00640 1 -- 172.21.15.100:0/3144088952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7d8068ad0 con 0x7fc7d80cc3a0 2024-04-23T14:33:13.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.914+0000 7fc7cf400640 1 -- 172.21.15.100:0/3144088952 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc7c401b020 con 0x7fc7d80cc3a0 2024-04-23T14:33:13.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.914+0000 7fc7cf400640 1 --2- 172.21.15.100:0/3144088952 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc7b80683b0 0x7fc7b806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:13.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.914+0000 7fc7cf400640 1 -- 172.21.15.100:0/3144088952 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc7c4086c90 con 0x7fc7d80cc3a0 2024-04-23T14:33:13.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.914+0000 7fc7d7400640 1 --2- 172.21.15.100:0/3144088952 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc7b80683b0 0x7fc7b806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:13.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.914+0000 7fc7d7400640 1 --2- 172.21.15.100:0/3144088952 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc7b80683b0 0x7fc7b806a870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc7c80ad000 tx=0x7fc7c80b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:13.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:13.918+0000 7fc7cf400640 1 -- 172.21.15.100:0/3144088952 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc7c4056d20 con 0x7fc7d80cc3a0 2024-04-23T14:33:14.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.078+0000 7fc7dca00640 1 -- 172.21.15.100:0/3144088952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc7d80cfc30 con 0x7fc7d80cc3a0 2024-04-23T14:33:14.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.078+0000 7fc7cf400640 1 -- 172.21.15.100:0/3144088952 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fc7c4056620 con 0x7fc7d80cc3a0 2024-04-23T14:33:14.079 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:33:14.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.078+0000 7fc7dca00640 1 -- 172.21.15.100:0/3144088952 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc7b80683b0 msgr2=0x7fc7b806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:14.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.078+0000 7fc7dca00640 1 --2- 172.21.15.100:0/3144088952 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc7b80683b0 0x7fc7b806a870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc7c80ad000 tx=0x7fc7c80b3040 comp rx=0 tx=0).stop 2024-04-23T14:33:14.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.078+0000 7fc7dca00640 1 -- 172.21.15.100:0/3144088952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7d80cc3a0 msgr2=0x7fc7d81571b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:14.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.078+0000 7fc7dca00640 1 --2- 172.21.15.100:0/3144088952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7d80cc3a0 0x7fc7d81571b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400d800 tx=0x7fc7c400dcc0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.078+0000 7fc7dca00640 1 -- 172.21.15.100:0/3144088952 shutdown_connections 2024-04-23T14:33:14.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.078+0000 7fc7dca00640 1 --2- 172.21.15.100:0/3144088952 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc7b80683b0 0x7fc7b806a870 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.078+0000 7fc7dca00640 1 --2- 172.21.15.100:0/3144088952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7d815b370 0x7fc7d80ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.078+0000 7fc7dca00640 1 --2- 172.21.15.100:0/3144088952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7d80cc3a0 0x7fc7d81571b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.078+0000 7fc7dca00640 1 --2- 172.21.15.100:0/3144088952 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc7d8073390 0x7fc7d8157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.078+0000 7fc7dca00640 1 -- 172.21.15.100:0/3144088952 >> 172.21.15.100:0/3144088952 conn(0x7fc7d80b1d00 msgr2=0x7fc7d806ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:14.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.078+0000 7fc7dca00640 1 -- 172.21.15.100:0/3144088952 shutdown_connections 2024-04-23T14:33:14.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.078+0000 7fc7dca00640 1 -- 172.21.15.100:0/3144088952 wait complete. 2024-04-23T14:33:14.100 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-23T14:33:14.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.234+0000 7fbe70800640 1 Processor -- start 2024-04-23T14:33:14.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.234+0000 7fbe70800640 1 -- start start 2024-04-23T14:33:14.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.238+0000 7fbe70800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbe6815e860 0x7fbe680b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:14.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.238+0000 7fbe70800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fbe680adb90 0x7fbe680adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:14.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.238+0000 7fbe70800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe6806e5f0 con 0x7fbe6806f950 2024-04-23T14:33:14.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.238+0000 7fbe70800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fbe6806e8b0 con 0x7fbe6815e860 2024-04-23T14:33:14.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.238+0000 7fbe70800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fbe6806eb90 con 0x7fbe680adb90 2024-04-23T14:33:14.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.238+0000 7fbe6de00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbe6815e860 0x7fbe680b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:14.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.238+0000 7fbe6de00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbe6815e860 0x7fbe680b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60914/0 (socket says 172.21.15.100:60914) 2024-04-23T14:33:14.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.238+0000 7fbe6de00640 1 -- 172.21.15.100:0/1907188761 learned_addr learned my addr 172.21.15.100:0/1907188761 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:14.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.238+0000 7fbe6d400640 1 --2- 172.21.15.100:0/1907188761 >> v2:172.21.15.151:3300/0 conn(0x7fbe680adb90 0x7fbe680adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:14.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.238+0000 7fbe6de00640 1 -- 172.21.15.100:0/1907188761 >> v2:172.21.15.151:3300/0 conn(0x7fbe680adb90 msgr2=0x7fbe680adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:14.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.238+0000 7fbe6de00640 1 --2- 172.21.15.100:0/1907188761 >> v2:172.21.15.151:3300/0 conn(0x7fbe680adb90 0x7fbe680adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.238+0000 7fbe6de00640 1 -- 172.21.15.100:0/1907188761 >> v1:172.21.15.151:6789/0 conn(0x7fbe6806f950 legacy=0x7fbe6806d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:14.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.238+0000 7fbe6de00640 1 -- 172.21.15.100:0/1907188761 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6806ee70 con 0x7fbe6815e860 2024-04-23T14:33:14.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.242+0000 7fbe6de00640 1 --2- 172.21.15.100:0/1907188761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe6815e860 0x7fbe680b3ab0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbe680d0aa0 tx=0x7fbe5c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22693f7d4de9f7ba server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:14.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.242+0000 7fbe6ca00640 1 -- 172.21.15.100:0/1907188761 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c0aa610 con 0x7fbe6815e860 2024-04-23T14:33:14.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.242+0000 7fbe6ca00640 1 -- 172.21.15.100:0/1907188761 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe5c0af070 con 0x7fbe6815e860 2024-04-23T14:33:14.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.242+0000 7fbe6ca00640 1 -- 172.21.15.100:0/1907188761 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c0aa9d0 con 0x7fbe6815e860 2024-04-23T14:33:14.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.242+0000 7fbe70800640 1 -- 172.21.15.100:0/1907188761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe6815e860 msgr2=0x7fbe680b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:14.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.242+0000 7fbe70800640 1 --2- 172.21.15.100:0/1907188761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe6815e860 0x7fbe680b3ab0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbe680d0aa0 tx=0x7fbe5c01c4b0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.242+0000 7fbe70800640 1 -- 172.21.15.100:0/1907188761 shutdown_connections 2024-04-23T14:33:14.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.242+0000 7fbe70800640 1 --2- 172.21.15.100:0/1907188761 >> v2:172.21.15.151:3300/0 conn(0x7fbe680adb90 0x7fbe680adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.242+0000 7fbe70800640 1 --2- 172.21.15.100:0/1907188761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe6815e860 0x7fbe680b3ab0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.242+0000 7fbe70800640 1 -- 172.21.15.100:0/1907188761 >> 172.21.15.100:0/1907188761 conn(0x7fbe68072590 msgr2=0x7fbe680729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:14.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.242+0000 7fbe70800640 1 -- 172.21.15.100:0/1907188761 shutdown_connections 2024-04-23T14:33:14.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.242+0000 7fbe70800640 1 -- 172.21.15.100:0/1907188761 wait complete. 2024-04-23T14:33:14.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.242+0000 7fbe70800640 1 Processor -- start 2024-04-23T14:33:14.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.242+0000 7fbe70800640 1 -- start start 2024-04-23T14:33:14.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe70800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe6806f950 0x7fbe6815bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:14.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe70800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe680adb90 0x7fbe68142400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:14.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe70800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbe6815e860 0x7fbe6814b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:14.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe70800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6815ec40 con 0x7fbe680adb90 2024-04-23T14:33:14.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe70800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbe68073790 con 0x7fbe6806f950 2024-04-23T14:33:14.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe70800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fbe6806da60 con 0x7fbe6815e860 2024-04-23T14:33:14.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe6d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe680adb90 0x7fbe68142400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:14.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe6de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe6806f950 0x7fbe6815bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:14.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe6d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe680adb90 0x7fbe68142400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60916/0 (socket says 172.21.15.100:60916) 2024-04-23T14:33:14.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe6d400640 1 -- 172.21.15.100:0/3669017234 learned_addr learned my addr 172.21.15.100:0/3669017234 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:14.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe6e800640 1 --2- 172.21.15.100:0/3669017234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbe6815e860 0x7fbe6814b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:14.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe6d400640 1 -- 172.21.15.100:0/3669017234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbe6815e860 msgr2=0x7fbe6814b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:14.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe6d400640 1 --2- 172.21.15.100:0/3669017234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbe6815e860 0x7fbe6814b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe6d400640 1 -- 172.21.15.100:0/3669017234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe6806f950 msgr2=0x7fbe6815bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:14.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe6d400640 1 --2- 172.21.15.100:0/3669017234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe6806f950 0x7fbe6815bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe6d400640 1 -- 172.21.15.100:0/3669017234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6814c210 con 0x7fbe680adb90 2024-04-23T14:33:14.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe6d400640 1 --2- 172.21.15.100:0/3669017234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe680adb90 0x7fbe68142400 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800d800 tx=0x7fbe5800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:14.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe62000640 1 -- 172.21.15.100:0/3669017234 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5800b6c0 con 0x7fbe680adb90 2024-04-23T14:33:14.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe62000640 1 -- 172.21.15.100:0/3669017234 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe58019070 con 0x7fbe680adb90 2024-04-23T14:33:14.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe70800640 1 -- 172.21.15.100:0/3669017234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6814c4f0 con 0x7fbe680adb90 2024-04-23T14:33:14.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe62000640 1 -- 172.21.15.100:0/3669017234 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5800ba60 con 0x7fbe680adb90 2024-04-23T14:33:14.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe70800640 1 -- 172.21.15.100:0/3669017234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe68142c30 con 0x7fbe680adb90 2024-04-23T14:33:14.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe70800640 1 -- 172.21.15.100:0/3669017234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe6814ca10 con 0x7fbe680adb90 2024-04-23T14:33:14.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.246+0000 7fbe62000640 1 -- 172.21.15.100:0/3669017234 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbe5800bc00 con 0x7fbe680adb90 2024-04-23T14:33:14.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.250+0000 7fbe62000640 1 --2- 172.21.15.100:0/3669017234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fbe4c068380 0x7fbe4c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:14.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.250+0000 7fbe62000640 1 -- 172.21.15.100:0/3669017234 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fbe58087190 con 0x7fbe680adb90 2024-04-23T14:33:14.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.250+0000 7fbe6de00640 1 --2- 172.21.15.100:0/3669017234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fbe4c068380 0x7fbe4c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:14.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.250+0000 7fbe6de00640 1 --2- 172.21.15.100:0/3669017234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fbe4c068380 0x7fbe4c06a840 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0ae000 tx=0x7fbe5c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:14.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.254+0000 7fbe62000640 1 -- 172.21.15.100:0/3669017234 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbe58056c60 con 0x7fbe680adb90 2024-04-23T14:33:14.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.410+0000 7fbe70800640 1 -- 172.21.15.100:0/3669017234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fbe68068ad0 con 0x7fbe680adb90 2024-04-23T14:33:14.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.410+0000 7fbe62000640 1 -- 172.21.15.100:0/3669017234 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fbe5805af60 con 0x7fbe680adb90 2024-04-23T14:33:14.412 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:14.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.410+0000 7fbe70800640 1 -- 172.21.15.100:0/3669017234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fbe4c068380 msgr2=0x7fbe4c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:14.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.410+0000 7fbe70800640 1 --2- 172.21.15.100:0/3669017234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fbe4c068380 0x7fbe4c06a840 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0ae000 tx=0x7fbe5c0a7000 comp rx=0 tx=0).stop 2024-04-23T14:33:14.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.410+0000 7fbe70800640 1 -- 172.21.15.100:0/3669017234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe680adb90 msgr2=0x7fbe68142400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:14.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.410+0000 7fbe70800640 1 --2- 172.21.15.100:0/3669017234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe680adb90 0x7fbe68142400 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800d800 tx=0x7fbe5800dcc0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.410+0000 7fbe70800640 1 -- 172.21.15.100:0/3669017234 shutdown_connections 2024-04-23T14:33:14.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.410+0000 7fbe70800640 1 --2- 172.21.15.100:0/3669017234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fbe4c068380 0x7fbe4c06a840 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.410+0000 7fbe70800640 1 --2- 172.21.15.100:0/3669017234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbe6815e860 0x7fbe6814b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.410+0000 7fbe70800640 1 --2- 172.21.15.100:0/3669017234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe680adb90 0x7fbe68142400 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.410+0000 7fbe70800640 1 --2- 172.21.15.100:0/3669017234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe6806f950 0x7fbe6815bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.410+0000 7fbe70800640 1 -- 172.21.15.100:0/3669017234 >> 172.21.15.100:0/3669017234 conn(0x7fbe68072590 msgr2=0x7fbe6806c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:14.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.410+0000 7fbe70800640 1 -- 172.21.15.100:0/3669017234 shutdown_connections 2024-04-23T14:33:14.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.410+0000 7fbe70800640 1 -- 172.21.15.100:0/3669017234 wait complete. 2024-04-23T14:33:14.432 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:33:14.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.566+0000 7f0e02800640 1 Processor -- start 2024-04-23T14:33:14.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.566+0000 7f0e02800640 1 -- start start 2024-04-23T14:33:14.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.566+0000 7f0e02800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f0dfc0cde20 0x7f0dfc0cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:14.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.566+0000 7f0e02800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dfc0ce500 con 0x7f0dfc15d0c0 2024-04-23T14:33:14.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.566+0000 7f0e02800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dfc0ce7e0 con 0x7f0dfc0cbee0 2024-04-23T14:33:14.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.566+0000 7f0e02800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f0dfc0cff50 con 0x7f0dfc0cde20 2024-04-23T14:33:14.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.566+0000 7f0dfb400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f0dfc15d0c0 0x7f0dfc15d4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:36314/0 (socket says 172.21.15.100:36314) 2024-04-23T14:33:14.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.566+0000 7f0dfb400640 1 -- 172.21.15.100:0/3458773585 learned_addr learned my addr 172.21.15.100:0/3458773585 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:14.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.566+0000 7f0df3e00640 1 --2- 172.21.15.100:0/3458773585 >> v2:172.21.15.151:3300/0 conn(0x7f0dfc0cde20 0x7f0dfc0cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:14.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.566+0000 7f0dfaa00640 1 -- 172.21.15.100:0/3458773585 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 747037979 0 0) 0x7f0dfc0ce500 con 0x7f0dfc15d0c0 2024-04-23T14:33:14.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.566+0000 7f0dfaa00640 1 -- 172.21.15.100:0/3458773585 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0de0003530 con 0x7f0dfc15d0c0 2024-04-23T14:33:14.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.566+0000 7f0dfaa00640 1 -- 172.21.15.100:0/3458773585 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 57381703 0 0) 0x7f0dfc0ce7e0 con 0x7f0dfc0cbee0 2024-04-23T14:33:14.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.566+0000 7f0dfaa00640 1 -- 172.21.15.100:0/3458773585 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0dfc0ce500 con 0x7f0dfc0cbee0 2024-04-23T14:33:14.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0dfaa00640 1 -- 172.21.15.100:0/3458773585 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3439467025 0 0) 0x7f0de0003530 con 0x7f0dfc15d0c0 2024-04-23T14:33:14.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0dfaa00640 1 -- 172.21.15.100:0/3458773585 >> v2:172.21.15.151:3300/0 conn(0x7f0dfc0cde20 msgr2=0x7f0dfc0cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:14.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0dfaa00640 1 --2- 172.21.15.100:0/3458773585 >> v2:172.21.15.151:3300/0 conn(0x7f0dfc0cde20 0x7f0dfc0cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0dfaa00640 1 -- 172.21.15.100:0/3458773585 >> v1:172.21.15.151:6789/0 conn(0x7f0dfc0cbee0 legacy=0x7f0dfc0cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:14.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0dfaa00640 1 -- 172.21.15.100:0/3458773585 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dfc0ceac0 con 0x7f0dfc15d0c0 2024-04-23T14:33:14.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0dfaa00640 1 -- 172.21.15.100:0/3458773585 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f0dec001680 con 0x7f0dfc15d0c0 2024-04-23T14:33:14.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0df3e00640 1 --2- 172.21.15.100:0/3458773585 >> v2:172.21.15.151:3300/0 conn(0x7f0dfc0cde20 0x7f0dfc0cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:33:14.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0dfaa00640 1 -- 172.21.15.100:0/3458773585 >> v1:172.21.15.100:6789/0 conn(0x7f0dfc15d0c0 legacy=0x7f0dfc15d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:14.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0dfaa00640 1 --2- 172.21.15.100:0/3458773585 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0de0003ba0 0x7f0de0003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:14.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0dfaa00640 1 --2- 172.21.15.100:0/3458773585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de0004810 0x7f0de0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:14.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0dfaa00640 1 --2- 172.21.15.100:0/3458773585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0de0005220 0x7f0de000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:14.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0dfaa00640 1 -- 172.21.15.100:0/3458773585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc0cff50 con 0x7f0de0004810 2024-04-23T14:33:14.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0dfaa00640 1 -- 172.21.15.100:0/3458773585 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0de000c150 con 0x7f0de0005220 2024-04-23T14:33:14.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0dfaa00640 1 -- 172.21.15.100:0/3458773585 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0de000c350 con 0x7f0de0003ba0 2024-04-23T14:33:14.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0df3e00640 1 --2- 172.21.15.100:0/3458773585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de0004810 0x7f0de0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:14.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0dfb400640 1 --2- 172.21.15.100:0/3458773585 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0de0003ba0 0x7f0de0003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:14.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0dfbe00640 1 --2- 172.21.15.100:0/3458773585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0de0005220 0x7f0de000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:14.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0df3e00640 1 -- 172.21.15.100:0/3458773585 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0de0003ba0 msgr2=0x7f0de0003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:14.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0df3e00640 1 --2- 172.21.15.100:0/3458773585 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0de0003ba0 0x7f0de0003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0df3e00640 1 -- 172.21.15.100:0/3458773585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0de0005220 msgr2=0x7f0de000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:14.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0df3e00640 1 --2- 172.21.15.100:0/3458773585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0de0005220 0x7f0de000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0dfbe00640 1 -- 172.21.15.100:0/3458773585 reap_dead start 2024-04-23T14:33:14.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0df3e00640 1 -- 172.21.15.100:0/3458773585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de0003530 con 0x7f0de0004810 2024-04-23T14:33:14.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.570+0000 7f0df3e00640 1 --2- 172.21.15.100:0/3458773585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de0004810 0x7f0de0004c90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009cb0 tx=0x7f0de401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f974bdeadaaacb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:14.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0dfaa00640 1 -- 172.21.15.100:0/3458773585 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de40b2070 con 0x7f0de0004810 2024-04-23T14:33:14.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0dfaa00640 1 -- 172.21.15.100:0/3458773585 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0de40a5670 con 0x7f0de0004810 2024-04-23T14:33:14.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0dfaa00640 1 -- 172.21.15.100:0/3458773585 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4009880 con 0x7f0de0004810 2024-04-23T14:33:14.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0e02800640 1 -- 172.21.15.100:0/3458773585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de0004810 msgr2=0x7f0de0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:14.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0e02800640 1 --2- 172.21.15.100:0/3458773585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de0004810 0x7f0de0004c90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009cb0 tx=0x7f0de401cc10 comp rx=0 tx=0).stop 2024-04-23T14:33:14.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0e02800640 1 -- 172.21.15.100:0/3458773585 shutdown_connections 2024-04-23T14:33:14.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0e02800640 1 --2- 172.21.15.100:0/3458773585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de0004810 0x7f0de0004c90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0e02800640 1 -- 172.21.15.100:0/3458773585 >> 172.21.15.100:0/3458773585 conn(0x7f0dfc0b5310 msgr2=0x7f0dfc0b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:14.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0e02800640 1 -- 172.21.15.100:0/3458773585 shutdown_connections 2024-04-23T14:33:14.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0e02800640 1 -- 172.21.15.100:0/3458773585 wait complete. 2024-04-23T14:33:14.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0e02800640 1 Processor -- start 2024-04-23T14:33:14.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0e02800640 1 -- start start 2024-04-23T14:33:14.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0e02800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de0004810 0x7f0dfc156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:14.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0e02800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0dfc071db0 0x7f0dfc0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:14.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0e02800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dfc0c6330 0x7f0dfc0b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:14.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0e02800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc0cf4f0 con 0x7f0de0004810 2024-04-23T14:33:14.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0e02800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc0cf370 con 0x7f0dfc0c6330 2024-04-23T14:33:14.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0e02800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0dfc0cf670 con 0x7f0dfc071db0 2024-04-23T14:33:14.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0dfb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de0004810 0x7f0dfc156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:14.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0dfb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de0004810 0x7f0dfc156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60926/0 (socket says 172.21.15.100:60926) 2024-04-23T14:33:14.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0dfb400640 1 -- 172.21.15.100:0/152844704 learned_addr learned my addr 172.21.15.100:0/152844704 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:14.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0dfbe00640 1 --2- 172.21.15.100:0/152844704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dfc0c6330 0x7f0dfc0b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:14.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.574+0000 7f0df3e00640 1 --2- 172.21.15.100:0/152844704 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0dfc071db0 0x7f0dfc0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:14.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.578+0000 7f0dfb400640 1 -- 172.21.15.100:0/152844704 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0dfc071db0 msgr2=0x7f0dfc0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:14.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.578+0000 7f0dfb400640 1 --2- 172.21.15.100:0/152844704 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0dfc071db0 0x7f0dfc0b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.578+0000 7f0dfb400640 1 -- 172.21.15.100:0/152844704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dfc0c6330 msgr2=0x7f0dfc0b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:14.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.578+0000 7f0dfb400640 1 --2- 172.21.15.100:0/152844704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dfc0c6330 0x7f0dfc0b2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.578+0000 7f0dfb400640 1 -- 172.21.15.100:0/152844704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dfc0b0ce0 con 0x7f0de0004810 2024-04-23T14:33:14.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.578+0000 7f0dfb400640 1 --2- 172.21.15.100:0/152844704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de0004810 0x7f0dfc156380 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0dec000c10 tx=0x7f0dec007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:14.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.578+0000 7f0df3400640 1 -- 172.21.15.100:0/152844704 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec00f720 con 0x7f0de0004810 2024-04-23T14:33:14.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.578+0000 7f0df3400640 1 -- 172.21.15.100:0/152844704 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dec018070 con 0x7f0de0004810 2024-04-23T14:33:14.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.578+0000 7f0df3400640 1 -- 172.21.15.100:0/152844704 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec00fb40 con 0x7f0de0004810 2024-04-23T14:33:14.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.578+0000 7f0e02800640 1 -- 172.21.15.100:0/152844704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dfc0c5290 con 0x7f0de0004810 2024-04-23T14:33:14.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.578+0000 7f0e02800640 1 -- 172.21.15.100:0/152844704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dfc0c5850 con 0x7f0de0004810 2024-04-23T14:33:14.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.578+0000 7f0e02800640 1 -- 172.21.15.100:0/152844704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dfc068ad0 con 0x7f0de0004810 2024-04-23T14:33:14.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.578+0000 7f0df3400640 1 -- 172.21.15.100:0/152844704 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0dec00fce0 con 0x7f0de0004810 2024-04-23T14:33:14.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.578+0000 7f0df3400640 1 --2- 172.21.15.100:0/152844704 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0dd80683b0 0x7f0dd806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:14.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.578+0000 7f0df3400640 1 -- 172.21.15.100:0/152844704 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f0dec086bc0 con 0x7f0de0004810 2024-04-23T14:33:14.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.578+0000 7f0df3e00640 1 --2- 172.21.15.100:0/152844704 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0dd80683b0 0x7f0dd806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:14.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.582+0000 7f0df3e00640 1 --2- 172.21.15.100:0/152844704 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0dd80683b0 0x7f0dd806a870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0de40af000 tx=0x7f0de40046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:14.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.582+0000 7f0df3400640 1 -- 172.21.15.100:0/152844704 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0dec056ce0 con 0x7f0de0004810 2024-04-23T14:33:14.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.750+0000 7f0e02800640 1 -- 172.21.15.100:0/152844704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0dfc06c730 con 0x7f0de0004810 2024-04-23T14:33:14.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.750+0000 7f0df3400640 1 -- 172.21.15.100:0/152844704 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f0dec0565e0 con 0x7f0de0004810 2024-04-23T14:33:14.754 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-04-23T14:33:14.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.754+0000 7f0e02800640 1 -- 172.21.15.100:0/152844704 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0dd80683b0 msgr2=0x7f0dd806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:14.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.754+0000 7f0e02800640 1 --2- 172.21.15.100:0/152844704 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0dd80683b0 0x7f0dd806a870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0de40af000 tx=0x7f0de40046d0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.754+0000 7f0e02800640 1 -- 172.21.15.100:0/152844704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de0004810 msgr2=0x7f0dfc156380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:14.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.754+0000 7f0e02800640 1 --2- 172.21.15.100:0/152844704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de0004810 0x7f0dfc156380 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0dec000c10 tx=0x7f0dec007430 comp rx=0 tx=0).stop 2024-04-23T14:33:14.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.754+0000 7f0e02800640 1 -- 172.21.15.100:0/152844704 shutdown_connections 2024-04-23T14:33:14.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.754+0000 7f0e02800640 1 --2- 172.21.15.100:0/152844704 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0dd80683b0 0x7f0dd806a870 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.754+0000 7f0e02800640 1 --2- 172.21.15.100:0/152844704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dfc0c6330 0x7f0dfc0b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.754+0000 7f0e02800640 1 --2- 172.21.15.100:0/152844704 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0dfc071db0 0x7f0dfc0b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.754+0000 7f0e02800640 1 --2- 172.21.15.100:0/152844704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de0004810 0x7f0dfc156380 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:14.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.754+0000 7f0e02800640 1 -- 172.21.15.100:0/152844704 >> 172.21.15.100:0/152844704 conn(0x7f0dfc0b5310 msgr2=0x7f0dfc0d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:14.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.754+0000 7f0e02800640 1 -- 172.21.15.100:0/152844704 shutdown_connections 2024-04-23T14:33:14.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:14.754+0000 7f0e02800640 1 -- 172.21.15.100:0/152844704 wait complete. 2024-04-23T14:33:14.775 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi151.front.sepia.ceph.com... 2024-04-23T14:33:14.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:14.776 DEBUG:teuthology.orchestra.run.smithi151:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-23T14:33:14.809 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:33:14.809 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-04-23T14:33:14.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:14.809 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-23T14:33:14.864 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:33:14.864 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:14.873 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:33:14.873 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:14.922 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:15.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.094+0000 7f062f000640 1 Processor -- start 2024-04-23T14:33:15.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.094+0000 7f062f000640 1 -- start start 2024-04-23T14:33:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.098+0000 7f062f000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f062806d350 0x7f0628073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.098+0000 7f062f000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f062806ef80 0x7f062806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.098+0000 7f062f000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0628073880 con 0x7f062815fa90 2024-04-23T14:33:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.098+0000 7f062f000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f062806eaf0 con 0x7f062806ef80 2024-04-23T14:33:15.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.098+0000 7f062f000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f062806e530 con 0x7f062806d350 2024-04-23T14:33:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.098+0000 7f0627e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f062806d350 0x7f0628073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.098+0000 7f062d000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f062806ef80 0x7f062806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.098+0000 7f062d000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f062806ef80 0x7f062806f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:39084/0 (socket says 172.21.15.100:39084) 2024-04-23T14:33:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.098+0000 7f062d000640 1 -- 172.21.15.100:0/1398834995 learned_addr learned my addr 172.21.15.100:0/1398834995 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.098+0000 7f0627400640 1 -- 172.21.15.100:0/1398834995 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 982555216 0 0) 0x7f0628073880 con 0x7f062815fa90 2024-04-23T14:33:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.098+0000 7f0627e00640 1 -- 172.21.15.100:0/1398834995 >> v2:172.21.15.151:3300/0 conn(0x7f062806ef80 msgr2=0x7f062806f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.098+0000 7f0627e00640 1 --2- 172.21.15.100:0/1398834995 >> v2:172.21.15.151:3300/0 conn(0x7f062806ef80 0x7f062806f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.098+0000 7f0627e00640 1 -- 172.21.15.100:0/1398834995 >> v1:172.21.15.151:6789/0 conn(0x7f062815fa90 legacy=0x7f062806d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.098+0000 7f0627e00640 1 -- 172.21.15.100:0/1398834995 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06280b47f0 con 0x7f062806d350 2024-04-23T14:33:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.098+0000 7f062d000640 1 --2- 172.21.15.100:0/1398834995 >> v2:172.21.15.151:3300/0 conn(0x7f062806ef80 0x7f062806f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:33:15.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.102+0000 7f0627e00640 1 --2- 172.21.15.100:0/1398834995 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f062806d350 0x7f0628073340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f06180097c0 tx=0x7f061801c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eae8b7a9fd44d83c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:15.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.102+0000 7f0627400640 1 -- 172.21.15.100:0/1398834995 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06180b1070 con 0x7f062806d350 2024-04-23T14:33:15.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.102+0000 7f0627400640 1 -- 172.21.15.100:0/1398834995 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06180a8050 con 0x7f062806d350 2024-04-23T14:33:15.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.102+0000 7f0627400640 1 -- 172.21.15.100:0/1398834995 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06180ac860 con 0x7f062806d350 2024-04-23T14:33:15.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.102+0000 7f062f000640 1 -- 172.21.15.100:0/1398834995 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f062806d350 msgr2=0x7f0628073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:15.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.102+0000 7f062f000640 1 --2- 172.21.15.100:0/1398834995 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f062806d350 0x7f0628073340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f06180097c0 tx=0x7f061801c3e0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.102+0000 7f062f000640 1 -- 172.21.15.100:0/1398834995 shutdown_connections 2024-04-23T14:33:15.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.102+0000 7f062f000640 1 --2- 172.21.15.100:0/1398834995 >> v2:172.21.15.151:3300/0 conn(0x7f062806ef80 0x7f062806f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.102+0000 7f062f000640 1 --2- 172.21.15.100:0/1398834995 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f062806d350 0x7f0628073340 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.102+0000 7f062f000640 1 -- 172.21.15.100:0/1398834995 >> 172.21.15.100:0/1398834995 conn(0x7f0628072600 msgr2=0x7f0628072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:15.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.102+0000 7f062f000640 1 -- 172.21.15.100:0/1398834995 shutdown_connections 2024-04-23T14:33:15.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.102+0000 7f062f000640 1 -- 172.21.15.100:0/1398834995 wait complete. 2024-04-23T14:33:15.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.102+0000 7f062f000640 1 Processor -- start 2024-04-23T14:33:15.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.102+0000 7f062f000640 1 -- start start 2024-04-23T14:33:15.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f062f000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f062806d350 0x7f06281558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:15.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f062f000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f062806ef80 0x7f062813cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:15.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f062f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f062815fa90 0x7f062815df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:15.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f062f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f062816bb10 con 0x7f062815fa90 2024-04-23T14:33:15.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f062f000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f062816b990 con 0x7f062806ef80 2024-04-23T14:33:15.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f062f000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f062816bc90 con 0x7f062806d350 2024-04-23T14:33:15.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f062d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f062815fa90 0x7f062815df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:15.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f062d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f062815fa90 0x7f062815df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60940/0 (socket says 172.21.15.100:60940) 2024-04-23T14:33:15.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f062d000640 1 -- 172.21.15.100:0/3255989339 learned_addr learned my addr 172.21.15.100:0/3255989339 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:15.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f0627e00640 1 --2- 172.21.15.100:0/3255989339 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f062806d350 0x7f06281558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:15.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f061f400640 1 --2- 172.21.15.100:0/3255989339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f062806ef80 0x7f062813cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:15.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f062d000640 1 -- 172.21.15.100:0/3255989339 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f062806d350 msgr2=0x7f06281558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:15.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f062d000640 1 --2- 172.21.15.100:0/3255989339 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f062806d350 0x7f06281558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f062d000640 1 -- 172.21.15.100:0/3255989339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f062806ef80 msgr2=0x7f062813cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:15.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f062d000640 1 --2- 172.21.15.100:0/3255989339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f062806ef80 0x7f062813cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f062d000640 1 -- 172.21.15.100:0/3255989339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062815eaa0 con 0x7f062815fa90 2024-04-23T14:33:15.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f062d000640 1 --2- 172.21.15.100:0/3255989339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f062815fa90 0x7f062815df50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f062000e980 tx=0x7f062000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:15.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f0624c00640 1 -- 172.21.15.100:0/3255989339 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0620017070 con 0x7f062815fa90 2024-04-23T14:33:15.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f0624c00640 1 -- 172.21.15.100:0/3255989339 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f062000ca00 con 0x7f062815fa90 2024-04-23T14:33:15.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f062f000640 1 -- 172.21.15.100:0/3255989339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f062815ed80 con 0x7f062815fa90 2024-04-23T14:33:15.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f0624c00640 1 -- 172.21.15.100:0/3255989339 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062000cdd0 con 0x7f062815fa90 2024-04-23T14:33:15.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f062f000640 1 -- 172.21.15.100:0/3255989339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0628151a30 con 0x7f062815fa90 2024-04-23T14:33:15.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f062f000640 1 -- 172.21.15.100:0/3255989339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0628151c30 con 0x7f062815fa90 2024-04-23T14:33:15.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.106+0000 7f0624c00640 1 -- 172.21.15.100:0/3255989339 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0620013470 con 0x7f062815fa90 2024-04-23T14:33:15.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.110+0000 7f0624c00640 1 --2- 172.21.15.100:0/3255989339 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0604068380 0x7f060406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:15.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.110+0000 7f0624c00640 1 -- 172.21.15.100:0/3255989339 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f0620086c60 con 0x7f062815fa90 2024-04-23T14:33:15.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.110+0000 7f0627e00640 1 --2- 172.21.15.100:0/3255989339 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0604068380 0x7f060406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:15.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.110+0000 7f0627e00640 1 --2- 172.21.15.100:0/3255989339 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0604068380 0x7f060406a840 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f06180ae000 tx=0x7f06180a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:15.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.114+0000 7f0624c00640 1 -- 172.21.15.100:0/3255989339 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f06200567c0 con 0x7f062815fa90 2024-04-23T14:33:15.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.270+0000 7f062f000640 1 -- 172.21.15.100:0/3255989339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0628068ad0 con 0x7f062815fa90 2024-04-23T14:33:15.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.274+0000 7f0624c00640 1 -- 172.21.15.100:0/3255989339 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18859 (secure 0 0 0) 0x7f062005aac0 con 0x7f062815fa90 2024-04-23T14:33:15.275 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:15.275 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:49.784608+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:15.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.274+0000 7f062f000640 1 -- 172.21.15.100:0/3255989339 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0604068380 msgr2=0x7f060406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:15.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.274+0000 7f062f000640 1 --2- 172.21.15.100:0/3255989339 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0604068380 0x7f060406a840 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f06180ae000 tx=0x7f06180a7000 comp rx=0 tx=0).stop 2024-04-23T14:33:15.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.274+0000 7f062f000640 1 -- 172.21.15.100:0/3255989339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f062815fa90 msgr2=0x7f062815df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:15.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.274+0000 7f062f000640 1 --2- 172.21.15.100:0/3255989339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f062815fa90 0x7f062815df50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f062000e980 tx=0x7f062000ee40 comp rx=0 tx=0).stop 2024-04-23T14:33:15.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.274+0000 7f062f000640 1 -- 172.21.15.100:0/3255989339 shutdown_connections 2024-04-23T14:33:15.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.274+0000 7f062f000640 1 --2- 172.21.15.100:0/3255989339 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0604068380 0x7f060406a840 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.274+0000 7f062f000640 1 --2- 172.21.15.100:0/3255989339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f062815fa90 0x7f062815df50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.274+0000 7f062f000640 1 --2- 172.21.15.100:0/3255989339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f062806ef80 0x7f062813cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.274+0000 7f062f000640 1 --2- 172.21.15.100:0/3255989339 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f062806d350 0x7f06281558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.274+0000 7f062f000640 1 -- 172.21.15.100:0/3255989339 >> 172.21.15.100:0/3255989339 conn(0x7f0628072600 msgr2=0x7f06280af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:15.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.274+0000 7f062f000640 1 -- 172.21.15.100:0/3255989339 shutdown_connections 2024-04-23T14:33:15.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.274+0000 7f062f000640 1 -- 172.21.15.100:0/3255989339 wait complete. 2024-04-23T14:33:15.296 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:32:06.835560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:32:07.930489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-23T14:32:11.636814+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-23T14:33:15.297 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:33:15.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1ff7600640 1 Processor -- start 2024-04-23T14:33:15.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1ff7600640 1 -- start start 2024-04-23T14:33:15.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1ff7600640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f1ff006bb20 0x7f1ff00ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:15.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1ff7600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ff00cd540 con 0x7f1ff00cd130 2024-04-23T14:33:15.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1ff7600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ff006cbb0 con 0x7f1ff015c7a0 2024-04-23T14:33:15.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1ff7600640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f1ff006d450 con 0x7f1ff006bb20 2024-04-23T14:33:15.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1ff5600640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1ff00cd130 0x7f1ff00ad430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:36320/0 (socket says 172.21.15.100:36320) 2024-04-23T14:33:15.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1ff5600640 1 -- 172.21.15.100:0/3752650544 learned_addr learned my addr 172.21.15.100:0/3752650544 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:15.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1fefe00640 1 --2- 172.21.15.100:0/3752650544 >> v2:172.21.15.151:3300/0 conn(0x7f1ff006bb20 0x7f1ff00ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:15.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1fef400640 1 -- 172.21.15.100:0/3752650544 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2614085410 0 0) 0x7f1fe80014a0 con 0x7f1ff00cd130 2024-04-23T14:33:15.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1fef400640 1 -- 172.21.15.100:0/3752650544 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fd8003530 con 0x7f1ff00cd130 2024-04-23T14:33:15.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1fef400640 1 -- 172.21.15.100:0/3752650544 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3546489967 0 0) 0x7f1fd8003530 con 0x7f1ff00cd130 2024-04-23T14:33:15.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1fef400640 1 -- 172.21.15.100:0/3752650544 >> v2:172.21.15.151:3300/0 conn(0x7f1ff006bb20 msgr2=0x7f1ff00ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:15.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1fef400640 1 --2- 172.21.15.100:0/3752650544 >> v2:172.21.15.151:3300/0 conn(0x7f1ff006bb20 0x7f1ff00ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1fef400640 1 -- 172.21.15.100:0/3752650544 >> v1:172.21.15.151:6789/0 conn(0x7f1ff015c7a0 legacy=0x7f1ff00af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:15.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1fef400640 1 -- 172.21.15.100:0/3752650544 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff006ce90 con 0x7f1ff00cd130 2024-04-23T14:33:15.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1fef400640 1 -- 172.21.15.100:0/3752650544 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f1fe8001bd0 con 0x7f1ff00cd130 2024-04-23T14:33:15.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1fef400640 1 -- 172.21.15.100:0/3752650544 >> v1:172.21.15.100:6789/0 conn(0x7f1ff00cd130 legacy=0x7f1ff00ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:15.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1fef400640 1 --2- 172.21.15.100:0/3752650544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fd80039c0 0x7f1fd8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:15.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1fef400640 1 --2- 172.21.15.100:0/3752650544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd8004810 0x7f1fd8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:15.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1fef400640 1 --2- 172.21.15.100:0/3752650544 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1fd80051d0 0x7f1fd800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:15.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1fef400640 1 -- 172.21.15.100:0/3752650544 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff006d450 con 0x7f1fd8004810 2024-04-23T14:33:15.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1fef400640 1 -- 172.21.15.100:0/3752650544 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd800c170 con 0x7f1fd80039c0 2024-04-23T14:33:15.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1fef400640 1 -- 172.21.15.100:0/3752650544 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1fd800c370 con 0x7f1fd80051d0 2024-04-23T14:33:15.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1fefe00640 1 --2- 172.21.15.100:0/3752650544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd8004810 0x7f1fd8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:15.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1ff5600640 1 --2- 172.21.15.100:0/3752650544 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1fd80051d0 0x7f1fd800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:15.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1ff4c00640 1 --2- 172.21.15.100:0/3752650544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fd80039c0 0x7f1fd8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:15.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1fefe00640 1 -- 172.21.15.100:0/3752650544 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1fd80051d0 msgr2=0x7f1fd800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:15.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1fefe00640 1 --2- 172.21.15.100:0/3752650544 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1fd80051d0 0x7f1fd800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1fefe00640 1 -- 172.21.15.100:0/3752650544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fd80039c0 msgr2=0x7f1fd8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:15.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1fefe00640 1 --2- 172.21.15.100:0/3752650544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fd80039c0 0x7f1fd8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1fefe00640 1 -- 172.21.15.100:0/3752650544 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd8003530 con 0x7f1fd8004810 2024-04-23T14:33:15.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.434+0000 7f1ff5600640 1 -- 172.21.15.100:0/3752650544 reap_dead start 2024-04-23T14:33:15.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.438+0000 7f1fefe00640 1 --2- 172.21.15.100:0/3752650544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd8004810 0x7f1fd8004c90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009ab0 tx=0x7f1fdc01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49eb069f22549961 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.438+0000 7f1fef400640 1 -- 172.21.15.100:0/3752650544 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc0096b0 con 0x7f1fd8004810 2024-04-23T14:33:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.438+0000 7f1fef400640 1 -- 172.21.15.100:0/3752650544 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fdc0a8460 con 0x7f1fd8004810 2024-04-23T14:33:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.438+0000 7f1fef400640 1 -- 172.21.15.100:0/3752650544 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc0a8850 con 0x7f1fd8004810 2024-04-23T14:33:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.438+0000 7f1ff7600640 1 -- 172.21.15.100:0/3752650544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd8004810 msgr2=0x7f1fd8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.438+0000 7f1ff7600640 1 --2- 172.21.15.100:0/3752650544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd8004810 0x7f1fd8004c90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009ab0 tx=0x7f1fdc01ca50 comp rx=0 tx=0).stop 2024-04-23T14:33:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.438+0000 7f1ff7600640 1 -- 172.21.15.100:0/3752650544 shutdown_connections 2024-04-23T14:33:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.438+0000 7f1ff7600640 1 --2- 172.21.15.100:0/3752650544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd8004810 0x7f1fd8004c90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.438+0000 7f1ff7600640 1 -- 172.21.15.100:0/3752650544 >> 172.21.15.100:0/3752650544 conn(0x7f1ff00b5200 msgr2=0x7f1ff00b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.438+0000 7f1ff7600640 1 -- 172.21.15.100:0/3752650544 shutdown_connections 2024-04-23T14:33:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.438+0000 7f1ff7600640 1 -- 172.21.15.100:0/3752650544 wait complete. 2024-04-23T14:33:15.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1ff7600640 1 Processor -- start 2024-04-23T14:33:15.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1ff7600640 1 -- start start 2024-04-23T14:33:15.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1ff7600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fd8004810 0x7f1ff00c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:15.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1ff7600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1ff0073740 0x7f1ff00998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:15.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1ff7600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ff00c6210 0x7f1ff00c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:15.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1ff7600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff006e620 con 0x7f1ff00c6210 2024-04-23T14:33:15.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1ff7600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff006e4a0 con 0x7f1fd8004810 2024-04-23T14:33:15.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1ff7600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff006e7a0 con 0x7f1ff0073740 2024-04-23T14:33:15.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1ff5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ff00c6210 0x7f1ff00c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:15.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1ff5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ff00c6210 0x7f1ff00c6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60964/0 (socket says 172.21.15.100:60964) 2024-04-23T14:33:15.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1ff5600640 1 -- 172.21.15.100:0/729380236 learned_addr learned my addr 172.21.15.100:0/729380236 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:15.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1fefe00640 1 --2- 172.21.15.100:0/729380236 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1ff0073740 0x7f1ff00998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:15.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1ff4c00640 1 --2- 172.21.15.100:0/729380236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fd8004810 0x7f1ff00c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:15.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1ff5600640 1 -- 172.21.15.100:0/729380236 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1ff0073740 msgr2=0x7f1ff00998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:15.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1ff5600640 1 --2- 172.21.15.100:0/729380236 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1ff0073740 0x7f1ff00998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1ff5600640 1 -- 172.21.15.100:0/729380236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fd8004810 msgr2=0x7f1ff00c7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:15.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1ff5600640 1 --2- 172.21.15.100:0/729380236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fd8004810 0x7f1ff00c7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1ff5600640 1 -- 172.21.15.100:0/729380236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff00c8280 con 0x7f1ff00c6210 2024-04-23T14:33:15.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1ff5600640 1 --2- 172.21.15.100:0/729380236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ff00c6210 0x7f1ff00c6690 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1ff01600f0 tx=0x7f1fe800af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:15.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1fecc00640 1 -- 172.21.15.100:0/729380236 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8007800 con 0x7f1ff00c6210 2024-04-23T14:33:15.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1fecc00640 1 -- 172.21.15.100:0/729380236 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe8018070 con 0x7f1ff00c6210 2024-04-23T14:33:15.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1fecc00640 1 -- 172.21.15.100:0/729380236 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80133f0 con 0x7f1ff00c6210 2024-04-23T14:33:15.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1ff7600640 1 -- 172.21.15.100:0/729380236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff00c8560 con 0x7f1ff00c6210 2024-04-23T14:33:15.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1ff7600640 1 -- 172.21.15.100:0/729380236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff00c8b20 con 0x7f1ff00c6210 2024-04-23T14:33:15.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.442+0000 7f1ff7600640 1 -- 172.21.15.100:0/729380236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff0068ad0 con 0x7f1ff00c6210 2024-04-23T14:33:15.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.446+0000 7f1fecc00640 1 -- 172.21.15.100:0/729380236 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1fe8013590 con 0x7f1ff00c6210 2024-04-23T14:33:15.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.446+0000 7f1fecc00640 1 --2- 172.21.15.100:0/729380236 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1fd0068380 0x7f1fd006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:15.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.446+0000 7f1fecc00640 1 -- 172.21.15.100:0/729380236 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1fe80873a0 con 0x7f1ff00c6210 2024-04-23T14:33:15.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.446+0000 7f1ff4c00640 1 --2- 172.21.15.100:0/729380236 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1fd0068380 0x7f1fd006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:15.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.446+0000 7f1ff4c00640 1 --2- 172.21.15.100:0/729380236 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1fd0068380 0x7f1fd006a840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4007fd0 tx=0x7f1fe4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:15.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.450+0000 7f1fecc00640 1 -- 172.21.15.100:0/729380236 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1ff00cd540 con 0x7f1ff00c6210 2024-04-23T14:33:15.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.606+0000 7f1ff7600640 1 -- 172.21.15.100:0/729380236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1ff00ce810 con 0x7f1ff00c6210 2024-04-23T14:33:15.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.606+0000 7f1fecc00640 1 -- 172.21.15.100:0/729380236 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f1fe8057550 con 0x7f1ff00c6210 2024-04-23T14:33:15.608 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:33:15.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.606+0000 7f1ff7600640 1 -- 172.21.15.100:0/729380236 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1fd0068380 msgr2=0x7f1fd006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:15.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.606+0000 7f1ff7600640 1 --2- 172.21.15.100:0/729380236 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1fd0068380 0x7f1fd006a840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4007fd0 tx=0x7f1fe4009040 comp rx=0 tx=0).stop 2024-04-23T14:33:15.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.606+0000 7f1ff7600640 1 -- 172.21.15.100:0/729380236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ff00c6210 msgr2=0x7f1ff00c6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:15.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.606+0000 7f1ff7600640 1 --2- 172.21.15.100:0/729380236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ff00c6210 0x7f1ff00c6690 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1ff01600f0 tx=0x7f1fe800af60 comp rx=0 tx=0).stop 2024-04-23T14:33:15.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.606+0000 7f1ff7600640 1 -- 172.21.15.100:0/729380236 shutdown_connections 2024-04-23T14:33:15.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.606+0000 7f1ff7600640 1 --2- 172.21.15.100:0/729380236 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1fd0068380 0x7f1fd006a840 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.606+0000 7f1ff7600640 1 --2- 172.21.15.100:0/729380236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ff00c6210 0x7f1ff00c6690 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.606+0000 7f1ff7600640 1 --2- 172.21.15.100:0/729380236 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1ff0073740 0x7f1ff00998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.606+0000 7f1ff7600640 1 --2- 172.21.15.100:0/729380236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fd8004810 0x7f1ff00c7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.606+0000 7f1ff7600640 1 -- 172.21.15.100:0/729380236 >> 172.21.15.100:0/729380236 conn(0x7f1ff00b5200 msgr2=0x7f1ff015ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:15.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.606+0000 7f1ff7600640 1 -- 172.21.15.100:0/729380236 shutdown_connections 2024-04-23T14:33:15.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.606+0000 7f1ff7600640 1 -- 172.21.15.100:0/729380236 wait complete. 2024-04-23T14:33:15.628 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:33:15.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f9200640 1 Processor -- start 2024-04-23T14:33:15.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f9200640 1 -- start start 2024-04-23T14:33:15.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f9200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34f415e950 0x7f34f40cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:15.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f9200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34f40cecc0 con 0x7f34f40cd9a0 2024-04-23T14:33:15.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f9200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34f40cef60 con 0x7f34f406db60 2024-04-23T14:33:15.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f9200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f34f40d0300 con 0x7f34f415e950 2024-04-23T14:33:15.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34e9400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f34f40cd9a0 0x7f34f406d450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:36328/0 (socket says 172.21.15.100:36328) 2024-04-23T14:33:15.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34e9400640 1 -- 172.21.15.100:0/2376367151 learned_addr learned my addr 172.21.15.100:0/2376367151 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:15.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f1e00640 1 --2- 172.21.15.100:0/2376367151 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34f415e950 0x7f34f40cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:15.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f1400640 1 -- 172.21.15.100:0/2376367151 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3773107314 0 0) 0x7f34f40cecc0 con 0x7f34f40cd9a0 2024-04-23T14:33:15.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f1400640 1 -- 172.21.15.100:0/2376367151 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34d8003530 con 0x7f34f40cd9a0 2024-04-23T14:33:15.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f1400640 1 -- 172.21.15.100:0/2376367151 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3737428389 0 0) 0x7f34f40cef60 con 0x7f34f406db60 2024-04-23T14:33:15.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f1400640 1 -- 172.21.15.100:0/2376367151 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34f40cecc0 con 0x7f34f406db60 2024-04-23T14:33:15.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f1400640 1 -- 172.21.15.100:0/2376367151 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2617326918 0 0) 0x7f34d8003530 con 0x7f34f40cd9a0 2024-04-23T14:33:15.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f1400640 1 -- 172.21.15.100:0/2376367151 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34f415e950 msgr2=0x7f34f40cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:15.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f1400640 1 --2- 172.21.15.100:0/2376367151 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34f415e950 0x7f34f40cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f1400640 1 -- 172.21.15.100:0/2376367151 >> v1:172.21.15.151:6789/0 conn(0x7f34f406db60 legacy=0x7f34f40d0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:15.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f1400640 1 -- 172.21.15.100:0/2376367151 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f40cf240 con 0x7f34f40cd9a0 2024-04-23T14:33:15.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f1400640 1 -- 172.21.15.100:0/2376367151 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f34dc001a00 con 0x7f34f40cd9a0 2024-04-23T14:33:15.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f1400640 1 -- 172.21.15.100:0/2376367151 >> v1:172.21.15.100:6789/0 conn(0x7f34f40cd9a0 legacy=0x7f34f406d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:15.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f1400640 1 --2- 172.21.15.100:0/2376367151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34d8003b80 0x7f34d8003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:15.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f1400640 1 --2- 172.21.15.100:0/2376367151 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34f415e950 0x7f34d8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:15.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f1400640 1 --2- 172.21.15.100:0/2376367151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34d8004d50 0x7f34d80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:15.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f1400640 1 -- 172.21.15.100:0/2376367151 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f34f40d0300 con 0x7f34d8004d50 2024-04-23T14:33:15.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f1400640 1 -- 172.21.15.100:0/2376367151 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f34d800bb60 con 0x7f34d8003b80 2024-04-23T14:33:15.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.766+0000 7f34f1400640 1 -- 172.21.15.100:0/2376367151 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f34d800bd30 con 0x7f34f415e950 2024-04-23T14:33:15.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.770+0000 7f34f2800640 1 --2- 172.21.15.100:0/2376367151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34d8004d50 0x7f34d80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:15.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.770+0000 7f34f1e00640 1 --2- 172.21.15.100:0/2376367151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34d8003b80 0x7f34d8003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:15.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.770+0000 7f34e9400640 1 --2- 172.21.15.100:0/2376367151 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34f415e950 0x7f34d8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:15.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.770+0000 7f34f2800640 1 -- 172.21.15.100:0/2376367151 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34f415e950 msgr2=0x7f34d8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:15.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.770+0000 7f34f2800640 1 --2- 172.21.15.100:0/2376367151 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34f415e950 0x7f34d8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.770+0000 7f34f2800640 1 -- 172.21.15.100:0/2376367151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34d8003b80 msgr2=0x7f34d8003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:15.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.770+0000 7f34f2800640 1 --2- 172.21.15.100:0/2376367151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34d8003b80 0x7f34d8003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.770+0000 7f34f2800640 1 -- 172.21.15.100:0/2376367151 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d8003530 con 0x7f34d8004d50 2024-04-23T14:33:15.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.770+0000 7f34f2800640 1 --2- 172.21.15.100:0/2376367151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34d8004d50 0x7f34d80051d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f34ec001900 tx=0x7f34ec01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349091c22c61214b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:15.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.770+0000 7f34f1400640 1 -- 172.21.15.100:0/2376367151 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec001520 con 0x7f34d8004d50 2024-04-23T14:33:15.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.770+0000 7f34f1400640 1 -- 172.21.15.100:0/2376367151 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34ec0052f0 con 0x7f34d8004d50 2024-04-23T14:33:15.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.770+0000 7f34f1400640 1 -- 172.21.15.100:0/2376367151 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec0a8950 con 0x7f34d8004d50 2024-04-23T14:33:15.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34f9200640 1 -- 172.21.15.100:0/2376367151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34d8004d50 msgr2=0x7f34d80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:15.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34f9200640 1 --2- 172.21.15.100:0/2376367151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34d8004d50 0x7f34d80051d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f34ec001900 tx=0x7f34ec01ecf0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34f2800640 1 -- 172.21.15.100:0/2376367151 reap_dead start 2024-04-23T14:33:15.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34f9200640 1 -- 172.21.15.100:0/2376367151 shutdown_connections 2024-04-23T14:33:15.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34f9200640 1 -- 172.21.15.100:0/2376367151 >> 172.21.15.100:0/2376367151 conn(0x7f34f40719d0 msgr2=0x7f34f4071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:15.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34f9200640 1 -- 172.21.15.100:0/2376367151 shutdown_connections 2024-04-23T14:33:15.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34f9200640 1 -- 172.21.15.100:0/2376367151 wait complete. 2024-04-23T14:33:15.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34f9200640 1 Processor -- start 2024-04-23T14:33:15.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34f9200640 1 -- start start 2024-04-23T14:33:15.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34f9200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34d8004d50 0x7f34f4156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:15.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34f9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34f40c8be0 0x7f34f40caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:15.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34f9200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34f40cb020 0x7f34f41f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:15.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34f9200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f34f40cd460 con 0x7f34f40c8be0 2024-04-23T14:33:15.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34f9200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f34f40cf800 con 0x7f34f40cb020 2024-04-23T14:33:15.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34f9200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f34f40cd5e0 con 0x7f34d8004d50 2024-04-23T14:33:15.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34e9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34f40c8be0 0x7f34f40caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:15.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34e9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34f40c8be0 0x7f34f40caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60994/0 (socket says 172.21.15.100:60994) 2024-04-23T14:33:15.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34e9400640 1 -- 172.21.15.100:0/2639246581 learned_addr learned my addr 172.21.15.100:0/2639246581 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:15.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34f2800640 1 --2- 172.21.15.100:0/2639246581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34f40cb020 0x7f34f41f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:15.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34f1e00640 1 --2- 172.21.15.100:0/2639246581 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34d8004d50 0x7f34f4156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:15.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34e9400640 1 -- 172.21.15.100:0/2639246581 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34d8004d50 msgr2=0x7f34f4156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:15.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34e9400640 1 --2- 172.21.15.100:0/2639246581 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34d8004d50 0x7f34f4156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34e9400640 1 -- 172.21.15.100:0/2639246581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34f40cb020 msgr2=0x7f34f41f7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:15.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34e9400640 1 --2- 172.21.15.100:0/2639246581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34f40cb020 0x7f34f41f7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34e9400640 1 -- 172.21.15.100:0/2639246581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f415ef80 con 0x7f34f40c8be0 2024-04-23T14:33:15.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.774+0000 7f34e9400640 1 --2- 172.21.15.100:0/2639246581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34f40c8be0 0x7f34f40caae0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0079d0 tx=0x7f34dc00a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:15.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.778+0000 7f34eaa00640 1 -- 172.21.15.100:0/2639246581 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc017020 con 0x7f34f40c8be0 2024-04-23T14:33:15.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.778+0000 7f34eaa00640 1 -- 172.21.15.100:0/2639246581 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34dc0019c0 con 0x7f34f40c8be0 2024-04-23T14:33:15.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.778+0000 7f34eaa00640 1 -- 172.21.15.100:0/2639246581 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc013610 con 0x7f34f40c8be0 2024-04-23T14:33:15.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.778+0000 7f34f9200640 1 -- 172.21.15.100:0/2639246581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f415f260 con 0x7f34f40c8be0 2024-04-23T14:33:15.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.778+0000 7f34f9200640 1 -- 172.21.15.100:0/2639246581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f415f820 con 0x7f34f40c8be0 2024-04-23T14:33:15.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.778+0000 7f34f9200640 1 -- 172.21.15.100:0/2639246581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f40cb430 con 0x7f34f40c8be0 2024-04-23T14:33:15.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.778+0000 7f34eaa00640 1 -- 172.21.15.100:0/2639246581 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f34dc01a020 con 0x7f34f40c8be0 2024-04-23T14:33:15.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.778+0000 7f34eaa00640 1 --2- 172.21.15.100:0/2639246581 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f34d0068380 0x7f34d006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:15.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.778+0000 7f34eaa00640 1 -- 172.21.15.100:0/2639246581 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f34dc087840 con 0x7f34f40c8be0 2024-04-23T14:33:15.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.778+0000 7f34f1e00640 1 --2- 172.21.15.100:0/2639246581 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f34d0068380 0x7f34d006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:15.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.778+0000 7f34f1e00640 1 --2- 172.21.15.100:0/2639246581 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f34d0068380 0x7f34d006a840 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f34e4009b20 tx=0x7f34e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:15.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.782+0000 7f34eaa00640 1 -- 172.21.15.100:0/2639246581 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f34f40cb430 con 0x7f34f40c8be0 2024-04-23T14:33:15.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.942+0000 7f34f9200640 1 -- 172.21.15.100:0/2639246581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f34f4068ad0 con 0x7f34f40c8be0 2024-04-23T14:33:15.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.942+0000 7f34eaa00640 1 -- 172.21.15.100:0/2639246581 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f34dc021020 con 0x7f34f40c8be0 2024-04-23T14:33:15.942 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:15.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.942+0000 7f34f9200640 1 -- 172.21.15.100:0/2639246581 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f34d0068380 msgr2=0x7f34d006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:15.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.942+0000 7f34f9200640 1 --2- 172.21.15.100:0/2639246581 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f34d0068380 0x7f34d006a840 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f34e4009b20 tx=0x7f34e4008040 comp rx=0 tx=0).stop 2024-04-23T14:33:15.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.942+0000 7f34f9200640 1 -- 172.21.15.100:0/2639246581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34f40c8be0 msgr2=0x7f34f40caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:15.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.942+0000 7f34f9200640 1 --2- 172.21.15.100:0/2639246581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34f40c8be0 0x7f34f40caae0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0079d0 tx=0x7f34dc00a600 comp rx=0 tx=0).stop 2024-04-23T14:33:15.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.942+0000 7f34f9200640 1 -- 172.21.15.100:0/2639246581 shutdown_connections 2024-04-23T14:33:15.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.942+0000 7f34f9200640 1 --2- 172.21.15.100:0/2639246581 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f34d0068380 0x7f34d006a840 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.942+0000 7f34f9200640 1 --2- 172.21.15.100:0/2639246581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34f40cb020 0x7f34f41f7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.942+0000 7f34f9200640 1 --2- 172.21.15.100:0/2639246581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34f40c8be0 0x7f34f40caae0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.942+0000 7f34f9200640 1 --2- 172.21.15.100:0/2639246581 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34d8004d50 0x7f34f4156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:15.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.942+0000 7f34f9200640 1 -- 172.21.15.100:0/2639246581 >> 172.21.15.100:0/2639246581 conn(0x7f34f40719d0 msgr2=0x7f34f40d2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:15.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.942+0000 7f34f9200640 1 -- 172.21.15.100:0/2639246581 shutdown_connections 2024-04-23T14:33:15.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:15.942+0000 7f34f9200640 1 -- 172.21.15.100:0/2639246581 wait complete. 2024-04-23T14:33:15.963 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:33:16.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.098+0000 7f8083000640 1 Processor -- start 2024-04-23T14:33:16.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.098+0000 7f8083000640 1 -- start start 2024-04-23T14:33:16.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.098+0000 7f8083000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f807c15c7a0 0x7f807c0af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.098+0000 7f8083000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f807c06cbb0 con 0x7f807c06bb20 2024-04-23T14:33:16.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.098+0000 7f8083000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f807c06ce90 con 0x7f807c0cd300 2024-04-23T14:33:16.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.098+0000 7f8083000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f807c06d450 con 0x7f807c15c7a0 2024-04-23T14:33:16.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.102+0000 7f807be00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f807c15c7a0 0x7f807c0af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.102+0000 7f8073400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f807c06bb20 0x7f807c0ccbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:36336/0 (socket says 172.21.15.100:36336) 2024-04-23T14:33:16.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.102+0000 7f8073400640 1 -- 172.21.15.100:0/2549478892 learned_addr learned my addr 172.21.15.100:0/2549478892 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:16.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.102+0000 7f807be00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f807c15c7a0 0x7f807c0af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32774/0 (socket says 172.21.15.100:32774) 2024-04-23T14:33:16.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.102+0000 7f807be00640 1 -- 172.21.15.100:0/2549478892 >> v1:172.21.15.151:6789/0 conn(0x7f807c0cd300 legacy=0x7f807c0ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:16.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.102+0000 7f807be00640 1 -- 172.21.15.100:0/2549478892 >> v1:172.21.15.100:6789/0 conn(0x7f807c06bb20 legacy=0x7f807c0ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.102+0000 7f807be00640 1 -- 172.21.15.100:0/2549478892 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f807c06d170 con 0x7f807c15c7a0 2024-04-23T14:33:16.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.102+0000 7f807b400640 1 -- 172.21.15.100:0/2549478892 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2137944371 0 0) 0x7f807c06cbb0 con 0x7f807c06bb20 2024-04-23T14:33:16.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.102+0000 7f807be00640 1 --2- 172.21.15.100:0/2549478892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807c15c7a0 0x7f807c0af190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f806c0097c0 tx=0x7f806c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdb5ddace73a450 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:16.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.102+0000 7f807b400640 1 -- 172.21.15.100:0/2549478892 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c0b0070 con 0x7f807c15c7a0 2024-04-23T14:33:16.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.102+0000 7f807b400640 1 -- 172.21.15.100:0/2549478892 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f806c0ab600 con 0x7f807c15c7a0 2024-04-23T14:33:16.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.102+0000 7f807b400640 1 -- 172.21.15.100:0/2549478892 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c0a4e60 con 0x7f807c15c7a0 2024-04-23T14:33:16.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f8083000640 1 -- 172.21.15.100:0/2549478892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807c15c7a0 msgr2=0x7f807c0af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f8083000640 1 --2- 172.21.15.100:0/2549478892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807c15c7a0 0x7f807c0af190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f806c0097c0 tx=0x7f806c01c3d0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f8083000640 1 -- 172.21.15.100:0/2549478892 shutdown_connections 2024-04-23T14:33:16.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f8083000640 1 --2- 172.21.15.100:0/2549478892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807c15c7a0 0x7f807c0af190 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f8083000640 1 -- 172.21.15.100:0/2549478892 >> 172.21.15.100:0/2549478892 conn(0x7f807c0b5200 msgr2=0x7f807c0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:16.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f8083000640 1 -- 172.21.15.100:0/2549478892 shutdown_connections 2024-04-23T14:33:16.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f8083000640 1 -- 172.21.15.100:0/2549478892 wait complete. 2024-04-23T14:33:16.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f8083000640 1 Processor -- start 2024-04-23T14:33:16.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f8083000640 1 -- start start 2024-04-23T14:33:16.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f8083000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807c06bb20 0x7f807c15a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f8083000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f807c0cd300 0x7f807c14da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f8083000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f807c15c7a0 0x7f807c099050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f8083000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f807c0ae760 con 0x7f807c06bb20 2024-04-23T14:33:16.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f8083000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f807c0ae5e0 con 0x7f807c15c7a0 2024-04-23T14:33:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f8083000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f807c16a210 con 0x7f807c0cd300 2024-04-23T14:33:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f807be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807c06bb20 0x7f807c15a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f807be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807c06bb20 0x7f807c15a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32790/0 (socket says 172.21.15.100:32790) 2024-04-23T14:33:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f807be00640 1 -- 172.21.15.100:0/2759551902 learned_addr learned my addr 172.21.15.100:0/2759551902 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f8073400640 1 --2- 172.21.15.100:0/2759551902 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f807c0cd300 0x7f807c14da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f8081000640 1 --2- 172.21.15.100:0/2759551902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f807c15c7a0 0x7f807c099050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f807be00640 1 -- 172.21.15.100:0/2759551902 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f807c0cd300 msgr2=0x7f807c14da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f807be00640 1 --2- 172.21.15.100:0/2759551902 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f807c0cd300 0x7f807c14da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f807be00640 1 -- 172.21.15.100:0/2759551902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f807c15c7a0 msgr2=0x7f807c099050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f807be00640 1 --2- 172.21.15.100:0/2759551902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f807c15c7a0 0x7f807c099050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f807be00640 1 -- 172.21.15.100:0/2759551902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f807c15ac20 con 0x7f807c06bb20 2024-04-23T14:33:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f8081000640 1 --2- 172.21.15.100:0/2759551902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f807c15c7a0 0x7f807c099050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.106+0000 7f807be00640 1 --2- 172.21.15.100:0/2759551902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807c06bb20 0x7f807c15a5f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f806c0097c0 tx=0x7f806c019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.110+0000 7f8078c00640 1 -- 172.21.15.100:0/2759551902 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c0b8020 con 0x7f807c06bb20 2024-04-23T14:33:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.110+0000 7f8078c00640 1 -- 172.21.15.100:0/2759551902 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f806c0a4e60 con 0x7f807c06bb20 2024-04-23T14:33:16.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.110+0000 7f8083000640 1 -- 172.21.15.100:0/2759551902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f807c14c7b0 con 0x7f807c06bb20 2024-04-23T14:33:16.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.110+0000 7f8078c00640 1 -- 172.21.15.100:0/2759551902 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c0b0040 con 0x7f807c06bb20 2024-04-23T14:33:16.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.110+0000 7f8083000640 1 -- 172.21.15.100:0/2759551902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f807c14cd70 con 0x7f807c06bb20 2024-04-23T14:33:16.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.110+0000 7f8083000640 1 -- 172.21.15.100:0/2759551902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f807c068ad0 con 0x7f807c06bb20 2024-04-23T14:33:16.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.110+0000 7f8078c00640 1 -- 172.21.15.100:0/2759551902 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f806c0bb020 con 0x7f807c06bb20 2024-04-23T14:33:16.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.110+0000 7f8078c00640 1 --2- 172.21.15.100:0/2759551902 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f80580683b0 0x7f805806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.110+0000 7f8078c00640 1 -- 172.21.15.100:0/2759551902 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f806c120a80 con 0x7f807c06bb20 2024-04-23T14:33:16.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.110+0000 7f8073400640 1 --2- 172.21.15.100:0/2759551902 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f80580683b0 0x7f805806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.110+0000 7f8073400640 1 --2- 172.21.15.100:0/2759551902 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f80580683b0 0x7f805806a870 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f807c0ce3f0 tx=0x7f8064007b80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:16.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.114+0000 7f8078c00640 1 -- 172.21.15.100:0/2759551902 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f806c122e20 con 0x7f807c06bb20 2024-04-23T14:33:16.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.270+0000 7f8083000640 1 -- 172.21.15.100:0/2759551902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f807c0cee20 con 0x7f807c06bb20 2024-04-23T14:33:16.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.270+0000 7f8078c00640 1 -- 172.21.15.100:0/2759551902 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f806c0c2030 con 0x7f807c06bb20 2024-04-23T14:33:16.273 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:33:16.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.274+0000 7f8083000640 1 -- 172.21.15.100:0/2759551902 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f80580683b0 msgr2=0x7f805806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.274+0000 7f8083000640 1 --2- 172.21.15.100:0/2759551902 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f80580683b0 0x7f805806a870 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f807c0ce3f0 tx=0x7f8064007b80 comp rx=0 tx=0).stop 2024-04-23T14:33:16.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.274+0000 7f8083000640 1 -- 172.21.15.100:0/2759551902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807c06bb20 msgr2=0x7f807c15a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.274+0000 7f8083000640 1 --2- 172.21.15.100:0/2759551902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807c06bb20 0x7f807c15a5f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f806c0097c0 tx=0x7f806c019170 comp rx=0 tx=0).stop 2024-04-23T14:33:16.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.274+0000 7f8083000640 1 -- 172.21.15.100:0/2759551902 shutdown_connections 2024-04-23T14:33:16.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.274+0000 7f8083000640 1 --2- 172.21.15.100:0/2759551902 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f80580683b0 0x7f805806a870 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.274+0000 7f8083000640 1 --2- 172.21.15.100:0/2759551902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f807c15c7a0 0x7f807c099050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.274+0000 7f8083000640 1 --2- 172.21.15.100:0/2759551902 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f807c0cd300 0x7f807c14da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.274+0000 7f8083000640 1 --2- 172.21.15.100:0/2759551902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807c06bb20 0x7f807c15a5f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.274+0000 7f8083000640 1 -- 172.21.15.100:0/2759551902 >> 172.21.15.100:0/2759551902 conn(0x7f807c0b5200 msgr2=0x7f807c15b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:16.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.274+0000 7f8083000640 1 -- 172.21.15.100:0/2759551902 shutdown_connections 2024-04-23T14:33:16.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.274+0000 7f8083000640 1 -- 172.21.15.100:0/2759551902 wait complete. 2024-04-23T14:33:16.294 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-23T14:33:16.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.430+0000 7f5dfda00640 1 Processor -- start 2024-04-23T14:33:16.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.430+0000 7f5dfda00640 1 -- start start 2024-04-23T14:33:16.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.430+0000 7f5dfda00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f5df815e950 0x7f5df80cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.430+0000 7f5dfda00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5df806db60 0x7f5df80d0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.430+0000 7f5dfda00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5df80cecc0 con 0x7f5df80cd9a0 2024-04-23T14:33:16.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.430+0000 7f5dfda00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f5df80d0300 con 0x7f5df806db60 2024-04-23T14:33:16.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.430+0000 7f5dfda00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f5df80cef60 con 0x7f5df815e950 2024-04-23T14:33:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.430+0000 7f5df7400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5df806db60 0x7f5df80d0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.434+0000 7f5df7400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5df806db60 0x7f5df80d0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32802/0 (socket says 172.21.15.100:32802) 2024-04-23T14:33:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.434+0000 7f5df7400640 1 -- 172.21.15.100:0/3669313224 learned_addr learned my addr 172.21.15.100:0/3669313224 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.434+0000 7f5dee000640 1 --1- 172.21.15.100:0/3669313224 >> v1:172.21.15.100:6789/0 conn(0x7f5df80cd9a0 0x7f5df806d450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:36348/0 (socket says 172.21.15.100:36348) 2024-04-23T14:33:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.434+0000 7f5df6a00640 1 --2- 172.21.15.100:0/3669313224 >> v2:172.21.15.151:3300/0 conn(0x7f5df815e950 0x7f5df80cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.434+0000 7f5df7400640 1 -- 172.21.15.100:0/3669313224 >> v2:172.21.15.151:3300/0 conn(0x7f5df815e950 msgr2=0x7f5df80cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.434+0000 7f5df7400640 1 --2- 172.21.15.100:0/3669313224 >> v2:172.21.15.151:3300/0 conn(0x7f5df815e950 0x7f5df80cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.434+0000 7f5df7400640 1 -- 172.21.15.100:0/3669313224 >> v1:172.21.15.100:6789/0 conn(0x7f5df80cd9a0 legacy=0x7f5df806d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.434+0000 7f5df7400640 1 -- 172.21.15.100:0/3669313224 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df80cf240 con 0x7f5df806db60 2024-04-23T14:33:16.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.434+0000 7f5df6000640 1 -- 172.21.15.100:0/3669313224 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3908994015 0 0) 0x7f5df80cecc0 con 0x7f5df80cd9a0 2024-04-23T14:33:16.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.434+0000 7f5df7400640 1 --2- 172.21.15.100:0/3669313224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df806db60 0x7f5df80d0790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5df806f630 tx=0x7f5df001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b373d4642543e74 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:16.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.434+0000 7f5df6000640 1 -- 172.21.15.100:0/3669313224 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df00a5730 con 0x7f5df806db60 2024-04-23T14:33:16.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.434+0000 7f5df6000640 1 -- 172.21.15.100:0/3669313224 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df00a58d0 con 0x7f5df806db60 2024-04-23T14:33:16.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.434+0000 7f5df6000640 1 -- 172.21.15.100:0/3669313224 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df00a5ca0 con 0x7f5df806db60 2024-04-23T14:33:16.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dfda00640 1 -- 172.21.15.100:0/3669313224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df806db60 msgr2=0x7f5df80d0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dfda00640 1 --2- 172.21.15.100:0/3669313224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df806db60 0x7f5df80d0790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5df806f630 tx=0x7f5df001c960 comp rx=0 tx=0).stop 2024-04-23T14:33:16.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dfda00640 1 -- 172.21.15.100:0/3669313224 shutdown_connections 2024-04-23T14:33:16.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dfda00640 1 --2- 172.21.15.100:0/3669313224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df806db60 0x7f5df80d0790 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dfda00640 1 --2- 172.21.15.100:0/3669313224 >> v2:172.21.15.151:3300/0 conn(0x7f5df815e950 0x7f5df80cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dfda00640 1 -- 172.21.15.100:0/3669313224 >> 172.21.15.100:0/3669313224 conn(0x7f5df80719d0 msgr2=0x7f5df8071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:16.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dfda00640 1 -- 172.21.15.100:0/3669313224 shutdown_connections 2024-04-23T14:33:16.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dfda00640 1 -- 172.21.15.100:0/3669313224 wait complete. 2024-04-23T14:33:16.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dfda00640 1 Processor -- start 2024-04-23T14:33:16.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dfda00640 1 -- start start 2024-04-23T14:33:16.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dfda00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5df806db60 0x7f5df80add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dfda00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df80cd9a0 0x7f5df8155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dfda00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5df815e950 0x7f5df81548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dfda00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5df80d1870 con 0x7f5df80cd9a0 2024-04-23T14:33:16.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dfda00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5df80d16f0 con 0x7f5df806db60 2024-04-23T14:33:16.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dfda00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5df80d19f0 con 0x7f5df815e950 2024-04-23T14:33:16.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dee000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df80cd9a0 0x7f5df8155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dee000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df80cd9a0 0x7f5df8155170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32818/0 (socket says 172.21.15.100:32818) 2024-04-23T14:33:16.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dee000640 1 -- 172.21.15.100:0/3048033309 learned_addr learned my addr 172.21.15.100:0/3048033309 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:16.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5df6a00640 1 --2- 172.21.15.100:0/3048033309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5df806db60 0x7f5df80add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5df7400640 1 --2- 172.21.15.100:0/3048033309 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5df815e950 0x7f5df81548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dee000640 1 -- 172.21.15.100:0/3048033309 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5df815e950 msgr2=0x7f5df81548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dee000640 1 --2- 172.21.15.100:0/3048033309 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5df815e950 0x7f5df81548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dee000640 1 -- 172.21.15.100:0/3048033309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5df806db60 msgr2=0x7f5df80add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dee000640 1 --2- 172.21.15.100:0/3048033309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5df806db60 0x7f5df80add10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dee000640 1 -- 172.21.15.100:0/3048033309 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df81569f0 con 0x7f5df80cd9a0 2024-04-23T14:33:16.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5df7400640 1 --2- 172.21.15.100:0/3048033309 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5df815e950 0x7f5df81548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:16.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.438+0000 7f5dee000640 1 --2- 172.21.15.100:0/3048033309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df80cd9a0 0x7f5df8155170 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5de000c4e0 tx=0x7f5de000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:16.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.442+0000 7f5def400640 1 -- 172.21.15.100:0/3048033309 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0018070 con 0x7f5df80cd9a0 2024-04-23T14:33:16.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.442+0000 7f5def400640 1 -- 172.21.15.100:0/3048033309 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de0007500 con 0x7f5df80cd9a0 2024-04-23T14:33:16.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.442+0000 7f5def400640 1 -- 172.21.15.100:0/3048033309 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0004a30 con 0x7f5df80cd9a0 2024-04-23T14:33:16.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.442+0000 7f5dfda00640 1 -- 172.21.15.100:0/3048033309 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df8155800 con 0x7f5df80cd9a0 2024-04-23T14:33:16.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.442+0000 7f5dfda00640 1 -- 172.21.15.100:0/3048033309 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df8155dc0 con 0x7f5df80cd9a0 2024-04-23T14:33:16.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.442+0000 7f5dfda00640 1 -- 172.21.15.100:0/3048033309 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df8068ad0 con 0x7f5df80cd9a0 2024-04-23T14:33:16.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.442+0000 7f5def400640 1 -- 172.21.15.100:0/3048033309 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5de0004bd0 con 0x7f5df80cd9a0 2024-04-23T14:33:16.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.442+0000 7f5def400640 1 --2- 172.21.15.100:0/3048033309 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5dd4068380 0x7f5dd406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.442+0000 7f5def400640 1 -- 172.21.15.100:0/3048033309 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5de0086140 con 0x7f5df80cd9a0 2024-04-23T14:33:16.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.442+0000 7f5df6a00640 1 --2- 172.21.15.100:0/3048033309 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5dd4068380 0x7f5dd406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.442+0000 7f5df6a00640 1 --2- 172.21.15.100:0/3048033309 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5dd4068380 0x7f5dd406a840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5de8007a60 tx=0x7f5de8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:16.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.446+0000 7f5def400640 1 -- 172.21.15.100:0/3048033309 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5de0055cd0 con 0x7f5df80cd9a0 2024-04-23T14:33:16.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.602+0000 7f5dfda00640 1 -- 172.21.15.100:0/3048033309 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5df8157bb0 con 0x7f5df80cd9a0 2024-04-23T14:33:16.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.602+0000 7f5def400640 1 -- 172.21.15.100:0/3048033309 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f5de0059fd0 con 0x7f5df80cd9a0 2024-04-23T14:33:16.605 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:16.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.606+0000 7f5dfda00640 1 -- 172.21.15.100:0/3048033309 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5dd4068380 msgr2=0x7f5dd406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.606+0000 7f5dfda00640 1 --2- 172.21.15.100:0/3048033309 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5dd4068380 0x7f5dd406a840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5de8007a60 tx=0x7f5de8008040 comp rx=0 tx=0).stop 2024-04-23T14:33:16.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.606+0000 7f5dfda00640 1 -- 172.21.15.100:0/3048033309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df80cd9a0 msgr2=0x7f5df8155170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.606+0000 7f5dfda00640 1 --2- 172.21.15.100:0/3048033309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df80cd9a0 0x7f5df8155170 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5de000c4e0 tx=0x7f5de000c9a0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.606+0000 7f5dfda00640 1 -- 172.21.15.100:0/3048033309 shutdown_connections 2024-04-23T14:33:16.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.606+0000 7f5dfda00640 1 --2- 172.21.15.100:0/3048033309 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5dd4068380 0x7f5dd406a840 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.606+0000 7f5dfda00640 1 --2- 172.21.15.100:0/3048033309 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5df815e950 0x7f5df81548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.606+0000 7f5dfda00640 1 --2- 172.21.15.100:0/3048033309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df80cd9a0 0x7f5df8155170 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.606+0000 7f5dfda00640 1 --2- 172.21.15.100:0/3048033309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5df806db60 0x7f5df80add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.606+0000 7f5dfda00640 1 -- 172.21.15.100:0/3048033309 >> 172.21.15.100:0/3048033309 conn(0x7f5df80719d0 msgr2=0x7f5df8161340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:16.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.606+0000 7f5dfda00640 1 -- 172.21.15.100:0/3048033309 shutdown_connections 2024-04-23T14:33:16.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.606+0000 7f5dfda00640 1 -- 172.21.15.100:0/3048033309 wait complete. 2024-04-23T14:33:16.625 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:33:16.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.758+0000 7fd0fe800640 1 Processor -- start 2024-04-23T14:33:16.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.758+0000 7fd0fe800640 1 -- start start 2024-04-23T14:33:16.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.762+0000 7fd0fe800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd0f8147ee0 0x7fd0f80b3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.762+0000 7fd0fe800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd0f80c77e0 0x7fd0f806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.762+0000 7fd0fe800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0f806d7f0 con 0x7fd0f8147390 2024-04-23T14:33:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.762+0000 7fd0fe800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fd0f806dad0 con 0x7fd0f80c77e0 2024-04-23T14:33:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.762+0000 7fd0fe800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fd0f806ddb0 con 0x7fd0f8147ee0 2024-04-23T14:33:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.762+0000 7fd0fde00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd0f80c77e0 0x7fd0f806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.762+0000 7fd0fd400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fd0f8147390 0x7fd0f80c70d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:36356/0 (socket says 172.21.15.100:36356) 2024-04-23T14:33:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.762+0000 7fd0fd400640 1 -- 172.21.15.100:0/4214328759 learned_addr learned my addr 172.21.15.100:0/4214328759 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.762+0000 7fd0fca00640 1 --2- 172.21.15.100:0/4214328759 >> v2:172.21.15.151:3300/0 conn(0x7fd0f8147ee0 0x7fd0f80b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.762+0000 7fd0f7e00640 1 -- 172.21.15.100:0/4214328759 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1265001327 0 0) 0x7fd0f806d7f0 con 0x7fd0f8147390 2024-04-23T14:33:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.762+0000 7fd0f7e00640 1 -- 172.21.15.100:0/4214328759 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0e0003530 con 0x7fd0f8147390 2024-04-23T14:33:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.762+0000 7fd0fde00640 1 -- 172.21.15.100:0/4214328759 >> v2:172.21.15.151:3300/0 conn(0x7fd0f8147ee0 msgr2=0x7fd0f80b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:16.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.762+0000 7fd0fde00640 1 --2- 172.21.15.100:0/4214328759 >> v2:172.21.15.151:3300/0 conn(0x7fd0f8147ee0 0x7fd0f80b3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.762+0000 7fd0fde00640 1 -- 172.21.15.100:0/4214328759 >> v1:172.21.15.100:6789/0 conn(0x7fd0f8147390 legacy=0x7fd0f80c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.762+0000 7fd0fde00640 1 -- 172.21.15.100:0/4214328759 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f806e090 con 0x7fd0f80c77e0 2024-04-23T14:33:16.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.766+0000 7fd0fde00640 1 --2- 172.21.15.100:0/4214328759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0f80c77e0 0x7fd0f806e890 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8147d90 tx=0x7fd0f001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d694fa2a43a653 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:16.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.766+0000 7fd0f7e00640 1 -- 172.21.15.100:0/4214328759 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f00b2070 con 0x7fd0f80c77e0 2024-04-23T14:33:16.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.766+0000 7fd0f7e00640 1 -- 172.21.15.100:0/4214328759 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0f00a5920 con 0x7fd0f80c77e0 2024-04-23T14:33:16.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.766+0000 7fd0f7e00640 1 -- 172.21.15.100:0/4214328759 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f00a5cf0 con 0x7fd0f80c77e0 2024-04-23T14:33:16.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.766+0000 7fd0fe800640 1 -- 172.21.15.100:0/4214328759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0f80c77e0 msgr2=0x7fd0f806e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.766+0000 7fd0fe800640 1 --2- 172.21.15.100:0/4214328759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0f80c77e0 0x7fd0f806e890 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8147d90 tx=0x7fd0f001c930 comp rx=0 tx=0).stop 2024-04-23T14:33:16.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.766+0000 7fd0fe800640 1 -- 172.21.15.100:0/4214328759 shutdown_connections 2024-04-23T14:33:16.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.766+0000 7fd0fe800640 1 --2- 172.21.15.100:0/4214328759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0f80c77e0 0x7fd0f806e890 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.766+0000 7fd0fe800640 1 --2- 172.21.15.100:0/4214328759 >> v2:172.21.15.151:3300/0 conn(0x7fd0f8147ee0 0x7fd0f80b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.766+0000 7fd0fe800640 1 -- 172.21.15.100:0/4214328759 >> 172.21.15.100:0/4214328759 conn(0x7fd0f80b3e60 msgr2=0x7fd0f80b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:16.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.766+0000 7fd0fe800640 1 -- 172.21.15.100:0/4214328759 shutdown_connections 2024-04-23T14:33:16.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.766+0000 7fd0fe800640 1 -- 172.21.15.100:0/4214328759 wait complete. 2024-04-23T14:33:16.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.766+0000 7fd0fe800640 1 Processor -- start 2024-04-23T14:33:16.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.766+0000 7fd0fe800640 1 -- start start 2024-04-23T14:33:16.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.770+0000 7fd0fe800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0f80c77e0 0x7fd0f815f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.770+0000 7fd0fe800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0f8147390 0x7fd0f8140be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.770+0000 7fd0fe800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0f8147ee0 0x7fd0f80d0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.770+0000 7fd0fe800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f80cd440 con 0x7fd0f8147390 2024-04-23T14:33:16.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.770+0000 7fd0fe800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f80cd2c0 con 0x7fd0f80c77e0 2024-04-23T14:33:16.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.770+0000 7fd0fe800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f80c93c0 con 0x7fd0f8147ee0 2024-04-23T14:33:16.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.770+0000 7fd0fca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0f8147390 0x7fd0f8140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.770+0000 7fd0fca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0f8147390 0x7fd0f8140be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32836/0 (socket says 172.21.15.100:32836) 2024-04-23T14:33:16.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.770+0000 7fd0fca00640 1 -- 172.21.15.100:0/3774349085 learned_addr learned my addr 172.21.15.100:0/3774349085 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:16.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.770+0000 7fd0fca00640 1 -- 172.21.15.100:0/3774349085 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0f8147ee0 msgr2=0x7fd0f80d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.770+0000 7fd0fca00640 1 --2- 172.21.15.100:0/3774349085 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0f8147ee0 0x7fd0f80d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.770+0000 7fd0fca00640 1 -- 172.21.15.100:0/3774349085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0f80c77e0 msgr2=0x7fd0f815f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.770+0000 7fd0fca00640 1 --2- 172.21.15.100:0/3774349085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0f80c77e0 0x7fd0f815f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.770+0000 7fd0fca00640 1 -- 172.21.15.100:0/3774349085 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f80c9630 con 0x7fd0f8147390 2024-04-23T14:33:16.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.770+0000 7fd0fca00640 1 --2- 172.21.15.100:0/3774349085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0f8147390 0x7fd0f8140be0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400d620 tx=0x7fd0e400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:16.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.770+0000 7fd0f5600640 1 -- 172.21.15.100:0/3774349085 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e400b620 con 0x7fd0f8147390 2024-04-23T14:33:16.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.770+0000 7fd0f5600640 1 -- 172.21.15.100:0/3774349085 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e4019070 con 0x7fd0f8147390 2024-04-23T14:33:16.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.770+0000 7fd0f5600640 1 -- 172.21.15.100:0/3774349085 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e400ba20 con 0x7fd0f8147390 2024-04-23T14:33:16.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.770+0000 7fd0fe800640 1 -- 172.21.15.100:0/3774349085 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f80c9910 con 0x7fd0f8147390 2024-04-23T14:33:16.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.770+0000 7fd0fe800640 1 -- 172.21.15.100:0/3774349085 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f815f9c0 con 0x7fd0f8147390 2024-04-23T14:33:16.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.770+0000 7fd0fe800640 1 -- 172.21.15.100:0/3774349085 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0f8075c20 con 0x7fd0f8147390 2024-04-23T14:33:16.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.770+0000 7fd0f5600640 1 -- 172.21.15.100:0/3774349085 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd0e400bbc0 con 0x7fd0f8147390 2024-04-23T14:33:16.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.774+0000 7fd0f5600640 1 --2- 172.21.15.100:0/3774349085 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd0d80683b0 0x7fd0d806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.774+0000 7fd0f5600640 1 -- 172.21.15.100:0/3774349085 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd0e4087270 con 0x7fd0f8147390 2024-04-23T14:33:16.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.774+0000 7fd0fd400640 1 --2- 172.21.15.100:0/3774349085 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd0d80683b0 0x7fd0d806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.774+0000 7fd0fd400640 1 --2- 172.21.15.100:0/3774349085 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd0d80683b0 0x7fd0d806a870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec007fd0 tx=0x7fd0ec007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:16.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.778+0000 7fd0f5600640 1 -- 172.21.15.100:0/3774349085 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd0e4057390 con 0x7fd0f8147390 2024-04-23T14:33:16.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.946+0000 7fd0fe800640 1 -- 172.21.15.100:0/3774349085 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd0f80c8370 con 0x7fd0f8147390 2024-04-23T14:33:16.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.946+0000 7fd0f5600640 1 -- 172.21.15.100:0/3774349085 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fd0e4056c90 con 0x7fd0f8147390 2024-04-23T14:33:16.949 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-04-23T14:33:16.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.950+0000 7fd0fe800640 1 -- 172.21.15.100:0/3774349085 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd0d80683b0 msgr2=0x7fd0d806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.950+0000 7fd0fe800640 1 --2- 172.21.15.100:0/3774349085 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd0d80683b0 0x7fd0d806a870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec007fd0 tx=0x7fd0ec007900 comp rx=0 tx=0).stop 2024-04-23T14:33:16.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.950+0000 7fd0fe800640 1 -- 172.21.15.100:0/3774349085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0f8147390 msgr2=0x7fd0f8140be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.950+0000 7fd0fe800640 1 --2- 172.21.15.100:0/3774349085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0f8147390 0x7fd0f8140be0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400d620 tx=0x7fd0e400dae0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.950+0000 7fd0fe800640 1 -- 172.21.15.100:0/3774349085 shutdown_connections 2024-04-23T14:33:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.950+0000 7fd0fe800640 1 --2- 172.21.15.100:0/3774349085 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd0d80683b0 0x7fd0d806a870 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.950+0000 7fd0fe800640 1 --2- 172.21.15.100:0/3774349085 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0f8147ee0 0x7fd0f80d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.950+0000 7fd0fe800640 1 --2- 172.21.15.100:0/3774349085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0f8147390 0x7fd0f8140be0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.950+0000 7fd0fe800640 1 --2- 172.21.15.100:0/3774349085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0f80c77e0 0x7fd0f815f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.950+0000 7fd0fe800640 1 -- 172.21.15.100:0/3774349085 >> 172.21.15.100:0/3774349085 conn(0x7fd0f80b3e60 msgr2=0x7fd0f80cdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.950+0000 7fd0fe800640 1 -- 172.21.15.100:0/3774349085 shutdown_connections 2024-04-23T14:33:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:16.950+0000 7fd0fe800640 1 -- 172.21.15.100:0/3774349085 wait complete. 2024-04-23T14:33:16.971 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:33:16.971 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:16.980 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:17.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a5d400640 1 Processor -- start 2024-04-23T14:33:17.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a5d400640 1 -- start start 2024-04-23T14:33:17.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a5d400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a58160de0 0x7f8a580b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a5d400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a58160320 con 0x7f8a580ad870 2024-04-23T14:33:17.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a5d400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a580cc7e0 con 0x7f8a5815c470 2024-04-23T14:33:17.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a5d400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8a581605e0 con 0x7f8a58160de0 2024-04-23T14:33:17.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a55e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f8a580ad870 0x7f8a5806bab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:36362/0 (socket says 172.21.15.100:36362) 2024-04-23T14:33:17.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a55e00640 1 -- 172.21.15.100:0/1533605300 learned_addr learned my addr 172.21.15.100:0/1533605300 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:17.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a57200640 1 --2- 172.21.15.100:0/1533605300 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a58160de0 0x7f8a580b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a55400640 1 -- 172.21.15.100:0/1533605300 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 920776308 0 0) 0x7f8a58160320 con 0x7f8a580ad870 2024-04-23T14:33:17.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a55400640 1 -- 172.21.15.100:0/1533605300 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a3c003530 con 0x7f8a580ad870 2024-04-23T14:33:17.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a55400640 1 -- 172.21.15.100:0/1533605300 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1449724996 0 0) 0x7f8a3c003530 con 0x7f8a580ad870 2024-04-23T14:33:17.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a55400640 1 -- 172.21.15.100:0/1533605300 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a58160de0 msgr2=0x7f8a580b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:17.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a55400640 1 --2- 172.21.15.100:0/1533605300 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a58160de0 0x7f8a580b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a55400640 1 -- 172.21.15.100:0/1533605300 >> v1:172.21.15.151:6789/0 conn(0x7f8a5815c470 legacy=0x7f8a580ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:17.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a55400640 1 -- 172.21.15.100:0/1533605300 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a581608c0 con 0x7f8a580ad870 2024-04-23T14:33:17.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a55400640 1 -- 172.21.15.100:0/1533605300 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f8a440016a0 con 0x7f8a580ad870 2024-04-23T14:33:17.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a55400640 1 -- 172.21.15.100:0/1533605300 >> v1:172.21.15.100:6789/0 conn(0x7f8a580ad870 legacy=0x7f8a5806bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:17.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a55400640 1 --2- 172.21.15.100:0/1533605300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a3c0039f0 0x7f8a3c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a55400640 1 --2- 172.21.15.100:0/1533605300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a3c004810 0x7f8a3c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a55400640 1 --2- 172.21.15.100:0/1533605300 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a58160de0 0x7f8a3c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a55400640 1 -- 172.21.15.100:0/1533605300 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8a581605e0 con 0x7f8a3c0039f0 2024-04-23T14:33:17.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a55400640 1 -- 172.21.15.100:0/1533605300 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c00bb30 con 0x7f8a3c004810 2024-04-23T14:33:17.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a55400640 1 -- 172.21.15.100:0/1533605300 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8a3c00bd30 con 0x7f8a58160de0 2024-04-23T14:33:17.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a56800640 1 --2- 172.21.15.100:0/1533605300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a3c0039f0 0x7f8a3c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a55e00640 1 --2- 172.21.15.100:0/1533605300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a3c004810 0x7f8a3c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.158+0000 7f8a57200640 1 --2- 172.21.15.100:0/1533605300 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a58160de0 0x7f8a3c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.162+0000 7f8a56800640 1 -- 172.21.15.100:0/1533605300 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a58160de0 msgr2=0x7f8a3c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:17.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.162+0000 7f8a56800640 1 --2- 172.21.15.100:0/1533605300 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a58160de0 0x7f8a3c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.162+0000 7f8a56800640 1 -- 172.21.15.100:0/1533605300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a3c004810 msgr2=0x7f8a3c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:17.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.162+0000 7f8a56800640 1 --2- 172.21.15.100:0/1533605300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a3c004810 0x7f8a3c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.162+0000 7f8a56800640 1 -- 172.21.15.100:0/1533605300 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3c003530 con 0x7f8a3c0039f0 2024-04-23T14:33:17.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.162+0000 7f8a57200640 1 --2- 172.21.15.100:0/1533605300 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a58160de0 0x7f8a3c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:17.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.162+0000 7f8a56800640 1 --2- 172.21.15.100:0/1533605300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a3c0039f0 0x7f8a3c003e10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8a48001e70 tx=0x7f8a4801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77885ac753e01706 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:17.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.162+0000 7f8a55400640 1 -- 172.21.15.100:0/1533605300 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48001a90 con 0x7f8a3c0039f0 2024-04-23T14:33:17.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.162+0000 7f8a55400640 1 -- 172.21.15.100:0/1533605300 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a48003b80 con 0x7f8a3c0039f0 2024-04-23T14:33:17.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.162+0000 7f8a55400640 1 -- 172.21.15.100:0/1533605300 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48005080 con 0x7f8a3c0039f0 2024-04-23T14:33:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.162+0000 7f8a5d400640 1 -- 172.21.15.100:0/1533605300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a3c0039f0 msgr2=0x7f8a3c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.162+0000 7f8a5d400640 1 --2- 172.21.15.100:0/1533605300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a3c0039f0 0x7f8a3c003e10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8a48001e70 tx=0x7f8a4801c780 comp rx=0 tx=0).stop 2024-04-23T14:33:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.162+0000 7f8a57200640 1 -- 172.21.15.100:0/1533605300 reap_dead start 2024-04-23T14:33:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.162+0000 7f8a5d400640 1 -- 172.21.15.100:0/1533605300 shutdown_connections 2024-04-23T14:33:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.162+0000 7f8a5d400640 1 -- 172.21.15.100:0/1533605300 >> 172.21.15.100:0/1533605300 conn(0x7f8a580b1e80 msgr2=0x7f8a580b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.162+0000 7f8a5d400640 1 -- 172.21.15.100:0/1533605300 shutdown_connections 2024-04-23T14:33:17.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.162+0000 7f8a5d400640 1 -- 172.21.15.100:0/1533605300 wait complete. 2024-04-23T14:33:17.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a5d400640 1 Processor -- start 2024-04-23T14:33:17.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a5d400640 1 -- start start 2024-04-23T14:33:17.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a5d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a3c0039f0 0x7f8a58156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a5d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a5806eda0 0x7f8a580994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a5d400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a5814de20 0x7f8a58150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a5d400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5815ca60 con 0x7f8a5806eda0 2024-04-23T14:33:17.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a5d400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a580cc5c0 con 0x7f8a3c0039f0 2024-04-23T14:33:17.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a5d400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8a58168540 con 0x7f8a5814de20 2024-04-23T14:33:17.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a55e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a5806eda0 0x7f8a580994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a55e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a5806eda0 0x7f8a580994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32844/0 (socket says 172.21.15.100:32844) 2024-04-23T14:33:17.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a55e00640 1 -- 172.21.15.100:0/152267915 learned_addr learned my addr 172.21.15.100:0/152267915 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:17.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a56800640 1 --2- 172.21.15.100:0/152267915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a3c0039f0 0x7f8a58156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a55e00640 1 -- 172.21.15.100:0/152267915 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a5814de20 msgr2=0x7f8a58150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:17.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a57200640 1 --2- 172.21.15.100:0/152267915 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a5814de20 0x7f8a58150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a55e00640 1 --2- 172.21.15.100:0/152267915 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a5814de20 0x7f8a58150680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a55e00640 1 -- 172.21.15.100:0/152267915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a3c0039f0 msgr2=0x7f8a58156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:17.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a55e00640 1 --2- 172.21.15.100:0/152267915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a3c0039f0 0x7f8a58156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a55e00640 1 -- 172.21.15.100:0/152267915 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a58151310 con 0x7f8a5806eda0 2024-04-23T14:33:17.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a56800640 1 --2- 172.21.15.100:0/152267915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a3c0039f0 0x7f8a58156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:17.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a57200640 1 --2- 172.21.15.100:0/152267915 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a5814de20 0x7f8a58150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:17.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a55e00640 1 --2- 172.21.15.100:0/152267915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a5806eda0 0x7f8a580994e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8a44007340 tx=0x7f8a44007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:17.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a4ea00640 1 -- 172.21.15.100:0/152267915 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4401a070 con 0x7f8a5806eda0 2024-04-23T14:33:17.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a4ea00640 1 -- 172.21.15.100:0/152267915 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a4400f9c0 con 0x7f8a5806eda0 2024-04-23T14:33:17.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a4ea00640 1 -- 172.21.15.100:0/152267915 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4400fdb0 con 0x7f8a5806eda0 2024-04-23T14:33:17.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a5d400640 1 -- 172.21.15.100:0/152267915 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a581515f0 con 0x7f8a5806eda0 2024-04-23T14:33:17.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.166+0000 7f8a5d400640 1 -- 172.21.15.100:0/152267915 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a58156970 con 0x7f8a5806eda0 2024-04-23T14:33:17.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.170+0000 7f8a5d400640 1 -- 172.21.15.100:0/152267915 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a581502a0 con 0x7f8a5806eda0 2024-04-23T14:33:17.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.170+0000 7f8a4ea00640 1 -- 172.21.15.100:0/152267915 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8a4401d360 con 0x7f8a5806eda0 2024-04-23T14:33:17.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.170+0000 7f8a4ea00640 1 --2- 172.21.15.100:0/152267915 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8a38068380 0x7f8a3806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.170+0000 7f8a4ea00640 1 -- 172.21.15.100:0/152267915 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f8a440866e0 con 0x7f8a5806eda0 2024-04-23T14:33:17.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.170+0000 7f8a56800640 1 --2- 172.21.15.100:0/152267915 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8a38068380 0x7f8a3806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.170+0000 7f8a56800640 1 --2- 172.21.15.100:0/152267915 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8a38068380 0x7f8a3806a840 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8a48005000 tx=0x7f8a48001520 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:17.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.174+0000 7f8a4ea00640 1 -- 172.21.15.100:0/152267915 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8a44056800 con 0x7f8a5806eda0 2024-04-23T14:33:17.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.330+0000 7f8a5d400640 1 -- 172.21.15.100:0/152267915 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a580b2750 con 0x7f8a5806eda0 2024-04-23T14:33:17.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.330+0000 7f8a4ea00640 1 -- 172.21.15.100:0/152267915 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18859 (secure 0 0 0) 0x7f8a44056100 con 0x7f8a5806eda0 2024-04-23T14:33:17.333 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:17.333 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:49.784608+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:17.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.334+0000 7f8a5d400640 1 -- 172.21.15.100:0/152267915 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8a38068380 msgr2=0x7f8a3806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:17.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.334+0000 7f8a5d400640 1 --2- 172.21.15.100:0/152267915 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8a38068380 0x7f8a3806a840 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8a48005000 tx=0x7f8a48001520 comp rx=0 tx=0).stop 2024-04-23T14:33:17.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.334+0000 7f8a5d400640 1 -- 172.21.15.100:0/152267915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a5806eda0 msgr2=0x7f8a580994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:17.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.334+0000 7f8a5d400640 1 --2- 172.21.15.100:0/152267915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a5806eda0 0x7f8a580994e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8a44007340 tx=0x7f8a44007800 comp rx=0 tx=0).stop 2024-04-23T14:33:17.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.334+0000 7f8a5d400640 1 -- 172.21.15.100:0/152267915 shutdown_connections 2024-04-23T14:33:17.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.334+0000 7f8a5d400640 1 --2- 172.21.15.100:0/152267915 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8a38068380 0x7f8a3806a840 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.334+0000 7f8a5d400640 1 --2- 172.21.15.100:0/152267915 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a5814de20 0x7f8a58150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.334+0000 7f8a5d400640 1 --2- 172.21.15.100:0/152267915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a5806eda0 0x7f8a580994e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.334+0000 7f8a5d400640 1 --2- 172.21.15.100:0/152267915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a3c0039f0 0x7f8a58156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.334+0000 7f8a5d400640 1 -- 172.21.15.100:0/152267915 >> 172.21.15.100:0/152267915 conn(0x7f8a580b1e80 msgr2=0x7f8a58156d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:17.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.334+0000 7f8a5d400640 1 -- 172.21.15.100:0/152267915 shutdown_connections 2024-04-23T14:33:17.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.334+0000 7f8a5d400640 1 -- 172.21.15.100:0/152267915 wait complete. 2024-04-23T14:33:17.354 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:32:06.835560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:32:07.930489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-23T14:32:11.636814+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-23T14:33:17.355 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:33:17.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.490+0000 7f6a7aa00640 1 Processor -- start 2024-04-23T14:33:17.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.490+0000 7f6a7aa00640 1 -- start start 2024-04-23T14:33:17.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a7aa00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f6a7406f8c0 0x7f6a7406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a7aa00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a7406dd60 con 0x7f6a740ce7f0 2024-04-23T14:33:17.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a7aa00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a740d09c0 con 0x7f6a740d0170 2024-04-23T14:33:17.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a7aa00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f6a740ad050 con 0x7f6a7406f8c0 2024-04-23T14:33:17.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a78a00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f6a740ce7f0 0x7f6a740add80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:36366/0 (socket says 172.21.15.100:36366) 2024-04-23T14:33:17.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a78a00640 1 -- 172.21.15.100:0/3271513640 learned_addr learned my addr 172.21.15.100:0/3271513640 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:17.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a73e00640 1 --2- 172.21.15.100:0/3271513640 >> v2:172.21.15.151:3300/0 conn(0x7f6a7406f8c0 0x7f6a7406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a73400640 1 -- 172.21.15.100:0/3271513640 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 198944116 0 0) 0x7f6a6c0014a0 con 0x7f6a740ce7f0 2024-04-23T14:33:17.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a73400640 1 -- 172.21.15.100:0/3271513640 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a58003530 con 0x7f6a740ce7f0 2024-04-23T14:33:17.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a73400640 1 -- 172.21.15.100:0/3271513640 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 632361581 0 0) 0x7f6a58003530 con 0x7f6a740ce7f0 2024-04-23T14:33:17.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a73400640 1 -- 172.21.15.100:0/3271513640 >> v2:172.21.15.151:3300/0 conn(0x7f6a7406f8c0 msgr2=0x7f6a7406c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:17.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a73400640 1 --2- 172.21.15.100:0/3271513640 >> v2:172.21.15.151:3300/0 conn(0x7f6a7406f8c0 0x7f6a7406c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a73400640 1 -- 172.21.15.100:0/3271513640 >> v1:172.21.15.151:6789/0 conn(0x7f6a740d0170 legacy=0x7f6a740ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:17.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a73400640 1 -- 172.21.15.100:0/3271513640 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a740ad2a0 con 0x7f6a740ce7f0 2024-04-23T14:33:17.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a73400640 1 -- 172.21.15.100:0/3271513640 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f6a6c001bd0 con 0x7f6a740ce7f0 2024-04-23T14:33:17.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a73400640 1 -- 172.21.15.100:0/3271513640 >> v1:172.21.15.100:6789/0 conn(0x7f6a740ce7f0 legacy=0x7f6a740add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:17.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a73400640 1 --2- 172.21.15.100:0/3271513640 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6a580039c0 0x7f6a58003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a73400640 1 --2- 172.21.15.100:0/3271513640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a58004810 0x7f6a58004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a73400640 1 --2- 172.21.15.100:0/3271513640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a580051d0 0x7f6a5800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a73400640 1 -- 172.21.15.100:0/3271513640 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6a740ad050 con 0x7f6a580051d0 2024-04-23T14:33:17.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a73400640 1 -- 172.21.15.100:0/3271513640 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5800c070 con 0x7f6a58004810 2024-04-23T14:33:17.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a73400640 1 -- 172.21.15.100:0/3271513640 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f6a5800c270 con 0x7f6a580039c0 2024-04-23T14:33:17.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a73e00640 1 --2- 172.21.15.100:0/3271513640 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6a580039c0 0x7f6a58003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a78a00640 1 --2- 172.21.15.100:0/3271513640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a580051d0 0x7f6a5800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a6b400640 1 --2- 172.21.15.100:0/3271513640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a58004810 0x7f6a58004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a78a00640 1 -- 172.21.15.100:0/3271513640 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6a580039c0 msgr2=0x7f6a58003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:17.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a78a00640 1 --2- 172.21.15.100:0/3271513640 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6a580039c0 0x7f6a58003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a78a00640 1 -- 172.21.15.100:0/3271513640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a58004810 msgr2=0x7f6a58004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:17.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a78a00640 1 --2- 172.21.15.100:0/3271513640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a58004810 0x7f6a58004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.494+0000 7f6a78a00640 1 -- 172.21.15.100:0/3271513640 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a58003530 con 0x7f6a580051d0 2024-04-23T14:33:17.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.498+0000 7f6a78a00640 1 -- 172.21.15.100:0/3271513640 reap_dead start 2024-04-23T14:33:17.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.498+0000 7f6a78a00640 1 --2- 172.21.15.100:0/3271513640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a580051d0 0x7f6a5800b9d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6a7406ff50 tx=0x7f6a6c01d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177b8b0001d787bd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:17.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.498+0000 7f6a73400640 1 -- 172.21.15.100:0/3271513640 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c00a640 con 0x7f6a580051d0 2024-04-23T14:33:17.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.498+0000 7f6a73400640 1 -- 172.21.15.100:0/3271513640 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a6c0ac640 con 0x7f6a580051d0 2024-04-23T14:33:17.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.498+0000 7f6a73400640 1 -- 172.21.15.100:0/3271513640 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c0aca30 con 0x7f6a580051d0 2024-04-23T14:33:17.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.498+0000 7f6a7aa00640 1 -- 172.21.15.100:0/3271513640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a580051d0 msgr2=0x7f6a5800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:17.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.498+0000 7f6a7aa00640 1 --2- 172.21.15.100:0/3271513640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a580051d0 0x7f6a5800b9d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6a7406ff50 tx=0x7f6a6c01d300 comp rx=0 tx=0).stop 2024-04-23T14:33:17.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.498+0000 7f6a7aa00640 1 -- 172.21.15.100:0/3271513640 shutdown_connections 2024-04-23T14:33:17.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.498+0000 7f6a7aa00640 1 --2- 172.21.15.100:0/3271513640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a580051d0 0x7f6a5800b9d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.498+0000 7f6a7aa00640 1 -- 172.21.15.100:0/3271513640 >> 172.21.15.100:0/3271513640 conn(0x7f6a740b5200 msgr2=0x7f6a740b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:17.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.498+0000 7f6a7aa00640 1 -- 172.21.15.100:0/3271513640 shutdown_connections 2024-04-23T14:33:17.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.498+0000 7f6a7aa00640 1 -- 172.21.15.100:0/3271513640 wait complete. 2024-04-23T14:33:17.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.498+0000 7f6a7aa00640 1 Processor -- start 2024-04-23T14:33:17.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.498+0000 7f6a7aa00640 1 -- start start 2024-04-23T14:33:17.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a7aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a580051d0 0x7f6a74152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a7aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a740cb810 0x7f6a740caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a7aa00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6a740cbc70 0x7f6a740c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a7aa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6a740d05b0 con 0x7f6a740cb810 2024-04-23T14:33:17.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a7aa00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6a740ad8f0 con 0x7f6a580051d0 2024-04-23T14:33:17.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a7aa00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f6a740d0730 con 0x7f6a740cbc70 2024-04-23T14:33:17.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a6b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a740cb810 0x7f6a740caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a6b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a740cb810 0x7f6a740caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32848/0 (socket says 172.21.15.100:32848) 2024-04-23T14:33:17.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a6b400640 1 -- 172.21.15.100:0/3138746474 learned_addr learned my addr 172.21.15.100:0/3138746474 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:17.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a78a00640 1 --2- 172.21.15.100:0/3138746474 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6a740cbc70 0x7f6a740c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a73e00640 1 --2- 172.21.15.100:0/3138746474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a580051d0 0x7f6a74152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a6b400640 1 -- 172.21.15.100:0/3138746474 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6a740cbc70 msgr2=0x7f6a740c59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:17.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a6b400640 1 --2- 172.21.15.100:0/3138746474 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6a740cbc70 0x7f6a740c59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a6b400640 1 -- 172.21.15.100:0/3138746474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a580051d0 msgr2=0x7f6a74152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:17.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a6b400640 1 --2- 172.21.15.100:0/3138746474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a580051d0 0x7f6a74152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a6b400640 1 -- 172.21.15.100:0/3138746474 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a74151a60 con 0x7f6a740cb810 2024-04-23T14:33:17.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a73e00640 1 --2- 172.21.15.100:0/3138746474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a580051d0 0x7f6a74152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:17.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a6b400640 1 --2- 172.21.15.100:0/3138746474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a740cb810 0x7f6a740caa70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00f5f0 tx=0x7f6a5c00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:17.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a70c00640 1 -- 172.21.15.100:0/3138746474 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c0052e0 con 0x7f6a740cb810 2024-04-23T14:33:17.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a70c00640 1 -- 172.21.15.100:0/3138746474 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a5c018070 con 0x7f6a740cb810 2024-04-23T14:33:17.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a7aa00640 1 -- 172.21.15.100:0/3138746474 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a74151d40 con 0x7f6a740cb810 2024-04-23T14:33:17.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a70c00640 1 -- 172.21.15.100:0/3138746474 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c003820 con 0x7f6a740cb810 2024-04-23T14:33:17.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a7aa00640 1 -- 172.21.15.100:0/3138746474 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a74152300 con 0x7f6a740cb810 2024-04-23T14:33:17.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a7aa00640 1 -- 172.21.15.100:0/3138746474 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a74068ad0 con 0x7f6a740cb810 2024-04-23T14:33:17.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.502+0000 7f6a70c00640 1 -- 172.21.15.100:0/3138746474 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6a5c0039c0 con 0x7f6a740cb810 2024-04-23T14:33:17.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.506+0000 7f6a70c00640 1 --2- 172.21.15.100:0/3138746474 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6a500683b0 0x7f6a5006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.506+0000 7f6a70c00640 1 -- 172.21.15.100:0/3138746474 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f6a5c086bb0 con 0x7f6a740cb810 2024-04-23T14:33:17.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.506+0000 7f6a73e00640 1 --2- 172.21.15.100:0/3138746474 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6a500683b0 0x7f6a5006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.506+0000 7f6a73e00640 1 --2- 172.21.15.100:0/3138746474 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6a500683b0 0x7f6a5006a870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6a64009b60 tx=0x7f6a64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:17.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.510+0000 7f6a70c00640 1 -- 172.21.15.100:0/3138746474 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6a5c056cd0 con 0x7f6a740cb810 2024-04-23T14:33:17.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.666+0000 7f6a7aa00640 1 -- 172.21.15.100:0/3138746474 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6a74153d80 con 0x7f6a740cb810 2024-04-23T14:33:17.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.666+0000 7f6a70c00640 1 -- 172.21.15.100:0/3138746474 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f6a5c005480 con 0x7f6a740cb810 2024-04-23T14:33:17.670 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:33:17.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.670+0000 7f6a7aa00640 1 -- 172.21.15.100:0/3138746474 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6a500683b0 msgr2=0x7f6a5006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:17.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.670+0000 7f6a7aa00640 1 --2- 172.21.15.100:0/3138746474 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6a500683b0 0x7f6a5006a870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6a64009b60 tx=0x7f6a64008040 comp rx=0 tx=0).stop 2024-04-23T14:33:17.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.670+0000 7f6a7aa00640 1 -- 172.21.15.100:0/3138746474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a740cb810 msgr2=0x7f6a740caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:17.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.670+0000 7f6a7aa00640 1 --2- 172.21.15.100:0/3138746474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a740cb810 0x7f6a740caa70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00f5f0 tx=0x7f6a5c00fab0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.670+0000 7f6a7aa00640 1 -- 172.21.15.100:0/3138746474 shutdown_connections 2024-04-23T14:33:17.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.670+0000 7f6a7aa00640 1 --2- 172.21.15.100:0/3138746474 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6a500683b0 0x7f6a5006a870 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.670+0000 7f6a7aa00640 1 --2- 172.21.15.100:0/3138746474 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6a740cbc70 0x7f6a740c59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.670+0000 7f6a7aa00640 1 --2- 172.21.15.100:0/3138746474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a740cb810 0x7f6a740caa70 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.670+0000 7f6a7aa00640 1 --2- 172.21.15.100:0/3138746474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a580051d0 0x7f6a74152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.670+0000 7f6a7aa00640 1 -- 172.21.15.100:0/3138746474 >> 172.21.15.100:0/3138746474 conn(0x7f6a740b5200 msgr2=0x7f6a740afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:17.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.670+0000 7f6a7aa00640 1 -- 172.21.15.100:0/3138746474 shutdown_connections 2024-04-23T14:33:17.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.670+0000 7f6a7aa00640 1 -- 172.21.15.100:0/3138746474 wait complete. 2024-04-23T14:33:17.692 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:33:17.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.830+0000 7f52b7400640 1 Processor -- start 2024-04-23T14:33:17.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.830+0000 7f52b7400640 1 -- start start 2024-04-23T14:33:17.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.830+0000 7f52b7400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f52b006c6c0 0x7f52b00ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.830+0000 7f52b7400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f52b00ce620 0x7f52b00add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.830+0000 7f52b7400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b006dd60 con 0x7f52b006f8c0 2024-04-23T14:33:17.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.830+0000 7f52b7400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f52b00ad050 con 0x7f52b006c6c0 2024-04-23T14:33:17.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.830+0000 7f52b7400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f52b00ad8f0 con 0x7f52b00ce620 2024-04-23T14:33:17.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.830+0000 7f52b4a00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f52b006f8c0 0x7f52b00d0170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:36380/0 (socket says 172.21.15.100:36380) 2024-04-23T14:33:17.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.830+0000 7f52b4a00640 1 -- 172.21.15.100:0/1040373941 learned_addr learned my addr 172.21.15.100:0/1040373941 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:17.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52afe00640 1 --2- 172.21.15.100:0/1040373941 >> v2:172.21.15.151:3300/0 conn(0x7f52b006c6c0 0x7f52b00ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52b5400640 1 --2- 172.21.15.100:0/1040373941 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f52b00ce620 0x7f52b00add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52af400640 1 -- 172.21.15.100:0/1040373941 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 183627080 0 0) 0x7f52a0001120 con 0x7f52b006f8c0 2024-04-23T14:33:17.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52af400640 1 -- 172.21.15.100:0/1040373941 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5294005740 con 0x7f52b006f8c0 2024-04-23T14:33:17.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52af400640 1 -- 172.21.15.100:0/1040373941 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3547711032 0 0) 0x7f5294005740 con 0x7f52b006f8c0 2024-04-23T14:33:17.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52af400640 1 -- 172.21.15.100:0/1040373941 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f52b00ce620 msgr2=0x7f52b00add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:17.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52af400640 1 --2- 172.21.15.100:0/1040373941 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f52b00ce620 0x7f52b00add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52af400640 1 -- 172.21.15.100:0/1040373941 >> v2:172.21.15.151:3300/0 conn(0x7f52b006c6c0 msgr2=0x7f52b00ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:17.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52af400640 1 --2- 172.21.15.100:0/1040373941 >> v2:172.21.15.151:3300/0 conn(0x7f52b006c6c0 0x7f52b00ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52af400640 1 -- 172.21.15.100:0/1040373941 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b00d09c0 con 0x7f52b006f8c0 2024-04-23T14:33:17.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52af400640 1 -- 172.21.15.100:0/1040373941 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f52a0001850 con 0x7f52b006f8c0 2024-04-23T14:33:17.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52afe00640 1 --2- 172.21.15.100:0/1040373941 >> v2:172.21.15.151:3300/0 conn(0x7f52b006c6c0 0x7f52b00ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:33:17.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52af400640 1 -- 172.21.15.100:0/1040373941 >> v1:172.21.15.100:6789/0 conn(0x7f52b006f8c0 legacy=0x7f52b00d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:17.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52af400640 1 --2- 172.21.15.100:0/1040373941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5294005bd0 0x7f5294005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52af400640 1 --2- 172.21.15.100:0/1040373941 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f52b00ce620 0x7f5294006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52af400640 1 --2- 172.21.15.100:0/1040373941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5294006f60 0x7f52940073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52af400640 1 -- 172.21.15.100:0/1040373941 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f52b00ad050 con 0x7f5294005bd0 2024-04-23T14:33:17.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52af400640 1 -- 172.21.15.100:0/1040373941 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f52b00ad8f0 con 0x7f5294006f60 2024-04-23T14:33:17.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52af400640 1 -- 172.21.15.100:0/1040373941 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f529400dbe0 con 0x7f52b00ce620 2024-04-23T14:33:17.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52b4a00640 1 --2- 172.21.15.100:0/1040373941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5294005bd0 0x7f5294005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52afe00640 1 --2- 172.21.15.100:0/1040373941 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f52b00ce620 0x7f5294006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52b5400640 1 --2- 172.21.15.100:0/1040373941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5294006f60 0x7f52940073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52b4a00640 1 -- 172.21.15.100:0/1040373941 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f52b00ce620 msgr2=0x7f5294006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:17.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52b4a00640 1 --2- 172.21.15.100:0/1040373941 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f52b00ce620 0x7f5294006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52b4a00640 1 -- 172.21.15.100:0/1040373941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5294006f60 msgr2=0x7f52940073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:17.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52b4a00640 1 --2- 172.21.15.100:0/1040373941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5294006f60 0x7f52940073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52b4a00640 1 -- 172.21.15.100:0/1040373941 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5294005740 con 0x7f5294005bd0 2024-04-23T14:33:17.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.834+0000 7f52b5400640 1 --2- 172.21.15.100:0/1040373941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5294006f60 0x7f52940073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:17.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.838+0000 7f52b4a00640 1 --2- 172.21.15.100:0/1040373941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5294005bd0 0x7f5294005ff0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f52a000c4f0 tx=0x7f52a001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708a426e47f77b20 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:17.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.838+0000 7f52af400640 1 -- 172.21.15.100:0/1040373941 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a000c0e0 con 0x7f5294005bd0 2024-04-23T14:33:17.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.838+0000 7f52af400640 1 -- 172.21.15.100:0/1040373941 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a00a9020 con 0x7f5294005bd0 2024-04-23T14:33:17.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.838+0000 7f52af400640 1 -- 172.21.15.100:0/1040373941 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0005ae0 con 0x7f5294005bd0 2024-04-23T14:33:17.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.838+0000 7f52b7400640 1 -- 172.21.15.100:0/1040373941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5294005bd0 msgr2=0x7f5294005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:17.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.838+0000 7f52b7400640 1 --2- 172.21.15.100:0/1040373941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5294005bd0 0x7f5294005ff0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f52a000c4f0 tx=0x7f52a001ccc0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.838+0000 7f52b5400640 1 -- 172.21.15.100:0/1040373941 reap_dead start 2024-04-23T14:33:17.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.838+0000 7f52b7400640 1 -- 172.21.15.100:0/1040373941 shutdown_connections 2024-04-23T14:33:17.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.838+0000 7f52b7400640 1 -- 172.21.15.100:0/1040373941 >> 172.21.15.100:0/1040373941 conn(0x7f52b00b5200 msgr2=0x7f52b00b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:17.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.838+0000 7f52b7400640 1 -- 172.21.15.100:0/1040373941 shutdown_connections 2024-04-23T14:33:17.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.838+0000 7f52b7400640 1 -- 172.21.15.100:0/1040373941 wait complete. 2024-04-23T14:33:17.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.838+0000 7f52b7400640 1 Processor -- start 2024-04-23T14:33:17.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.838+0000 7f52b7400640 1 -- start start 2024-04-23T14:33:17.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.838+0000 7f52b7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5294005bd0 0x7f52b00997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.838+0000 7f52b7400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f52b0160ca0 0x7f52b0162480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.838+0000 7f52b7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52b01629c0 0x7f52b00c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.838+0000 7f52b7400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f52b00cd800 con 0x7f5294005bd0 2024-04-23T14:33:17.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.838+0000 7f52b7400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f52b00cd680 con 0x7f52b01629c0 2024-04-23T14:33:17.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.838+0000 7f52b7400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f52b00cd980 con 0x7f52b0160ca0 2024-04-23T14:33:17.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.838+0000 7f52b4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5294005bd0 0x7f52b00997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.838+0000 7f52b4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5294005bd0 0x7f52b00997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32870/0 (socket says 172.21.15.100:32870) 2024-04-23T14:33:17.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.838+0000 7f52b4a00640 1 -- 172.21.15.100:0/1347064955 learned_addr learned my addr 172.21.15.100:0/1347064955 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:17.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.842+0000 7f52afe00640 1 --2- 172.21.15.100:0/1347064955 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f52b0160ca0 0x7f52b0162480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.842+0000 7f52b5400640 1 --2- 172.21.15.100:0/1347064955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52b01629c0 0x7f52b00c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.842+0000 7f52b4a00640 1 -- 172.21.15.100:0/1347064955 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f52b0160ca0 msgr2=0x7f52b0162480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:17.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.842+0000 7f52b4a00640 1 --2- 172.21.15.100:0/1347064955 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f52b0160ca0 0x7f52b0162480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.842+0000 7f52b4a00640 1 -- 172.21.15.100:0/1347064955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52b01629c0 msgr2=0x7f52b00c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:17.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.842+0000 7f52b4a00640 1 --2- 172.21.15.100:0/1347064955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52b01629c0 0x7f52b00c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.842+0000 7f52b4a00640 1 -- 172.21.15.100:0/1347064955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b0162e60 con 0x7f5294005bd0 2024-04-23T14:33:17.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.842+0000 7f52afe00640 1 --2- 172.21.15.100:0/1347064955 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f52b0160ca0 0x7f52b0162480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:17.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.842+0000 7f52b5400640 1 --2- 172.21.15.100:0/1347064955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52b01629c0 0x7f52b00c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:17.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.842+0000 7f52b4a00640 1 --2- 172.21.15.100:0/1347064955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5294005bd0 0x7f52b00997f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f52a000c4f0 tx=0x7f5294007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:17.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.842+0000 7f52acc00640 1 -- 172.21.15.100:0/1347064955 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a00047f0 con 0x7f5294005bd0 2024-04-23T14:33:17.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.842+0000 7f52acc00640 1 -- 172.21.15.100:0/1347064955 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a0004990 con 0x7f5294005bd0 2024-04-23T14:33:17.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.842+0000 7f52acc00640 1 -- 172.21.15.100:0/1347064955 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0004d40 con 0x7f5294005bd0 2024-04-23T14:33:17.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.842+0000 7f52b7400640 1 -- 172.21.15.100:0/1347064955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b0163140 con 0x7f5294005bd0 2024-04-23T14:33:17.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.842+0000 7f52b7400640 1 -- 172.21.15.100:0/1347064955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b0163700 con 0x7f5294005bd0 2024-04-23T14:33:17.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.842+0000 7f52b7400640 1 -- 172.21.15.100:0/1347064955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b0068ad0 con 0x7f5294005bd0 2024-04-23T14:33:17.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.842+0000 7f52acc00640 1 -- 172.21.15.100:0/1347064955 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f52a00a57b0 con 0x7f5294005bd0 2024-04-23T14:33:17.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.846+0000 7f52acc00640 1 --2- 172.21.15.100:0/1347064955 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f52900683b0 0x7f529006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:17.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.846+0000 7f52acc00640 1 -- 172.21.15.100:0/1347064955 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f52a01202b0 con 0x7f5294005bd0 2024-04-23T14:33:17.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.846+0000 7f52afe00640 1 --2- 172.21.15.100:0/1347064955 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f52900683b0 0x7f529006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:17.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.846+0000 7f52afe00640 1 --2- 172.21.15.100:0/1347064955 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f52900683b0 0x7f529006a870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f52b00cbe70 tx=0x7f529c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:17.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:17.850+0000 7f52acc00640 1 -- 172.21.15.100:0/1347064955 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f52b006dd60 con 0x7f5294005bd0 2024-04-23T14:33:18.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.006+0000 7f52b7400640 1 -- 172.21.15.100:0/1347064955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f52b006cc00 con 0x7f5294005bd0 2024-04-23T14:33:18.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.006+0000 7f52acc00640 1 -- 172.21.15.100:0/1347064955 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f52a00b6340 con 0x7f5294005bd0 2024-04-23T14:33:18.008 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:18.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.006+0000 7f52b7400640 1 -- 172.21.15.100:0/1347064955 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f52900683b0 msgr2=0x7f529006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:18.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.006+0000 7f52b7400640 1 --2- 172.21.15.100:0/1347064955 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f52900683b0 0x7f529006a870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f52b00cbe70 tx=0x7f529c008040 comp rx=0 tx=0).stop 2024-04-23T14:33:18.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.006+0000 7f52b7400640 1 -- 172.21.15.100:0/1347064955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5294005bd0 msgr2=0x7f52b00997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:18.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.006+0000 7f52b7400640 1 --2- 172.21.15.100:0/1347064955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5294005bd0 0x7f52b00997f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f52a000c4f0 tx=0x7f5294007a50 comp rx=0 tx=0).stop 2024-04-23T14:33:18.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.006+0000 7f52b7400640 1 -- 172.21.15.100:0/1347064955 shutdown_connections 2024-04-23T14:33:18.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.006+0000 7f52b7400640 1 --2- 172.21.15.100:0/1347064955 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f52900683b0 0x7f529006a870 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.006+0000 7f52b7400640 1 --2- 172.21.15.100:0/1347064955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52b01629c0 0x7f52b00c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.006+0000 7f52b7400640 1 --2- 172.21.15.100:0/1347064955 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f52b0160ca0 0x7f52b0162480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.006+0000 7f52b7400640 1 --2- 172.21.15.100:0/1347064955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5294005bd0 0x7f52b00997f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.006+0000 7f52b7400640 1 -- 172.21.15.100:0/1347064955 >> 172.21.15.100:0/1347064955 conn(0x7f52b00b5200 msgr2=0x7f52b00aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:18.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.006+0000 7f52b7400640 1 -- 172.21.15.100:0/1347064955 shutdown_connections 2024-04-23T14:33:18.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.006+0000 7f52b7400640 1 -- 172.21.15.100:0/1347064955 wait complete. 2024-04-23T14:33:18.029 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:33:18.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.162+0000 7f10edc00640 1 Processor -- start 2024-04-23T14:33:18.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.166+0000 7f10edc00640 1 -- start start 2024-04-23T14:33:18.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.166+0000 7f10edc00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f10e80cd300 0x7f10e81671f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:18.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.166+0000 7f10edc00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10e806cbb0 con 0x7f10e806bb20 2024-04-23T14:33:18.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.166+0000 7f10edc00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10e806ce90 con 0x7f10e815c7a0 2024-04-23T14:33:18.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.166+0000 7f10edc00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f10e806d450 con 0x7f10e80cd300 2024-04-23T14:33:18.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.166+0000 7f10e6000640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f10e806bb20 0x7f10e80ccbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:36382/0 (socket says 172.21.15.100:36382) 2024-04-23T14:33:18.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.166+0000 7f10e6000640 1 -- 172.21.15.100:0/3488361039 learned_addr learned my addr 172.21.15.100:0/3488361039 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:18.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.166+0000 7f10e7400640 1 --2- 172.21.15.100:0/3488361039 >> v2:172.21.15.151:3300/0 conn(0x7f10e80cd300 0x7f10e81671f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:18.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.166+0000 7f10e5600640 1 -- 172.21.15.100:0/3488361039 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 550581340 0 0) 0x7f10e806cbb0 con 0x7f10e806bb20 2024-04-23T14:33:18.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.166+0000 7f10e5600640 1 -- 172.21.15.100:0/3488361039 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10cc003530 con 0x7f10e806bb20 2024-04-23T14:33:18.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.166+0000 7f10e5600640 1 -- 172.21.15.100:0/3488361039 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2299716505 0 0) 0x7f10cc003530 con 0x7f10e806bb20 2024-04-23T14:33:18.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.166+0000 7f10e7400640 1 -- 172.21.15.100:0/3488361039 >> v1:172.21.15.151:6789/0 conn(0x7f10e815c7a0 legacy=0x7f10e80af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:18.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.166+0000 7f10e7400640 1 -- 172.21.15.100:0/3488361039 >> v1:172.21.15.100:6789/0 conn(0x7f10e806bb20 legacy=0x7f10e80ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:18.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.166+0000 7f10e7400640 1 -- 172.21.15.100:0/3488361039 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e806d170 con 0x7f10e80cd300 2024-04-23T14:33:18.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.170+0000 7f10e7400640 1 --2- 172.21.15.100:0/3488361039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e80cd300 0x7f10e81671f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009d90 tx=0x7f10e001c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c08d0fa157c93d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:18.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.170+0000 7f10e5600640 1 -- 172.21.15.100:0/3488361039 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4001680 con 0x7f10e80cd300 2024-04-23T14:33:18.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.170+0000 7f10e5600640 1 -- 172.21.15.100:0/3488361039 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10e00a6450 con 0x7f10e80cd300 2024-04-23T14:33:18.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.170+0000 7f10e5600640 1 -- 172.21.15.100:0/3488361039 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e00a6820 con 0x7f10e80cd300 2024-04-23T14:33:18.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.170+0000 7f10edc00640 1 -- 172.21.15.100:0/3488361039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e80cd300 msgr2=0x7f10e81671f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:18.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.170+0000 7f10edc00640 1 --2- 172.21.15.100:0/3488361039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e80cd300 0x7f10e81671f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009d90 tx=0x7f10e001c800 comp rx=0 tx=0).stop 2024-04-23T14:33:18.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.170+0000 7f10edc00640 1 -- 172.21.15.100:0/3488361039 shutdown_connections 2024-04-23T14:33:18.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.170+0000 7f10edc00640 1 --2- 172.21.15.100:0/3488361039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e80cd300 0x7f10e81671f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.170+0000 7f10edc00640 1 -- 172.21.15.100:0/3488361039 >> 172.21.15.100:0/3488361039 conn(0x7f10e80b5200 msgr2=0x7f10e80b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:18.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.170+0000 7f10edc00640 1 -- 172.21.15.100:0/3488361039 shutdown_connections 2024-04-23T14:33:18.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.170+0000 7f10edc00640 1 -- 172.21.15.100:0/3488361039 wait complete. 2024-04-23T14:33:18.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.170+0000 7f10edc00640 1 Processor -- start 2024-04-23T14:33:18.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.170+0000 7f10edc00640 1 -- start start 2024-04-23T14:33:18.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10edc00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f10e806bb20 0x7f10e80c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:18.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10edc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10e80cd300 0x7f10e80c3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:18.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10edc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e815c7a0 0x7f10e814e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:18.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10edc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f10e816a2e0 con 0x7f10e80cd300 2024-04-23T14:33:18.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10edc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f10e816a160 con 0x7f10e815c7a0 2024-04-23T14:33:18.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10edc00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f10e816a460 con 0x7f10e806bb20 2024-04-23T14:33:18.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10e6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10e80cd300 0x7f10e80c3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:18.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10e6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10e80cd300 0x7f10e80c3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32886/0 (socket says 172.21.15.100:32886) 2024-04-23T14:33:18.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10e6000640 1 -- 172.21.15.100:0/3161345130 learned_addr learned my addr 172.21.15.100:0/3161345130 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:18.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10e7400640 1 --2- 172.21.15.100:0/3161345130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e815c7a0 0x7f10e814e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:18.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10e6a00640 1 --2- 172.21.15.100:0/3161345130 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f10e806bb20 0x7f10e80c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:18.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10e6000640 1 -- 172.21.15.100:0/3161345130 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f10e806bb20 msgr2=0x7f10e80c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:18.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10e6000640 1 --2- 172.21.15.100:0/3161345130 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f10e806bb20 0x7f10e80c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10e6000640 1 -- 172.21.15.100:0/3161345130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e815c7a0 msgr2=0x7f10e814e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:18.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10e6000640 1 --2- 172.21.15.100:0/3161345130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e815c7a0 0x7f10e814e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10e6000640 1 -- 172.21.15.100:0/3161345130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e80734e0 con 0x7f10e80cd300 2024-04-23T14:33:18.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10e6000640 1 --2- 172.21.15.100:0/3161345130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10e80cd300 0x7f10e80c3dc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f10d400c730 tx=0x7f10d400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:18.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10dea00640 1 -- 172.21.15.100:0/3161345130 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4017070 con 0x7f10e80cd300 2024-04-23T14:33:18.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10dea00640 1 -- 172.21.15.100:0/3161345130 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d4005ae0 con 0x7f10e80cd300 2024-04-23T14:33:18.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10edc00640 1 -- 172.21.15.100:0/3161345130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e80737c0 con 0x7f10e80cd300 2024-04-23T14:33:18.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10dea00640 1 -- 172.21.15.100:0/3161345130 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4007070 con 0x7f10e80cd300 2024-04-23T14:33:18.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10edc00640 1 -- 172.21.15.100:0/3161345130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e8073d80 con 0x7f10e80cd300 2024-04-23T14:33:18.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10edc00640 1 -- 172.21.15.100:0/3161345130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e8068ad0 con 0x7f10e80cd300 2024-04-23T14:33:18.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.174+0000 7f10dea00640 1 -- 172.21.15.100:0/3161345130 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f10d401a070 con 0x7f10e80cd300 2024-04-23T14:33:18.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.178+0000 7f10dea00640 1 --2- 172.21.15.100:0/3161345130 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f10c80683b0 0x7f10c806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:18.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.178+0000 7f10dea00640 1 -- 172.21.15.100:0/3161345130 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f10d40863e0 con 0x7f10e80cd300 2024-04-23T14:33:18.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.178+0000 7f10e6a00640 1 --2- 172.21.15.100:0/3161345130 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f10c80683b0 0x7f10c806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:18.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.178+0000 7f10e6a00640 1 --2- 172.21.15.100:0/3161345130 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f10c80683b0 0x7f10c806a870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f10d8000f70 tx=0x7f10d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:18.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.182+0000 7f10dea00640 1 -- 172.21.15.100:0/3161345130 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f10d4055da0 con 0x7f10e80cd300 2024-04-23T14:33:18.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.334+0000 7f10edc00640 1 -- 172.21.15.100:0/3161345130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f10e8073f80 con 0x7f10e80cd300 2024-04-23T14:33:18.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.338+0000 7f10dea00640 1 -- 172.21.15.100:0/3161345130 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f10d405a0a0 con 0x7f10e80cd300 2024-04-23T14:33:18.338 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:33:18.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.338+0000 7f10edc00640 1 -- 172.21.15.100:0/3161345130 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f10c80683b0 msgr2=0x7f10c806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:18.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.338+0000 7f10edc00640 1 --2- 172.21.15.100:0/3161345130 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f10c80683b0 0x7f10c806a870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f10d8000f70 tx=0x7f10d8009040 comp rx=0 tx=0).stop 2024-04-23T14:33:18.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.338+0000 7f10edc00640 1 -- 172.21.15.100:0/3161345130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10e80cd300 msgr2=0x7f10e80c3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:18.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.338+0000 7f10edc00640 1 --2- 172.21.15.100:0/3161345130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10e80cd300 0x7f10e80c3dc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f10d400c730 tx=0x7f10d400cbf0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.338+0000 7f10edc00640 1 -- 172.21.15.100:0/3161345130 shutdown_connections 2024-04-23T14:33:18.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.338+0000 7f10edc00640 1 --2- 172.21.15.100:0/3161345130 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f10c80683b0 0x7f10c806a870 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.338+0000 7f10edc00640 1 --2- 172.21.15.100:0/3161345130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e815c7a0 0x7f10e814e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.338+0000 7f10edc00640 1 --2- 172.21.15.100:0/3161345130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10e80cd300 0x7f10e80c3dc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.338+0000 7f10edc00640 1 --2- 172.21.15.100:0/3161345130 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f10e806bb20 0x7f10e80c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.338+0000 7f10edc00640 1 -- 172.21.15.100:0/3161345130 >> 172.21.15.100:0/3161345130 conn(0x7f10e80b5200 msgr2=0x7f10e80cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:18.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.338+0000 7f10edc00640 1 -- 172.21.15.100:0/3161345130 shutdown_connections 2024-04-23T14:33:18.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.338+0000 7f10edc00640 1 -- 172.21.15.100:0/3161345130 wait complete. 2024-04-23T14:33:18.359 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-23T14:33:18.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f2905200640 1 Processor -- start 2024-04-23T14:33:18.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f2905200640 1 -- start start 2024-04-23T14:33:18.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f2905200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f290006f8c0 0x7f290006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:18.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f2905200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f29000ce7f0 0x7f29000add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:18.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f2905200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f290006dd60 con 0x7f29000d0170 2024-04-23T14:33:18.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f2905200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f29000ad050 con 0x7f29000ce7f0 2024-04-23T14:33:18.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f2905200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f29000ad8f0 con 0x7f290006f8c0 2024-04-23T14:33:18.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fd400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f29000d0170 0x7f29000ce0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:36398/0 (socket says 172.21.15.100:36398) 2024-04-23T14:33:18.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fd400640 1 -- 172.21.15.100:0/330162154 learned_addr learned my addr 172.21.15.100:0/330162154 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:18.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fde00640 1 --2- 172.21.15.100:0/330162154 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f290006f8c0 0x7f290006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:18.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fe800640 1 --2- 172.21.15.100:0/330162154 >> v2:172.21.15.151:3300/0 conn(0x7f29000ce7f0 0x7f29000add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:18.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fca00640 1 -- 172.21.15.100:0/330162154 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 724206218 0 0) 0x7f28ec001120 con 0x7f29000d0170 2024-04-23T14:33:18.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fca00640 1 -- 172.21.15.100:0/330162154 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28e4003530 con 0x7f29000d0170 2024-04-23T14:33:18.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fca00640 1 -- 172.21.15.100:0/330162154 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3643153809 0 0) 0x7f28e4003530 con 0x7f29000d0170 2024-04-23T14:33:18.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fca00640 1 -- 172.21.15.100:0/330162154 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f290006f8c0 msgr2=0x7f290006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:18.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fca00640 1 --2- 172.21.15.100:0/330162154 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f290006f8c0 0x7f290006c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fca00640 1 -- 172.21.15.100:0/330162154 >> v2:172.21.15.151:3300/0 conn(0x7f29000ce7f0 msgr2=0x7f29000add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:18.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fca00640 1 --2- 172.21.15.100:0/330162154 >> v2:172.21.15.151:3300/0 conn(0x7f29000ce7f0 0x7f29000add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fca00640 1 -- 172.21.15.100:0/330162154 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29000d09c0 con 0x7f29000d0170 2024-04-23T14:33:18.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fca00640 1 -- 172.21.15.100:0/330162154 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f28ec001850 con 0x7f29000d0170 2024-04-23T14:33:18.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fde00640 1 --2- 172.21.15.100:0/330162154 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f290006f8c0 0x7f290006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:33:18.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fca00640 1 -- 172.21.15.100:0/330162154 >> v1:172.21.15.100:6789/0 conn(0x7f29000d0170 legacy=0x7f29000ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:18.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fca00640 1 --2- 172.21.15.100:0/330162154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28e40039c0 0x7f28e4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:18.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fca00640 1 --2- 172.21.15.100:0/330162154 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f290006f8c0 0x7f28e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:18.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fca00640 1 --2- 172.21.15.100:0/330162154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e4004d50 0x7f28e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:18.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fca00640 1 -- 172.21.15.100:0/330162154 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f29000ad050 con 0x7f28e4004d50 2024-04-23T14:33:18.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fca00640 1 -- 172.21.15.100:0/330162154 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f29000ad8f0 con 0x7f28e40039c0 2024-04-23T14:33:18.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fca00640 1 -- 172.21.15.100:0/330162154 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f28e400b9d0 con 0x7f290006f8c0 2024-04-23T14:33:18.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fe800640 1 --2- 172.21.15.100:0/330162154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e4004d50 0x7f28e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:18.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fde00640 1 --2- 172.21.15.100:0/330162154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28e40039c0 0x7f28e4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:18.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fd400640 1 --2- 172.21.15.100:0/330162154 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f290006f8c0 0x7f28e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:18.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fe800640 1 -- 172.21.15.100:0/330162154 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f290006f8c0 msgr2=0x7f28e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:18.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fe800640 1 --2- 172.21.15.100:0/330162154 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f290006f8c0 0x7f28e4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fe800640 1 -- 172.21.15.100:0/330162154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28e40039c0 msgr2=0x7f28e4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:18.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fe800640 1 --2- 172.21.15.100:0/330162154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28e40039c0 0x7f28e4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fe800640 1 -- 172.21.15.100:0/330162154 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e4003530 con 0x7f28e4004d50 2024-04-23T14:33:18.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.498+0000 7f28fd400640 1 --2- 172.21.15.100:0/330162154 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f290006f8c0 0x7f28e4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:18.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.502+0000 7f28fe800640 1 --2- 172.21.15.100:0/330162154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e4004d50 0x7f28e40051d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009fc0 tx=0x7f28f801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3126f41ea42ee208 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:18.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.502+0000 7f28fca00640 1 -- 172.21.15.100:0/330162154 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f80b2070 con 0x7f28e4004d50 2024-04-23T14:33:18.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.502+0000 7f28fca00640 1 -- 172.21.15.100:0/330162154 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28f80a5cd0 con 0x7f28e4004d50 2024-04-23T14:33:18.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.502+0000 7f28fca00640 1 -- 172.21.15.100:0/330162154 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8009c10 con 0x7f28e4004d50 2024-04-23T14:33:18.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.502+0000 7f2905200640 1 -- 172.21.15.100:0/330162154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e4004d50 msgr2=0x7f28e40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:18.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.502+0000 7f2905200640 1 --2- 172.21.15.100:0/330162154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e4004d50 0x7f28e40051d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009fc0 tx=0x7f28f801ccc0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.502+0000 7f28fe800640 1 -- 172.21.15.100:0/330162154 reap_dead start 2024-04-23T14:33:18.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.502+0000 7f2905200640 1 -- 172.21.15.100:0/330162154 shutdown_connections 2024-04-23T14:33:18.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.502+0000 7f2905200640 1 -- 172.21.15.100:0/330162154 >> 172.21.15.100:0/330162154 conn(0x7f29000b5200 msgr2=0x7f29000b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:18.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.502+0000 7f2905200640 1 -- 172.21.15.100:0/330162154 shutdown_connections 2024-04-23T14:33:18.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.502+0000 7f2905200640 1 -- 172.21.15.100:0/330162154 wait complete. 2024-04-23T14:33:18.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f2905200640 1 Processor -- start 2024-04-23T14:33:18.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f2905200640 1 -- start start 2024-04-23T14:33:18.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f2905200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e4004d50 0x7f2900099aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:18.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f2905200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29000b0de0 0x7f29000afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:18.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f2905200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f29000b0140 0x7f29001533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:18.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f2905200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f29000d05b0 con 0x7f28e4004d50 2024-04-23T14:33:18.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f2905200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f29000ad580 con 0x7f29000b0de0 2024-04-23T14:33:18.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f2905200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f29000d0730 con 0x7f29000b0140 2024-04-23T14:33:18.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f28fde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e4004d50 0x7f2900099aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:18.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f28fde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e4004d50 0x7f2900099aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32902/0 (socket says 172.21.15.100:32902) 2024-04-23T14:33:18.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f28fde00640 1 -- 172.21.15.100:0/4186150617 learned_addr learned my addr 172.21.15.100:0/4186150617 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:18.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f28fe800640 1 --2- 172.21.15.100:0/4186150617 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f29000b0140 0x7f29001533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:18.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f28fd400640 1 --2- 172.21.15.100:0/4186150617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29000b0de0 0x7f29000afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:18.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f28fde00640 1 -- 172.21.15.100:0/4186150617 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f29000b0140 msgr2=0x7f29001533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:18.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f28fde00640 1 --2- 172.21.15.100:0/4186150617 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f29000b0140 0x7f29001533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f28fde00640 1 -- 172.21.15.100:0/4186150617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29000b0de0 msgr2=0x7f29000afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:18.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f28fde00640 1 --2- 172.21.15.100:0/4186150617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29000b0de0 0x7f29000afc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f28fde00640 1 -- 172.21.15.100:0/4186150617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29000b0650 con 0x7f28e4004d50 2024-04-23T14:33:18.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f28fde00640 1 --2- 172.21.15.100:0/4186150617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e4004d50 0x7f2900099aa0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f28f0011910 tx=0x7f28f0011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:18.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f28f6000640 1 -- 172.21.15.100:0/4186150617 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f000fa90 con 0x7f28e4004d50 2024-04-23T14:33:18.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f28f6000640 1 -- 172.21.15.100:0/4186150617 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28f000fc30 con 0x7f28e4004d50 2024-04-23T14:33:18.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f28f6000640 1 -- 172.21.15.100:0/4186150617 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f0004030 con 0x7f28e4004d50 2024-04-23T14:33:18.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f2905200640 1 -- 172.21.15.100:0/4186150617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29000b0930 con 0x7f28e4004d50 2024-04-23T14:33:18.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.506+0000 7f2905200640 1 -- 172.21.15.100:0/4186150617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2900155890 con 0x7f28e4004d50 2024-04-23T14:33:18.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.510+0000 7f2905200640 1 -- 172.21.15.100:0/4186150617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2900068ad0 con 0x7f28e4004d50 2024-04-23T14:33:18.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.510+0000 7f28f6000640 1 -- 172.21.15.100:0/4186150617 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f28f0012040 con 0x7f28e4004d50 2024-04-23T14:33:18.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.510+0000 7f28f6000640 1 --2- 172.21.15.100:0/4186150617 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f28e0068380 0x7f28e006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:18.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.510+0000 7f28f6000640 1 -- 172.21.15.100:0/4186150617 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f28f00871b0 con 0x7f28e4004d50 2024-04-23T14:33:18.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.510+0000 7f28fd400640 1 --2- 172.21.15.100:0/4186150617 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f28e0068380 0x7f28e006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:18.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.510+0000 7f28fd400640 1 --2- 172.21.15.100:0/4186150617 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f28e0068380 0x7f28e006a840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f28ec000c10 tx=0x7f28ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:18.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.514+0000 7f28f6000640 1 -- 172.21.15.100:0/4186150617 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f28f0019070 con 0x7f28e4004d50 2024-04-23T14:33:18.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.674+0000 7f2905200640 1 -- 172.21.15.100:0/4186150617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f290006cc00 con 0x7f28e4004d50 2024-04-23T14:33:18.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.674+0000 7f28f6000640 1 -- 172.21.15.100:0/4186150617 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f28f00572d0 con 0x7f28e4004d50 2024-04-23T14:33:18.675 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:18.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.674+0000 7f2905200640 1 -- 172.21.15.100:0/4186150617 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f28e0068380 msgr2=0x7f28e006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:18.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.674+0000 7f2905200640 1 --2- 172.21.15.100:0/4186150617 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f28e0068380 0x7f28e006a840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f28ec000c10 tx=0x7f28ec009040 comp rx=0 tx=0).stop 2024-04-23T14:33:18.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.674+0000 7f2905200640 1 -- 172.21.15.100:0/4186150617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e4004d50 msgr2=0x7f2900099aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:18.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.674+0000 7f2905200640 1 --2- 172.21.15.100:0/4186150617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e4004d50 0x7f2900099aa0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f28f0011910 tx=0x7f28f0011dd0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.674+0000 7f2905200640 1 -- 172.21.15.100:0/4186150617 shutdown_connections 2024-04-23T14:33:18.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.674+0000 7f2905200640 1 --2- 172.21.15.100:0/4186150617 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f28e0068380 0x7f28e006a840 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.674+0000 7f2905200640 1 --2- 172.21.15.100:0/4186150617 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f29000b0140 0x7f29001533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.674+0000 7f2905200640 1 --2- 172.21.15.100:0/4186150617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29000b0de0 0x7f29000afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.674+0000 7f2905200640 1 --2- 172.21.15.100:0/4186150617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e4004d50 0x7f2900099aa0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.674+0000 7f2905200640 1 -- 172.21.15.100:0/4186150617 >> 172.21.15.100:0/4186150617 conn(0x7f29000b5200 msgr2=0x7f29000caa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:18.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.674+0000 7f2905200640 1 -- 172.21.15.100:0/4186150617 shutdown_connections 2024-04-23T14:33:18.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.674+0000 7f2905200640 1 -- 172.21.15.100:0/4186150617 wait complete. 2024-04-23T14:33:18.696 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:33:18.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.830+0000 7f4871000640 1 Processor -- start 2024-04-23T14:33:18.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.830+0000 7f4871000640 1 -- start start 2024-04-23T14:33:18.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.834+0000 7f4871000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f486c15b660 0x7f486c0c6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:18.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.834+0000 7f4871000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f486c0c6d60 0x7f486c0d0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:18.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.834+0000 7f4871000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f486c0cee00 con 0x7f486c0d06d0 2024-04-23T14:33:18.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.834+0000 7f4871000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f486c0d0b60 con 0x7f486c15b660 2024-04-23T14:33:18.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.834+0000 7f4871000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f486c15ba40 con 0x7f486c0c6d60 2024-04-23T14:33:18.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.834+0000 7f486a800640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f486c0d06d0 0x7f486c15f0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57964/0 (socket says 172.21.15.100:57964) 2024-04-23T14:33:18.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.834+0000 7f486a800640 1 -- 172.21.15.100:0/1162402103 learned_addr learned my addr 172.21.15.100:0/1162402103 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:18.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.834+0000 7f4869e00640 1 --2- 172.21.15.100:0/1162402103 >> v2:172.21.15.151:3300/0 conn(0x7f486c15b660 0x7f486c0c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:18.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.834+0000 7f4869400640 1 --2- 172.21.15.100:0/1162402103 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f486c0c6d60 0x7f486c0d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:18.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.834+0000 7f4868a00640 1 -- 172.21.15.100:0/1162402103 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 129846209 0 0) 0x7f4864001120 con 0x7f486c0d06d0 2024-04-23T14:33:18.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.834+0000 7f4868a00640 1 -- 172.21.15.100:0/1162402103 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4850003530 con 0x7f486c0d06d0 2024-04-23T14:33:18.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.834+0000 7f4868a00640 1 -- 172.21.15.100:0/1162402103 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2887508592 0 0) 0x7f4850003530 con 0x7f486c0d06d0 2024-04-23T14:33:18.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.834+0000 7f4869e00640 1 -- 172.21.15.100:0/1162402103 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f486c0c6d60 msgr2=0x7f486c0d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:18.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.834+0000 7f4869e00640 1 --2- 172.21.15.100:0/1162402103 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f486c0c6d60 0x7f486c0d0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.834+0000 7f4869e00640 1 -- 172.21.15.100:0/1162402103 >> v1:172.21.15.100:6789/0 conn(0x7f486c0d06d0 legacy=0x7f486c15f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:18.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.834+0000 7f4869e00640 1 -- 172.21.15.100:0/1162402103 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c06bc50 con 0x7f486c15b660 2024-04-23T14:33:18.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.838+0000 7f4869e00640 1 --2- 172.21.15.100:0/1162402103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f486c15b660 0x7f486c0c6820 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f485c009d90 tx=0x7f485c01c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d628f0ccdb11ee6c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:18.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.838+0000 7f4868a00640 1 -- 172.21.15.100:0/1162402103 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c0b2070 con 0x7f486c15b660 2024-04-23T14:33:18.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.838+0000 7f4868a00640 1 -- 172.21.15.100:0/1162402103 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f485c0a5570 con 0x7f486c15b660 2024-04-23T14:33:18.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.838+0000 7f4868a00640 1 -- 172.21.15.100:0/1162402103 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4864001850 con 0x7f486c15b660 2024-04-23T14:33:18.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.838+0000 7f4871000640 1 -- 172.21.15.100:0/1162402103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f486c15b660 msgr2=0x7f486c0c6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:18.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.838+0000 7f4871000640 1 --2- 172.21.15.100:0/1162402103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f486c15b660 0x7f486c0c6820 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f485c009d90 tx=0x7f485c01c860 comp rx=0 tx=0).stop 2024-04-23T14:33:18.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.838+0000 7f4871000640 1 -- 172.21.15.100:0/1162402103 shutdown_connections 2024-04-23T14:33:18.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.838+0000 7f4871000640 1 --2- 172.21.15.100:0/1162402103 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f486c0c6d60 0x7f486c0d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.838+0000 7f4871000640 1 --2- 172.21.15.100:0/1162402103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f486c15b660 0x7f486c0c6820 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.838+0000 7f4871000640 1 -- 172.21.15.100:0/1162402103 >> 172.21.15.100:0/1162402103 conn(0x7f486c0b5310 msgr2=0x7f486c0b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:18.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.838+0000 7f4871000640 1 -- 172.21.15.100:0/1162402103 shutdown_connections 2024-04-23T14:33:18.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.838+0000 7f4871000640 1 -- 172.21.15.100:0/1162402103 wait complete. 2024-04-23T14:33:18.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.838+0000 7f4871000640 1 Processor -- start 2024-04-23T14:33:18.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.838+0000 7f4871000640 1 -- start start 2024-04-23T14:33:18.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4871000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f486c0c6d60 0x7f486c0ca950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:18.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4871000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f486c0d06d0 0x7f486c14a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:18.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4871000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f486c15b660 0x7f486c151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:18.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4871000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f486c0cccd0 con 0x7f486c0d06d0 2024-04-23T14:33:18.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4871000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f486c06c7d0 con 0x7f486c0c6d60 2024-04-23T14:33:18.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4871000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f486c0cce50 con 0x7f486c15b660 2024-04-23T14:33:18.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4869400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f486c0d06d0 0x7f486c14a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:18.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4869400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f486c0d06d0 0x7f486c14a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47972/0 (socket says 172.21.15.100:47972) 2024-04-23T14:33:18.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4869400640 1 -- 172.21.15.100:0/1102882308 learned_addr learned my addr 172.21.15.100:0/1102882308 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:18.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f486a800640 1 --2- 172.21.15.100:0/1102882308 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f486c15b660 0x7f486c151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:18.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4869e00640 1 --2- 172.21.15.100:0/1102882308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f486c0c6d60 0x7f486c0ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:18.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4869400640 1 -- 172.21.15.100:0/1102882308 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f486c15b660 msgr2=0x7f486c151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:18.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4869400640 1 --2- 172.21.15.100:0/1102882308 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f486c15b660 0x7f486c151c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4869400640 1 -- 172.21.15.100:0/1102882308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f486c0c6d60 msgr2=0x7f486c0ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:18.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4869400640 1 --2- 172.21.15.100:0/1102882308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f486c0c6d60 0x7f486c0ca950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:18.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4869400640 1 -- 172.21.15.100:0/1102882308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c1504a0 con 0x7f486c0d06d0 2024-04-23T14:33:18.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4869400640 1 --2- 172.21.15.100:0/1102882308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f486c0d06d0 0x7f486c14a550 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f485800e930 tx=0x7f485800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:18.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4862000640 1 -- 172.21.15.100:0/1102882308 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4858017070 con 0x7f486c0d06d0 2024-04-23T14:33:18.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4862000640 1 -- 172.21.15.100:0/1102882308 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f485800ca00 con 0x7f486c0d06d0 2024-04-23T14:33:18.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4862000640 1 -- 172.21.15.100:0/1102882308 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485800cdb0 con 0x7f486c0d06d0 2024-04-23T14:33:18.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4871000640 1 -- 172.21.15.100:0/1102882308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486c099d50 con 0x7f486c0d06d0 2024-04-23T14:33:18.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4871000640 1 -- 172.21.15.100:0/1102882308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f486c14b100 con 0x7f486c0d06d0 2024-04-23T14:33:18.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4871000640 1 -- 172.21.15.100:0/1102882308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f486c068ad0 con 0x7f486c0d06d0 2024-04-23T14:33:18.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.842+0000 7f4862000640 1 -- 172.21.15.100:0/1102882308 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f485801a070 con 0x7f486c0d06d0 2024-04-23T14:33:18.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.846+0000 7f4862000640 1 --2- 172.21.15.100:0/1102882308 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f484c068380 0x7f484c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:18.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.846+0000 7f4862000640 1 -- 172.21.15.100:0/1102882308 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f4858086e10 con 0x7f486c0d06d0 2024-04-23T14:33:18.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.846+0000 7f4869e00640 1 --2- 172.21.15.100:0/1102882308 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f484c068380 0x7f484c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:18.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.846+0000 7f4869e00640 1 --2- 172.21.15.100:0/1102882308 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f484c068380 0x7f484c06a840 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f485c0af000 tx=0x7f485c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:18.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:18.850+0000 7f4862000640 1 -- 172.21.15.100:0/1102882308 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f485808a030 con 0x7f486c0d06d0 2024-04-23T14:33:19.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.022+0000 7f4871000640 1 -- 172.21.15.100:0/1102882308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f486c0aedb0 con 0x7f486c0d06d0 2024-04-23T14:33:19.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.022+0000 7f4862000640 1 -- 172.21.15.100:0/1102882308 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f48580569a0 con 0x7f486c0d06d0 2024-04-23T14:33:19.023 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-04-23T14:33:19.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.022+0000 7f4871000640 1 -- 172.21.15.100:0/1102882308 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f484c068380 msgr2=0x7f484c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:19.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.022+0000 7f4871000640 1 --2- 172.21.15.100:0/1102882308 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f484c068380 0x7f484c06a840 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f485c0af000 tx=0x7f485c0a7070 comp rx=0 tx=0).stop 2024-04-23T14:33:19.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.022+0000 7f4871000640 1 -- 172.21.15.100:0/1102882308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f486c0d06d0 msgr2=0x7f486c14a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:19.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.022+0000 7f4871000640 1 --2- 172.21.15.100:0/1102882308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f486c0d06d0 0x7f486c14a550 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f485800e930 tx=0x7f485800edf0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.022+0000 7f4871000640 1 -- 172.21.15.100:0/1102882308 shutdown_connections 2024-04-23T14:33:19.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.022+0000 7f4871000640 1 --2- 172.21.15.100:0/1102882308 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f484c068380 0x7f484c06a840 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.022+0000 7f4871000640 1 --2- 172.21.15.100:0/1102882308 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f486c15b660 0x7f486c151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.022+0000 7f4871000640 1 --2- 172.21.15.100:0/1102882308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f486c0d06d0 0x7f486c14a550 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.022+0000 7f4871000640 1 --2- 172.21.15.100:0/1102882308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f486c0c6d60 0x7f486c0ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.022+0000 7f4871000640 1 -- 172.21.15.100:0/1102882308 >> 172.21.15.100:0/1102882308 conn(0x7f486c0b5310 msgr2=0x7f486c072aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:19.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.022+0000 7f4871000640 1 -- 172.21.15.100:0/1102882308 shutdown_connections 2024-04-23T14:33:19.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.022+0000 7f4871000640 1 -- 172.21.15.100:0/1102882308 wait complete. 2024-04-23T14:33:19.045 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi100.front.sepia.ceph.com... 2024-04-23T14:33:19.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:19.046 DEBUG:teuthology.orchestra.run.smithi100:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-23T14:33:19.065 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:33:19.065 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:33:19.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:19.066 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-23T14:33:19.118 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:33:19.118 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:19.127 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:33:19.127 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:19.179 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:19.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.310+0000 7fe656800640 1 Processor -- start 2024-04-23T14:33:19.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.310+0000 7fe656800640 1 -- start start 2024-04-23T14:33:19.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.314+0000 7fe656800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fe65006d350 0x7fe650073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:19.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.314+0000 7fe656800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe65015fa90 0x7fe65006d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:19.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.314+0000 7fe656800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe650073880 con 0x7fe650162390 2024-04-23T14:33:19.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.314+0000 7fe656800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fe65006eaf0 con 0x7fe65015fa90 2024-04-23T14:33:19.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.314+0000 7fe656800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fe65006e530 con 0x7fe65006d350 2024-04-23T14:33:19.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.314+0000 7fe64ea00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe65015fa90 0x7fe65006d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:19.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.314+0000 7fe64fe00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe650162390 0x7fe650162840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57968/0 (socket says 172.21.15.100:57968) 2024-04-23T14:33:19.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.314+0000 7fe64fe00640 1 -- 172.21.15.100:0/3499202034 learned_addr learned my addr 172.21.15.100:0/3499202034 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:19.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.314+0000 7fe64f400640 1 --2- 172.21.15.100:0/3499202034 >> v2:172.21.15.151:3300/0 conn(0x7fe65006d350 0x7fe650073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:19.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.314+0000 7fe64ea00640 1 -- 172.21.15.100:0/3499202034 >> v2:172.21.15.151:3300/0 conn(0x7fe65006d350 msgr2=0x7fe650073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:19.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.314+0000 7fe64ea00640 1 --2- 172.21.15.100:0/3499202034 >> v2:172.21.15.151:3300/0 conn(0x7fe65006d350 0x7fe650073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.314+0000 7fe64ea00640 1 -- 172.21.15.100:0/3499202034 >> v1:172.21.15.100:6789/0 conn(0x7fe650162390 legacy=0x7fe650162840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:19.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.314+0000 7fe64ea00640 1 -- 172.21.15.100:0/3499202034 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65006dc70 con 0x7fe65015fa90 2024-04-23T14:33:19.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.314+0000 7fe64f400640 1 --2- 172.21.15.100:0/3499202034 >> v2:172.21.15.151:3300/0 conn(0x7fe65006d350 0x7fe650073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:19.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.314+0000 7fe64e000640 1 -- 172.21.15.100:0/3499202034 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3559039743 0 0) 0x7fe650073880 con 0x7fe650162390 2024-04-23T14:33:19.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe64ea00640 1 --2- 172.21.15.100:0/3499202034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe65015fa90 0x7fe65006d730 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe6500b5960 tx=0x7fe63c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6edbc5f4820bf0f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:19.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe64e000640 1 -- 172.21.15.100:0/3499202034 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c0b1070 con 0x7fe65015fa90 2024-04-23T14:33:19.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe64e000640 1 -- 172.21.15.100:0/3499202034 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe63c0ac620 con 0x7fe65015fa90 2024-04-23T14:33:19.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe64e000640 1 -- 172.21.15.100:0/3499202034 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c0ac9f0 con 0x7fe65015fa90 2024-04-23T14:33:19.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe656800640 1 -- 172.21.15.100:0/3499202034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe65015fa90 msgr2=0x7fe65006d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:19.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe656800640 1 --2- 172.21.15.100:0/3499202034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe65015fa90 0x7fe65006d730 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe6500b5960 tx=0x7fe63c01c3c0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe656800640 1 -- 172.21.15.100:0/3499202034 shutdown_connections 2024-04-23T14:33:19.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe656800640 1 --2- 172.21.15.100:0/3499202034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe65015fa90 0x7fe65006d730 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe656800640 1 --2- 172.21.15.100:0/3499202034 >> v2:172.21.15.151:3300/0 conn(0x7fe65006d350 0x7fe650073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe656800640 1 -- 172.21.15.100:0/3499202034 >> 172.21.15.100:0/3499202034 conn(0x7fe650072600 msgr2=0x7fe650072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:19.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe656800640 1 -- 172.21.15.100:0/3499202034 shutdown_connections 2024-04-23T14:33:19.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe656800640 1 -- 172.21.15.100:0/3499202034 wait complete. 2024-04-23T14:33:19.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe656800640 1 Processor -- start 2024-04-23T14:33:19.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe656800640 1 -- start start 2024-04-23T14:33:19.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe656800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe65006d350 0x7fe650147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:19.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe656800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe65015fa90 0x7fe6501477b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:19.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe656800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe650162390 0x7fe6501519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:19.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe656800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe65016c110 con 0x7fe65006d350 2024-04-23T14:33:19.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe656800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe65016bf90 con 0x7fe65015fa90 2024-04-23T14:33:19.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe656800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe65016c290 con 0x7fe650162390 2024-04-23T14:33:19.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe64f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe65006d350 0x7fe650147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:19.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe64f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe65006d350 0x7fe650147f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47996/0 (socket says 172.21.15.100:47996) 2024-04-23T14:33:19.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.318+0000 7fe64f400640 1 -- 172.21.15.100:0/4277799841 learned_addr learned my addr 172.21.15.100:0/4277799841 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:19.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.322+0000 7fe64ea00640 1 --2- 172.21.15.100:0/4277799841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe65015fa90 0x7fe6501477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:19.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.322+0000 7fe64fe00640 1 --2- 172.21.15.100:0/4277799841 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe650162390 0x7fe6501519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:19.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.322+0000 7fe64f400640 1 -- 172.21.15.100:0/4277799841 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe650162390 msgr2=0x7fe6501519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:19.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.322+0000 7fe64f400640 1 --2- 172.21.15.100:0/4277799841 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe650162390 0x7fe6501519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.322+0000 7fe64f400640 1 -- 172.21.15.100:0/4277799841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe65015fa90 msgr2=0x7fe6501477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:19.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.322+0000 7fe64f400640 1 --2- 172.21.15.100:0/4277799841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe65015fa90 0x7fe6501477b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.322+0000 7fe64f400640 1 -- 172.21.15.100:0/4277799841 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6501485e0 con 0x7fe65006d350 2024-04-23T14:33:19.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.322+0000 7fe64f400640 1 --2- 172.21.15.100:0/4277799841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe65006d350 0x7fe650147f50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe64000ce30 tx=0x7fe64000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:19.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.322+0000 7fe647400640 1 -- 172.21.15.100:0/4277799841 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64000f890 con 0x7fe65006d350 2024-04-23T14:33:19.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.322+0000 7fe647400640 1 -- 172.21.15.100:0/4277799841 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe64000fa30 con 0x7fe65006d350 2024-04-23T14:33:19.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.322+0000 7fe647400640 1 -- 172.21.15.100:0/4277799841 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64000fe00 con 0x7fe65006d350 2024-04-23T14:33:19.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.322+0000 7fe656800640 1 -- 172.21.15.100:0/4277799841 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6501488c0 con 0x7fe65006d350 2024-04-23T14:33:19.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.322+0000 7fe656800640 1 -- 172.21.15.100:0/4277799841 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe650148e80 con 0x7fe65006d350 2024-04-23T14:33:19.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.322+0000 7fe656800640 1 -- 172.21.15.100:0/4277799841 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe650149260 con 0x7fe65006d350 2024-04-23T14:33:19.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.322+0000 7fe647400640 1 -- 172.21.15.100:0/4277799841 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe640018400 con 0x7fe65006d350 2024-04-23T14:33:19.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.322+0000 7fe647400640 1 --2- 172.21.15.100:0/4277799841 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe630068380 0x7fe63006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:19.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.322+0000 7fe647400640 1 -- 172.21.15.100:0/4277799841 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe640087b40 con 0x7fe65006d350 2024-04-23T14:33:19.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.326+0000 7fe64ea00640 1 --2- 172.21.15.100:0/4277799841 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe630068380 0x7fe63006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:19.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.326+0000 7fe64ea00640 1 --2- 172.21.15.100:0/4277799841 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe630068380 0x7fe63006a840 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe63c0ae000 tx=0x7fe63c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:19.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.330+0000 7fe647400640 1 -- 172.21.15.100:0/4277799841 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe650149260 con 0x7fe65006d350 2024-04-23T14:33:19.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.486+0000 7fe656800640 1 -- 172.21.15.100:0/4277799841 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe650068ad0 con 0x7fe65006d350 2024-04-23T14:33:19.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.486+0000 7fe647400640 1 -- 172.21.15.100:0/4277799841 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18859 (secure 0 0 0) 0x7fe640014030 con 0x7fe65006d350 2024-04-23T14:33:19.489 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:19.489 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:49.784608+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:19.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.490+0000 7fe656800640 1 -- 172.21.15.100:0/4277799841 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe630068380 msgr2=0x7fe63006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:19.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.490+0000 7fe656800640 1 --2- 172.21.15.100:0/4277799841 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe630068380 0x7fe63006a840 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe63c0ae000 tx=0x7fe63c008040 comp rx=0 tx=0).stop 2024-04-23T14:33:19.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.490+0000 7fe656800640 1 -- 172.21.15.100:0/4277799841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe65006d350 msgr2=0x7fe650147f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:19.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.490+0000 7fe656800640 1 --2- 172.21.15.100:0/4277799841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe65006d350 0x7fe650147f50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe64000ce30 tx=0x7fe64000a650 comp rx=0 tx=0).stop 2024-04-23T14:33:19.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.490+0000 7fe656800640 1 -- 172.21.15.100:0/4277799841 shutdown_connections 2024-04-23T14:33:19.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.490+0000 7fe656800640 1 --2- 172.21.15.100:0/4277799841 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe630068380 0x7fe63006a840 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.490+0000 7fe656800640 1 --2- 172.21.15.100:0/4277799841 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe650162390 0x7fe6501519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.490+0000 7fe656800640 1 --2- 172.21.15.100:0/4277799841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe65015fa90 0x7fe6501477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.490+0000 7fe656800640 1 --2- 172.21.15.100:0/4277799841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe65006d350 0x7fe650147f50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.490+0000 7fe656800640 1 -- 172.21.15.100:0/4277799841 >> 172.21.15.100:0/4277799841 conn(0x7fe650072600 msgr2=0x7fe650160880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:19.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.490+0000 7fe656800640 1 -- 172.21.15.100:0/4277799841 shutdown_connections 2024-04-23T14:33:19.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.490+0000 7fe656800640 1 -- 172.21.15.100:0/4277799841 wait complete. 2024-04-23T14:33:19.509 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:32:06.835560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:32:07.930489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-23T14:32:11.636814+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-23T14:33:19.510 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:33:19.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.646+0000 7fd02ec00640 1 Processor -- start 2024-04-23T14:33:19.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.646+0000 7fd02ec00640 1 -- start start 2024-04-23T14:33:19.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.646+0000 7fd02ec00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd02806f8c0 0x7fd02806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:19.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.646+0000 7fd02ec00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0280d0170 0x7fd0280ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:19.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.646+0000 7fd02ec00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd02806dd60 con 0x7fd0280ce620 2024-04-23T14:33:19.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.646+0000 7fd02ec00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fd0280ad050 con 0x7fd02806f8c0 2024-04-23T14:33:19.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.646+0000 7fd02ec00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd0280ad8f0 con 0x7fd0280d0170 2024-04-23T14:33:19.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.646+0000 7fd027e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd02806f8c0 0x7fd02806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:19.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.646+0000 7fd01f400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0280d0170 0x7fd0280ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:19.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.646+0000 7fd02cc00640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7fd0280ce620 0x7fd02815f0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:58932/0 (socket says 172.21.15.100:58932) 2024-04-23T14:33:19.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.646+0000 7fd02cc00640 1 -- 172.21.15.100:0/4075828620 learned_addr learned my addr 172.21.15.100:0/4075828620 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:19.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.646+0000 7fd027e00640 1 -- 172.21.15.100:0/4075828620 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0280d0170 msgr2=0x7fd0280ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:19.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.646+0000 7fd027e00640 1 --2- 172.21.15.100:0/4075828620 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0280d0170 0x7fd0280ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.646+0000 7fd027e00640 1 -- 172.21.15.100:0/4075828620 >> v1:172.21.15.151:6789/0 conn(0x7fd0280ce620 legacy=0x7fd02815f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:19.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.646+0000 7fd027e00640 1 -- 172.21.15.100:0/4075828620 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0280d09c0 con 0x7fd02806f8c0 2024-04-23T14:33:19.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.650+0000 7fd027e00640 1 --2- 172.21.15.100:0/4075828620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd02806f8c0 0x7fd02806c6c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd0180097c0 tx=0x7fd01801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ddf491c24f57a81 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:19.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.650+0000 7fd027400640 1 -- 172.21.15.100:0/4075828620 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0180b0070 con 0x7fd02806f8c0 2024-04-23T14:33:19.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.650+0000 7fd027400640 1 -- 172.21.15.100:0/4075828620 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0180ab600 con 0x7fd02806f8c0 2024-04-23T14:33:19.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.650+0000 7fd027400640 1 -- 172.21.15.100:0/4075828620 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0180ab9d0 con 0x7fd02806f8c0 2024-04-23T14:33:19.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.650+0000 7fd02ec00640 1 -- 172.21.15.100:0/4075828620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd02806f8c0 msgr2=0x7fd02806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:19.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.650+0000 7fd02ec00640 1 --2- 172.21.15.100:0/4075828620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd02806f8c0 0x7fd02806c6c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd0180097c0 tx=0x7fd01801c4a0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.650+0000 7fd02ec00640 1 -- 172.21.15.100:0/4075828620 shutdown_connections 2024-04-23T14:33:19.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.650+0000 7fd02ec00640 1 --2- 172.21.15.100:0/4075828620 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0280d0170 0x7fd0280ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.650+0000 7fd02ec00640 1 --2- 172.21.15.100:0/4075828620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd02806f8c0 0x7fd02806c6c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.650+0000 7fd02ec00640 1 -- 172.21.15.100:0/4075828620 >> 172.21.15.100:0/4075828620 conn(0x7fd0280b5200 msgr2=0x7fd0280b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:19.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.650+0000 7fd02ec00640 1 -- 172.21.15.100:0/4075828620 shutdown_connections 2024-04-23T14:33:19.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.650+0000 7fd02ec00640 1 -- 172.21.15.100:0/4075828620 wait complete. 2024-04-23T14:33:19.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd02ec00640 1 Processor -- start 2024-04-23T14:33:19.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd02ec00640 1 -- start start 2024-04-23T14:33:19.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd02ec00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd02806f8c0 0x7fd028167400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:19.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd02ec00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0280ce620 0x7fd028098280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:19.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd02ec00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0280d0170 0x7fd0281554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:19.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd02ec00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd0280d06d0 con 0x7fd02806f8c0 2024-04-23T14:33:19.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd02ec00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd0280d0550 con 0x7fd0280d0170 2024-04-23T14:33:19.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd02ec00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd0281663f0 con 0x7fd0280ce620 2024-04-23T14:33:19.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd027e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd02806f8c0 0x7fd028167400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:19.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd027e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd02806f8c0 0x7fd028167400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48002/0 (socket says 172.21.15.100:48002) 2024-04-23T14:33:19.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd027e00640 1 -- 172.21.15.100:0/1249824904 learned_addr learned my addr 172.21.15.100:0/1249824904 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:19.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd02cc00640 1 --2- 172.21.15.100:0/1249824904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0280d0170 0x7fd0281554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:19.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd01f400640 1 --2- 172.21.15.100:0/1249824904 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0280ce620 0x7fd028098280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:19.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd027e00640 1 -- 172.21.15.100:0/1249824904 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0280ce620 msgr2=0x7fd028098280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:19.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd027e00640 1 --2- 172.21.15.100:0/1249824904 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0280ce620 0x7fd028098280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd027e00640 1 -- 172.21.15.100:0/1249824904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0280d0170 msgr2=0x7fd0281554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:19.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd027e00640 1 --2- 172.21.15.100:0/1249824904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0280d0170 0x7fd0281554f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd027e00640 1 -- 172.21.15.100:0/1249824904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0280b2ba0 con 0x7fd02806f8c0 2024-04-23T14:33:19.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd02cc00640 1 --2- 172.21.15.100:0/1249824904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0280d0170 0x7fd0281554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:19.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd01f400640 1 --2- 172.21.15.100:0/1249824904 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0280ce620 0x7fd028098280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:19.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd027e00640 1 --2- 172.21.15.100:0/1249824904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd02806f8c0 0x7fd028167400 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd0180ae040 tx=0x7fd01801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:19.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd024c00640 1 -- 172.21.15.100:0/1249824904 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0180b0050 con 0x7fd02806f8c0 2024-04-23T14:33:19.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd024c00640 1 -- 172.21.15.100:0/1249824904 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0180bc070 con 0x7fd02806f8c0 2024-04-23T14:33:19.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd02ec00640 1 -- 172.21.15.100:0/1249824904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0280987c0 con 0x7fd02806f8c0 2024-04-23T14:33:19.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd024c00640 1 -- 172.21.15.100:0/1249824904 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0180b5870 con 0x7fd02806f8c0 2024-04-23T14:33:19.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd02ec00640 1 -- 172.21.15.100:0/1249824904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd028167b00 con 0x7fd02806f8c0 2024-04-23T14:33:19.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.654+0000 7fd02ec00640 1 -- 172.21.15.100:0/1249824904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd028068ad0 con 0x7fd02806f8c0 2024-04-23T14:33:19.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.658+0000 7fd024c00640 1 -- 172.21.15.100:0/1249824904 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd0180b5a10 con 0x7fd02806f8c0 2024-04-23T14:33:19.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.658+0000 7fd024c00640 1 --2- 172.21.15.100:0/1249824904 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd0040683b0 0x7fd00406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:19.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.658+0000 7fd024c00640 1 -- 172.21.15.100:0/1249824904 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd018121570 con 0x7fd02806f8c0 2024-04-23T14:33:19.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.658+0000 7fd01f400640 1 --2- 172.21.15.100:0/1249824904 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd0040683b0 0x7fd00406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:19.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.658+0000 7fd01f400640 1 --2- 172.21.15.100:0/1249824904 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd0040683b0 0x7fd00406a870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd02815e900 tx=0x7fd01000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:19.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.662+0000 7fd024c00640 1 -- 172.21.15.100:0/1249824904 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd018016090 con 0x7fd02806f8c0 2024-04-23T14:33:19.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.822+0000 7fd02ec00640 1 -- 172.21.15.100:0/1249824904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd02806cc00 con 0x7fd02806f8c0 2024-04-23T14:33:19.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.822+0000 7fd024c00640 1 -- 172.21.15.100:0/1249824904 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fd0180f1690 con 0x7fd02806f8c0 2024-04-23T14:33:19.824 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:33:19.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.822+0000 7fd02ec00640 1 -- 172.21.15.100:0/1249824904 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd0040683b0 msgr2=0x7fd00406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:19.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.822+0000 7fd02ec00640 1 --2- 172.21.15.100:0/1249824904 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd0040683b0 0x7fd00406a870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd02815e900 tx=0x7fd01000b040 comp rx=0 tx=0).stop 2024-04-23T14:33:19.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.822+0000 7fd02ec00640 1 -- 172.21.15.100:0/1249824904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd02806f8c0 msgr2=0x7fd028167400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:19.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.822+0000 7fd02ec00640 1 --2- 172.21.15.100:0/1249824904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd02806f8c0 0x7fd028167400 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd0180ae040 tx=0x7fd01801ace0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.822+0000 7fd02ec00640 1 -- 172.21.15.100:0/1249824904 shutdown_connections 2024-04-23T14:33:19.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.822+0000 7fd02ec00640 1 --2- 172.21.15.100:0/1249824904 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd0040683b0 0x7fd00406a870 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.822+0000 7fd02ec00640 1 --2- 172.21.15.100:0/1249824904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0280d0170 0x7fd0281554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.822+0000 7fd02ec00640 1 --2- 172.21.15.100:0/1249824904 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0280ce620 0x7fd028098280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.822+0000 7fd02ec00640 1 --2- 172.21.15.100:0/1249824904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd02806f8c0 0x7fd028167400 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.822+0000 7fd02ec00640 1 -- 172.21.15.100:0/1249824904 >> 172.21.15.100:0/1249824904 conn(0x7fd0280b5200 msgr2=0x7fd0281529c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:19.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.822+0000 7fd02ec00640 1 -- 172.21.15.100:0/1249824904 shutdown_connections 2024-04-23T14:33:19.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.822+0000 7fd02ec00640 1 -- 172.21.15.100:0/1249824904 wait complete. 2024-04-23T14:33:19.845 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:33:19.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.981+0000 7ff54b000640 1 Processor -- start 2024-04-23T14:33:19.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.981+0000 7ff54b000640 1 -- start start 2024-04-23T14:33:19.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.981+0000 7ff54b000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff54406c6c0 0x7ff5440ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:19.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.981+0000 7ff54b000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff54406dd60 con 0x7ff54406f8c0 2024-04-23T14:33:19.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.981+0000 7ff54b000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5440d09c0 con 0x7ff5440ce620 2024-04-23T14:33:19.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.981+0000 7ff54b000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7ff5440ad050 con 0x7ff54406c6c0 2024-04-23T14:33:19.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.981+0000 7ff543e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff54406f8c0 0x7ff5440d0170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57978/0 (socket says 172.21.15.100:57978) 2024-04-23T14:33:19.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.981+0000 7ff543e00640 1 -- 172.21.15.100:0/3054457311 learned_addr learned my addr 172.21.15.100:0/3054457311 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:19.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.981+0000 7ff53b400640 1 --2- 172.21.15.100:0/3054457311 >> v2:172.21.15.151:3300/0 conn(0x7ff54406c6c0 0x7ff5440ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:19.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.981+0000 7ff543400640 1 -- 172.21.15.100:0/3054457311 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 493463174 0 0) 0x7ff5340014a0 con 0x7ff54406f8c0 2024-04-23T14:33:19.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.981+0000 7ff543400640 1 -- 172.21.15.100:0/3054457311 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff528003530 con 0x7ff54406f8c0 2024-04-23T14:33:19.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff543400640 1 -- 172.21.15.100:0/3054457311 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1683892521 0 0) 0x7ff528003530 con 0x7ff54406f8c0 2024-04-23T14:33:19.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff543400640 1 -- 172.21.15.100:0/3054457311 >> v2:172.21.15.151:3300/0 conn(0x7ff54406c6c0 msgr2=0x7ff5440ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:19.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff543400640 1 --2- 172.21.15.100:0/3054457311 >> v2:172.21.15.151:3300/0 conn(0x7ff54406c6c0 0x7ff5440ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff543400640 1 -- 172.21.15.100:0/3054457311 >> v1:172.21.15.151:6789/0 conn(0x7ff5440ce620 legacy=0x7ff5440add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:19.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff543400640 1 -- 172.21.15.100:0/3054457311 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5440ad2a0 con 0x7ff54406f8c0 2024-04-23T14:33:19.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff543400640 1 -- 172.21.15.100:0/3054457311 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7ff534001bd0 con 0x7ff54406f8c0 2024-04-23T14:33:19.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff543400640 1 -- 172.21.15.100:0/3054457311 >> v1:172.21.15.100:6789/0 conn(0x7ff54406f8c0 legacy=0x7ff5440d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:19.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff543400640 1 --2- 172.21.15.100:0/3054457311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5280039c0 0x7ff528003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:19.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff543400640 1 --2- 172.21.15.100:0/3054457311 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff528004810 0x7ff528004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:19.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff543400640 1 --2- 172.21.15.100:0/3054457311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5280051d0 0x7ff52800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:19.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff543400640 1 -- 172.21.15.100:0/3054457311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff5440ad050 con 0x7ff5280051d0 2024-04-23T14:33:19.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff543400640 1 -- 172.21.15.100:0/3054457311 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff52800e280 con 0x7ff5280039c0 2024-04-23T14:33:19.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff543400640 1 -- 172.21.15.100:0/3054457311 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff52800e480 con 0x7ff528004810 2024-04-23T14:33:19.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff549000640 1 --2- 172.21.15.100:0/3054457311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5280051d0 0x7ff52800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:19.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff53b400640 1 --2- 172.21.15.100:0/3054457311 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff528004810 0x7ff528004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:19.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff543e00640 1 --2- 172.21.15.100:0/3054457311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5280039c0 0x7ff528003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:19.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff549000640 1 -- 172.21.15.100:0/3054457311 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff528004810 msgr2=0x7ff528004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:19.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff549000640 1 --2- 172.21.15.100:0/3054457311 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff528004810 0x7ff528004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff549000640 1 -- 172.21.15.100:0/3054457311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5280039c0 msgr2=0x7ff528003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:19.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff549000640 1 --2- 172.21.15.100:0/3054457311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5280039c0 0x7ff528003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff549000640 1 -- 172.21.15.100:0/3054457311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff528003530 con 0x7ff5280051d0 2024-04-23T14:33:19.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff549000640 1 -- 172.21.15.100:0/3054457311 reap_dead start 2024-04-23T14:33:19.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.985+0000 7ff549000640 1 --2- 172.21.15.100:0/3054457311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5280051d0 0x7ff52800dbe0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff53c001780 tx=0x7ff53c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4868cc5f0ee10155 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:19.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.989+0000 7ff543400640 1 -- 172.21.15.100:0/3054457311 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0013a0 con 0x7ff5280051d0 2024-04-23T14:33:19.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.989+0000 7ff543400640 1 -- 172.21.15.100:0/3054457311 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff53c0a9070 con 0x7ff5280051d0 2024-04-23T14:33:19.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.989+0000 7ff543400640 1 -- 172.21.15.100:0/3054457311 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c003600 con 0x7ff5280051d0 2024-04-23T14:33:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.989+0000 7ff54b000640 1 -- 172.21.15.100:0/3054457311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5280051d0 msgr2=0x7ff52800dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.989+0000 7ff54b000640 1 --2- 172.21.15.100:0/3054457311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5280051d0 0x7ff52800dbe0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff53c001780 tx=0x7ff53c01c420 comp rx=0 tx=0).stop 2024-04-23T14:33:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.989+0000 7ff54b000640 1 -- 172.21.15.100:0/3054457311 shutdown_connections 2024-04-23T14:33:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.989+0000 7ff54b000640 1 --2- 172.21.15.100:0/3054457311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5280051d0 0x7ff52800dbe0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.989+0000 7ff54b000640 1 -- 172.21.15.100:0/3054457311 >> 172.21.15.100:0/3054457311 conn(0x7ff5440b5200 msgr2=0x7ff5440b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.989+0000 7ff54b000640 1 -- 172.21.15.100:0/3054457311 shutdown_connections 2024-04-23T14:33:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.989+0000 7ff54b000640 1 -- 172.21.15.100:0/3054457311 wait complete. 2024-04-23T14:33:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.989+0000 7ff54b000640 1 Processor -- start 2024-04-23T14:33:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.989+0000 7ff54b000640 1 -- start start 2024-04-23T14:33:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.989+0000 7ff54b000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff5280051d0 0x7ff54409dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.989+0000 7ff54b000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff54409a160 0x7ff5440c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.989+0000 7ff54b000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff54409a5e0 0x7ff5440ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.989+0000 7ff54b000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff5440ce0e0 con 0x7ff54409a5e0 2024-04-23T14:33:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.989+0000 7ff54b000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff5440ad8f0 con 0x7ff54409a160 2024-04-23T14:33:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.989+0000 7ff54b000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff5440ce260 con 0x7ff5280051d0 2024-04-23T14:33:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.989+0000 7ff549000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff54409a5e0 0x7ff5440ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.989+0000 7ff549000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff54409a5e0 0x7ff5440ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48022/0 (socket says 172.21.15.100:48022) 2024-04-23T14:33:19.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.989+0000 7ff549000640 1 -- 172.21.15.100:0/1667674143 learned_addr learned my addr 172.21.15.100:0/1667674143 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.993+0000 7ff543e00640 1 --2- 172.21.15.100:0/1667674143 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff5280051d0 0x7ff54409dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.993+0000 7ff53b400640 1 --2- 172.21.15.100:0/1667674143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff54409a160 0x7ff5440c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.993+0000 7ff549000640 1 -- 172.21.15.100:0/1667674143 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff5280051d0 msgr2=0x7ff54409dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.993+0000 7ff549000640 1 --2- 172.21.15.100:0/1667674143 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff5280051d0 0x7ff54409dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.993+0000 7ff549000640 1 -- 172.21.15.100:0/1667674143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff54409a160 msgr2=0x7ff5440c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.993+0000 7ff549000640 1 --2- 172.21.15.100:0/1667674143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff54409a160 0x7ff5440c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.993+0000 7ff549000640 1 -- 172.21.15.100:0/1667674143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54409cc80 con 0x7ff54409a5e0 2024-04-23T14:33:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.993+0000 7ff549000640 1 --2- 172.21.15.100:0/1667674143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff54409a5e0 0x7ff5440ca010 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff53c01ccc0 tx=0x7ff528005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.993+0000 7ff540c00640 1 -- 172.21.15.100:0/1667674143 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0013a0 con 0x7ff54409a5e0 2024-04-23T14:33:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.993+0000 7ff540c00640 1 -- 172.21.15.100:0/1667674143 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff53c004b50 con 0x7ff54409a5e0 2024-04-23T14:33:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.993+0000 7ff54b000640 1 -- 172.21.15.100:0/1667674143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff54409cf60 con 0x7ff54409a5e0 2024-04-23T14:33:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.993+0000 7ff540c00640 1 -- 172.21.15.100:0/1667674143 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c007a40 con 0x7ff54409a5e0 2024-04-23T14:33:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.993+0000 7ff54b000640 1 -- 172.21.15.100:0/1667674143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff54409d520 con 0x7ff54409a5e0 2024-04-23T14:33:19.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.993+0000 7ff54b000640 1 -- 172.21.15.100:0/1667674143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff544068ad0 con 0x7ff54409a5e0 2024-04-23T14:33:19.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.993+0000 7ff540c00640 1 -- 172.21.15.100:0/1667674143 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff53c0ab070 con 0x7ff54409a5e0 2024-04-23T14:33:19.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.993+0000 7ff540c00640 1 --2- 172.21.15.100:0/1667674143 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff520068380 0x7ff52006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:19.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.993+0000 7ff540c00640 1 -- 172.21.15.100:0/1667674143 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff53c120250 con 0x7ff54409a5e0 2024-04-23T14:33:19.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.993+0000 7ff543e00640 1 --2- 172.21.15.100:0/1667674143 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff520068380 0x7ff52006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:19.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.997+0000 7ff543e00640 1 --2- 172.21.15.100:0/1667674143 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff520068380 0x7ff52006a840 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff534001640 tx=0x7ff534006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:20.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:19.997+0000 7ff540c00640 1 -- 172.21.15.100:0/1667674143 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff53c016130 con 0x7ff54409a5e0 2024-04-23T14:33:20.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.153+0000 7ff54b000640 1 -- 172.21.15.100:0/1667674143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff54406cc00 con 0x7ff54409a5e0 2024-04-23T14:33:20.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.153+0000 7ff540c00640 1 -- 172.21.15.100:0/1667674143 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7ff53c0f6070 con 0x7ff54409a5e0 2024-04-23T14:33:20.157 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:20.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.157+0000 7ff54b000640 1 -- 172.21.15.100:0/1667674143 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff520068380 msgr2=0x7ff52006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:20.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.157+0000 7ff54b000640 1 --2- 172.21.15.100:0/1667674143 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff520068380 0x7ff52006a840 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff534001640 tx=0x7ff534006040 comp rx=0 tx=0).stop 2024-04-23T14:33:20.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.157+0000 7ff54b000640 1 -- 172.21.15.100:0/1667674143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff54409a5e0 msgr2=0x7ff5440ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:20.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.157+0000 7ff54b000640 1 --2- 172.21.15.100:0/1667674143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff54409a5e0 0x7ff5440ca010 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff53c01ccc0 tx=0x7ff528005840 comp rx=0 tx=0).stop 2024-04-23T14:33:20.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.157+0000 7ff54b000640 1 -- 172.21.15.100:0/1667674143 shutdown_connections 2024-04-23T14:33:20.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.157+0000 7ff54b000640 1 --2- 172.21.15.100:0/1667674143 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff520068380 0x7ff52006a840 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.157+0000 7ff54b000640 1 --2- 172.21.15.100:0/1667674143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff54409a5e0 0x7ff5440ca010 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.157+0000 7ff54b000640 1 --2- 172.21.15.100:0/1667674143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff54409a160 0x7ff5440c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.157+0000 7ff54b000640 1 --2- 172.21.15.100:0/1667674143 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff5280051d0 0x7ff54409dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.157+0000 7ff54b000640 1 -- 172.21.15.100:0/1667674143 >> 172.21.15.100:0/1667674143 conn(0x7ff5440b5200 msgr2=0x7ff5440add80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:20.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.157+0000 7ff54b000640 1 -- 172.21.15.100:0/1667674143 shutdown_connections 2024-04-23T14:33:20.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.157+0000 7ff54b000640 1 -- 172.21.15.100:0/1667674143 wait complete. 2024-04-23T14:33:20.178 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:33:20.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.313+0000 7fde05600640 1 Processor -- start 2024-04-23T14:33:20.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.313+0000 7fde05600640 1 -- start start 2024-04-23T14:33:20.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.313+0000 7fde05600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fde0006bb20 0x7fde000ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:20.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.313+0000 7fde05600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde000cd130 0x7fde000ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:20.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.313+0000 7fde05600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde0006cbb0 con 0x7fde0015c7a0 2024-04-23T14:33:20.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.313+0000 7fde05600640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fde0006d450 con 0x7fde0006bb20 2024-04-23T14:33:20.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.313+0000 7fde05600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fde000cd580 con 0x7fde000cd130 2024-04-23T14:33:20.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.313+0000 7fddfe000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fde0006bb20 0x7fde000ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:20.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.313+0000 7fddfe000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fde0006bb20 0x7fde000ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48030/0 (socket says 172.21.15.100:48030) 2024-04-23T14:33:20.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.313+0000 7fddfe000640 1 -- 172.21.15.100:0/1392153060 learned_addr learned my addr 172.21.15.100:0/1392153060 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:20.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.313+0000 7fddff400640 1 --2- 172.21.15.100:0/1392153060 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde000cd130 0x7fde000ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:20.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.313+0000 7fddfe000640 1 -- 172.21.15.100:0/1392153060 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde000cd130 msgr2=0x7fde000ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:20.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.313+0000 7fddfe000640 1 --2- 172.21.15.100:0/1392153060 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde000cd130 0x7fde000ad430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.313+0000 7fddfe000640 1 -- 172.21.15.100:0/1392153060 >> v1:172.21.15.151:6789/0 conn(0x7fde0015c7a0 legacy=0x7fde000af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:20.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.313+0000 7fddfe000640 1 -- 172.21.15.100:0/1392153060 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0006ce90 con 0x7fde0006bb20 2024-04-23T14:33:20.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.317+0000 7fddfe000640 1 --2- 172.21.15.100:0/1392153060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde0006bb20 0x7fde000ccbf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fddec009870 tx=0x7fddec01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dff0cd1ef4af430 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:20.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.317+0000 7fddfd600640 1 -- 172.21.15.100:0/1392153060 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec0b0070 con 0x7fde0006bb20 2024-04-23T14:33:20.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.317+0000 7fddfd600640 1 -- 172.21.15.100:0/1392153060 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddec0ab600 con 0x7fde0006bb20 2024-04-23T14:33:20.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.317+0000 7fddfd600640 1 -- 172.21.15.100:0/1392153060 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec0ab9d0 con 0x7fde0006bb20 2024-04-23T14:33:20.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.317+0000 7fde05600640 1 -- 172.21.15.100:0/1392153060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde0006bb20 msgr2=0x7fde000ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:20.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.317+0000 7fde05600640 1 --2- 172.21.15.100:0/1392153060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde0006bb20 0x7fde000ccbf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fddec009870 tx=0x7fddec01c4d0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.317+0000 7fde05600640 1 -- 172.21.15.100:0/1392153060 shutdown_connections 2024-04-23T14:33:20.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.317+0000 7fde05600640 1 --2- 172.21.15.100:0/1392153060 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde000cd130 0x7fde000ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.317+0000 7fde05600640 1 --2- 172.21.15.100:0/1392153060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde0006bb20 0x7fde000ccbf0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.317+0000 7fde05600640 1 -- 172.21.15.100:0/1392153060 >> 172.21.15.100:0/1392153060 conn(0x7fde000b5200 msgr2=0x7fde000b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:20.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.317+0000 7fde05600640 1 -- 172.21.15.100:0/1392153060 shutdown_connections 2024-04-23T14:33:20.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.317+0000 7fde05600640 1 -- 172.21.15.100:0/1392153060 wait complete. 2024-04-23T14:33:20.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fde05600640 1 Processor -- start 2024-04-23T14:33:20.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fde05600640 1 -- start start 2024-04-23T14:33:20.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fde05600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde0006bb20 0x7fde000c3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:20.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fde05600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde000cd130 0x7fde000735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:20.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fde05600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde0015c7a0 0x7fde00073b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:20.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fde05600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fde000ae510 con 0x7fde000cd130 2024-04-23T14:33:20.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fde05600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde000ae390 con 0x7fde0015c7a0 2024-04-23T14:33:20.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fde05600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fde000ae690 con 0x7fde0006bb20 2024-04-23T14:33:20.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fddfe000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde000cd130 0x7fde000735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:20.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fddfe000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde000cd130 0x7fde000735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48038/0 (socket says 172.21.15.100:48038) 2024-04-23T14:33:20.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fddfe000640 1 -- 172.21.15.100:0/2532496239 learned_addr learned my addr 172.21.15.100:0/2532496239 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:20.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fddff400640 1 --2- 172.21.15.100:0/2532496239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde0015c7a0 0x7fde00073b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:20.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fddfea00640 1 --2- 172.21.15.100:0/2532496239 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde0006bb20 0x7fde000c3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:20.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fddfe000640 1 -- 172.21.15.100:0/2532496239 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde0006bb20 msgr2=0x7fde000c3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:20.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fddfe000640 1 --2- 172.21.15.100:0/2532496239 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde0006bb20 0x7fde000c3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fddfe000640 1 -- 172.21.15.100:0/2532496239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde0015c7a0 msgr2=0x7fde00073b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:20.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fddfe000640 1 --2- 172.21.15.100:0/2532496239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde0015c7a0 0x7fde00073b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fddfe000640 1 -- 172.21.15.100:0/2532496239 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde000c4250 con 0x7fde000cd130 2024-04-23T14:33:20.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fddfe000640 1 --2- 172.21.15.100:0/2532496239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde000cd130 0x7fde000735d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fddec0ae040 tx=0x7fddec019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:20.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fddf6a00640 1 -- 172.21.15.100:0/2532496239 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec0b0040 con 0x7fde000cd130 2024-04-23T14:33:20.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fddf6a00640 1 -- 172.21.15.100:0/2532496239 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddec0b4440 con 0x7fde000cd130 2024-04-23T14:33:20.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fde05600640 1 -- 172.21.15.100:0/2532496239 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde000c58f0 con 0x7fde000cd130 2024-04-23T14:33:20.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fddf6a00640 1 -- 172.21.15.100:0/2532496239 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec0b4830 con 0x7fde000cd130 2024-04-23T14:33:20.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fde05600640 1 -- 172.21.15.100:0/2532496239 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde000c5eb0 con 0x7fde000cd130 2024-04-23T14:33:20.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.321+0000 7fde05600640 1 -- 172.21.15.100:0/2532496239 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde00068ad0 con 0x7fde000cd130 2024-04-23T14:33:20.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.325+0000 7fddf6a00640 1 -- 172.21.15.100:0/2532496239 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fddec0bb020 con 0x7fde000cd130 2024-04-23T14:33:20.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.325+0000 7fddf6a00640 1 --2- 172.21.15.100:0/2532496239 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdde00683b0 0x7fdde006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:20.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.325+0000 7fddf6a00640 1 -- 172.21.15.100:0/2532496239 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fddec120370 con 0x7fde000cd130 2024-04-23T14:33:20.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.325+0000 7fddfea00640 1 --2- 172.21.15.100:0/2532496239 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdde00683b0 0x7fdde006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:20.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.325+0000 7fddfea00640 1 --2- 172.21.15.100:0/2532496239 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdde00683b0 0x7fdde006a870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fddf0001d50 tx=0x7fddf0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:20.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.329+0000 7fddf6a00640 1 -- 172.21.15.100:0/2532496239 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fddec0c2020 con 0x7fde000cd130 2024-04-23T14:33:20.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.485+0000 7fde05600640 1 -- 172.21.15.100:0/2532496239 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fde000ca590 con 0x7fde000cd130 2024-04-23T14:33:20.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.485+0000 7fddf6a00640 1 -- 172.21.15.100:0/2532496239 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fddec016150 con 0x7fde000cd130 2024-04-23T14:33:20.488 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:33:20.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.485+0000 7fde05600640 1 -- 172.21.15.100:0/2532496239 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdde00683b0 msgr2=0x7fdde006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:20.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.485+0000 7fde05600640 1 --2- 172.21.15.100:0/2532496239 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdde00683b0 0x7fdde006a870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fddf0001d50 tx=0x7fddf0009040 comp rx=0 tx=0).stop 2024-04-23T14:33:20.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.485+0000 7fde05600640 1 -- 172.21.15.100:0/2532496239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde000cd130 msgr2=0x7fde000735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:20.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.485+0000 7fde05600640 1 --2- 172.21.15.100:0/2532496239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde000cd130 0x7fde000735d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fddec0ae040 tx=0x7fddec019270 comp rx=0 tx=0).stop 2024-04-23T14:33:20.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.485+0000 7fde05600640 1 -- 172.21.15.100:0/2532496239 shutdown_connections 2024-04-23T14:33:20.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.485+0000 7fde05600640 1 --2- 172.21.15.100:0/2532496239 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdde00683b0 0x7fdde006a870 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.485+0000 7fde05600640 1 --2- 172.21.15.100:0/2532496239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde0015c7a0 0x7fde00073b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.485+0000 7fde05600640 1 --2- 172.21.15.100:0/2532496239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde000cd130 0x7fde000735d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.485+0000 7fde05600640 1 --2- 172.21.15.100:0/2532496239 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde0006bb20 0x7fde000c3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.485+0000 7fde05600640 1 -- 172.21.15.100:0/2532496239 >> 172.21.15.100:0/2532496239 conn(0x7fde000b5200 msgr2=0x7fde000d0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:20.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.485+0000 7fde05600640 1 -- 172.21.15.100:0/2532496239 shutdown_connections 2024-04-23T14:33:20.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.485+0000 7fde05600640 1 -- 172.21.15.100:0/2532496239 wait complete. 2024-04-23T14:33:20.508 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-23T14:33:20.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.645+0000 7fa46ec00640 1 Processor -- start 2024-04-23T14:33:20.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.645+0000 7fa46ec00640 1 -- start start 2024-04-23T14:33:20.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.645+0000 7fa46ec00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4680afd90 0x7fa46815c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:20.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.645+0000 7fa46ec00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa46806de30 0x7fa46806cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:20.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.645+0000 7fa46ec00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa46806e640 con 0x7fa46806d000 2024-04-23T14:33:20.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.645+0000 7fa46ec00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fa46806e920 con 0x7fa46806de30 2024-04-23T14:33:20.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.645+0000 7fa46ec00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa46806ec00 con 0x7fa4680afd90 2024-04-23T14:33:20.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.649+0000 7fa46e200640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fa46806d000 0x7fa46815fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57982/0 (socket says 172.21.15.100:57982) 2024-04-23T14:33:20.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.649+0000 7fa46ce00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa46806de30 0x7fa46806cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:20.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.649+0000 7fa46e200640 1 -- 172.21.15.100:0/2407801870 learned_addr learned my addr 172.21.15.100:0/2407801870 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:20.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.649+0000 7fa46d800640 1 --2- 172.21.15.100:0/2407801870 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4680afd90 0x7fa46815c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:20.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.649+0000 7fa46ce00640 1 -- 172.21.15.100:0/2407801870 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4680afd90 msgr2=0x7fa46815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:20.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.649+0000 7fa46ce00640 1 --2- 172.21.15.100:0/2407801870 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4680afd90 0x7fa46815c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.649+0000 7fa46ce00640 1 -- 172.21.15.100:0/2407801870 >> v1:172.21.15.100:6789/0 conn(0x7fa46806d000 legacy=0x7fa46815fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:20.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.649+0000 7fa46ce00640 1 -- 172.21.15.100:0/2407801870 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4680cd0f0 con 0x7fa46806de30 2024-04-23T14:33:20.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.649+0000 7fa467e00640 1 -- 172.21.15.100:0/2407801870 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1289130104 0 0) 0x7fa46806e640 con 0x7fa46806d000 2024-04-23T14:33:20.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.649+0000 7fa46ce00640 1 --2- 172.21.15.100:0/2407801870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa46806de30 0x7fa46806cac0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa4540097c0 tx=0x7fa45401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3469d00f4c3aa5f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:20.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.649+0000 7fa467e00640 1 -- 172.21.15.100:0/2407801870 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4540b0070 con 0x7fa46806de30 2024-04-23T14:33:20.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.649+0000 7fa467e00640 1 -- 172.21.15.100:0/2407801870 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4540ab600 con 0x7fa46806de30 2024-04-23T14:33:20.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.649+0000 7fa467e00640 1 -- 172.21.15.100:0/2407801870 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4540ab9d0 con 0x7fa46806de30 2024-04-23T14:33:20.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46ec00640 1 -- 172.21.15.100:0/2407801870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa46806de30 msgr2=0x7fa46806cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:20.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46ec00640 1 --2- 172.21.15.100:0/2407801870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa46806de30 0x7fa46806cac0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa4540097c0 tx=0x7fa45401c410 comp rx=0 tx=0).stop 2024-04-23T14:33:20.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46ec00640 1 -- 172.21.15.100:0/2407801870 shutdown_connections 2024-04-23T14:33:20.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46ec00640 1 --2- 172.21.15.100:0/2407801870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa46806de30 0x7fa46806cac0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46ec00640 1 --2- 172.21.15.100:0/2407801870 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4680afd90 0x7fa46815c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46ec00640 1 -- 172.21.15.100:0/2407801870 >> 172.21.15.100:0/2407801870 conn(0x7fa4680b2900 msgr2=0x7fa4680720b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:20.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46ec00640 1 -- 172.21.15.100:0/2407801870 shutdown_connections 2024-04-23T14:33:20.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46ec00640 1 -- 172.21.15.100:0/2407801870 wait complete. 2024-04-23T14:33:20.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46ec00640 1 Processor -- start 2024-04-23T14:33:20.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46ec00640 1 -- start start 2024-04-23T14:33:20.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46ec00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa46806d000 0x7fa46815f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:20.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46ec00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa46806de30 0x7fa46814b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:20.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46ec00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4680afd90 0x7fa46814a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:20.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46ec00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa468169540 con 0x7fa4680afd90 2024-04-23T14:33:20.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46ec00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa4681693c0 con 0x7fa46806d000 2024-04-23T14:33:20.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46ec00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa4681696c0 con 0x7fa46806de30 2024-04-23T14:33:20.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4680afd90 0x7fa46814a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:20.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4680afd90 0x7fa46814a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48068/0 (socket says 172.21.15.100:48068) 2024-04-23T14:33:20.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46e200640 1 -- 172.21.15.100:0/3213360410 learned_addr learned my addr 172.21.15.100:0/3213360410 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:20.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46ce00640 1 --2- 172.21.15.100:0/3213360410 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa46806de30 0x7fa46814b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:20.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46d800640 1 --2- 172.21.15.100:0/3213360410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa46806d000 0x7fa46815f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:20.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46e200640 1 -- 172.21.15.100:0/3213360410 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa46806de30 msgr2=0x7fa46814b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:20.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46e200640 1 --2- 172.21.15.100:0/3213360410 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa46806de30 0x7fa46814b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46e200640 1 -- 172.21.15.100:0/3213360410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa46806d000 msgr2=0x7fa46815f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:20.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46e200640 1 --2- 172.21.15.100:0/3213360410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa46806d000 0x7fa46815f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.653+0000 7fa46e200640 1 -- 172.21.15.100:0/3213360410 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa46814bcc0 con 0x7fa4680afd90 2024-04-23T14:33:20.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.657+0000 7fa46e200640 1 --2- 172.21.15.100:0/3213360410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4680afd90 0x7fa46814a2c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa46000c7a0 tx=0x7fa46000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:20.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.657+0000 7fa465600640 1 -- 172.21.15.100:0/3213360410 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460018070 con 0x7fa4680afd90 2024-04-23T14:33:20.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.657+0000 7fa465600640 1 -- 172.21.15.100:0/3213360410 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa46000a640 con 0x7fa4680afd90 2024-04-23T14:33:20.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.657+0000 7fa465600640 1 -- 172.21.15.100:0/3213360410 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46000aab0 con 0x7fa4680afd90 2024-04-23T14:33:20.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.657+0000 7fa46ec00640 1 -- 172.21.15.100:0/3213360410 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4681fa670 con 0x7fa4680afd90 2024-04-23T14:33:20.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.657+0000 7fa46ec00640 1 -- 172.21.15.100:0/3213360410 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4681fac30 con 0x7fa4680afd90 2024-04-23T14:33:20.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.657+0000 7fa46ec00640 1 -- 172.21.15.100:0/3213360410 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4680691e0 con 0x7fa4680afd90 2024-04-23T14:33:20.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.657+0000 7fa465600640 1 -- 172.21.15.100:0/3213360410 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa46001b070 con 0x7fa4680afd90 2024-04-23T14:33:20.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.657+0000 7fa465600640 1 --2- 172.21.15.100:0/3213360410 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa448068380 0x7fa44806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:20.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.657+0000 7fa465600640 1 -- 172.21.15.100:0/3213360410 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa460086370 con 0x7fa4680afd90 2024-04-23T14:33:20.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.657+0000 7fa46d800640 1 --2- 172.21.15.100:0/3213360410 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa448068380 0x7fa44806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:20.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.661+0000 7fa46d800640 1 --2- 172.21.15.100:0/3213360410 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa448068380 0x7fa44806a840 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa45c00b930 tx=0x7fa45c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:20.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.661+0000 7fa465600640 1 -- 172.21.15.100:0/3213360410 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa460055ed0 con 0x7fa4680afd90 2024-04-23T14:33:20.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.821+0000 7fa46ec00640 1 -- 172.21.15.100:0/3213360410 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa468073180 con 0x7fa4680afd90 2024-04-23T14:33:20.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.821+0000 7fa465600640 1 -- 172.21.15.100:0/3213360410 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fa46005a1d0 con 0x7fa4680afd90 2024-04-23T14:33:20.825 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:20.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.821+0000 7fa46ec00640 1 -- 172.21.15.100:0/3213360410 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa448068380 msgr2=0x7fa44806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:20.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.821+0000 7fa46ec00640 1 --2- 172.21.15.100:0/3213360410 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa448068380 0x7fa44806a840 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa45c00b930 tx=0x7fa45c008040 comp rx=0 tx=0).stop 2024-04-23T14:33:20.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.821+0000 7fa46ec00640 1 -- 172.21.15.100:0/3213360410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4680afd90 msgr2=0x7fa46814a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:20.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.821+0000 7fa46ec00640 1 --2- 172.21.15.100:0/3213360410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4680afd90 0x7fa46814a2c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa46000c7a0 tx=0x7fa46000cc60 comp rx=0 tx=0).stop 2024-04-23T14:33:20.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.825+0000 7fa46ec00640 1 -- 172.21.15.100:0/3213360410 shutdown_connections 2024-04-23T14:33:20.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.825+0000 7fa46ec00640 1 --2- 172.21.15.100:0/3213360410 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa448068380 0x7fa44806a840 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.825+0000 7fa46ec00640 1 --2- 172.21.15.100:0/3213360410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4680afd90 0x7fa46814a2c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.825+0000 7fa46ec00640 1 --2- 172.21.15.100:0/3213360410 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa46806de30 0x7fa46814b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.825+0000 7fa46ec00640 1 --2- 172.21.15.100:0/3213360410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa46806d000 0x7fa46815f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.825+0000 7fa46ec00640 1 -- 172.21.15.100:0/3213360410 >> 172.21.15.100:0/3213360410 conn(0x7fa4680b2900 msgr2=0x7fa4680b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:20.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.825+0000 7fa46ec00640 1 -- 172.21.15.100:0/3213360410 shutdown_connections 2024-04-23T14:33:20.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.825+0000 7fa46ec00640 1 -- 172.21.15.100:0/3213360410 wait complete. 2024-04-23T14:33:20.846 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:33:20.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.981+0000 7f6062200640 1 Processor -- start 2024-04-23T14:33:20.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.981+0000 7f6062200640 1 -- start start 2024-04-23T14:33:20.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.981+0000 7f6062200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f605c06bab0 0x7f605c06bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:20.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.981+0000 7f6062200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f605c0724b0 0x7f605c0728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:20.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.981+0000 7f6062200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f605c15fa70 con 0x7f605c15e680 2024-04-23T14:33:20.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.981+0000 7f6062200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f605c0b3b50 con 0x7f605c06bab0 2024-04-23T14:33:20.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.981+0000 7f6062200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f605c0b46f0 con 0x7f605c0724b0 2024-04-23T14:33:20.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.981+0000 7f605aa00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f605c06bab0 0x7f605c06bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:20.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.981+0000 7f605aa00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f605c06bab0 0x7f605c06bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48072/0 (socket says 172.21.15.100:48072) 2024-04-23T14:33:20.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.981+0000 7f605aa00640 1 -- 172.21.15.100:0/2728405813 learned_addr learned my addr 172.21.15.100:0/2728405813 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:20.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.981+0000 7f605be00640 1 --2- 172.21.15.100:0/2728405813 >> v2:172.21.15.151:3300/0 conn(0x7f605c0724b0 0x7f605c0728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:20.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.981+0000 7f605aa00640 1 -- 172.21.15.100:0/2728405813 >> v2:172.21.15.151:3300/0 conn(0x7f605c0724b0 msgr2=0x7f605c0728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:20.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.981+0000 7f605aa00640 1 --2- 172.21.15.100:0/2728405813 >> v2:172.21.15.151:3300/0 conn(0x7f605c0724b0 0x7f605c0728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.981+0000 7f605aa00640 1 -- 172.21.15.100:0/2728405813 >> v1:172.21.15.151:6789/0 conn(0x7f605c15e680 legacy=0x7f605c0b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:20.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.981+0000 7f605aa00640 1 -- 172.21.15.100:0/2728405813 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605c06c5b0 con 0x7f605c06bab0 2024-04-23T14:33:20.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.985+0000 7f605aa00640 1 --2- 172.21.15.100:0/2728405813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f605c06bab0 0x7f605c06bef0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f605c06d550 tx=0x7f604801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3487085e4b1ec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:20.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.985+0000 7f605a000640 1 -- 172.21.15.100:0/2728405813 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60480b2070 con 0x7f605c06bab0 2024-04-23T14:33:20.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.985+0000 7f605a000640 1 -- 172.21.15.100:0/2728405813 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60480ad680 con 0x7f605c06bab0 2024-04-23T14:33:20.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.985+0000 7f605a000640 1 -- 172.21.15.100:0/2728405813 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60480ada50 con 0x7f605c06bab0 2024-04-23T14:33:20.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.985+0000 7f6062200640 1 -- 172.21.15.100:0/2728405813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f605c06bab0 msgr2=0x7f605c06bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:20.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.985+0000 7f6062200640 1 --2- 172.21.15.100:0/2728405813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f605c06bab0 0x7f605c06bef0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f605c06d550 tx=0x7f604801c490 comp rx=0 tx=0).stop 2024-04-23T14:33:20.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.985+0000 7f6062200640 1 -- 172.21.15.100:0/2728405813 shutdown_connections 2024-04-23T14:33:20.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.985+0000 7f6062200640 1 --2- 172.21.15.100:0/2728405813 >> v2:172.21.15.151:3300/0 conn(0x7f605c0724b0 0x7f605c0728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.985+0000 7f6062200640 1 --2- 172.21.15.100:0/2728405813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f605c06bab0 0x7f605c06bef0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.985+0000 7f6062200640 1 -- 172.21.15.100:0/2728405813 >> 172.21.15.100:0/2728405813 conn(0x7f605c0b2600 msgr2=0x7f605c0b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:20.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.985+0000 7f6062200640 1 -- 172.21.15.100:0/2728405813 shutdown_connections 2024-04-23T14:33:20.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.985+0000 7f6062200640 1 -- 172.21.15.100:0/2728405813 wait complete. 2024-04-23T14:33:20.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.985+0000 7f6062200640 1 Processor -- start 2024-04-23T14:33:20.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.985+0000 7f6062200640 1 -- start start 2024-04-23T14:33:20.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f6062200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f605c06bab0 0x7f605c0cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:20.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f6062200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f605c0724b0 0x7f605c153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:20.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f6062200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f605c15e680 0x7f605c13be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:20.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f6062200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f605c16b340 con 0x7f605c06bab0 2024-04-23T14:33:20.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f6062200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f605c06fed0 con 0x7f605c15e680 2024-04-23T14:33:20.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f6062200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f605c16b4c0 con 0x7f605c0724b0 2024-04-23T14:33:20.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f605b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f605c06bab0 0x7f605c0cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:20.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f605b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f605c06bab0 0x7f605c0cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48074/0 (socket says 172.21.15.100:48074) 2024-04-23T14:33:20.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f605b400640 1 -- 172.21.15.100:0/308916840 learned_addr learned my addr 172.21.15.100:0/308916840 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:20.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f605be00640 1 --2- 172.21.15.100:0/308916840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f605c15e680 0x7f605c13be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:20.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f605aa00640 1 --2- 172.21.15.100:0/308916840 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f605c0724b0 0x7f605c153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:20.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f605b400640 1 -- 172.21.15.100:0/308916840 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f605c0724b0 msgr2=0x7f605c153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:20.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f605b400640 1 --2- 172.21.15.100:0/308916840 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f605c0724b0 0x7f605c153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f605b400640 1 -- 172.21.15.100:0/308916840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f605c15e680 msgr2=0x7f605c13be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:20.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f605b400640 1 --2- 172.21.15.100:0/308916840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f605c15e680 0x7f605c13be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:20.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f605b400640 1 -- 172.21.15.100:0/308916840 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605c155be0 con 0x7f605c06bab0 2024-04-23T14:33:20.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f605be00640 1 --2- 172.21.15.100:0/308916840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f605c15e680 0x7f605c13be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:20.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f605b400640 1 --2- 172.21.15.100:0/308916840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f605c06bab0 0x7f605c0cfcd0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f604c00c370 tx=0x7f604c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:20.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f6053400640 1 -- 172.21.15.100:0/308916840 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c0052d0 con 0x7f605c06bab0 2024-04-23T14:33:20.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f6053400640 1 -- 172.21.15.100:0/308916840 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f604c005470 con 0x7f605c06bab0 2024-04-23T14:33:20.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f6062200640 1 -- 172.21.15.100:0/308916840 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f605c155ec0 con 0x7f605c06bab0 2024-04-23T14:33:20.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f6053400640 1 -- 172.21.15.100:0/308916840 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c007260 con 0x7f605c06bab0 2024-04-23T14:33:20.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f6062200640 1 -- 172.21.15.100:0/308916840 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f605c1564b0 con 0x7f605c06bab0 2024-04-23T14:33:20.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.989+0000 7f6062200640 1 -- 172.21.15.100:0/308916840 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f605c068ad0 con 0x7f605c06bab0 2024-04-23T14:33:20.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.993+0000 7f6053400640 1 -- 172.21.15.100:0/308916840 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f604c007400 con 0x7f605c06bab0 2024-04-23T14:33:20.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.993+0000 7f6053400640 1 --2- 172.21.15.100:0/308916840 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f603c068380 0x7f603c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:20.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.993+0000 7f6053400640 1 -- 172.21.15.100:0/308916840 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f604c086370 con 0x7f605c06bab0 2024-04-23T14:33:20.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.993+0000 7f605aa00640 1 --2- 172.21.15.100:0/308916840 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f603c068380 0x7f603c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:20.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.993+0000 7f605aa00640 1 --2- 172.21.15.100:0/308916840 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f603c068380 0x7f603c06a840 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f60480af000 tx=0x7f6048008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:20.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:20.997+0000 7f6053400640 1 -- 172.21.15.100:0/308916840 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f604c056400 con 0x7f605c06bab0 2024-04-23T14:33:21.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.165+0000 7f6062200640 1 -- 172.21.15.100:0/308916840 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f605c168de0 con 0x7f605c06bab0 2024-04-23T14:33:21.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.165+0000 7f6053400640 1 -- 172.21.15.100:0/308916840 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f604c055d00 con 0x7f605c06bab0 2024-04-23T14:33:21.168 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-04-23T14:33:21.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.165+0000 7f6062200640 1 -- 172.21.15.100:0/308916840 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f603c068380 msgr2=0x7f603c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:21.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.165+0000 7f6062200640 1 --2- 172.21.15.100:0/308916840 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f603c068380 0x7f603c06a840 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f60480af000 tx=0x7f6048008040 comp rx=0 tx=0).stop 2024-04-23T14:33:21.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.165+0000 7f6062200640 1 -- 172.21.15.100:0/308916840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f605c06bab0 msgr2=0x7f605c0cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:21.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.165+0000 7f6062200640 1 --2- 172.21.15.100:0/308916840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f605c06bab0 0x7f605c0cfcd0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f604c00c370 tx=0x7f604c00c830 comp rx=0 tx=0).stop 2024-04-23T14:33:21.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.165+0000 7f6062200640 1 -- 172.21.15.100:0/308916840 shutdown_connections 2024-04-23T14:33:21.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.165+0000 7f6062200640 1 --2- 172.21.15.100:0/308916840 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f603c068380 0x7f603c06a840 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:21.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.165+0000 7f6062200640 1 --2- 172.21.15.100:0/308916840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f605c15e680 0x7f605c13be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:21.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.165+0000 7f6062200640 1 --2- 172.21.15.100:0/308916840 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f605c0724b0 0x7f605c153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:21.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.165+0000 7f6062200640 1 --2- 172.21.15.100:0/308916840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f605c06bab0 0x7f605c0cfcd0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:21.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.165+0000 7f6062200640 1 -- 172.21.15.100:0/308916840 >> 172.21.15.100:0/308916840 conn(0x7f605c0b2600 msgr2=0x7f605c0b3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:21.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.165+0000 7f6062200640 1 -- 172.21.15.100:0/308916840 shutdown_connections 2024-04-23T14:33:21.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.165+0000 7f6062200640 1 -- 172.21.15.100:0/308916840 wait complete. 2024-04-23T14:33:21.189 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:33:21.189 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:21.199 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:21.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.329+0000 7f54caa00640 1 Processor -- start 2024-04-23T14:33:21.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.329+0000 7f54caa00640 1 -- start start 2024-04-23T14:33:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.333+0000 7f54caa00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f54c40b2a10 0x7f54c40adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.333+0000 7f54caa00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f54c4073610 0x7f54c406df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.333+0000 7f54caa00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54c40af080 con 0x7f54c415b640 2024-04-23T14:33:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.333+0000 7f54caa00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f54c40afbc0 con 0x7f54c40b2a10 2024-04-23T14:33:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.333+0000 7f54caa00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f54c415ba20 con 0x7f54c4073610 2024-04-23T14:33:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.333+0000 7f54bbe00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f54c40b2a10 0x7f54c40adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.333+0000 7f54bbe00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f54c40b2a10 0x7f54c40adfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48082/0 (socket says 172.21.15.100:48082) 2024-04-23T14:33:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.333+0000 7f54bbe00640 1 -- 172.21.15.100:0/3720989522 learned_addr learned my addr 172.21.15.100:0/3720989522 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.333+0000 7f54c8a00640 1 --2- 172.21.15.100:0/3720989522 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f54c4073610 0x7f54c406df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.333+0000 7f54bbe00640 1 -- 172.21.15.100:0/3720989522 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f54c4073610 msgr2=0x7f54c406df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:21.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.333+0000 7f54bbe00640 1 --2- 172.21.15.100:0/3720989522 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f54c4073610 0x7f54c406df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:21.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.333+0000 7f54bbe00640 1 -- 172.21.15.100:0/3720989522 >> v1:172.21.15.151:6789/0 conn(0x7f54c415b640 legacy=0x7f54c4072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:21.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.333+0000 7f54bbe00640 1 -- 172.21.15.100:0/3720989522 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c40af320 con 0x7f54c40b2a10 2024-04-23T14:33:21.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.337+0000 7f54bbe00640 1 --2- 172.21.15.100:0/3720989522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c40b2a10 0x7f54c40adfc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009870 tx=0x7f54ac01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed474cf41300e00e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:21.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.337+0000 7f54c3400640 1 -- 172.21.15.100:0/3720989522 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac0a95f0 con 0x7f54c40b2a10 2024-04-23T14:33:21.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.337+0000 7f54c3400640 1 -- 172.21.15.100:0/3720989522 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54ac0a9790 con 0x7f54c40b2a10 2024-04-23T14:33:21.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.337+0000 7f54c3400640 1 -- 172.21.15.100:0/3720989522 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac0a9b60 con 0x7f54c40b2a10 2024-04-23T14:33:21.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.337+0000 7f54caa00640 1 -- 172.21.15.100:0/3720989522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c40b2a10 msgr2=0x7f54c40adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:21.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.337+0000 7f54caa00640 1 --2- 172.21.15.100:0/3720989522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c40b2a10 0x7f54c40adfc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009870 tx=0x7f54ac01c530 comp rx=0 tx=0).stop 2024-04-23T14:33:21.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.337+0000 7f54caa00640 1 -- 172.21.15.100:0/3720989522 shutdown_connections 2024-04-23T14:33:21.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.337+0000 7f54caa00640 1 --2- 172.21.15.100:0/3720989522 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f54c4073610 0x7f54c406df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:21.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.337+0000 7f54caa00640 1 --2- 172.21.15.100:0/3720989522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c40b2a10 0x7f54c40adfc0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:21.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.337+0000 7f54caa00640 1 -- 172.21.15.100:0/3720989522 >> 172.21.15.100:0/3720989522 conn(0x7f54c40b1e80 msgr2=0x7f54c40b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:21.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.337+0000 7f54caa00640 1 -- 172.21.15.100:0/3720989522 shutdown_connections 2024-04-23T14:33:21.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.337+0000 7f54caa00640 1 -- 172.21.15.100:0/3720989522 wait complete. 2024-04-23T14:33:21.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.337+0000 7f54caa00640 1 Processor -- start 2024-04-23T14:33:21.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.337+0000 7f54caa00640 1 -- start start 2024-04-23T14:33:21.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54caa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54c4073610 0x7f54c414a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:21.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54caa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c40b2a10 0x7f54c415d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:21.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54caa00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f54c415b640 0x7f54c415dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:21.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54caa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f54c406f0d0 con 0x7f54c40b2a10 2024-04-23T14:33:21.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54caa00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f54c406ef50 con 0x7f54c4073610 2024-04-23T14:33:21.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54caa00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f54c406f250 con 0x7f54c415b640 2024-04-23T14:33:21.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54bbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c40b2a10 0x7f54c415d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:21.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54c8a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f54c415b640 0x7f54c415dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:21.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54c3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54c4073610 0x7f54c414a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:21.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54c8a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f54c415b640 0x7f54c415dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:60144/0 (socket says 172.21.15.100:60144) 2024-04-23T14:33:21.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54c3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54c4073610 0x7f54c414a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:44646/0 (socket says 172.21.15.100:44646) 2024-04-23T14:33:21.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54c8a00640 1 -- 172.21.15.100:0/1214332270 learned_addr learned my addr 172.21.15.100:0/1214332270 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:21.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54bbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c40b2a10 0x7f54c415d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48092/0 (socket says 172.21.15.100:48092) 2024-04-23T14:33:21.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54bbe00640 1 -- 172.21.15.100:0/1214332270 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f54c415b640 msgr2=0x7f54c415dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:21.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54bbe00640 1 --2- 172.21.15.100:0/1214332270 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f54c415b640 0x7f54c415dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:21.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54bbe00640 1 -- 172.21.15.100:0/1214332270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54c4073610 msgr2=0x7f54c414a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:21.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54bbe00640 1 --2- 172.21.15.100:0/1214332270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54c4073610 0x7f54c414a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:21.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54bbe00640 1 -- 172.21.15.100:0/1214332270 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c415e490 con 0x7f54c40b2a10 2024-04-23T14:33:21.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54c8a00640 1 --2- 172.21.15.100:0/1214332270 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f54c415b640 0x7f54c415dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:33:21.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54bbe00640 1 --2- 172.21.15.100:0/1214332270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c40b2a10 0x7f54c415d7c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f54ac0aa000 tx=0x7f54ac01ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:21.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54c0c00640 1 -- 172.21.15.100:0/1214332270 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac0bc650 con 0x7f54c40b2a10 2024-04-23T14:33:21.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54c0c00640 1 -- 172.21.15.100:0/1214332270 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54ac0bc7f0 con 0x7f54c40b2a10 2024-04-23T14:33:21.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54c0c00640 1 -- 172.21.15.100:0/1214332270 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac0bcbe0 con 0x7f54c40b2a10 2024-04-23T14:33:21.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54caa00640 1 -- 172.21.15.100:0/1214332270 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c415bc60 con 0x7f54c40b2a10 2024-04-23T14:33:21.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54caa00640 1 -- 172.21.15.100:0/1214332270 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c415e730 con 0x7f54c40b2a10 2024-04-23T14:33:21.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54caa00640 1 -- 172.21.15.100:0/1214332270 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c4068ad0 con 0x7f54c40b2a10 2024-04-23T14:33:21.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.341+0000 7f54c0c00640 1 -- 172.21.15.100:0/1214332270 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f54ac0b13f0 con 0x7f54c40b2a10 2024-04-23T14:33:21.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.345+0000 7f54c0c00640 1 --2- 172.21.15.100:0/1214332270 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f54a0068380 0x7f54a006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:21.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.345+0000 7f54c0c00640 1 -- 172.21.15.100:0/1214332270 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f54ac121580 con 0x7f54c40b2a10 2024-04-23T14:33:21.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.345+0000 7f54c3e00640 1 --2- 172.21.15.100:0/1214332270 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f54a0068380 0x7f54a006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:21.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.345+0000 7f54c3e00640 1 --2- 172.21.15.100:0/1214332270 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f54a0068380 0x7f54a006a840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f54c40723b0 tx=0x7f54b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:21.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.349+0000 7f54c0c00640 1 -- 172.21.15.100:0/1214332270 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f54ac0c3030 con 0x7f54c40b2a10 2024-04-23T14:33:21.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.505+0000 7f54caa00640 1 -- 172.21.15.100:0/1214332270 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f54c415e240 con 0x7f54c40b2a10 2024-04-23T14:33:21.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.505+0000 7f54c0c00640 1 -- 172.21.15.100:0/1214332270 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18859 (secure 0 0 0) 0x7f54ac0ad020 con 0x7f54c40b2a10 2024-04-23T14:33:21.509 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:21.509 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:49.784608+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:21.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.509+0000 7f54caa00640 1 -- 172.21.15.100:0/1214332270 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f54a0068380 msgr2=0x7f54a006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:21.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.509+0000 7f54caa00640 1 --2- 172.21.15.100:0/1214332270 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f54a0068380 0x7f54a006a840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f54c40723b0 tx=0x7f54b4009040 comp rx=0 tx=0).stop 2024-04-23T14:33:21.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.509+0000 7f54caa00640 1 -- 172.21.15.100:0/1214332270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c40b2a10 msgr2=0x7f54c415d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:21.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.509+0000 7f54caa00640 1 --2- 172.21.15.100:0/1214332270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c40b2a10 0x7f54c415d7c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f54ac0aa000 tx=0x7f54ac01ad70 comp rx=0 tx=0).stop 2024-04-23T14:33:21.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.509+0000 7f54caa00640 1 -- 172.21.15.100:0/1214332270 shutdown_connections 2024-04-23T14:33:21.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.509+0000 7f54caa00640 1 --2- 172.21.15.100:0/1214332270 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f54a0068380 0x7f54a006a840 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:21.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.509+0000 7f54caa00640 1 --2- 172.21.15.100:0/1214332270 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f54c415b640 0x7f54c415dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:21.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.509+0000 7f54caa00640 1 --2- 172.21.15.100:0/1214332270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c40b2a10 0x7f54c415d7c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:21.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.509+0000 7f54caa00640 1 --2- 172.21.15.100:0/1214332270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54c4073610 0x7f54c414a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:21.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.509+0000 7f54caa00640 1 -- 172.21.15.100:0/1214332270 >> 172.21.15.100:0/1214332270 conn(0x7f54c40b1e80 msgr2=0x7f54c4159c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:21.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.509+0000 7f54caa00640 1 -- 172.21.15.100:0/1214332270 shutdown_connections 2024-04-23T14:33:21.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.509+0000 7f54caa00640 1 -- 172.21.15.100:0/1214332270 wait complete. 2024-04-23T14:33:21.530 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:32:06.835560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:32:07.930489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-23T14:32:11.636814+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-23T14:33:21.530 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:33:21.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.669+0000 7f1677000640 1 Processor -- start 2024-04-23T14:33:21.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.669+0000 7f1677000640 1 -- start start 2024-04-23T14:33:21.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.669+0000 7f1677000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f167006c6c0 0x7f16700ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:21.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.669+0000 7f1677000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f16700ce620 0x7f16700add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:21.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.669+0000 7f1677000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f167006dd60 con 0x7f167006f8c0 2024-04-23T14:33:21.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.669+0000 7f1677000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f16700ad050 con 0x7f16700ce620 2024-04-23T14:33:21.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.669+0000 7f1677000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f16700ad8f0 con 0x7f167006c6c0 2024-04-23T14:33:21.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.669+0000 7f1675000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f16700ce620 0x7f16700add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:21.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.669+0000 7f166fe00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f167006f8c0 0x7f16700d0170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57986/0 (socket says 172.21.15.100:57986) 2024-04-23T14:33:21.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.669+0000 7f166fe00640 1 -- 172.21.15.100:0/3086893692 learned_addr learned my addr 172.21.15.100:0/3086893692 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:21.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.669+0000 7f166f400640 1 --2- 172.21.15.100:0/3086893692 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f167006c6c0 0x7f16700ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:21.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.669+0000 7f1675000640 1 -- 172.21.15.100:0/3086893692 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f167006c6c0 msgr2=0x7f16700ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:21.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.669+0000 7f1675000640 1 --2- 172.21.15.100:0/3086893692 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f167006c6c0 0x7f16700ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:21.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.669+0000 7f1675000640 1 -- 172.21.15.100:0/3086893692 >> v1:172.21.15.100:6789/0 conn(0x7f167006f8c0 legacy=0x7f16700d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:21.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.669+0000 7f1675000640 1 -- 172.21.15.100:0/3086893692 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16700d09c0 con 0x7f16700ce620 2024-04-23T14:33:21.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.673+0000 7f1675000640 1 --2- 172.21.15.100:0/3086893692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16700ce620 0x7f16700add80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f166800bfa0 tx=0x7f166801ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517ed12e73b49d22 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:21.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.673+0000 7f166ea00640 1 -- 172.21.15.100:0/3086893692 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166800b2d0 con 0x7f16700ce620 2024-04-23T14:33:21.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.673+0000 7f166ea00640 1 -- 172.21.15.100:0/3086893692 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16680a7720 con 0x7f16700ce620 2024-04-23T14:33:21.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.673+0000 7f166ea00640 1 -- 172.21.15.100:0/3086893692 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16680a7af0 con 0x7f16700ce620 2024-04-23T14:33:21.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.673+0000 7f1677000640 1 -- 172.21.15.100:0/3086893692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16700ce620 msgr2=0x7f16700add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:21.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.673+0000 7f1677000640 1 --2- 172.21.15.100:0/3086893692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16700ce620 0x7f16700add80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f166800bfa0 tx=0x7f166801ea00 comp rx=0 tx=0).stop 2024-04-23T14:33:21.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.673+0000 7f1677000640 1 -- 172.21.15.100:0/3086893692 shutdown_connections 2024-04-23T14:33:21.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.673+0000 7f1677000640 1 --2- 172.21.15.100:0/3086893692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16700ce620 0x7f16700add80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:21.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.673+0000 7f1677000640 1 --2- 172.21.15.100:0/3086893692 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f167006c6c0 0x7f16700ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:21.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.673+0000 7f1677000640 1 -- 172.21.15.100:0/3086893692 >> 172.21.15.100:0/3086893692 conn(0x7f16700b5200 msgr2=0x7f16700b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:21.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.673+0000 7f1677000640 1 -- 172.21.15.100:0/3086893692 shutdown_connections 2024-04-23T14:33:21.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.673+0000 7f1677000640 1 -- 172.21.15.100:0/3086893692 wait complete. 2024-04-23T14:33:21.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.673+0000 7f1677000640 1 Processor -- start 2024-04-23T14:33:21.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.673+0000 7f1677000640 1 -- start start 2024-04-23T14:33:21.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f1677000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f167006c6c0 0x7f16700afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:21.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f1677000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f167006f8c0 0x7f16700c2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:21.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f1677000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16700ce620 0x7f1670099770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:21.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f1677000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f16700aeea0 con 0x7f167006c6c0 2024-04-23T14:33:21.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f1677000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f16700aed20 con 0x7f167006f8c0 2024-04-23T14:33:21.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f1677000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f16700ccc60 con 0x7f16700ce620 2024-04-23T14:33:21.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f166fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f167006c6c0 0x7f16700afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:21.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f166fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f167006c6c0 0x7f16700afc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48120/0 (socket says 172.21.15.100:48120) 2024-04-23T14:33:21.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f166fe00640 1 -- 172.21.15.100:0/3084298507 learned_addr learned my addr 172.21.15.100:0/3084298507 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:21.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f166f400640 1 --2- 172.21.15.100:0/3084298507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f167006f8c0 0x7f16700c2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:21.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f1675000640 1 --2- 172.21.15.100:0/3084298507 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16700ce620 0x7f1670099770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:21.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f166fe00640 1 -- 172.21.15.100:0/3084298507 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16700ce620 msgr2=0x7f1670099770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:21.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f166fe00640 1 --2- 172.21.15.100:0/3084298507 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16700ce620 0x7f1670099770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:21.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f166fe00640 1 -- 172.21.15.100:0/3084298507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f167006f8c0 msgr2=0x7f16700c2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:21.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f166fe00640 1 --2- 172.21.15.100:0/3084298507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f167006f8c0 0x7f16700c2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:21.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f166fe00640 1 -- 172.21.15.100:0/3084298507 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16700c3680 con 0x7f167006c6c0 2024-04-23T14:33:21.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f166fe00640 1 --2- 172.21.15.100:0/3084298507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f167006c6c0 0x7f16700afc00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f166000c840 tx=0x7f166000cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:21.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f1667e00640 1 -- 172.21.15.100:0/3084298507 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1660018070 con 0x7f167006c6c0 2024-04-23T14:33:21.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f1667e00640 1 -- 172.21.15.100:0/3084298507 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16600074e0 con 0x7f167006c6c0 2024-04-23T14:33:21.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f1667e00640 1 -- 172.21.15.100:0/3084298507 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1660004a30 con 0x7f167006c6c0 2024-04-23T14:33:21.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f1677000640 1 -- 172.21.15.100:0/3084298507 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16700c3960 con 0x7f167006c6c0 2024-04-23T14:33:21.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f1677000640 1 -- 172.21.15.100:0/3084298507 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16700b0140 con 0x7f167006c6c0 2024-04-23T14:33:21.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f1677000640 1 -- 172.21.15.100:0/3084298507 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16700c0500 con 0x7f167006c6c0 2024-04-23T14:33:21.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.677+0000 7f1667e00640 1 -- 172.21.15.100:0/3084298507 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f166001b070 con 0x7f167006c6c0 2024-04-23T14:33:21.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.681+0000 7f1667e00640 1 --2- 172.21.15.100:0/3084298507 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1650068380 0x7f165006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:21.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.681+0000 7f1667e00640 1 -- 172.21.15.100:0/3084298507 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f16600864e0 con 0x7f167006c6c0 2024-04-23T14:33:21.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.681+0000 7f166f400640 1 --2- 172.21.15.100:0/3084298507 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1650068380 0x7f165006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:21.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.681+0000 7f166f400640 1 --2- 172.21.15.100:0/3084298507 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1650068380 0x7f165006a840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f167006d0e0 tx=0x7f165c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:21.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.685+0000 7f1667e00640 1 -- 172.21.15.100:0/3084298507 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f167006dd60 con 0x7f167006c6c0 2024-04-23T14:33:21.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.841+0000 7f1677000640 1 -- 172.21.15.100:0/3084298507 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1670068ad0 con 0x7f167006c6c0 2024-04-23T14:33:21.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.845+0000 7f1667e00640 1 -- 172.21.15.100:0/3084298507 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f1660056600 con 0x7f167006c6c0 2024-04-23T14:33:21.847 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:33:21.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.845+0000 7f1677000640 1 -- 172.21.15.100:0/3084298507 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1650068380 msgr2=0x7f165006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:21.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.845+0000 7f1677000640 1 --2- 172.21.15.100:0/3084298507 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1650068380 0x7f165006a840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f167006d0e0 tx=0x7f165c008040 comp rx=0 tx=0).stop 2024-04-23T14:33:21.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.845+0000 7f1677000640 1 -- 172.21.15.100:0/3084298507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f167006c6c0 msgr2=0x7f16700afc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:21.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.845+0000 7f1677000640 1 --2- 172.21.15.100:0/3084298507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f167006c6c0 0x7f16700afc00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f166000c840 tx=0x7f166000cd00 comp rx=0 tx=0).stop 2024-04-23T14:33:21.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.845+0000 7f1677000640 1 -- 172.21.15.100:0/3084298507 shutdown_connections 2024-04-23T14:33:21.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.845+0000 7f1677000640 1 --2- 172.21.15.100:0/3084298507 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1650068380 0x7f165006a840 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:21.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.845+0000 7f1677000640 1 --2- 172.21.15.100:0/3084298507 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16700ce620 0x7f1670099770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:21.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.845+0000 7f1677000640 1 --2- 172.21.15.100:0/3084298507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f167006f8c0 0x7f16700c2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:21.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.845+0000 7f1677000640 1 --2- 172.21.15.100:0/3084298507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f167006c6c0 0x7f16700afc00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:21.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.845+0000 7f1677000640 1 -- 172.21.15.100:0/3084298507 >> 172.21.15.100:0/3084298507 conn(0x7f16700b5200 msgr2=0x7f16700ccff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:21.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.845+0000 7f1677000640 1 -- 172.21.15.100:0/3084298507 shutdown_connections 2024-04-23T14:33:21.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:21.845+0000 7f1677000640 1 -- 172.21.15.100:0/3084298507 wait complete. 2024-04-23T14:33:21.867 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:33:22.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.001+0000 7ff52f000640 1 Processor -- start 2024-04-23T14:33:22.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.005+0000 7ff52f000640 1 -- start start 2024-04-23T14:33:22.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.005+0000 7ff52f000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff52806bb20 0x7ff5280ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:22.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.005+0000 7ff52f000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5280cd540 con 0x7ff52815c7a0 2024-04-23T14:33:22.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.005+0000 7ff52f000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff52806cbb0 con 0x7ff5280cd130 2024-04-23T14:33:22.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.005+0000 7ff52f000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff52806d450 con 0x7ff52806bb20 2024-04-23T14:33:22.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.005+0000 7ff527e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff52815c7a0 0x7ff5280af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57994/0 (socket says 172.21.15.100:57994) 2024-04-23T14:33:22.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.005+0000 7ff527e00640 1 -- 172.21.15.100:0/3199745544 learned_addr learned my addr 172.21.15.100:0/3199745544 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:22.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.005+0000 7ff51f400640 1 --2- 172.21.15.100:0/3199745544 >> v2:172.21.15.100:3300/0 conn(0x7ff52806bb20 0x7ff5280ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:22.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.005+0000 7ff527400640 1 -- 172.21.15.100:0/3199745544 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 8989589 0 0) 0x7ff518001120 con 0x7ff52815c7a0 2024-04-23T14:33:22.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.005+0000 7ff527400640 1 -- 172.21.15.100:0/3199745544 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff50c003530 con 0x7ff52815c7a0 2024-04-23T14:33:22.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.005+0000 7ff51f400640 1 -- 172.21.15.100:0/3199745544 >> v1:172.21.15.151:6789/0 conn(0x7ff5280cd130 legacy=0x7ff5280ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:22.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.005+0000 7ff51f400640 1 -- 172.21.15.100:0/3199745544 >> v1:172.21.15.100:6789/0 conn(0x7ff52815c7a0 legacy=0x7ff5280af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:22.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.005+0000 7ff51f400640 1 -- 172.21.15.100:0/3199745544 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52806ce90 con 0x7ff52806bb20 2024-04-23T14:33:22.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.009+0000 7ff51f400640 1 --2- 172.21.15.100:0/3199745544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff52806bb20 0x7ff5280ccbf0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff5100097c0 tx=0x7ff51001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb74616fdd83db8f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:22.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.009+0000 7ff527400640 1 -- 172.21.15.100:0/3199745544 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5100b0070 con 0x7ff52806bb20 2024-04-23T14:33:22.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.009+0000 7ff527400640 1 -- 172.21.15.100:0/3199745544 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5100ab600 con 0x7ff52806bb20 2024-04-23T14:33:22.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.009+0000 7ff527400640 1 -- 172.21.15.100:0/3199745544 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5100ab9d0 con 0x7ff52806bb20 2024-04-23T14:33:22.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.009+0000 7ff52f000640 1 -- 172.21.15.100:0/3199745544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff52806bb20 msgr2=0x7ff5280ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:22.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.009+0000 7ff52f000640 1 --2- 172.21.15.100:0/3199745544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff52806bb20 0x7ff5280ccbf0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff5100097c0 tx=0x7ff51001c410 comp rx=0 tx=0).stop 2024-04-23T14:33:22.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.009+0000 7ff52f000640 1 -- 172.21.15.100:0/3199745544 shutdown_connections 2024-04-23T14:33:22.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.009+0000 7ff52f000640 1 --2- 172.21.15.100:0/3199745544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff52806bb20 0x7ff5280ccbf0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.009+0000 7ff52f000640 1 -- 172.21.15.100:0/3199745544 >> 172.21.15.100:0/3199745544 conn(0x7ff5280b5200 msgr2=0x7ff5280b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:22.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.009+0000 7ff52f000640 1 -- 172.21.15.100:0/3199745544 shutdown_connections 2024-04-23T14:33:22.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.009+0000 7ff52f000640 1 -- 172.21.15.100:0/3199745544 wait complete. 2024-04-23T14:33:22.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.009+0000 7ff52f000640 1 Processor -- start 2024-04-23T14:33:22.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.009+0000 7ff52f000640 1 -- start start 2024-04-23T14:33:22.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff52f000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff52806bb20 0x7ff528099570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:22.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff52f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5280cd130 0x7ff5280afa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:22.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff52f000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff52815c7a0 0x7ff528155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:22.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff52f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff5280ae720 con 0x7ff5280cd130 2024-04-23T14:33:22.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff52f000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff5280ae5a0 con 0x7ff52815c7a0 2024-04-23T14:33:22.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff52f000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff52816a210 con 0x7ff52806bb20 2024-04-23T14:33:22.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff51f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5280cd130 0x7ff5280afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:22.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff51f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5280cd130 0x7ff5280afa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48126/0 (socket says 172.21.15.100:48126) 2024-04-23T14:33:22.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff51f400640 1 -- 172.21.15.100:0/392151850 learned_addr learned my addr 172.21.15.100:0/392151850 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:22.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff527e00640 1 --2- 172.21.15.100:0/392151850 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff52806bb20 0x7ff528099570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:22.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff52d000640 1 --2- 172.21.15.100:0/392151850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff52815c7a0 0x7ff528155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:22.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff51f400640 1 -- 172.21.15.100:0/392151850 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff52806bb20 msgr2=0x7ff528099570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:22.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff51f400640 1 --2- 172.21.15.100:0/392151850 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff52806bb20 0x7ff528099570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff51f400640 1 -- 172.21.15.100:0/392151850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff52815c7a0 msgr2=0x7ff528155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:22.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff51f400640 1 --2- 172.21.15.100:0/392151850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff52815c7a0 0x7ff528155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff51f400640 1 -- 172.21.15.100:0/392151850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52815b250 con 0x7ff5280cd130 2024-04-23T14:33:22.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff51f400640 1 --2- 172.21.15.100:0/392151850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5280cd130 0x7ff5280afa90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff5100ae040 tx=0x7ff5100191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:22.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff524c00640 1 -- 172.21.15.100:0/392151850 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5100b0040 con 0x7ff5280cd130 2024-04-23T14:33:22.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff524c00640 1 -- 172.21.15.100:0/392151850 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5100b9070 con 0x7ff5280cd130 2024-04-23T14:33:22.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff52f000640 1 -- 172.21.15.100:0/392151850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff52815b760 con 0x7ff5280cd130 2024-04-23T14:33:22.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff524c00640 1 -- 172.21.15.100:0/392151850 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5100b4850 con 0x7ff5280cd130 2024-04-23T14:33:22.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff52f000640 1 -- 172.21.15.100:0/392151850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5280c4190 con 0x7ff5280cd130 2024-04-23T14:33:22.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff52f000640 1 -- 172.21.15.100:0/392151850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff528068ad0 con 0x7ff5280cd130 2024-04-23T14:33:22.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.013+0000 7ff524c00640 1 -- 172.21.15.100:0/392151850 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff5100b49f0 con 0x7ff5280cd130 2024-04-23T14:33:22.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.017+0000 7ff524c00640 1 --2- 172.21.15.100:0/392151850 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff504068380 0x7ff50406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:22.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.017+0000 7ff524c00640 1 -- 172.21.15.100:0/392151850 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff510120cb0 con 0x7ff5280cd130 2024-04-23T14:33:22.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.017+0000 7ff527e00640 1 --2- 172.21.15.100:0/392151850 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff504068380 0x7ff50406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:22.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.017+0000 7ff527e00640 1 --2- 172.21.15.100:0/392151850 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff504068380 0x7ff50406a840 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff5180012c0 tx=0x7ff518009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:22.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.021+0000 7ff524c00640 1 -- 172.21.15.100:0/392151850 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff510016090 con 0x7ff5280cd130 2024-04-23T14:33:22.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.177+0000 7ff52f000640 1 -- 172.21.15.100:0/392151850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff528167a70 con 0x7ff5280cd130 2024-04-23T14:33:22.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.177+0000 7ff524c00640 1 -- 172.21.15.100:0/392151850 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7ff5100f0dd0 con 0x7ff5280cd130 2024-04-23T14:33:22.180 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:22.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.177+0000 7ff52f000640 1 -- 172.21.15.100:0/392151850 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff504068380 msgr2=0x7ff50406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:22.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.177+0000 7ff52f000640 1 --2- 172.21.15.100:0/392151850 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff504068380 0x7ff50406a840 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff5180012c0 tx=0x7ff518009040 comp rx=0 tx=0).stop 2024-04-23T14:33:22.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.177+0000 7ff52f000640 1 -- 172.21.15.100:0/392151850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5280cd130 msgr2=0x7ff5280afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:22.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.177+0000 7ff52f000640 1 --2- 172.21.15.100:0/392151850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5280cd130 0x7ff5280afa90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff5100ae040 tx=0x7ff5100191b0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.177+0000 7ff52f000640 1 -- 172.21.15.100:0/392151850 shutdown_connections 2024-04-23T14:33:22.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.177+0000 7ff52f000640 1 --2- 172.21.15.100:0/392151850 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff504068380 0x7ff50406a840 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.177+0000 7ff52f000640 1 --2- 172.21.15.100:0/392151850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff52815c7a0 0x7ff528155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.177+0000 7ff52f000640 1 --2- 172.21.15.100:0/392151850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5280cd130 0x7ff5280afa90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.177+0000 7ff52f000640 1 --2- 172.21.15.100:0/392151850 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff52806bb20 0x7ff528099570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.177+0000 7ff52f000640 1 -- 172.21.15.100:0/392151850 >> 172.21.15.100:0/392151850 conn(0x7ff5280b5200 msgr2=0x7ff52815ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:22.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.177+0000 7ff52f000640 1 -- 172.21.15.100:0/392151850 shutdown_connections 2024-04-23T14:33:22.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.177+0000 7ff52f000640 1 -- 172.21.15.100:0/392151850 wait complete. 2024-04-23T14:33:22.201 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:33:22.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.337+0000 7f0320a00640 1 Processor -- start 2024-04-23T14:33:22.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.337+0000 7f0320a00640 1 -- start start 2024-04-23T14:33:22.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.337+0000 7f0320a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f031c15e760 0x7f031c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:22.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.337+0000 7f0320a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f031c06d5f0 0x7f031c0cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:22.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.337+0000 7f0320a00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f031c0b56d0 con 0x7f031c0cf8c0 2024-04-23T14:33:22.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.337+0000 7f0320a00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f031c0d0260 con 0x7f031c06d5f0 2024-04-23T14:33:22.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.337+0000 7f0320a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f031c0b59b0 con 0x7f031c15e760 2024-04-23T14:33:22.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.337+0000 7f031a800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f031c06d5f0 0x7f031c0cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:22.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.337+0000 7f0319400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f031c0cf8c0 0x7f031c162320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58006/0 (socket says 172.21.15.100:58006) 2024-04-23T14:33:22.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.337+0000 7f0319400640 1 -- 172.21.15.100:0/3966193656 learned_addr learned my addr 172.21.15.100:0/3966193656 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:22.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.337+0000 7f0319e00640 1 --2- 172.21.15.100:0/3966193656 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f031c15e760 0x7f031c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:22.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.337+0000 7f031a800640 1 -- 172.21.15.100:0/3966193656 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f031c15e760 msgr2=0x7f031c0b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:22.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.337+0000 7f031a800640 1 --2- 172.21.15.100:0/3966193656 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f031c15e760 0x7f031c0b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.337+0000 7f0318a00640 1 -- 172.21.15.100:0/3966193656 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3203317889 0 0) 0x7f031c0b56d0 con 0x7f031c0cf8c0 2024-04-23T14:33:22.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.337+0000 7f031a800640 1 -- 172.21.15.100:0/3966193656 >> v1:172.21.15.100:6789/0 conn(0x7f031c0cf8c0 legacy=0x7f031c162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:22.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.337+0000 7f031a800640 1 -- 172.21.15.100:0/3966193656 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031c0b5c90 con 0x7f031c06d5f0 2024-04-23T14:33:22.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.341+0000 7f031a800640 1 --2- 172.21.15.100:0/3966193656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f031c06d5f0 0x7f031c0cfd20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0314009d90 tx=0x7f031401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e0eacaa92cdca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:22.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.341+0000 7f0318a00640 1 -- 172.21.15.100:0/3966193656 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03140b2070 con 0x7f031c06d5f0 2024-04-23T14:33:22.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.341+0000 7f0318a00640 1 -- 172.21.15.100:0/3966193656 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03140a5860 con 0x7f031c06d5f0 2024-04-23T14:33:22.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.341+0000 7f0318a00640 1 -- 172.21.15.100:0/3966193656 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03140a5c30 con 0x7f031c06d5f0 2024-04-23T14:33:22.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.341+0000 7f0320a00640 1 -- 172.21.15.100:0/3966193656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f031c06d5f0 msgr2=0x7f031c0cfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:22.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.341+0000 7f0320a00640 1 --2- 172.21.15.100:0/3966193656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f031c06d5f0 0x7f031c0cfd20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0314009d90 tx=0x7f031401c970 comp rx=0 tx=0).stop 2024-04-23T14:33:22.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.341+0000 7f0320a00640 1 -- 172.21.15.100:0/3966193656 shutdown_connections 2024-04-23T14:33:22.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.341+0000 7f0320a00640 1 --2- 172.21.15.100:0/3966193656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f031c06d5f0 0x7f031c0cfd20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.341+0000 7f0320a00640 1 --2- 172.21.15.100:0/3966193656 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f031c15e760 0x7f031c0b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.341+0000 7f0320a00640 1 -- 172.21.15.100:0/3966193656 >> 172.21.15.100:0/3966193656 conn(0x7f031c070670 msgr2=0x7f031c070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:22.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.341+0000 7f0320a00640 1 -- 172.21.15.100:0/3966193656 shutdown_connections 2024-04-23T14:33:22.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.341+0000 7f0320a00640 1 -- 172.21.15.100:0/3966193656 wait complete. 2024-04-23T14:33:22.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0320a00640 1 Processor -- start 2024-04-23T14:33:22.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0320a00640 1 -- start start 2024-04-23T14:33:22.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0320a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f031c06d5f0 0x7f031c160150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:22.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0320a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f031c0cf8c0 0x7f031c1511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:22.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0320a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f031c15e760 0x7f031c1587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:22.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0320a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f031c06fc90 con 0x7f031c06d5f0 2024-04-23T14:33:22.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0320a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f031c06fb10 con 0x7f031c0cf8c0 2024-04-23T14:33:22.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0320a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f031c16b470 con 0x7f031c15e760 2024-04-23T14:33:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0319e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f031c06d5f0 0x7f031c160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0319e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f031c06d5f0 0x7f031c160150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48148/0 (socket says 172.21.15.100:48148) 2024-04-23T14:33:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0319e00640 1 -- 172.21.15.100:0/38908622 learned_addr learned my addr 172.21.15.100:0/38908622 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0319400640 1 --2- 172.21.15.100:0/38908622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f031c0cf8c0 0x7f031c1511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f031a800640 1 --2- 172.21.15.100:0/38908622 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f031c15e760 0x7f031c1587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0319e00640 1 -- 172.21.15.100:0/38908622 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f031c15e760 msgr2=0x7f031c1587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0319e00640 1 --2- 172.21.15.100:0/38908622 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f031c15e760 0x7f031c1587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0319e00640 1 -- 172.21.15.100:0/38908622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f031c0cf8c0 msgr2=0x7f031c1511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0319e00640 1 --2- 172.21.15.100:0/38908622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f031c0cf8c0 0x7f031c1511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0319e00640 1 -- 172.21.15.100:0/38908622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031c1591f0 con 0x7f031c06d5f0 2024-04-23T14:33:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0319e00640 1 --2- 172.21.15.100:0/38908622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f031c06d5f0 0x7f031c160150 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f030c00d5f0 tx=0x7f030c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0312000640 1 -- 172.21.15.100:0/38908622 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c0125f0 con 0x7f031c06d5f0 2024-04-23T14:33:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0312000640 1 -- 172.21.15.100:0/38908622 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f030c019070 con 0x7f031c06d5f0 2024-04-23T14:33:22.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0312000640 1 -- 172.21.15.100:0/38908622 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c0129b0 con 0x7f031c06d5f0 2024-04-23T14:33:22.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0320a00640 1 -- 172.21.15.100:0/38908622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f031c1594d0 con 0x7f031c06d5f0 2024-04-23T14:33:22.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0320a00640 1 -- 172.21.15.100:0/38908622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f031c158d20 con 0x7f031c06d5f0 2024-04-23T14:33:22.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.345+0000 7f0320a00640 1 -- 172.21.15.100:0/38908622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f031c0681c0 con 0x7f031c06d5f0 2024-04-23T14:33:22.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.349+0000 7f0312000640 1 -- 172.21.15.100:0/38908622 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f030c012b50 con 0x7f031c06d5f0 2024-04-23T14:33:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.349+0000 7f0312000640 1 --2- 172.21.15.100:0/38908622 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f02fc068380 0x7f02fc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.349+0000 7f0312000640 1 -- 172.21.15.100:0/38908622 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f030c0872a0 con 0x7f031c06d5f0 2024-04-23T14:33:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.349+0000 7f0319400640 1 --2- 172.21.15.100:0/38908622 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f02fc068380 0x7f02fc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.349+0000 7f0319400640 1 --2- 172.21.15.100:0/38908622 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f02fc068380 0x7f02fc06a840 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f031c1600f0 tx=0x7f0308007b80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:22.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.353+0000 7f0312000640 1 -- 172.21.15.100:0/38908622 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f030c0573c0 con 0x7f031c06d5f0 2024-04-23T14:33:22.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.509+0000 7f0320a00640 1 -- 172.21.15.100:0/38908622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f031c068ad0 con 0x7f031c06d5f0 2024-04-23T14:33:22.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.509+0000 7f0312000640 1 -- 172.21.15.100:0/38908622 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f030c056cc0 con 0x7f031c06d5f0 2024-04-23T14:33:22.511 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:33:22.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.509+0000 7f0320a00640 1 -- 172.21.15.100:0/38908622 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f02fc068380 msgr2=0x7f02fc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:22.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.509+0000 7f0320a00640 1 --2- 172.21.15.100:0/38908622 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f02fc068380 0x7f02fc06a840 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f031c1600f0 tx=0x7f0308007b80 comp rx=0 tx=0).stop 2024-04-23T14:33:22.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.509+0000 7f0320a00640 1 -- 172.21.15.100:0/38908622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f031c06d5f0 msgr2=0x7f031c160150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:22.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.509+0000 7f0320a00640 1 --2- 172.21.15.100:0/38908622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f031c06d5f0 0x7f031c160150 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f030c00d5f0 tx=0x7f030c00dab0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.509+0000 7f0320a00640 1 -- 172.21.15.100:0/38908622 shutdown_connections 2024-04-23T14:33:22.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.509+0000 7f0320a00640 1 --2- 172.21.15.100:0/38908622 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f02fc068380 0x7f02fc06a840 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.509+0000 7f0320a00640 1 --2- 172.21.15.100:0/38908622 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f031c15e760 0x7f031c1587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.509+0000 7f0320a00640 1 --2- 172.21.15.100:0/38908622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f031c0cf8c0 0x7f031c1511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.509+0000 7f0320a00640 1 --2- 172.21.15.100:0/38908622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f031c06d5f0 0x7f031c160150 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.509+0000 7f0320a00640 1 -- 172.21.15.100:0/38908622 >> 172.21.15.100:0/38908622 conn(0x7f031c070670 msgr2=0x7f031c06ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:22.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.509+0000 7f0320a00640 1 -- 172.21.15.100:0/38908622 shutdown_connections 2024-04-23T14:33:22.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.509+0000 7f0320a00640 1 -- 172.21.15.100:0/38908622 wait complete. 2024-04-23T14:33:22.533 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-23T14:33:22.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.669+0000 7fc439200640 1 Processor -- start 2024-04-23T14:33:22.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.669+0000 7fc439200640 1 -- start start 2024-04-23T14:33:22.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.669+0000 7fc439200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc43406c6c0 0x7fc4340ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:22.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.669+0000 7fc439200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc4340ce620 0x7fc4340add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:22.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.669+0000 7fc439200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc43406dd60 con 0x7fc43406f8c0 2024-04-23T14:33:22.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.669+0000 7fc439200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fc4340ad050 con 0x7fc4340ce620 2024-04-23T14:33:22.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.669+0000 7fc439200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc4340ad8f0 con 0x7fc43406c6c0 2024-04-23T14:33:22.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.669+0000 7fc431400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc43406c6c0 0x7fc4340ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:22.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.669+0000 7fc432800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc4340ce620 0x7fc4340add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:22.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.669+0000 7fc431400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc43406c6c0 0x7fc4340ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:60200/0 (socket says 172.21.15.100:60200) 2024-04-23T14:33:22.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.669+0000 7fc432800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc4340ce620 0x7fc4340add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:44676/0 (socket says 172.21.15.100:44676) 2024-04-23T14:33:22.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.669+0000 7fc431400640 1 -- 172.21.15.100:0/2930521911 learned_addr learned my addr 172.21.15.100:0/2930521911 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:22.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.669+0000 7fc431400640 1 -- 172.21.15.100:0/2930521911 >> v2:172.21.15.151:3300/0 conn(0x7fc4340ce620 msgr2=0x7fc4340add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:22.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.669+0000 7fc431400640 1 --2- 172.21.15.100:0/2930521911 >> v2:172.21.15.151:3300/0 conn(0x7fc4340ce620 0x7fc4340add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.669+0000 7fc431400640 1 -- 172.21.15.100:0/2930521911 >> v1:172.21.15.151:6789/0 conn(0x7fc43406f8c0 legacy=0x7fc4340d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:22.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.669+0000 7fc430a00640 1 -- 172.21.15.100:0/2930521911 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2327032418 0 0) 0x7fc424001120 con 0x7fc43406f8c0 2024-04-23T14:33:22.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.669+0000 7fc431400640 1 -- 172.21.15.100:0/2930521911 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4340d09c0 con 0x7fc43406c6c0 2024-04-23T14:33:22.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.669+0000 7fc432800640 1 --2- 172.21.15.100:0/2930521911 >> v2:172.21.15.151:3300/0 conn(0x7fc4340ce620 0x7fc4340add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:33:22.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.673+0000 7fc431400640 1 --2- 172.21.15.100:0/2930521911 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc43406c6c0 0x7fc4340ce0e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc420009870 tx=0x7fc42001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91ae8d1beeac7f12 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:22.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.673+0000 7fc430a00640 1 -- 172.21.15.100:0/2930521911 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4200a7070 con 0x7fc43406c6c0 2024-04-23T14:33:22.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.673+0000 7fc430a00640 1 -- 172.21.15.100:0/2930521911 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4200b0070 con 0x7fc43406c6c0 2024-04-23T14:33:22.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.673+0000 7fc430a00640 1 -- 172.21.15.100:0/2930521911 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4200ab890 con 0x7fc43406c6c0 2024-04-23T14:33:22.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.673+0000 7fc439200640 1 -- 172.21.15.100:0/2930521911 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc43406c6c0 msgr2=0x7fc4340ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:22.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.673+0000 7fc439200640 1 --2- 172.21.15.100:0/2930521911 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc43406c6c0 0x7fc4340ce0e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc420009870 tx=0x7fc42001c560 comp rx=0 tx=0).stop 2024-04-23T14:33:22.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.673+0000 7fc439200640 1 -- 172.21.15.100:0/2930521911 shutdown_connections 2024-04-23T14:33:22.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.673+0000 7fc439200640 1 --2- 172.21.15.100:0/2930521911 >> v2:172.21.15.151:3300/0 conn(0x7fc4340ce620 0x7fc4340add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.673+0000 7fc439200640 1 --2- 172.21.15.100:0/2930521911 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc43406c6c0 0x7fc4340ce0e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.673+0000 7fc439200640 1 -- 172.21.15.100:0/2930521911 >> 172.21.15.100:0/2930521911 conn(0x7fc4340b5200 msgr2=0x7fc4340b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:22.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.673+0000 7fc439200640 1 -- 172.21.15.100:0/2930521911 shutdown_connections 2024-04-23T14:33:22.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.673+0000 7fc439200640 1 -- 172.21.15.100:0/2930521911 wait complete. 2024-04-23T14:33:22.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc439200640 1 Processor -- start 2024-04-23T14:33:22.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc439200640 1 -- start start 2024-04-23T14:33:22.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc439200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc43406c6c0 0x7fc4340c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:22.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc439200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc43406f8c0 0x7fc434071e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:22.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc439200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4340ce620 0x7fc4340723d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:22.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc439200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc4340aeea0 con 0x7fc43406f8c0 2024-04-23T14:33:22.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc439200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc4340aed20 con 0x7fc4340ce620 2024-04-23T14:33:22.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc439200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc434168400 con 0x7fc43406c6c0 2024-04-23T14:33:22.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc431400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc43406f8c0 0x7fc434071e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:22.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc431400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc43406f8c0 0x7fc434071e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48154/0 (socket says 172.21.15.100:48154) 2024-04-23T14:33:22.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc431400640 1 -- 172.21.15.100:0/964436943 learned_addr learned my addr 172.21.15.100:0/964436943 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:22.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc432800640 1 --2- 172.21.15.100:0/964436943 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4340ce620 0x7fc4340723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:22.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc431e00640 1 --2- 172.21.15.100:0/964436943 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc43406c6c0 0x7fc4340c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:22.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc431400640 1 -- 172.21.15.100:0/964436943 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc43406c6c0 msgr2=0x7fc4340c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:22.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc431400640 1 --2- 172.21.15.100:0/964436943 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc43406c6c0 0x7fc4340c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc431400640 1 -- 172.21.15.100:0/964436943 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4340ce620 msgr2=0x7fc4340723d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:22.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc431400640 1 --2- 172.21.15.100:0/964436943 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4340ce620 0x7fc4340723d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc431400640 1 -- 172.21.15.100:0/964436943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4340c6300 con 0x7fc43406f8c0 2024-04-23T14:33:22.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc431400640 1 --2- 172.21.15.100:0/964436943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc43406f8c0 0x7fc434071e90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc42001a190 tx=0x7fc42001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:22.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc42a000640 1 -- 172.21.15.100:0/964436943 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4200ba070 con 0x7fc43406f8c0 2024-04-23T14:33:22.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc42a000640 1 -- 172.21.15.100:0/964436943 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc420004220 con 0x7fc43406f8c0 2024-04-23T14:33:22.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc42a000640 1 -- 172.21.15.100:0/964436943 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4200ab970 con 0x7fc43406f8c0 2024-04-23T14:33:22.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc439200640 1 -- 172.21.15.100:0/964436943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc43415a480 con 0x7fc43406f8c0 2024-04-23T14:33:22.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc439200640 1 -- 172.21.15.100:0/964436943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc43415aa40 con 0x7fc43406f8c0 2024-04-23T14:33:22.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.677+0000 7fc439200640 1 -- 172.21.15.100:0/964436943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc434068ad0 con 0x7fc43406f8c0 2024-04-23T14:33:22.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.681+0000 7fc42a000640 1 -- 172.21.15.100:0/964436943 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc4200abb10 con 0x7fc43406f8c0 2024-04-23T14:33:22.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.681+0000 7fc42a000640 1 --2- 172.21.15.100:0/964436943 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc414068380 0x7fc41406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:22.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.681+0000 7fc42a000640 1 -- 172.21.15.100:0/964436943 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc420120db0 con 0x7fc43406f8c0 2024-04-23T14:33:22.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.681+0000 7fc431e00640 1 --2- 172.21.15.100:0/964436943 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc414068380 0x7fc41406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:22.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.681+0000 7fc431e00640 1 --2- 172.21.15.100:0/964436943 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc414068380 0x7fc41406a840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc424000f70 tx=0x7fc424009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:22.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.685+0000 7fc42a000640 1 -- 172.21.15.100:0/964436943 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc420125070 con 0x7fc43406f8c0 2024-04-23T14:33:22.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.841+0000 7fc439200640 1 -- 172.21.15.100:0/964436943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc43406cc00 con 0x7fc43406f8c0 2024-04-23T14:33:22.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.841+0000 7fc42a000640 1 -- 172.21.15.100:0/964436943 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fc420016150 con 0x7fc43406f8c0 2024-04-23T14:33:22.845 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:22.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.845+0000 7fc439200640 1 -- 172.21.15.100:0/964436943 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc414068380 msgr2=0x7fc41406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:22.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.845+0000 7fc439200640 1 --2- 172.21.15.100:0/964436943 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc414068380 0x7fc41406a840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc424000f70 tx=0x7fc424009040 comp rx=0 tx=0).stop 2024-04-23T14:33:22.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.845+0000 7fc439200640 1 -- 172.21.15.100:0/964436943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc43406f8c0 msgr2=0x7fc434071e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:22.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.845+0000 7fc439200640 1 --2- 172.21.15.100:0/964436943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc43406f8c0 0x7fc434071e90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc42001a190 tx=0x7fc42001ada0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.845+0000 7fc439200640 1 -- 172.21.15.100:0/964436943 shutdown_connections 2024-04-23T14:33:22.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.845+0000 7fc439200640 1 --2- 172.21.15.100:0/964436943 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc414068380 0x7fc41406a840 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.845+0000 7fc439200640 1 --2- 172.21.15.100:0/964436943 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4340ce620 0x7fc4340723d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.845+0000 7fc439200640 1 --2- 172.21.15.100:0/964436943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc43406f8c0 0x7fc434071e90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.845+0000 7fc439200640 1 --2- 172.21.15.100:0/964436943 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc43406c6c0 0x7fc4340c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:22.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.845+0000 7fc439200640 1 -- 172.21.15.100:0/964436943 >> 172.21.15.100:0/964436943 conn(0x7fc4340b5200 msgr2=0x7fc434164aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:22.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.845+0000 7fc439200640 1 -- 172.21.15.100:0/964436943 shutdown_connections 2024-04-23T14:33:22.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:22.845+0000 7fc439200640 1 -- 172.21.15.100:0/964436943 wait complete. 2024-04-23T14:33:22.866 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:33:23.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f7760800640 1 Processor -- start 2024-04-23T14:33:23.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f7760800640 1 -- start start 2024-04-23T14:33:23.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f7760800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f77580d0190 0x7f77580d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:23.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f7760800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77580cee00 con 0x7f775815b660 2024-04-23T14:33:23.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f7760800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f775806bc50 con 0x7f77580c5160 2024-04-23T14:33:23.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f7760800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f775815ba40 con 0x7f77580d0190 2024-04-23T14:33:23.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775de00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f775815b660 0x7f77580c6820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58022/0 (socket says 172.21.15.100:58022) 2024-04-23T14:33:23.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775de00640 1 -- 172.21.15.100:0/2692005955 learned_addr learned my addr 172.21.15.100:0/2692005955 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:23.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775d400640 1 --2- 172.21.15.100:0/2692005955 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f77580d0190 0x7f77580d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:23.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775ca00640 1 -- 172.21.15.100:0/2692005955 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1553353701 0 0) 0x7f774c001120 con 0x7f775815b660 2024-04-23T14:33:23.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775ca00640 1 -- 172.21.15.100:0/2692005955 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7740003530 con 0x7f775815b660 2024-04-23T14:33:23.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775ca00640 1 -- 172.21.15.100:0/2692005955 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 931048692 0 0) 0x7f775806bc50 con 0x7f77580c5160 2024-04-23T14:33:23.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775ca00640 1 -- 172.21.15.100:0/2692005955 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f774c001120 con 0x7f77580c5160 2024-04-23T14:33:23.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775ca00640 1 -- 172.21.15.100:0/2692005955 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2269090060 0 0) 0x7f7740003530 con 0x7f775815b660 2024-04-23T14:33:23.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775ca00640 1 -- 172.21.15.100:0/2692005955 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f77580d0190 msgr2=0x7f77580d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:23.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775ca00640 1 --2- 172.21.15.100:0/2692005955 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f77580d0190 0x7f77580d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775ca00640 1 -- 172.21.15.100:0/2692005955 >> v1:172.21.15.151:6789/0 conn(0x7f77580c5160 legacy=0x7f77580c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:23.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775ca00640 1 -- 172.21.15.100:0/2692005955 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775806bf30 con 0x7f775815b660 2024-04-23T14:33:23.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775ca00640 1 -- 172.21.15.100:0/2692005955 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f774c001850 con 0x7f775815b660 2024-04-23T14:33:23.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775ca00640 1 -- 172.21.15.100:0/2692005955 >> v1:172.21.15.100:6789/0 conn(0x7f775815b660 legacy=0x7f77580c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:23.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775ca00640 1 --2- 172.21.15.100:0/2692005955 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f77580d0190 0x7f7740003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:23.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775ca00640 1 --2- 172.21.15.100:0/2692005955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77400040e0 0x7f7740004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:23.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775ca00640 1 --2- 172.21.15.100:0/2692005955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7740004d50 0x7f77400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:23.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775ca00640 1 -- 172.21.15.100:0/2692005955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f774000dc90 con 0x7f77400040e0 2024-04-23T14:33:23.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775ca00640 1 -- 172.21.15.100:0/2692005955 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f774000de40 con 0x7f7740004d50 2024-04-23T14:33:23.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775ca00640 1 -- 172.21.15.100:0/2692005955 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f774000e040 con 0x7f77580d0190 2024-04-23T14:33:23.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775d400640 1 --2- 172.21.15.100:0/2692005955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77400040e0 0x7f7740004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:23.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775de00640 1 --2- 172.21.15.100:0/2692005955 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f77580d0190 0x7f7740003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:23.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775e800640 1 --2- 172.21.15.100:0/2692005955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7740004d50 0x7f77400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:23.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775d400640 1 -- 172.21.15.100:0/2692005955 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f77580d0190 msgr2=0x7f7740003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:23.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775d400640 1 --2- 172.21.15.100:0/2692005955 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f77580d0190 0x7f7740003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775d400640 1 -- 172.21.15.100:0/2692005955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7740004d50 msgr2=0x7f77400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:23.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775d400640 1 --2- 172.21.15.100:0/2692005955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7740004d50 0x7f77400051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775d400640 1 -- 172.21.15.100:0/2692005955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7740003530 con 0x7f77400040e0 2024-04-23T14:33:23.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.001+0000 7f775de00640 1 --2- 172.21.15.100:0/2692005955 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f77580d0190 0x7f7740003b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:23.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.005+0000 7f775d400640 1 --2- 172.21.15.100:0/2692005955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77400040e0 0x7f7740004810 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7748009ae0 tx=0x7f774801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81c16407785305e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:23.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.005+0000 7f775ca00640 1 -- 172.21.15.100:0/2692005955 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77480b2070 con 0x7f77400040e0 2024-04-23T14:33:23.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.005+0000 7f775ca00640 1 -- 172.21.15.100:0/2692005955 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77480a5660 con 0x7f77400040e0 2024-04-23T14:33:23.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.005+0000 7f775ca00640 1 -- 172.21.15.100:0/2692005955 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77480096b0 con 0x7f77400040e0 2024-04-23T14:33:23.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.005+0000 7f7760800640 1 -- 172.21.15.100:0/2692005955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77400040e0 msgr2=0x7f7740004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:23.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.005+0000 7f7760800640 1 --2- 172.21.15.100:0/2692005955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77400040e0 0x7f7740004810 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7748009ae0 tx=0x7f774801ca70 comp rx=0 tx=0).stop 2024-04-23T14:33:23.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.005+0000 7f775e800640 1 -- 172.21.15.100:0/2692005955 reap_dead start 2024-04-23T14:33:23.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.005+0000 7f7760800640 1 -- 172.21.15.100:0/2692005955 shutdown_connections 2024-04-23T14:33:23.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.005+0000 7f7760800640 1 -- 172.21.15.100:0/2692005955 >> 172.21.15.100:0/2692005955 conn(0x7f77580b5310 msgr2=0x7f77580b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:23.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.005+0000 7f7760800640 1 -- 172.21.15.100:0/2692005955 shutdown_connections 2024-04-23T14:33:23.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.005+0000 7f7760800640 1 -- 172.21.15.100:0/2692005955 wait complete. 2024-04-23T14:33:23.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f7760800640 1 Processor -- start 2024-04-23T14:33:23.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f7760800640 1 -- start start 2024-04-23T14:33:23.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f7760800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77400040e0 0x7f77580b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:23.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f7760800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f775806f180 0x7f775814e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:23.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f7760800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f775806f600 0x7f775814e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:23.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f7760800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f775806c7d0 con 0x7f77400040e0 2024-04-23T14:33:23.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f7760800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f775806c4f0 con 0x7f775806f600 2024-04-23T14:33:23.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f7760800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f77580c6f30 con 0x7f775806f180 2024-04-23T14:33:23.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f775de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77400040e0 0x7f77580b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:23.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f775de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77400040e0 0x7f77580b0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48174/0 (socket says 172.21.15.100:48174) 2024-04-23T14:33:23.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f775de00640 1 -- 172.21.15.100:0/3676292629 learned_addr learned my addr 172.21.15.100:0/3676292629 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:23.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f775d400640 1 --2- 172.21.15.100:0/3676292629 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f775806f180 0x7f775814e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:23.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f775e800640 1 --2- 172.21.15.100:0/3676292629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f775806f600 0x7f775814e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:23.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f775de00640 1 -- 172.21.15.100:0/3676292629 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f775806f180 msgr2=0x7f775814e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:23.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f775de00640 1 --2- 172.21.15.100:0/3676292629 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f775806f180 0x7f775814e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f775de00640 1 -- 172.21.15.100:0/3676292629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f775806f600 msgr2=0x7f775814e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:23.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f775de00640 1 --2- 172.21.15.100:0/3676292629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f775806f600 0x7f775814e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f775de00640 1 -- 172.21.15.100:0/3676292629 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775815e030 con 0x7f77400040e0 2024-04-23T14:33:23.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f775de00640 1 --2- 172.21.15.100:0/3676292629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77400040e0 0x7f77580b0010 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f774c007360 tx=0x7f774c007820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:23.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f7752000640 1 -- 172.21.15.100:0/3676292629 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c00ea70 con 0x7f77400040e0 2024-04-23T14:33:23.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f7752000640 1 -- 172.21.15.100:0/3676292629 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f774c018070 con 0x7f77400040e0 2024-04-23T14:33:23.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f7752000640 1 -- 172.21.15.100:0/3676292629 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c00ee10 con 0x7f77400040e0 2024-04-23T14:33:23.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f7760800640 1 -- 172.21.15.100:0/3676292629 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f775815e310 con 0x7f77400040e0 2024-04-23T14:33:23.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.009+0000 7f7760800640 1 -- 172.21.15.100:0/3676292629 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f775815e8d0 con 0x7f77400040e0 2024-04-23T14:33:23.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.013+0000 7f7760800640 1 -- 172.21.15.100:0/3676292629 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7758068ad0 con 0x7f77400040e0 2024-04-23T14:33:23.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.013+0000 7f7752000640 1 -- 172.21.15.100:0/3676292629 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f774c01d460 con 0x7f77400040e0 2024-04-23T14:33:23.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.013+0000 7f7752000640 1 --2- 172.21.15.100:0/3676292629 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f773c0683b0 0x7f773c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:23.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.013+0000 7f7752000640 1 -- 172.21.15.100:0/3676292629 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f774c086fe0 con 0x7f77400040e0 2024-04-23T14:33:23.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.013+0000 7f775d400640 1 --2- 172.21.15.100:0/3676292629 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f773c0683b0 0x7f773c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:23.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.013+0000 7f775d400640 1 --2- 172.21.15.100:0/3676292629 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f773c0683b0 0x7f773c06a870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f77480af000 tx=0x7f77480046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:23.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.017+0000 7f7752000640 1 -- 172.21.15.100:0/3676292629 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f77580cee00 con 0x7f77400040e0 2024-04-23T14:33:23.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.185+0000 7f7760800640 1 -- 172.21.15.100:0/3676292629 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f775806d3a0 con 0x7f77400040e0 2024-04-23T14:33:23.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.185+0000 7f7752000640 1 -- 172.21.15.100:0/3676292629 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f774c0571b0 con 0x7f77400040e0 2024-04-23T14:33:23.188 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-04-23T14:33:23.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.185+0000 7f7760800640 1 -- 172.21.15.100:0/3676292629 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f773c0683b0 msgr2=0x7f773c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:23.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.185+0000 7f7760800640 1 --2- 172.21.15.100:0/3676292629 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f773c0683b0 0x7f773c06a870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f77480af000 tx=0x7f77480046d0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.185+0000 7f7760800640 1 -- 172.21.15.100:0/3676292629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77400040e0 msgr2=0x7f77580b0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:23.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.185+0000 7f7760800640 1 --2- 172.21.15.100:0/3676292629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77400040e0 0x7f77580b0010 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f774c007360 tx=0x7f774c007820 comp rx=0 tx=0).stop 2024-04-23T14:33:23.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.185+0000 7f7760800640 1 -- 172.21.15.100:0/3676292629 shutdown_connections 2024-04-23T14:33:23.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.185+0000 7f7760800640 1 --2- 172.21.15.100:0/3676292629 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f773c0683b0 0x7f773c06a870 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.185+0000 7f7760800640 1 --2- 172.21.15.100:0/3676292629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f775806f600 0x7f775814e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.185+0000 7f7760800640 1 --2- 172.21.15.100:0/3676292629 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f775806f180 0x7f775814e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.185+0000 7f7760800640 1 --2- 172.21.15.100:0/3676292629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77400040e0 0x7f77580b0010 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.185+0000 7f7760800640 1 -- 172.21.15.100:0/3676292629 >> 172.21.15.100:0/3676292629 conn(0x7f77580b5310 msgr2=0x7f77580d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:23.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.185+0000 7f7760800640 1 -- 172.21.15.100:0/3676292629 shutdown_connections 2024-04-23T14:33:23.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.185+0000 7f7760800640 1 -- 172.21.15.100:0/3676292629 wait complete. 2024-04-23T14:33:23.209 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi151.front.sepia.ceph.com... 2024-04-23T14:33:23.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:23.209 DEBUG:teuthology.orchestra.run.smithi151:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-23T14:33:23.233 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:33:23.233 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-04-23T14:33:23.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:23.234 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-23T14:33:23.284 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-23T14:33:23.285 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:33:23.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.417+0000 7ff131800640 1 Processor -- start 2024-04-23T14:33:23.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.417+0000 7ff131800640 1 -- start start 2024-04-23T14:33:23.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.417+0000 7ff131800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff12c073610 0x7ff12c0adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:23.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.417+0000 7ff131800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff12c0af080 con 0x7ff12c0b2a10 2024-04-23T14:33:23.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.417+0000 7ff131800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff12c0afbc0 con 0x7ff12c15b640 2024-04-23T14:33:23.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.417+0000 7ff131800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7ff12c15ba20 con 0x7ff12c073610 2024-04-23T14:33:23.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.421+0000 7ff122000640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff12c0b2a10 0x7ff12c06df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58024/0 (socket says 172.21.15.100:58024) 2024-04-23T14:33:23.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.421+0000 7ff122000640 1 -- 172.21.15.100:0/822283874 learned_addr learned my addr 172.21.15.100:0/822283874 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:23.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.421+0000 7ff12b400640 1 --2- 172.21.15.100:0/822283874 >> v2:172.21.15.151:3300/0 conn(0x7ff12c073610 0x7ff12c0adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:23.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.421+0000 7ff12a000640 1 -- 172.21.15.100:0/822283874 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 355499457 0 0) 0x7ff12c0af080 con 0x7ff12c0b2a10 2024-04-23T14:33:23.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.421+0000 7ff12a000640 1 -- 172.21.15.100:0/822283874 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff110005740 con 0x7ff12c0b2a10 2024-04-23T14:33:23.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.421+0000 7ff12b400640 1 -- 172.21.15.100:0/822283874 >> v1:172.21.15.151:6789/0 conn(0x7ff12c15b640 legacy=0x7ff12c072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:23.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.421+0000 7ff12b400640 1 -- 172.21.15.100:0/822283874 >> v1:172.21.15.100:6789/0 conn(0x7ff12c0b2a10 legacy=0x7ff12c06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:23.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.421+0000 7ff12b400640 1 -- 172.21.15.100:0/822283874 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12c0af320 con 0x7ff12c073610 2024-04-23T14:33:23.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.421+0000 7ff12b400640 1 --2- 172.21.15.100:0/822283874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff12c073610 0x7ff12c0adfc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff1240097c0 tx=0x7ff12401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbffb67ad24f124c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:23.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.425+0000 7ff12a000640 1 -- 172.21.15.100:0/822283874 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1240b1070 con 0x7ff12c073610 2024-04-23T14:33:23.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.425+0000 7ff12a000640 1 -- 172.21.15.100:0/822283874 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1240a8050 con 0x7ff12c073610 2024-04-23T14:33:23.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.425+0000 7ff12a000640 1 -- 172.21.15.100:0/822283874 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1240ac860 con 0x7ff12c073610 2024-04-23T14:33:23.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.425+0000 7ff131800640 1 -- 172.21.15.100:0/822283874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff12c073610 msgr2=0x7ff12c0adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:23.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.425+0000 7ff131800640 1 --2- 172.21.15.100:0/822283874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff12c073610 0x7ff12c0adfc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff1240097c0 tx=0x7ff12401c3e0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.425+0000 7ff131800640 1 -- 172.21.15.100:0/822283874 shutdown_connections 2024-04-23T14:33:23.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.425+0000 7ff131800640 1 --2- 172.21.15.100:0/822283874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff12c073610 0x7ff12c0adfc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.425+0000 7ff131800640 1 -- 172.21.15.100:0/822283874 >> 172.21.15.100:0/822283874 conn(0x7ff12c0b1e80 msgr2=0x7ff12c0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:23.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.425+0000 7ff131800640 1 -- 172.21.15.100:0/822283874 shutdown_connections 2024-04-23T14:33:23.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.425+0000 7ff131800640 1 -- 172.21.15.100:0/822283874 wait complete. 2024-04-23T14:33:23.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.425+0000 7ff131800640 1 Processor -- start 2024-04-23T14:33:23.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.425+0000 7ff131800640 1 -- start start 2024-04-23T14:33:23.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.425+0000 7ff131800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff12c073610 0x7ff12c145730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:23.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.425+0000 7ff131800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff12c0b2a10 0x7ff12c14cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:23.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.425+0000 7ff131800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff12c15b640 0x7ff12c1f8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:23.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.425+0000 7ff131800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c06c340 con 0x7ff12c15b640 2024-04-23T14:33:23.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.425+0000 7ff131800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c06f160 con 0x7ff12c073610 2024-04-23T14:33:23.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.425+0000 7ff131800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff12c06c4c0 con 0x7ff12c0b2a10 2024-04-23T14:33:23.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.425+0000 7ff12b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff12c15b640 0x7ff12c1f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:23.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.425+0000 7ff12b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff12c15b640 0x7ff12c1f8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48190/0 (socket says 172.21.15.100:48190) 2024-04-23T14:33:23.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.425+0000 7ff12b400640 1 -- 172.21.15.100:0/4281001965 learned_addr learned my addr 172.21.15.100:0/4281001965 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:23.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.429+0000 7ff12aa00640 1 --2- 172.21.15.100:0/4281001965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff12c073610 0x7ff12c145730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:23.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.429+0000 7ff122000640 1 --2- 172.21.15.100:0/4281001965 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff12c0b2a10 0x7ff12c14cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:23.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.429+0000 7ff12b400640 1 -- 172.21.15.100:0/4281001965 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff12c0b2a10 msgr2=0x7ff12c14cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:23.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.429+0000 7ff12b400640 1 --2- 172.21.15.100:0/4281001965 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff12c0b2a10 0x7ff12c14cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.429+0000 7ff12b400640 1 -- 172.21.15.100:0/4281001965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff12c073610 msgr2=0x7ff12c145730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:23.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.429+0000 7ff12b400640 1 --2- 172.21.15.100:0/4281001965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff12c073610 0x7ff12c145730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.429+0000 7ff12b400640 1 -- 172.21.15.100:0/4281001965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12c1f7ca0 con 0x7ff12c15b640 2024-04-23T14:33:23.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.429+0000 7ff122000640 1 --2- 172.21.15.100:0/4281001965 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff12c0b2a10 0x7ff12c14cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:23.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.429+0000 7ff12b400640 1 --2- 172.21.15.100:0/4281001965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff12c15b640 0x7ff12c1f8460 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff1240af040 tx=0x7ff124019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:23.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.429+0000 7ff123400640 1 -- 172.21.15.100:0/4281001965 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1240b9070 con 0x7ff12c15b640 2024-04-23T14:33:23.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.429+0000 7ff123400640 1 -- 172.21.15.100:0/4281001965 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1240accc0 con 0x7ff12c15b640 2024-04-23T14:33:23.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.429+0000 7ff131800640 1 -- 172.21.15.100:0/4281001965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12c1466c0 con 0x7ff12c15b640 2024-04-23T14:33:23.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.429+0000 7ff123400640 1 -- 172.21.15.100:0/4281001965 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1240b1040 con 0x7ff12c15b640 2024-04-23T14:33:23.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.429+0000 7ff131800640 1 -- 172.21.15.100:0/4281001965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff12c1f7f40 con 0x7ff12c15b640 2024-04-23T14:33:23.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.429+0000 7ff131800640 1 -- 172.21.15.100:0/4281001965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff12c068ad0 con 0x7ff12c15b640 2024-04-23T14:33:23.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.429+0000 7ff123400640 1 -- 172.21.15.100:0/4281001965 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff1240bc070 con 0x7ff12c15b640 2024-04-23T14:33:23.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.429+0000 7ff123400640 1 --2- 172.21.15.100:0/4281001965 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff108068380 0x7ff10806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:23.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.429+0000 7ff123400640 1 -- 172.21.15.100:0/4281001965 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff124120a60 con 0x7ff12c15b640 2024-04-23T14:33:23.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.429+0000 7ff12aa00640 1 --2- 172.21.15.100:0/4281001965 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff108068380 0x7ff10806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:23.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.433+0000 7ff12aa00640 1 --2- 172.21.15.100:0/4281001965 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff108068380 0x7ff10806a840 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff12c15e980 tx=0x7ff11c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:23.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.433+0000 7ff123400640 1 -- 172.21.15.100:0/4281001965 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff124016090 con 0x7ff12c15b640 2024-04-23T14:33:23.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.621+0000 7ff131800640 1 -- 172.21.15.100:0/4281001965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff12c1586e0 con 0x7ff12c15b640 2024-04-23T14:33:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.625+0000 7ff123400640 1 -- 172.21.15.100:0/4281001965 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7ff1240f0c30 con 0x7ff12c15b640 2024-04-23T14:33:23.627 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:23.627 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1456958549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1456958549},{"type":"v1","addr":"172.21.15.100:6836","nonce":1456958549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3655609551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3655609551},{"type":"v1","addr":"172.21.15.100:6837","nonce":3655609551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/4242180972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":4242180972},{"type":"v1","addr":"172.21.15.151:6834","nonce":4242180972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:32:08.976576+0000","modified":"2024-04-23T14:32:34.057740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1533322173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1533322173},{"type":"v1","addr":"172.21.15.151:6835","nonce":1533322173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-23T14:33:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 9 2024-04-23T14:33:23.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.625+0000 7ff131800640 1 -- 172.21.15.100:0/4281001965 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff108068380 msgr2=0x7ff10806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:23.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.625+0000 7ff131800640 1 --2- 172.21.15.100:0/4281001965 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff108068380 0x7ff10806a840 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff12c15e980 tx=0x7ff11c009040 comp rx=0 tx=0).stop 2024-04-23T14:33:23.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.625+0000 7ff131800640 1 -- 172.21.15.100:0/4281001965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff12c15b640 msgr2=0x7ff12c1f8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:23.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.625+0000 7ff131800640 1 --2- 172.21.15.100:0/4281001965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff12c15b640 0x7ff12c1f8460 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff1240af040 tx=0x7ff124019180 comp rx=0 tx=0).stop 2024-04-23T14:33:23.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.625+0000 7ff131800640 1 -- 172.21.15.100:0/4281001965 shutdown_connections 2024-04-23T14:33:23.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.625+0000 7ff131800640 1 --2- 172.21.15.100:0/4281001965 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff108068380 0x7ff10806a840 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.625+0000 7ff131800640 1 --2- 172.21.15.100:0/4281001965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff12c15b640 0x7ff12c1f8460 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.625+0000 7ff131800640 1 --2- 172.21.15.100:0/4281001965 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff12c0b2a10 0x7ff12c14cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.625+0000 7ff131800640 1 --2- 172.21.15.100:0/4281001965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff12c073610 0x7ff12c145730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.625+0000 7ff131800640 1 -- 172.21.15.100:0/4281001965 >> 172.21.15.100:0/4281001965 conn(0x7ff12c0b1e80 msgr2=0x7ff12c0adfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:23.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.625+0000 7ff131800640 1 -- 172.21.15.100:0/4281001965 shutdown_connections 2024-04-23T14:33:23.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.625+0000 7ff131800640 1 -- 172.21.15.100:0/4281001965 wait complete. 2024-04-23T14:33:23.647 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:23.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f218ae00640 1 Processor -- start 2024-04-23T14:33:23.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f218ae00640 1 -- start start 2024-04-23T14:33:23.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f218ae00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f218406fec0 0x7f21840702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:23.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f218ae00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f21840afaf0 0x7f218406f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:23.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f218ae00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f218406e520 con 0x7f218406da40 2024-04-23T14:33:23.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f218ae00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f218406ede0 con 0x7f21840afaf0 2024-04-23T14:33:23.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f218ae00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f21840b3f90 con 0x7f218406fec0 2024-04-23T14:33:23.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2189000640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f218406da40 0x7f21840b3880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58040/0 (socket says 172.21.15.100:58040) 2024-04-23T14:33:23.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2189000640 1 -- 172.21.15.100:0/1695051262 learned_addr learned my addr 172.21.15.100:0/1695051262 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:23.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f218a400640 1 --2- 172.21.15.100:0/1695051262 >> v2:172.21.15.151:3300/0 conn(0x7f21840afaf0 0x7f218406f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:23.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2189a00640 1 --2- 172.21.15.100:0/1695051262 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f218406fec0 0x7f21840702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:23.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2183e00640 1 -- 172.21.15.100:0/1695051262 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2461338676 0 0) 0x7f218406e520 con 0x7f218406da40 2024-04-23T14:33:23.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2183e00640 1 -- 172.21.15.100:0/1695051262 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f216c003530 con 0x7f218406da40 2024-04-23T14:33:23.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2183e00640 1 -- 172.21.15.100:0/1695051262 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1101330882 0 0) 0x7f216c003530 con 0x7f218406da40 2024-04-23T14:33:23.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2183e00640 1 -- 172.21.15.100:0/1695051262 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f218406fec0 msgr2=0x7f21840702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:23.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2183e00640 1 --2- 172.21.15.100:0/1695051262 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f218406fec0 0x7f21840702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2183e00640 1 -- 172.21.15.100:0/1695051262 >> v2:172.21.15.151:3300/0 conn(0x7f21840afaf0 msgr2=0x7f218406f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:23.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2183e00640 1 --2- 172.21.15.100:0/1695051262 >> v2:172.21.15.151:3300/0 conn(0x7f21840afaf0 0x7f218406f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2183e00640 1 -- 172.21.15.100:0/1695051262 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21840707e0 con 0x7f218406da40 2024-04-23T14:33:23.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2183e00640 1 -- 172.21.15.100:0/1695051262 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f2170001680 con 0x7f218406da40 2024-04-23T14:33:23.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2183e00640 1 -- 172.21.15.100:0/1695051262 >> v1:172.21.15.100:6789/0 conn(0x7f218406da40 legacy=0x7f21840b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:23.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2183e00640 1 --2- 172.21.15.100:0/1695051262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f216c0039c0 0x7f216c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:23.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2183e00640 1 --2- 172.21.15.100:0/1695051262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f216c004810 0x7f216c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:23.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2183e00640 1 --2- 172.21.15.100:0/1695051262 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f218406fec0 0x7f216c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:23.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2183e00640 1 -- 172.21.15.100:0/1695051262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f218406ede0 con 0x7f216c004810 2024-04-23T14:33:23.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2183e00640 1 -- 172.21.15.100:0/1695051262 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f21840b3f90 con 0x7f216c0039c0 2024-04-23T14:33:23.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2183e00640 1 -- 172.21.15.100:0/1695051262 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f216c00b9d0 con 0x7f218406fec0 2024-04-23T14:33:23.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2189000640 1 --2- 172.21.15.100:0/1695051262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f216c004810 0x7f216c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:23.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2189a00640 1 --2- 172.21.15.100:0/1695051262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f216c0039c0 0x7f216c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:23.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f218a400640 1 --2- 172.21.15.100:0/1695051262 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f218406fec0 0x7f216c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:23.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2189000640 1 -- 172.21.15.100:0/1695051262 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f218406fec0 msgr2=0x7f216c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:23.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2189000640 1 --2- 172.21.15.100:0/1695051262 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f218406fec0 0x7f216c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2189000640 1 -- 172.21.15.100:0/1695051262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f216c0039c0 msgr2=0x7f216c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:23.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2189000640 1 --2- 172.21.15.100:0/1695051262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f216c0039c0 0x7f216c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.781+0000 7f2189000640 1 -- 172.21.15.100:0/1695051262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216c003530 con 0x7f216c004810 2024-04-23T14:33:23.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.785+0000 7f2189000640 1 --2- 172.21.15.100:0/1695051262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f216c004810 0x7f216c004c90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2170001060 tx=0x7f217001f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261cde0949f773fe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:23.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.785+0000 7f2183e00640 1 -- 172.21.15.100:0/1695051262 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2170007d80 con 0x7f216c004810 2024-04-23T14:33:23.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.785+0000 7f2183e00640 1 -- 172.21.15.100:0/1695051262 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2170005b00 con 0x7f216c004810 2024-04-23T14:33:23.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.785+0000 7f2183e00640 1 -- 172.21.15.100:0/1695051262 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2170007070 con 0x7f216c004810 2024-04-23T14:33:23.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.785+0000 7f218ae00640 1 -- 172.21.15.100:0/1695051262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f216c004810 msgr2=0x7f216c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:23.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.785+0000 7f218ae00640 1 --2- 172.21.15.100:0/1695051262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f216c004810 0x7f216c004c90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2170001060 tx=0x7f217001f410 comp rx=0 tx=0).stop 2024-04-23T14:33:23.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.785+0000 7f218a400640 1 -- 172.21.15.100:0/1695051262 reap_dead start 2024-04-23T14:33:23.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.785+0000 7f218ae00640 1 -- 172.21.15.100:0/1695051262 shutdown_connections 2024-04-23T14:33:23.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.785+0000 7f218ae00640 1 -- 172.21.15.100:0/1695051262 >> 172.21.15.100:0/1695051262 conn(0x7f21840d0d10 msgr2=0x7f21840d1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:23.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.785+0000 7f218ae00640 1 -- 172.21.15.100:0/1695051262 shutdown_connections 2024-04-23T14:33:23.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.785+0000 7f218ae00640 1 -- 172.21.15.100:0/1695051262 wait complete. 2024-04-23T14:33:23.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f218ae00640 1 Processor -- start 2024-04-23T14:33:23.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f218ae00640 1 -- start start 2024-04-23T14:33:23.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f218ae00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f216c004810 0x7f21840aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:23.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f218ae00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2184145cd0 0x7f2184146130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:23.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f218ae00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21841f7950 0x7f21841f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:23.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f218ae00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f218415fb80 con 0x7f216c004810 2024-04-23T14:33:23.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f218ae00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f218406ea70 con 0x7f21841f7950 2024-04-23T14:33:23.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f218ae00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f218415c550 con 0x7f2184145cd0 2024-04-23T14:33:23.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f2189a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f216c004810 0x7f21840aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:23.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f2189a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f216c004810 0x7f21840aec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48208/0 (socket says 172.21.15.100:48208) 2024-04-23T14:33:23.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f2189a00640 1 -- 172.21.15.100:0/2968108398 learned_addr learned my addr 172.21.15.100:0/2968108398 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:23.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f2189000640 1 --2- 172.21.15.100:0/2968108398 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2184145cd0 0x7f2184146130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:23.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f218a400640 1 --2- 172.21.15.100:0/2968108398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21841f7950 0x7f21841f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:23.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f2189a00640 1 -- 172.21.15.100:0/2968108398 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2184145cd0 msgr2=0x7f2184146130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:23.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f2189a00640 1 --2- 172.21.15.100:0/2968108398 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2184145cd0 0x7f2184146130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f2189a00640 1 -- 172.21.15.100:0/2968108398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21841f7950 msgr2=0x7f21841f7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:23.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f2189a00640 1 --2- 172.21.15.100:0/2968108398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21841f7950 0x7f21841f7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f2189a00640 1 -- 172.21.15.100:0/2968108398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21841f86c0 con 0x7f216c004810 2024-04-23T14:33:23.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f2189a00640 1 --2- 172.21.15.100:0/2968108398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f216c004810 0x7f21840aec40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2178009f90 tx=0x7f217800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:23.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f2181600640 1 -- 172.21.15.100:0/2968108398 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217800fea0 con 0x7f216c004810 2024-04-23T14:33:23.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f2181600640 1 -- 172.21.15.100:0/2968108398 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2178015630 con 0x7f216c004810 2024-04-23T14:33:23.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f218ae00640 1 -- 172.21.15.100:0/2968108398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2184148c90 con 0x7f216c004810 2024-04-23T14:33:23.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f2181600640 1 -- 172.21.15.100:0/2968108398 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2178015a20 con 0x7f216c004810 2024-04-23T14:33:23.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.789+0000 7f218ae00640 1 -- 172.21.15.100:0/2968108398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2184149250 con 0x7f216c004810 2024-04-23T14:33:23.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.793+0000 7f218ae00640 1 -- 172.21.15.100:0/2968108398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2184075c20 con 0x7f216c004810 2024-04-23T14:33:23.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.793+0000 7f2181600640 1 -- 172.21.15.100:0/2968108398 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2178015bc0 con 0x7f216c004810 2024-04-23T14:33:23.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.793+0000 7f2181600640 1 --2- 172.21.15.100:0/2968108398 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2164068380 0x7f216406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:23.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.793+0000 7f2181600640 1 -- 172.21.15.100:0/2968108398 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2178087fb0 con 0x7f216c004810 2024-04-23T14:33:23.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.793+0000 7f2189000640 1 --2- 172.21.15.100:0/2968108398 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2164068380 0x7f216406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:23.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.793+0000 7f2189000640 1 --2- 172.21.15.100:0/2968108398 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2164068380 0x7f216406a840 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f21700ae000 tx=0x7f21700b4040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:23.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.797+0000 7f2181600640 1 -- 172.21.15.100:0/2968108398 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2178019030 con 0x7f216c004810 2024-04-23T14:33:23.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.957+0000 7f218ae00640 1 -- 172.21.15.100:0/2968108398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21840b01d0 con 0x7f216c004810 2024-04-23T14:33:23.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.957+0000 7f2181600640 1 -- 172.21.15.100:0/2968108398 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18859 (secure 0 0 0) 0x7f2178010040 con 0x7f216c004810 2024-04-23T14:33:23.961 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:23.961 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:49.784608+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.961+0000 7f218ae00640 1 -- 172.21.15.100:0/2968108398 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2164068380 msgr2=0x7f216406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.961+0000 7f218ae00640 1 --2- 172.21.15.100:0/2968108398 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2164068380 0x7f216406a840 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f21700ae000 tx=0x7f21700b4040 comp rx=0 tx=0).stop 2024-04-23T14:33:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.961+0000 7f218ae00640 1 -- 172.21.15.100:0/2968108398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f216c004810 msgr2=0x7f21840aec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.961+0000 7f218ae00640 1 --2- 172.21.15.100:0/2968108398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f216c004810 0x7f21840aec40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2178009f90 tx=0x7f217800f580 comp rx=0 tx=0).stop 2024-04-23T14:33:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.961+0000 7f218ae00640 1 -- 172.21.15.100:0/2968108398 shutdown_connections 2024-04-23T14:33:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.961+0000 7f218ae00640 1 --2- 172.21.15.100:0/2968108398 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2164068380 0x7f216406a840 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.961+0000 7f218ae00640 1 --2- 172.21.15.100:0/2968108398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21841f7950 0x7f21841f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.961+0000 7f218ae00640 1 --2- 172.21.15.100:0/2968108398 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2184145cd0 0x7f2184146130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.961+0000 7f218ae00640 1 --2- 172.21.15.100:0/2968108398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f216c004810 0x7f21840aec40 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:23.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.961+0000 7f218ae00640 1 -- 172.21.15.100:0/2968108398 >> 172.21.15.100:0/2968108398 conn(0x7f21840d0d10 msgr2=0x7f21840b4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:23.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.961+0000 7f218ae00640 1 -- 172.21.15.100:0/2968108398 shutdown_connections 2024-04-23T14:33:23.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:23.961+0000 7f218ae00640 1 -- 172.21.15.100:0/2968108398 wait complete. 2024-04-23T14:33:23.981 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:32:06.835560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:32:07.930489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-23T14:32:11.636814+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-23T14:33:23.982 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:33:24.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.117+0000 7f350be00640 1 Processor -- start 2024-04-23T14:33:24.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.117+0000 7f350be00640 1 -- start start 2024-04-23T14:33:24.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.117+0000 7f350be00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f350c0b19a0 0x7f350c0d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:24.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.117+0000 7f350be00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f350c0d0790 con 0x7f350c0d01f0 2024-04-23T14:33:24.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.117+0000 7f350be00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f350c0d0a70 con 0x7f350c156930 2024-04-23T14:33:24.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.117+0000 7f350be00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f350c0d0d50 con 0x7f350c0b19a0 2024-04-23T14:33:24.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.117+0000 7f350b400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f350c0d01f0 0x7f350c0b3840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58056/0 (socket says 172.21.15.100:58056) 2024-04-23T14:33:24.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.117+0000 7f350b400640 1 -- 172.21.15.100:0/2201792109 learned_addr learned my addr 172.21.15.100:0/2201792109 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:24.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.117+0000 7f3502000640 1 --2- 172.21.15.100:0/2201792109 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f350c0b19a0 0x7f350c0d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:24.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.117+0000 7f350a000640 1 -- 172.21.15.100:0/2201792109 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3043768992 0 0) 0x7f350c0d0790 con 0x7f350c0d01f0 2024-04-23T14:33:24.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.117+0000 7f350a000640 1 -- 172.21.15.100:0/2201792109 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34f0003530 con 0x7f350c0d01f0 2024-04-23T14:33:24.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.117+0000 7f350a000640 1 -- 172.21.15.100:0/2201792109 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4246694837 0 0) 0x7f350c0d0a70 con 0x7f350c156930 2024-04-23T14:33:24.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.117+0000 7f350a000640 1 -- 172.21.15.100:0/2201792109 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f350c0d0790 con 0x7f350c156930 2024-04-23T14:33:24.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.117+0000 7f3502000640 1 -- 172.21.15.100:0/2201792109 >> v1:172.21.15.151:6789/0 conn(0x7f350c156930 legacy=0x7f350c15d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:24.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.117+0000 7f3502000640 1 -- 172.21.15.100:0/2201792109 >> v1:172.21.15.100:6789/0 conn(0x7f350c0d01f0 legacy=0x7f350c0b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:24.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.117+0000 7f350a000640 1 -- 172.21.15.100:0/2201792109 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4081711696 0 0) 0x7f34f0003530 con 0x7f350c0d01f0 2024-04-23T14:33:24.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.117+0000 7f3502000640 1 -- 172.21.15.100:0/2201792109 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350c0d1030 con 0x7f350c0b19a0 2024-04-23T14:33:24.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.121+0000 7f3502000640 1 --2- 172.21.15.100:0/2201792109 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f350c0b19a0 0x7f350c0d15f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f34f40097f0 tx=0x7f34f401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89bb51322ab46746 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:24.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.121+0000 7f350a000640 1 -- 172.21.15.100:0/2201792109 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f40b0070 con 0x7f350c0b19a0 2024-04-23T14:33:24.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.121+0000 7f350a000640 1 -- 172.21.15.100:0/2201792109 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f40ab400 con 0x7f350c0b19a0 2024-04-23T14:33:24.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.121+0000 7f350a000640 1 -- 172.21.15.100:0/2201792109 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504001a00 con 0x7f350c0b19a0 2024-04-23T14:33:24.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.121+0000 7f350be00640 1 -- 172.21.15.100:0/2201792109 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f350c0b19a0 msgr2=0x7f350c0d15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:24.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.121+0000 7f350be00640 1 --2- 172.21.15.100:0/2201792109 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f350c0b19a0 0x7f350c0d15f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f34f40097f0 tx=0x7f34f401c400 comp rx=0 tx=0).stop 2024-04-23T14:33:24.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.121+0000 7f350be00640 1 -- 172.21.15.100:0/2201792109 shutdown_connections 2024-04-23T14:33:24.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.121+0000 7f350be00640 1 --2- 172.21.15.100:0/2201792109 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f350c0b19a0 0x7f350c0d15f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.121+0000 7f350be00640 1 -- 172.21.15.100:0/2201792109 >> 172.21.15.100:0/2201792109 conn(0x7f350c072420 msgr2=0x7f350c0ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:24.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.121+0000 7f350be00640 1 -- 172.21.15.100:0/2201792109 shutdown_connections 2024-04-23T14:33:24.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.121+0000 7f350be00640 1 -- 172.21.15.100:0/2201792109 wait complete. 2024-04-23T14:33:24.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f350be00640 1 Processor -- start 2024-04-23T14:33:24.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f350be00640 1 -- start start 2024-04-23T14:33:24.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f350be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350c0b19a0 0x7f350c1541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:24.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f350be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f350c0d01f0 0x7f350c14e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:24.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f350be00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f350c156930 0x7f350c159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:24.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f350be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f350c16a350 con 0x7f350c0b19a0 2024-04-23T14:33:24.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f350be00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f350c0d1f40 con 0x7f350c0d01f0 2024-04-23T14:33:24.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f350be00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f350c16a4d0 con 0x7f350c156930 2024-04-23T14:33:24.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f350aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350c0b19a0 0x7f350c1541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:24.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f350aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350c0b19a0 0x7f350c1541a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48218/0 (socket says 172.21.15.100:48218) 2024-04-23T14:33:24.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f350aa00640 1 -- 172.21.15.100:0/3421686393 learned_addr learned my addr 172.21.15.100:0/3421686393 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:24.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f3502000640 1 --2- 172.21.15.100:0/3421686393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f350c0d01f0 0x7f350c14e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:24.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f350b400640 1 --2- 172.21.15.100:0/3421686393 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f350c156930 0x7f350c159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:24.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f350aa00640 1 -- 172.21.15.100:0/3421686393 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f350c156930 msgr2=0x7f350c159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:24.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f350aa00640 1 --2- 172.21.15.100:0/3421686393 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f350c156930 0x7f350c159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f350aa00640 1 -- 172.21.15.100:0/3421686393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f350c0d01f0 msgr2=0x7f350c14e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:24.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f350aa00640 1 --2- 172.21.15.100:0/3421686393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f350c0d01f0 0x7f350c14e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f350aa00640 1 -- 172.21.15.100:0/3421686393 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350c153100 con 0x7f350c0b19a0 2024-04-23T14:33:24.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f350b400640 1 --2- 172.21.15.100:0/3421686393 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f350c156930 0x7f350c159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:24.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f3502000640 1 --2- 172.21.15.100:0/3421686393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f350c0d01f0 0x7f350c14e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:24.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f350aa00640 1 --2- 172.21.15.100:0/3421686393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350c0b19a0 0x7f350c1541a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00c460 tx=0x7f34fc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:24.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f3503400640 1 -- 172.21.15.100:0/3421686393 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc0052d0 con 0x7f350c0b19a0 2024-04-23T14:33:24.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f3503400640 1 -- 172.21.15.100:0/3421686393 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34fc005490 con 0x7f350c0b19a0 2024-04-23T14:33:24.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f3503400640 1 -- 172.21.15.100:0/3421686393 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc012850 con 0x7f350c0b19a0 2024-04-23T14:33:24.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f350be00640 1 -- 172.21.15.100:0/3421686393 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f350c1533e0 con 0x7f350c0b19a0 2024-04-23T14:33:24.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.125+0000 7f350be00640 1 -- 172.21.15.100:0/3421686393 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f350c1539a0 con 0x7f350c0b19a0 2024-04-23T14:33:24.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.129+0000 7f350be00640 1 -- 172.21.15.100:0/3421686393 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f350c068ad0 con 0x7f350c0b19a0 2024-04-23T14:33:24.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.129+0000 7f3503400640 1 -- 172.21.15.100:0/3421686393 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f34fc0129f0 con 0x7f350c0b19a0 2024-04-23T14:33:24.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.129+0000 7f3503400640 1 --2- 172.21.15.100:0/3421686393 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f34e80683b0 0x7f34e806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:24.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.129+0000 7f3503400640 1 -- 172.21.15.100:0/3421686393 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f34fc086a10 con 0x7f350c0b19a0 2024-04-23T14:33:24.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.129+0000 7f3502000640 1 --2- 172.21.15.100:0/3421686393 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f34e80683b0 0x7f34e806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:24.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.129+0000 7f3502000640 1 --2- 172.21.15.100:0/3421686393 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f34e80683b0 0x7f34e806a870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f34f40ad000 tx=0x7f34f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:24.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.133+0000 7f3503400640 1 -- 172.21.15.100:0/3421686393 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f34fc056b30 con 0x7f350c0b19a0 2024-04-23T14:33:24.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.289+0000 7f350be00640 1 -- 172.21.15.100:0/3421686393 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f350c0b31b0 con 0x7f350c0b19a0 2024-04-23T14:33:24.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.289+0000 7f3503400640 1 -- 172.21.15.100:0/3421686393 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f34fc056430 con 0x7f350c0b19a0 2024-04-23T14:33:24.293 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:33:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.293+0000 7f350be00640 1 -- 172.21.15.100:0/3421686393 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f34e80683b0 msgr2=0x7f34e806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.293+0000 7f350be00640 1 --2- 172.21.15.100:0/3421686393 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f34e80683b0 0x7f34e806a870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f34f40ad000 tx=0x7f34f4008040 comp rx=0 tx=0).stop 2024-04-23T14:33:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.293+0000 7f350be00640 1 -- 172.21.15.100:0/3421686393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350c0b19a0 msgr2=0x7f350c1541a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.293+0000 7f350be00640 1 --2- 172.21.15.100:0/3421686393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350c0b19a0 0x7f350c1541a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00c460 tx=0x7f34fc00c920 comp rx=0 tx=0).stop 2024-04-23T14:33:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.293+0000 7f350be00640 1 -- 172.21.15.100:0/3421686393 shutdown_connections 2024-04-23T14:33:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.293+0000 7f350be00640 1 --2- 172.21.15.100:0/3421686393 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f34e80683b0 0x7f34e806a870 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.293+0000 7f350be00640 1 --2- 172.21.15.100:0/3421686393 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f350c156930 0x7f350c159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.293+0000 7f350be00640 1 --2- 172.21.15.100:0/3421686393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f350c0d01f0 0x7f350c14e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.293+0000 7f350be00640 1 --2- 172.21.15.100:0/3421686393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350c0b19a0 0x7f350c1541a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.293+0000 7f350be00640 1 -- 172.21.15.100:0/3421686393 >> 172.21.15.100:0/3421686393 conn(0x7f350c072420 msgr2=0x7f350c0cc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:24.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.293+0000 7f350be00640 1 -- 172.21.15.100:0/3421686393 shutdown_connections 2024-04-23T14:33:24.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.293+0000 7f350be00640 1 -- 172.21.15.100:0/3421686393 wait complete. 2024-04-23T14:33:24.314 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:33:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.449+0000 7fc24e600640 1 Processor -- start 2024-04-23T14:33:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.449+0000 7fc24e600640 1 -- start start 2024-04-23T14:33:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.449+0000 7fc24e600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc24815c7a0 0x7fc2480af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.449+0000 7fc24e600640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc24806bb20 0x7fc2480ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.449+0000 7fc24e600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc2480cd130 0x7fc24806e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.449+0000 7fc24e600640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc24806d450 con 0x7fc24815c7a0 2024-04-23T14:33:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.449+0000 7fc24e600640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fc2480cd580 con 0x7fc24806bb20 2024-04-23T14:33:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.449+0000 7fc24e600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc24806cbb0 con 0x7fc2480cd130 2024-04-23T14:33:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.449+0000 7fc247400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc24815c7a0 0x7fc2480af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.449+0000 7fc247400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc24815c7a0 0x7fc2480af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48222/0 (socket says 172.21.15.100:48222) 2024-04-23T14:33:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.449+0000 7fc247400640 1 -- 172.21.15.100:0/257581379 learned_addr learned my addr 172.21.15.100:0/257581379 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.449+0000 7fc246a00640 1 --2- 172.21.15.100:0/257581379 >> v2:172.21.15.151:3300/0 conn(0x7fc24806bb20 0x7fc2480ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.449+0000 7fc247e00640 1 --2- 172.21.15.100:0/257581379 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc2480cd130 0x7fc24806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.449+0000 7fc247400640 1 -- 172.21.15.100:0/257581379 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc2480cd130 msgr2=0x7fc24806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.449+0000 7fc247400640 1 --2- 172.21.15.100:0/257581379 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc2480cd130 0x7fc24806e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.449+0000 7fc247400640 1 -- 172.21.15.100:0/257581379 >> v2:172.21.15.151:3300/0 conn(0x7fc24806bb20 msgr2=0x7fc2480ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:24.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.449+0000 7fc247400640 1 --2- 172.21.15.100:0/257581379 >> v2:172.21.15.151:3300/0 conn(0x7fc24806bb20 0x7fc2480ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.449+0000 7fc247400640 1 -- 172.21.15.100:0/257581379 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc24806ce90 con 0x7fc24815c7a0 2024-04-23T14:33:24.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.453+0000 7fc247400640 1 --2- 172.21.15.100:0/257581379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc24815c7a0 0x7fc2480af190 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc2480d1280 tx=0x7fc23801c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40943974846d628 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:24.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.453+0000 7fc246000640 1 -- 172.21.15.100:0/257581379 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2380b0070 con 0x7fc24815c7a0 2024-04-23T14:33:24.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.453+0000 7fc246000640 1 -- 172.21.15.100:0/257581379 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2380ab600 con 0x7fc24815c7a0 2024-04-23T14:33:24.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.453+0000 7fc246000640 1 -- 172.21.15.100:0/257581379 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2380ab9d0 con 0x7fc24815c7a0 2024-04-23T14:33:24.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.453+0000 7fc24e600640 1 -- 172.21.15.100:0/257581379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc24815c7a0 msgr2=0x7fc2480af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:24.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.453+0000 7fc24e600640 1 --2- 172.21.15.100:0/257581379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc24815c7a0 0x7fc2480af190 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc2480d1280 tx=0x7fc23801c1d0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.453+0000 7fc24e600640 1 -- 172.21.15.100:0/257581379 shutdown_connections 2024-04-23T14:33:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.453+0000 7fc24e600640 1 --2- 172.21.15.100:0/257581379 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc2480cd130 0x7fc24806e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.453+0000 7fc24e600640 1 --2- 172.21.15.100:0/257581379 >> v2:172.21.15.151:3300/0 conn(0x7fc24806bb20 0x7fc2480ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.453+0000 7fc24e600640 1 --2- 172.21.15.100:0/257581379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc24815c7a0 0x7fc2480af190 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.453+0000 7fc24e600640 1 -- 172.21.15.100:0/257581379 >> 172.21.15.100:0/257581379 conn(0x7fc2480b5200 msgr2=0x7fc2480b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.453+0000 7fc24e600640 1 -- 172.21.15.100:0/257581379 shutdown_connections 2024-04-23T14:33:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.453+0000 7fc24e600640 1 -- 172.21.15.100:0/257581379 wait complete. 2024-04-23T14:33:24.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.453+0000 7fc24e600640 1 Processor -- start 2024-04-23T14:33:24.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.453+0000 7fc24e600640 1 -- start start 2024-04-23T14:33:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc24e600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc24806bb20 0x7fc24815b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc24e600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2480cd130 0x7fc248154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc24e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc24815c7a0 0x7fc2480c5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc24e600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc2480ad800 con 0x7fc24815c7a0 2024-04-23T14:33:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc24e600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc2480ad680 con 0x7fc2480cd130 2024-04-23T14:33:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc24e600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc2480ad980 con 0x7fc24806bb20 2024-04-23T14:33:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc247e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc24815c7a0 0x7fc2480c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc247e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc24815c7a0 0x7fc2480c5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48224/0 (socket says 172.21.15.100:48224) 2024-04-23T14:33:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc247e00640 1 -- 172.21.15.100:0/150942482 learned_addr learned my addr 172.21.15.100:0/150942482 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc246a00640 1 --2- 172.21.15.100:0/150942482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2480cd130 0x7fc248154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc247400640 1 --2- 172.21.15.100:0/150942482 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc24806bb20 0x7fc24815b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc247e00640 1 -- 172.21.15.100:0/150942482 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc24806bb20 msgr2=0x7fc24815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc247e00640 1 --2- 172.21.15.100:0/150942482 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc24806bb20 0x7fc24815b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc247e00640 1 -- 172.21.15.100:0/150942482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2480cd130 msgr2=0x7fc248154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc247e00640 1 --2- 172.21.15.100:0/150942482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2480cd130 0x7fc248154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc247e00640 1 -- 172.21.15.100:0/150942482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2481598f0 con 0x7fc24815c7a0 2024-04-23T14:33:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc247e00640 1 --2- 172.21.15.100:0/150942482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc24815c7a0 0x7fc2480c5610 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc240007e70 tx=0x7fc24000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc23f400640 1 -- 172.21.15.100:0/150942482 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240014620 con 0x7fc24815c7a0 2024-04-23T14:33:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc23f400640 1 -- 172.21.15.100:0/150942482 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc240019070 con 0x7fc24815c7a0 2024-04-23T14:33:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc24e600640 1 -- 172.21.15.100:0/150942482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc248159bd0 con 0x7fc24815c7a0 2024-04-23T14:33:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc23f400640 1 -- 172.21.15.100:0/150942482 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240014a20 con 0x7fc24815c7a0 2024-04-23T14:33:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc24e600640 1 -- 172.21.15.100:0/150942482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc248073b90 con 0x7fc24815c7a0 2024-04-23T14:33:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc24e600640 1 -- 172.21.15.100:0/150942482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc248068ad0 con 0x7fc24815c7a0 2024-04-23T14:33:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.457+0000 7fc23f400640 1 -- 172.21.15.100:0/150942482 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc240014bc0 con 0x7fc24815c7a0 2024-04-23T14:33:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.461+0000 7fc23f400640 1 --2- 172.21.15.100:0/150942482 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc2280683b0 0x7fc22806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.461+0000 7fc23f400640 1 -- 172.21.15.100:0/150942482 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc2400874a0 con 0x7fc24815c7a0 2024-04-23T14:33:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.461+0000 7fc247400640 1 --2- 172.21.15.100:0/150942482 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc2280683b0 0x7fc22806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.461+0000 7fc247400640 1 --2- 172.21.15.100:0/150942482 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc2280683b0 0x7fc22806a870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc2380ad000 tx=0x7fc2380a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:24.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.465+0000 7fc23f400640 1 -- 172.21.15.100:0/150942482 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc240057000 con 0x7fc24815c7a0 2024-04-23T14:33:24.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.621+0000 7fc24e600640 1 -- 172.21.15.100:0/150942482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc248166e80 con 0x7fc24815c7a0 2024-04-23T14:33:24.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.621+0000 7fc23f400640 1 -- 172.21.15.100:0/150942482 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fc24005b300 con 0x7fc24815c7a0 2024-04-23T14:33:24.624 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:24.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.621+0000 7fc24e600640 1 -- 172.21.15.100:0/150942482 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc2280683b0 msgr2=0x7fc22806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:24.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.621+0000 7fc24e600640 1 --2- 172.21.15.100:0/150942482 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc2280683b0 0x7fc22806a870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc2380ad000 tx=0x7fc2380a6000 comp rx=0 tx=0).stop 2024-04-23T14:33:24.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.621+0000 7fc24e600640 1 -- 172.21.15.100:0/150942482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc24815c7a0 msgr2=0x7fc2480c5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:24.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.621+0000 7fc24e600640 1 --2- 172.21.15.100:0/150942482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc24815c7a0 0x7fc2480c5610 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc240007e70 tx=0x7fc24000e4b0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.621+0000 7fc24e600640 1 -- 172.21.15.100:0/150942482 shutdown_connections 2024-04-23T14:33:24.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.621+0000 7fc24e600640 1 --2- 172.21.15.100:0/150942482 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc2280683b0 0x7fc22806a870 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.621+0000 7fc24e600640 1 --2- 172.21.15.100:0/150942482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc24815c7a0 0x7fc2480c5610 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.621+0000 7fc24e600640 1 --2- 172.21.15.100:0/150942482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2480cd130 0x7fc248154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.621+0000 7fc24e600640 1 --2- 172.21.15.100:0/150942482 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc24806bb20 0x7fc24815b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.621+0000 7fc24e600640 1 -- 172.21.15.100:0/150942482 >> 172.21.15.100:0/150942482 conn(0x7fc2480b5200 msgr2=0x7fc248165d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:24.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.621+0000 7fc24e600640 1 -- 172.21.15.100:0/150942482 shutdown_connections 2024-04-23T14:33:24.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.621+0000 7fc24e600640 1 -- 172.21.15.100:0/150942482 wait complete. 2024-04-23T14:33:24.645 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:33:24.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.781+0000 7faa05c00640 1 Processor -- start 2024-04-23T14:33:24.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.781+0000 7faa05c00640 1 -- start start 2024-04-23T14:33:24.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.785+0000 7faa05c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa00156930 0x7faa0015d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:24.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.785+0000 7faa05c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faa000d1d00 0x7faa000b3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:24.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.785+0000 7faa05c00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa0015d9b0 con 0x7faa000b19a0 2024-04-23T14:33:24.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.785+0000 7faa05c00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7faa000d0790 con 0x7faa000d1d00 2024-04-23T14:33:24.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.785+0000 7faa05c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7faa000d0a70 con 0x7faa00156930 2024-04-23T14:33:24.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.785+0000 7faa05200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faa000d1d00 0x7faa000b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:24.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.785+0000 7faa05200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faa000d1d00 0x7faa000b3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48226/0 (socket says 172.21.15.100:48226) 2024-04-23T14:33:24.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.785+0000 7faa05200640 1 -- 172.21.15.100:0/224730790 learned_addr learned my addr 172.21.15.100:0/224730790 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:24.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.785+0000 7fa9ffe00640 1 --2- 172.21.15.100:0/224730790 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa00156930 0x7faa0015d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:24.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.785+0000 7faa05200640 1 -- 172.21.15.100:0/224730790 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa00156930 msgr2=0x7faa0015d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:24.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.785+0000 7faa05200640 1 --2- 172.21.15.100:0/224730790 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa00156930 0x7faa0015d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.785+0000 7faa05200640 1 -- 172.21.15.100:0/224730790 >> v1:172.21.15.151:6789/0 conn(0x7faa000b19a0 legacy=0x7faa000d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:24.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.785+0000 7faa05200640 1 -- 172.21.15.100:0/224730790 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa000d0d50 con 0x7faa000d1d00 2024-04-23T14:33:24.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.785+0000 7fa9ffe00640 1 --2- 172.21.15.100:0/224730790 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa00156930 0x7faa0015d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:24.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.785+0000 7faa05200640 1 --2- 172.21.15.100:0/224730790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa000d1d00 0x7faa000b3840 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009e50 tx=0x7fa9f801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af84798f868ac9a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:24.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7fa9ff400640 1 -- 172.21.15.100:0/224730790 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80a5780 con 0x7faa000d1d00 2024-04-23T14:33:24.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7fa9ff400640 1 -- 172.21.15.100:0/224730790 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f80a5920 con 0x7faa000d1d00 2024-04-23T14:33:24.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7fa9ff400640 1 -- 172.21.15.100:0/224730790 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80a5cf0 con 0x7faa000d1d00 2024-04-23T14:33:24.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7faa05c00640 1 -- 172.21.15.100:0/224730790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa000d1d00 msgr2=0x7faa000b3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:24.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7faa05c00640 1 --2- 172.21.15.100:0/224730790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa000d1d00 0x7faa000b3840 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009e50 tx=0x7fa9f801c9f0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7faa05c00640 1 -- 172.21.15.100:0/224730790 shutdown_connections 2024-04-23T14:33:24.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7faa05c00640 1 --2- 172.21.15.100:0/224730790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa000d1d00 0x7faa000b3840 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7faa05c00640 1 --2- 172.21.15.100:0/224730790 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa00156930 0x7faa0015d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7faa05c00640 1 -- 172.21.15.100:0/224730790 >> 172.21.15.100:0/224730790 conn(0x7faa00072420 msgr2=0x7faa000ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:24.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7faa05c00640 1 -- 172.21.15.100:0/224730790 shutdown_connections 2024-04-23T14:33:24.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7faa05c00640 1 -- 172.21.15.100:0/224730790 wait complete. 2024-04-23T14:33:24.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7faa05c00640 1 Processor -- start 2024-04-23T14:33:24.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7faa05c00640 1 -- start start 2024-04-23T14:33:24.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7faa05c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa000b19a0 0x7faa0006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:24.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7faa05c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa000d1d00 0x7faa00155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:24.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7faa05c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa00156930 0x7faa0006f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:24.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7faa05c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faa001694c0 con 0x7faa00156930 2024-04-23T14:33:24.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7faa05c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faa00169340 con 0x7faa000b19a0 2024-04-23T14:33:24.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7faa05c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7faa00169640 con 0x7faa000d1d00 2024-04-23T14:33:24.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7faa05200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa00156930 0x7faa0006f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:24.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7faa05200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa00156930 0x7faa0006f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48228/0 (socket says 172.21.15.100:48228) 2024-04-23T14:33:24.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7faa05200640 1 -- 172.21.15.100:0/2495859539 learned_addr learned my addr 172.21.15.100:0/2495859539 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:24.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7fa9ffe00640 1 --2- 172.21.15.100:0/2495859539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa000b19a0 0x7faa0006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:24.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.789+0000 7fa9f7400640 1 --2- 172.21.15.100:0/2495859539 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa000d1d00 0x7faa00155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:24.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.793+0000 7faa05200640 1 -- 172.21.15.100:0/2495859539 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa000d1d00 msgr2=0x7faa00155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:24.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.793+0000 7faa05200640 1 --2- 172.21.15.100:0/2495859539 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa000d1d00 0x7faa00155130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.793+0000 7faa05200640 1 -- 172.21.15.100:0/2495859539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa000b19a0 msgr2=0x7faa0006d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:24.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.793+0000 7faa05200640 1 --2- 172.21.15.100:0/2495859539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa000b19a0 0x7faa0006d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.793+0000 7faa05200640 1 -- 172.21.15.100:0/2495859539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa000702e0 con 0x7faa00156930 2024-04-23T14:33:24.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.793+0000 7faa05200640 1 --2- 172.21.15.100:0/2495859539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa00156930 0x7faa0006f9e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80af040 tx=0x7fa9f8019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:24.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.793+0000 7fa9fcc00640 1 -- 172.21.15.100:0/2495859539 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80bc020 con 0x7faa00156930 2024-04-23T14:33:24.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.793+0000 7fa9fcc00640 1 -- 172.21.15.100:0/2495859539 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f80b8670 con 0x7faa00156930 2024-04-23T14:33:24.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.793+0000 7fa9fcc00640 1 -- 172.21.15.100:0/2495859539 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80b8a60 con 0x7faa00156930 2024-04-23T14:33:24.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.793+0000 7faa05c00640 1 -- 172.21.15.100:0/2495859539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa000705c0 con 0x7faa00156930 2024-04-23T14:33:24.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.793+0000 7faa05c00640 1 -- 172.21.15.100:0/2495859539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa00070b80 con 0x7faa00156930 2024-04-23T14:33:24.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.793+0000 7faa05c00640 1 -- 172.21.15.100:0/2495859539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa00068ad0 con 0x7faa00156930 2024-04-23T14:33:24.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.793+0000 7fa9fcc00640 1 -- 172.21.15.100:0/2495859539 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa9f80bf020 con 0x7faa00156930 2024-04-23T14:33:24.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.793+0000 7fa9fcc00640 1 --2- 172.21.15.100:0/2495859539 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa9dc068380 0x7fa9dc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:24.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.793+0000 7fa9fcc00640 1 -- 172.21.15.100:0/2495859539 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa9f8121e80 con 0x7faa00156930 2024-04-23T14:33:24.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.793+0000 7fa9ffe00640 1 --2- 172.21.15.100:0/2495859539 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa9dc068380 0x7fa9dc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:24.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.797+0000 7fa9ffe00640 1 --2- 172.21.15.100:0/2495859539 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa9dc068380 0x7fa9dc06a840 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0006fd0 tx=0x7fa9f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:24.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.797+0000 7fa9fcc00640 1 -- 172.21.15.100:0/2495859539 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa9f80c6030 con 0x7faa00156930 2024-04-23T14:33:24.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.957+0000 7faa05c00640 1 -- 172.21.15.100:0/2495859539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7faa0015d0b0 con 0x7faa00156930 2024-04-23T14:33:24.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.957+0000 7fa9fcc00640 1 -- 172.21.15.100:0/2495859539 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fa9f80165e0 con 0x7faa00156930 2024-04-23T14:33:24.960 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:33:24.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.957+0000 7faa05c00640 1 -- 172.21.15.100:0/2495859539 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa9dc068380 msgr2=0x7fa9dc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:24.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.957+0000 7faa05c00640 1 --2- 172.21.15.100:0/2495859539 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa9dc068380 0x7fa9dc06a840 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0006fd0 tx=0x7fa9f0008040 comp rx=0 tx=0).stop 2024-04-23T14:33:24.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.957+0000 7faa05c00640 1 -- 172.21.15.100:0/2495859539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa00156930 msgr2=0x7faa0006f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:24.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.957+0000 7faa05c00640 1 --2- 172.21.15.100:0/2495859539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa00156930 0x7faa0006f9e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80af040 tx=0x7fa9f8019790 comp rx=0 tx=0).stop 2024-04-23T14:33:24.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.957+0000 7faa05c00640 1 -- 172.21.15.100:0/2495859539 shutdown_connections 2024-04-23T14:33:24.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.957+0000 7faa05c00640 1 --2- 172.21.15.100:0/2495859539 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa9dc068380 0x7fa9dc06a840 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.957+0000 7faa05c00640 1 --2- 172.21.15.100:0/2495859539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa00156930 0x7faa0006f9e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.957+0000 7faa05c00640 1 --2- 172.21.15.100:0/2495859539 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa000d1d00 0x7faa00155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.957+0000 7faa05c00640 1 --2- 172.21.15.100:0/2495859539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa000b19a0 0x7faa0006d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:24.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.957+0000 7faa05c00640 1 -- 172.21.15.100:0/2495859539 >> 172.21.15.100:0/2495859539 conn(0x7faa00072420 msgr2=0x7faa000c7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:24.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.957+0000 7faa05c00640 1 -- 172.21.15.100:0/2495859539 shutdown_connections 2024-04-23T14:33:24.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:24.957+0000 7faa05c00640 1 -- 172.21.15.100:0/2495859539 wait complete. 2024-04-23T14:33:24.981 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-23T14:33:25.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.113+0000 7f258ee00640 1 Processor -- start 2024-04-23T14:33:25.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.113+0000 7f258ee00640 1 -- start start 2024-04-23T14:33:25.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.117+0000 7f258ee00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f258815e7a0 0x7f25880aea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:25.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.117+0000 7f258ee00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f258806bc70 0x7f258806c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:25.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.117+0000 7f258ee00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25880ada30 con 0x7f25880cee40 2024-04-23T14:33:25.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.117+0000 7f258ee00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f258806f6e0 con 0x7f258806bc70 2024-04-23T14:33:25.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.117+0000 7f258ee00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f25880aefb0 con 0x7f258815e7a0 2024-04-23T14:33:25.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.117+0000 7f2587400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f25880cee40 0x7f25880cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58058/0 (socket says 172.21.15.100:58058) 2024-04-23T14:33:25.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.117+0000 7f2587400640 1 -- 172.21.15.100:0/710015414 learned_addr learned my addr 172.21.15.100:0/710015414 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:25.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.117+0000 7f258ce00640 1 --2- 172.21.15.100:0/710015414 >> v2:172.21.15.100:3300/0 conn(0x7f258806bc70 0x7f258806c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:25.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.117+0000 7f2587e00640 1 --2- 172.21.15.100:0/710015414 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f258815e7a0 0x7f25880aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:25.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.117+0000 7f258ce00640 1 -- 172.21.15.100:0/710015414 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f258815e7a0 msgr2=0x7f25880aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:25.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.117+0000 7f258ce00640 1 --2- 172.21.15.100:0/710015414 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f258815e7a0 0x7f25880aea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.117+0000 7f2586a00640 1 -- 172.21.15.100:0/710015414 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 678293829 0 0) 0x7f2574001120 con 0x7f25880cee40 2024-04-23T14:33:25.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.117+0000 7f258ce00640 1 -- 172.21.15.100:0/710015414 >> v1:172.21.15.100:6789/0 conn(0x7f25880cee40 legacy=0x7f25880cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:25.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.117+0000 7f258ce00640 1 -- 172.21.15.100:0/710015414 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25880adcb0 con 0x7f258806bc70 2024-04-23T14:33:25.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.121+0000 7f258ce00640 1 --2- 172.21.15.100:0/710015414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f258806bc70 0x7f258806c0e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f25800097c0 tx=0x7f258001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2c6851c83ba179 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:25.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.121+0000 7f2586a00640 1 -- 172.21.15.100:0/710015414 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25800a7070 con 0x7f258806bc70 2024-04-23T14:33:25.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.121+0000 7f2586a00640 1 -- 172.21.15.100:0/710015414 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25800b0070 con 0x7f258806bc70 2024-04-23T14:33:25.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.121+0000 7f2586a00640 1 -- 172.21.15.100:0/710015414 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25800ab890 con 0x7f258806bc70 2024-04-23T14:33:25.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.121+0000 7f258ee00640 1 -- 172.21.15.100:0/710015414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f258806bc70 msgr2=0x7f258806c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:25.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.121+0000 7f258ee00640 1 --2- 172.21.15.100:0/710015414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f258806bc70 0x7f258806c0e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f25800097c0 tx=0x7f258001c470 comp rx=0 tx=0).stop 2024-04-23T14:33:25.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.121+0000 7f258ee00640 1 -- 172.21.15.100:0/710015414 shutdown_connections 2024-04-23T14:33:25.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.121+0000 7f258ee00640 1 --2- 172.21.15.100:0/710015414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f258806bc70 0x7f258806c0e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.121+0000 7f258ee00640 1 --2- 172.21.15.100:0/710015414 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f258815e7a0 0x7f25880aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.121+0000 7f258ee00640 1 -- 172.21.15.100:0/710015414 >> 172.21.15.100:0/710015414 conn(0x7f25880b5200 msgr2=0x7f25880b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:25.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.121+0000 7f258ee00640 1 -- 172.21.15.100:0/710015414 shutdown_connections 2024-04-23T14:33:25.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.121+0000 7f258ee00640 1 -- 172.21.15.100:0/710015414 wait complete. 2024-04-23T14:33:25.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.121+0000 7f258ee00640 1 Processor -- start 2024-04-23T14:33:25.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.121+0000 7f258ee00640 1 -- start start 2024-04-23T14:33:25.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.121+0000 7f258ee00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f258806bc70 0x7f25880998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:25.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.121+0000 7f258ee00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25880cee40 0x7f25880c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:25.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.121+0000 7f258ee00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f258815e7a0 0x7f25880c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:25.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.121+0000 7f258ee00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f258815ed00 con 0x7f25880cee40 2024-04-23T14:33:25.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.121+0000 7f258ee00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f258815eb80 con 0x7f258806bc70 2024-04-23T14:33:25.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.121+0000 7f258ee00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f25880cf990 con 0x7f258815e7a0 2024-04-23T14:33:25.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.125+0000 7f2587400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25880cee40 0x7f25880c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:25.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.125+0000 7f2587400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25880cee40 0x7f25880c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48238/0 (socket says 172.21.15.100:48238) 2024-04-23T14:33:25.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.125+0000 7f2587400640 1 -- 172.21.15.100:0/269032406 learned_addr learned my addr 172.21.15.100:0/269032406 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:25.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.125+0000 7f258ce00640 1 --2- 172.21.15.100:0/269032406 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f258815e7a0 0x7f25880c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:25.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.125+0000 7f2587e00640 1 --2- 172.21.15.100:0/269032406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f258806bc70 0x7f25880998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:25.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.125+0000 7f2587400640 1 -- 172.21.15.100:0/269032406 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f258815e7a0 msgr2=0x7f25880c57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:25.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.125+0000 7f2587400640 1 --2- 172.21.15.100:0/269032406 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f258815e7a0 0x7f25880c57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.125+0000 7f2587400640 1 -- 172.21.15.100:0/269032406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f258806bc70 msgr2=0x7f25880998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:25.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.125+0000 7f2587400640 1 --2- 172.21.15.100:0/269032406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f258806bc70 0x7f25880998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.125+0000 7f2587400640 1 -- 172.21.15.100:0/269032406 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25880c3b80 con 0x7f25880cee40 2024-04-23T14:33:25.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.125+0000 7f2587400640 1 --2- 172.21.15.100:0/269032406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25880cee40 0x7f25880c6770 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f257400c780 tx=0x7f257400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:25.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.125+0000 7f257fe00640 1 -- 172.21.15.100:0/269032406 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2574018070 con 0x7f25880cee40 2024-04-23T14:33:25.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.125+0000 7f257fe00640 1 -- 172.21.15.100:0/269032406 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25740072f0 con 0x7f25880cee40 2024-04-23T14:33:25.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.125+0000 7f257fe00640 1 -- 172.21.15.100:0/269032406 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25740047f0 con 0x7f25880cee40 2024-04-23T14:33:25.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.125+0000 7f258ee00640 1 -- 172.21.15.100:0/269032406 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25880c3e20 con 0x7f25880cee40 2024-04-23T14:33:25.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.125+0000 7f258ee00640 1 -- 172.21.15.100:0/269032406 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25880c5cf0 con 0x7f25880cee40 2024-04-23T14:33:25.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.125+0000 7f258ee00640 1 -- 172.21.15.100:0/269032406 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25880c4340 con 0x7f25880cee40 2024-04-23T14:33:25.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.125+0000 7f257fe00640 1 -- 172.21.15.100:0/269032406 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f257401b070 con 0x7f25880cee40 2024-04-23T14:33:25.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.129+0000 7f257fe00640 1 --2- 172.21.15.100:0/269032406 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2568068380 0x7f256806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:25.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.129+0000 7f257fe00640 1 -- 172.21.15.100:0/269032406 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2574086160 con 0x7f25880cee40 2024-04-23T14:33:25.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.129+0000 7f2587e00640 1 --2- 172.21.15.100:0/269032406 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2568068380 0x7f256806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:25.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.129+0000 7f2587e00640 1 --2- 172.21.15.100:0/269032406 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2568068380 0x7f256806a840 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2578003b90 tx=0x7f25780098d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:25.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.133+0000 7f257fe00640 1 -- 172.21.15.100:0/269032406 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f25880ada30 con 0x7f25880cee40 2024-04-23T14:33:25.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.289+0000 7f258ee00640 1 -- 172.21.15.100:0/269032406 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2588152ea0 con 0x7f25880cee40 2024-04-23T14:33:25.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.289+0000 7f257fe00640 1 -- 172.21.15.100:0/269032406 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f2574055cc0 con 0x7f25880cee40 2024-04-23T14:33:25.294 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:25.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.293+0000 7f258ee00640 1 -- 172.21.15.100:0/269032406 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2568068380 msgr2=0x7f256806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:25.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.293+0000 7f258ee00640 1 --2- 172.21.15.100:0/269032406 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2568068380 0x7f256806a840 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2578003b90 tx=0x7f25780098d0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.293+0000 7f258ee00640 1 -- 172.21.15.100:0/269032406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25880cee40 msgr2=0x7f25880c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:25.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.293+0000 7f258ee00640 1 --2- 172.21.15.100:0/269032406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25880cee40 0x7f25880c6770 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f257400c780 tx=0x7f257400cc40 comp rx=0 tx=0).stop 2024-04-23T14:33:25.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.293+0000 7f258ee00640 1 -- 172.21.15.100:0/269032406 shutdown_connections 2024-04-23T14:33:25.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.293+0000 7f258ee00640 1 --2- 172.21.15.100:0/269032406 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2568068380 0x7f256806a840 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.293+0000 7f258ee00640 1 --2- 172.21.15.100:0/269032406 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f258815e7a0 0x7f25880c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.293+0000 7f258ee00640 1 --2- 172.21.15.100:0/269032406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25880cee40 0x7f25880c6770 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.293+0000 7f258ee00640 1 --2- 172.21.15.100:0/269032406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f258806bc70 0x7f25880998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.293+0000 7f258ee00640 1 -- 172.21.15.100:0/269032406 >> 172.21.15.100:0/269032406 conn(0x7f25880b5200 msgr2=0x7f25880d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:25.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.293+0000 7f258ee00640 1 -- 172.21.15.100:0/269032406 shutdown_connections 2024-04-23T14:33:25.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.293+0000 7f258ee00640 1 -- 172.21.15.100:0/269032406 wait complete. 2024-04-23T14:33:25.314 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:33:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.445+0000 7faddb600640 1 Processor -- start 2024-04-23T14:33:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.445+0000 7faddb600640 1 -- start start 2024-04-23T14:33:25.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.445+0000 7faddb600640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fadd415e730 0x7fadd40cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:25.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.445+0000 7faddb600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fadd415eb10 0x7fadd40d0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:25.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.445+0000 7faddb600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadd406f450 con 0x7fadd4163070 2024-04-23T14:33:25.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.445+0000 7faddb600640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fadd406f170 con 0x7fadd415eb10 2024-04-23T14:33:25.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.445+0000 7faddb600640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fadd4162590 con 0x7fadd415e730 2024-04-23T14:33:25.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.445+0000 7fadcbe00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fadd415eb10 0x7fadd40d0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:25.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.445+0000 7fadcbe00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fadd415eb10 0x7fadd40d0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48250/0 (socket says 172.21.15.100:48250) 2024-04-23T14:33:25.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.445+0000 7fadcbe00640 1 -- 172.21.15.100:0/1062831867 learned_addr learned my addr 172.21.15.100:0/1062831867 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:25.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.445+0000 7fadd8c00640 1 --2- 172.21.15.100:0/1062831867 >> v2:172.21.15.151:3300/0 conn(0x7fadd415e730 0x7fadd40cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:25.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.445+0000 7fadcbe00640 1 -- 172.21.15.100:0/1062831867 >> v2:172.21.15.151:3300/0 conn(0x7fadd415e730 msgr2=0x7fadd40cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:25.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.445+0000 7fadcbe00640 1 --2- 172.21.15.100:0/1062831867 >> v2:172.21.15.151:3300/0 conn(0x7fadd415e730 0x7fadd40cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.445+0000 7fadcbe00640 1 -- 172.21.15.100:0/1062831867 >> v1:172.21.15.100:6789/0 conn(0x7fadd4163070 legacy=0x7fadd40ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:25.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.445+0000 7fadcbe00640 1 -- 172.21.15.100:0/1062831867 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd4162870 con 0x7fadd415eb10 2024-04-23T14:33:25.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.449+0000 7fadcbe00640 1 --2- 172.21.15.100:0/1062831867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadd415eb10 0x7fadd40d0050 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fadd40b8610 tx=0x7fadbc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b968e250c423a9ca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:25.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.449+0000 7fadd3e00640 1 -- 172.21.15.100:0/1062831867 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc0b1070 con 0x7fadd415eb10 2024-04-23T14:33:25.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.449+0000 7fadd3e00640 1 -- 172.21.15.100:0/1062831867 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadbc0ac600 con 0x7fadd415eb10 2024-04-23T14:33:25.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.449+0000 7fadd3e00640 1 -- 172.21.15.100:0/1062831867 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc0ac9d0 con 0x7fadd415eb10 2024-04-23T14:33:25.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.449+0000 7faddb600640 1 -- 172.21.15.100:0/1062831867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadd415eb10 msgr2=0x7fadd40d0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:25.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.449+0000 7faddb600640 1 --2- 172.21.15.100:0/1062831867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadd415eb10 0x7fadd40d0050 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fadd40b8610 tx=0x7fadbc01c490 comp rx=0 tx=0).stop 2024-04-23T14:33:25.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.449+0000 7faddb600640 1 -- 172.21.15.100:0/1062831867 shutdown_connections 2024-04-23T14:33:25.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.449+0000 7faddb600640 1 --2- 172.21.15.100:0/1062831867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadd415eb10 0x7fadd40d0050 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.449+0000 7faddb600640 1 --2- 172.21.15.100:0/1062831867 >> v2:172.21.15.151:3300/0 conn(0x7fadd415e730 0x7fadd40cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.449+0000 7faddb600640 1 -- 172.21.15.100:0/1062831867 >> 172.21.15.100:0/1062831867 conn(0x7fadd40b4050 msgr2=0x7fadd40b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:25.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.449+0000 7faddb600640 1 -- 172.21.15.100:0/1062831867 shutdown_connections 2024-04-23T14:33:25.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.449+0000 7faddb600640 1 -- 172.21.15.100:0/1062831867 wait complete. 2024-04-23T14:33:25.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7faddb600640 1 Processor -- start 2024-04-23T14:33:25.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7faddb600640 1 -- start start 2024-04-23T14:33:25.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7faddb600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fadd415e730 0x7fadd415b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:25.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7faddb600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadd415eb10 0x7fadd40ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:25.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7faddb600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadd4163070 0x7fadd40ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:25.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7faddb600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fadd416bb70 con 0x7fadd415eb10 2024-04-23T14:33:25.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7faddb600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fadd416b9f0 con 0x7fadd4163070 2024-04-23T14:33:25.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7faddb600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fadd416bcf0 con 0x7fadd415e730 2024-04-23T14:33:25.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7fadcbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadd415eb10 0x7fadd40ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:25.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7fadcbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadd415eb10 0x7fadd40ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48254/0 (socket says 172.21.15.100:48254) 2024-04-23T14:33:25.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7fadcbe00640 1 -- 172.21.15.100:0/4229692105 learned_addr learned my addr 172.21.15.100:0/4229692105 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:25.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7fadd8c00640 1 --2- 172.21.15.100:0/4229692105 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fadd415e730 0x7fadd415b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:25.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7fadd9600640 1 --2- 172.21.15.100:0/4229692105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadd4163070 0x7fadd40ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:25.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7fadcbe00640 1 -- 172.21.15.100:0/4229692105 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fadd415e730 msgr2=0x7fadd415b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:25.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7fadcbe00640 1 --2- 172.21.15.100:0/4229692105 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fadd415e730 0x7fadd415b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7fadcbe00640 1 -- 172.21.15.100:0/4229692105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadd4163070 msgr2=0x7fadd40ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:25.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7fadcbe00640 1 --2- 172.21.15.100:0/4229692105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadd4163070 0x7fadd40ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7fadcbe00640 1 -- 172.21.15.100:0/4229692105 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd40cb520 con 0x7fadd415eb10 2024-04-23T14:33:25.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7fadcbe00640 1 --2- 172.21.15.100:0/4229692105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadd415eb10 0x7fadd40ad870 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fadbc01a0c0 tx=0x7fadbc01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:25.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7fadd1600640 1 -- 172.21.15.100:0/4229692105 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc0b1040 con 0x7fadd415eb10 2024-04-23T14:33:25.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7fadd1600640 1 -- 172.21.15.100:0/4229692105 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadbc0bc070 con 0x7fadd415eb10 2024-04-23T14:33:25.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7fadd1600640 1 -- 172.21.15.100:0/4229692105 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc0b78b0 con 0x7fadd415eb10 2024-04-23T14:33:25.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7faddb600640 1 -- 172.21.15.100:0/4229692105 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd41f7f80 con 0x7fadd415eb10 2024-04-23T14:33:25.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.453+0000 7faddb600640 1 -- 172.21.15.100:0/4229692105 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd40cc640 con 0x7fadd415eb10 2024-04-23T14:33:25.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.457+0000 7faddb600640 1 -- 172.21.15.100:0/4229692105 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadd4068ad0 con 0x7fadd415eb10 2024-04-23T14:33:25.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.457+0000 7fadd1600640 1 -- 172.21.15.100:0/4229692105 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fadbc0b7a50 con 0x7fadd415eb10 2024-04-23T14:33:25.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.457+0000 7fadd1600640 1 --2- 172.21.15.100:0/4229692105 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fadb0068380 0x7fadb006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:25.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.457+0000 7fadd1600640 1 -- 172.21.15.100:0/4229692105 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fadbc120cc0 con 0x7fadd415eb10 2024-04-23T14:33:25.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.457+0000 7fadd8c00640 1 --2- 172.21.15.100:0/4229692105 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fadb0068380 0x7fadb006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:25.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.457+0000 7fadd8c00640 1 --2- 172.21.15.100:0/4229692105 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fadb0068380 0x7fadb006a840 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fadc400b9d0 tx=0x7fadc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:25.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.461+0000 7fadd1600640 1 -- 172.21.15.100:0/4229692105 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fadbc127080 con 0x7fadd415eb10 2024-04-23T14:33:25.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.649+0000 7faddb600640 1 -- 172.21.15.100:0/4229692105 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fadd40b3ae0 con 0x7fadd415eb10 2024-04-23T14:33:25.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.653+0000 7fadd1600640 1 -- 172.21.15.100:0/4229692105 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7fadbc0c5020 con 0x7fadd415eb10 2024-04-23T14:33:25.655 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:25.655 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1456958549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1456958549},{"type":"v1","addr":"172.21.15.100:6836","nonce":1456958549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3655609551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3655609551},{"type":"v1","addr":"172.21.15.100:6837","nonce":3655609551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/4242180972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":4242180972},{"type":"v1","addr":"172.21.15.151:6834","nonce":4242180972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:32:08.976576+0000","modified":"2024-04-23T14:32:34.057740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1533322173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1533322173},{"type":"v1","addr":"172.21.15.151:6835","nonce":1533322173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-23T14:33:25.655 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 9 2024-04-23T14:33:25.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.653+0000 7faddb600640 1 -- 172.21.15.100:0/4229692105 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fadb0068380 msgr2=0x7fadb006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:25.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.653+0000 7faddb600640 1 --2- 172.21.15.100:0/4229692105 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fadb0068380 0x7fadb006a840 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fadc400b9d0 tx=0x7fadc4008040 comp rx=0 tx=0).stop 2024-04-23T14:33:25.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.653+0000 7faddb600640 1 -- 172.21.15.100:0/4229692105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadd415eb10 msgr2=0x7fadd40ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:25.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.653+0000 7faddb600640 1 --2- 172.21.15.100:0/4229692105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadd415eb10 0x7fadd40ad870 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fadbc01a0c0 tx=0x7fadbc01acd0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.653+0000 7faddb600640 1 -- 172.21.15.100:0/4229692105 shutdown_connections 2024-04-23T14:33:25.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.653+0000 7faddb600640 1 --2- 172.21.15.100:0/4229692105 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fadb0068380 0x7fadb006a840 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.653+0000 7faddb600640 1 --2- 172.21.15.100:0/4229692105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadd4163070 0x7fadd40ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.653+0000 7faddb600640 1 --2- 172.21.15.100:0/4229692105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadd415eb10 0x7fadd40ad870 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.653+0000 7faddb600640 1 --2- 172.21.15.100:0/4229692105 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fadd415e730 0x7fadd415b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.653+0000 7faddb600640 1 -- 172.21.15.100:0/4229692105 >> 172.21.15.100:0/4229692105 conn(0x7fadd40b4050 msgr2=0x7fadd406e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:25.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.653+0000 7faddb600640 1 -- 172.21.15.100:0/4229692105 shutdown_connections 2024-04-23T14:33:25.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.653+0000 7faddb600640 1 -- 172.21.15.100:0/4229692105 wait complete. 2024-04-23T14:33:25.677 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:25.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.809+0000 7f3bf4e00640 1 Processor -- start 2024-04-23T14:33:25.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.809+0000 7f3bf4e00640 1 -- start start 2024-04-23T14:33:25.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3bf4e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bf015b640 0x7f3bf00adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:25.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3bf4e00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bf00af080 con 0x7f3bf00b2a10 2024-04-23T14:33:25.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3bf4e00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bf00afbc0 con 0x7f3bf0073610 2024-04-23T14:33:25.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3bf4e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f3bf015ba20 con 0x7f3bf015b640 2024-04-23T14:33:25.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3bed400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f3bf00b2a10 0x7f3bf0072f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58084/0 (socket says 172.21.15.100:58084) 2024-04-23T14:33:25.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3bed400640 1 -- 172.21.15.100:0/198442664 learned_addr learned my addr 172.21.15.100:0/198442664 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:25.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3bede00640 1 --2- 172.21.15.100:0/198442664 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bf015b640 0x7f3bf00adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:25.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3beca00640 1 -- 172.21.15.100:0/198442664 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 244250447 0 0) 0x7f3bf00af080 con 0x7f3bf00b2a10 2024-04-23T14:33:25.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3beca00640 1 -- 172.21.15.100:0/198442664 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3bd4005740 con 0x7f3bf00b2a10 2024-04-23T14:33:25.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3beca00640 1 -- 172.21.15.100:0/198442664 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2965695417 0 0) 0x7f3bf00afbc0 con 0x7f3bf0073610 2024-04-23T14:33:25.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3beca00640 1 -- 172.21.15.100:0/198442664 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3bf00af080 con 0x7f3bf0073610 2024-04-23T14:33:25.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3beca00640 1 -- 172.21.15.100:0/198442664 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1617767189 0 0) 0x7f3bd4005740 con 0x7f3bf00b2a10 2024-04-23T14:33:25.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3beca00640 1 -- 172.21.15.100:0/198442664 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bf015b640 msgr2=0x7f3bf00adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:25.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3beca00640 1 --2- 172.21.15.100:0/198442664 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bf015b640 0x7f3bf00adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3beca00640 1 -- 172.21.15.100:0/198442664 >> v1:172.21.15.151:6789/0 conn(0x7f3bf0073610 legacy=0x7f3bf006df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:25.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3beca00640 1 -- 172.21.15.100:0/198442664 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf00af320 con 0x7f3bf00b2a10 2024-04-23T14:33:25.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3beca00640 1 -- 172.21.15.100:0/198442664 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f3be0001680 con 0x7f3bf00b2a10 2024-04-23T14:33:25.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3beca00640 1 -- 172.21.15.100:0/198442664 >> v1:172.21.15.100:6789/0 conn(0x7f3bf00b2a10 legacy=0x7f3bf0072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:25.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3beca00640 1 --2- 172.21.15.100:0/198442664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bd4005d90 0x7f3bd4006190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:25.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3beca00640 1 --2- 172.21.15.100:0/198442664 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bf015b640 0x7f3bd4006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:25.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3beca00640 1 --2- 172.21.15.100:0/198442664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bd4006f60 0x7f3bd40073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:25.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3beca00640 1 -- 172.21.15.100:0/198442664 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf015ba20 con 0x7f3bd4005d90 2024-04-23T14:33:25.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3beca00640 1 -- 172.21.15.100:0/198442664 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd400dc90 con 0x7f3bd4006f60 2024-04-23T14:33:25.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3beca00640 1 -- 172.21.15.100:0/198442664 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f3bd400de40 con 0x7f3bf015b640 2024-04-23T14:33:25.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3bede00640 1 --2- 172.21.15.100:0/198442664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bd4005d90 0x7f3bd4006190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:25.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3bee800640 1 --2- 172.21.15.100:0/198442664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bd4006f60 0x7f3bd40073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:25.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3bed400640 1 --2- 172.21.15.100:0/198442664 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bf015b640 0x7f3bd4006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:25.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3bede00640 1 -- 172.21.15.100:0/198442664 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bf015b640 msgr2=0x7f3bd4006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:25.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3bede00640 1 --2- 172.21.15.100:0/198442664 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bf015b640 0x7f3bd4006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3bede00640 1 -- 172.21.15.100:0/198442664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bd4006f60 msgr2=0x7f3bd40073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:25.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3bede00640 1 --2- 172.21.15.100:0/198442664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bd4006f60 0x7f3bd40073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3bede00640 1 -- 172.21.15.100:0/198442664 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd4005740 con 0x7f3bd4005d90 2024-04-23T14:33:25.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.813+0000 7f3bee800640 1 --2- 172.21.15.100:0/198442664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bd4006f60 0x7f3bd40073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:25.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.817+0000 7f3bede00640 1 --2- 172.21.15.100:0/198442664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bd4005d90 0x7f3bd4006190 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009ae0 tx=0x7f3bd801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4024831a39a57be8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:25.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.817+0000 7f3beca00640 1 -- 172.21.15.100:0/198442664 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd80b2070 con 0x7f3bd4005d90 2024-04-23T14:33:25.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.817+0000 7f3beca00640 1 -- 172.21.15.100:0/198442664 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bd80a5610 con 0x7f3bd4005d90 2024-04-23T14:33:25.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.817+0000 7f3beca00640 1 -- 172.21.15.100:0/198442664 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd80096b0 con 0x7f3bd4005d90 2024-04-23T14:33:25.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.817+0000 7f3bf4e00640 1 -- 172.21.15.100:0/198442664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bd4005d90 msgr2=0x7f3bd4006190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:25.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.817+0000 7f3bf4e00640 1 --2- 172.21.15.100:0/198442664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bd4005d90 0x7f3bd4006190 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009ae0 tx=0x7f3bd801ca40 comp rx=0 tx=0).stop 2024-04-23T14:33:25.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.817+0000 7f3bee800640 1 -- 172.21.15.100:0/198442664 reap_dead start 2024-04-23T14:33:25.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.817+0000 7f3bf4e00640 1 -- 172.21.15.100:0/198442664 shutdown_connections 2024-04-23T14:33:25.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.817+0000 7f3bf4e00640 1 -- 172.21.15.100:0/198442664 >> 172.21.15.100:0/198442664 conn(0x7f3bf00b1e80 msgr2=0x7f3bf00b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:25.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.817+0000 7f3bf4e00640 1 -- 172.21.15.100:0/198442664 shutdown_connections 2024-04-23T14:33:25.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.817+0000 7f3bf4e00640 1 -- 172.21.15.100:0/198442664 wait complete. 2024-04-23T14:33:25.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bf4e00640 1 Processor -- start 2024-04-23T14:33:25.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bf4e00640 1 -- start start 2024-04-23T14:33:25.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bf4e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bd4005d90 0x7f3bf01f8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:25.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bf4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bf009cb70 0x7f3bf00cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:25.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bf4e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bf00cbe70 0x7f3bf01fabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:25.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bf4e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf00ae3d0 con 0x7f3bf009cb70 2024-04-23T14:33:25.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bf4e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf006f160 con 0x7f3bd4005d90 2024-04-23T14:33:25.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bf4e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f3bf006c340 con 0x7f3bf00cbe70 2024-04-23T14:33:25.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bed400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bf009cb70 0x7f3bf00cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:25.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bed400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bf009cb70 0x7f3bf00cb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48272/0 (socket says 172.21.15.100:48272) 2024-04-23T14:33:25.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bed400640 1 -- 172.21.15.100:0/2724110551 learned_addr learned my addr 172.21.15.100:0/2724110551 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:25.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bede00640 1 --2- 172.21.15.100:0/2724110551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bd4005d90 0x7f3bf01f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:25.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bee800640 1 --2- 172.21.15.100:0/2724110551 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bf00cbe70 0x7f3bf01fabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:25.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bed400640 1 -- 172.21.15.100:0/2724110551 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bf00cbe70 msgr2=0x7f3bf01fabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:25.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bed400640 1 --2- 172.21.15.100:0/2724110551 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bf00cbe70 0x7f3bf01fabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bed400640 1 -- 172.21.15.100:0/2724110551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bd4005d90 msgr2=0x7f3bf01f8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:25.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bed400640 1 --2- 172.21.15.100:0/2724110551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bd4005d90 0x7f3bf01f8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bed400640 1 -- 172.21.15.100:0/2724110551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf01f85b0 con 0x7f3bf009cb70 2024-04-23T14:33:25.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bee800640 1 --2- 172.21.15.100:0/2724110551 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bf00cbe70 0x7f3bf01fabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:25.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bed400640 1 --2- 172.21.15.100:0/2724110551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bf009cb70 0x7f3bf00cb930 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3be000e6e0 tx=0x7f3be000eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:25.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3be6000640 1 -- 172.21.15.100:0/2724110551 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be000c6c0 con 0x7f3bf009cb70 2024-04-23T14:33:25.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3be6000640 1 -- 172.21.15.100:0/2724110551 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3be0018070 con 0x7f3bf009cb70 2024-04-23T14:33:25.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3be6000640 1 -- 172.21.15.100:0/2724110551 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be000ca60 con 0x7f3bf009cb70 2024-04-23T14:33:25.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bf4e00640 1 -- 172.21.15.100:0/2724110551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf009d110 con 0x7f3bf009cb70 2024-04-23T14:33:25.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bf4e00640 1 -- 172.21.15.100:0/2724110551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf009d6d0 con 0x7f3bf009cb70 2024-04-23T14:33:25.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.821+0000 7f3bf4e00640 1 -- 172.21.15.100:0/2724110551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bf0068ad0 con 0x7f3bf009cb70 2024-04-23T14:33:25.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.825+0000 7f3be6000640 1 -- 172.21.15.100:0/2724110551 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3be000cc00 con 0x7f3bf009cb70 2024-04-23T14:33:25.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.825+0000 7f3be6000640 1 --2- 172.21.15.100:0/2724110551 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3bcc068380 0x7f3bcc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:25.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.825+0000 7f3be6000640 1 -- 172.21.15.100:0/2724110551 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f3be0086bd0 con 0x7f3bf009cb70 2024-04-23T14:33:25.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.825+0000 7f3bede00640 1 --2- 172.21.15.100:0/2724110551 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3bcc068380 0x7f3bcc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:25.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.825+0000 7f3bede00640 1 --2- 172.21.15.100:0/2724110551 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3bcc068380 0x7f3bcc06a840 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80af000 tx=0x7f3bd80046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:25.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.829+0000 7f3be6000640 1 -- 172.21.15.100:0/2724110551 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3bf01f85b0 con 0x7f3bf009cb70 2024-04-23T14:33:25.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.985+0000 7f3bf4e00640 1 -- 172.21.15.100:0/2724110551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3bf015a820 con 0x7f3bf009cb70 2024-04-23T14:33:25.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.989+0000 7f3be6000640 1 -- 172.21.15.100:0/2724110551 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18859 (secure 0 0 0) 0x7f3be0056cf0 con 0x7f3bf009cb70 2024-04-23T14:33:25.991 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:25.991 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:49.784608+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:25.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.989+0000 7f3bf4e00640 1 -- 172.21.15.100:0/2724110551 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3bcc068380 msgr2=0x7f3bcc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:25.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.989+0000 7f3bf4e00640 1 --2- 172.21.15.100:0/2724110551 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3bcc068380 0x7f3bcc06a840 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80af000 tx=0x7f3bd80046d0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.989+0000 7f3bf4e00640 1 -- 172.21.15.100:0/2724110551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bf009cb70 msgr2=0x7f3bf00cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:25.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.989+0000 7f3bf4e00640 1 --2- 172.21.15.100:0/2724110551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bf009cb70 0x7f3bf00cb930 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3be000e6e0 tx=0x7f3be000eba0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.989+0000 7f3bf4e00640 1 -- 172.21.15.100:0/2724110551 shutdown_connections 2024-04-23T14:33:25.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.989+0000 7f3bf4e00640 1 --2- 172.21.15.100:0/2724110551 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3bcc068380 0x7f3bcc06a840 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.989+0000 7f3bf4e00640 1 --2- 172.21.15.100:0/2724110551 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3bf00cbe70 0x7f3bf01fabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.989+0000 7f3bf4e00640 1 --2- 172.21.15.100:0/2724110551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bf009cb70 0x7f3bf00cb930 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.989+0000 7f3bf4e00640 1 --2- 172.21.15.100:0/2724110551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bd4005d90 0x7f3bf01f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:25.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.989+0000 7f3bf4e00640 1 -- 172.21.15.100:0/2724110551 >> 172.21.15.100:0/2724110551 conn(0x7f3bf00b1e80 msgr2=0x7f3bf0072df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:25.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.989+0000 7f3bf4e00640 1 -- 172.21.15.100:0/2724110551 shutdown_connections 2024-04-23T14:33:25.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:25.989+0000 7f3bf4e00640 1 -- 172.21.15.100:0/2724110551 wait complete. 2024-04-23T14:33:26.012 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-23T14:33:26.013 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:33:26.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.145+0000 7fe9e2c00640 1 Processor -- start 2024-04-23T14:33:26.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.145+0000 7fe9e2c00640 1 -- start start 2024-04-23T14:33:26.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.145+0000 7fe9e2c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe9dc06ec60 0x7fe9dc06f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:26.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.145+0000 7fe9e2c00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9dc06f800 con 0x7fe9dc072e80 2024-04-23T14:33:26.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.145+0000 7fe9e2c00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9dc06fae0 con 0x7fe9dc15c580 2024-04-23T14:33:26.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.145+0000 7fe9e2c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe9dc0cfe70 con 0x7fe9dc06ec60 2024-04-23T14:33:26.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.145+0000 7fe9e0c00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe9dc072e80 0x7fe9dc06cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58094/0 (socket says 172.21.15.100:58094) 2024-04-23T14:33:26.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.145+0000 7fe9e0c00640 1 -- 172.21.15.100:0/1249489122 learned_addr learned my addr 172.21.15.100:0/1249489122 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:26.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.145+0000 7fe9d3400640 1 --2- 172.21.15.100:0/1249489122 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe9dc06ec60 0x7fe9dc06f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:26.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.145+0000 7fe9db400640 1 -- 172.21.15.100:0/1249489122 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1303406262 0 0) 0x7fe9dc06f800 con 0x7fe9dc072e80 2024-04-23T14:33:26.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.145+0000 7fe9db400640 1 -- 172.21.15.100:0/1249489122 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9c0003530 con 0x7fe9dc072e80 2024-04-23T14:33:26.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.145+0000 7fe9db400640 1 -- 172.21.15.100:0/1249489122 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2129743570 0 0) 0x7fe9dc06fae0 con 0x7fe9dc15c580 2024-04-23T14:33:26.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.145+0000 7fe9db400640 1 -- 172.21.15.100:0/1249489122 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9dc06f800 con 0x7fe9dc15c580 2024-04-23T14:33:26.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.145+0000 7fe9d3400640 1 -- 172.21.15.100:0/1249489122 >> v1:172.21.15.151:6789/0 conn(0x7fe9dc15c580 legacy=0x7fe9dc0cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:26.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.145+0000 7fe9d3400640 1 -- 172.21.15.100:0/1249489122 >> v1:172.21.15.100:6789/0 conn(0x7fe9dc072e80 legacy=0x7fe9dc06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:26.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.145+0000 7fe9db400640 1 -- 172.21.15.100:0/1249489122 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3233472618 0 0) 0x7fe9c0003530 con 0x7fe9dc072e80 2024-04-23T14:33:26.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.145+0000 7fe9d3400640 1 -- 172.21.15.100:0/1249489122 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9dc06c470 con 0x7fe9dc06ec60 2024-04-23T14:33:26.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.149+0000 7fe9d3400640 1 --2- 172.21.15.100:0/1249489122 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe9dc06ec60 0x7fe9dc06f0c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40097f0 tx=0x7fe9c401c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61bdd8116672d46a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:26.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.149+0000 7fe9db400640 1 -- 172.21.15.100:0/1249489122 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c40b0070 con 0x7fe9dc06ec60 2024-04-23T14:33:26.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.149+0000 7fe9db400640 1 -- 172.21.15.100:0/1249489122 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c40ab400 con 0x7fe9dc06ec60 2024-04-23T14:33:26.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.149+0000 7fe9db400640 1 -- 172.21.15.100:0/1249489122 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4001a00 con 0x7fe9dc06ec60 2024-04-23T14:33:26.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.149+0000 7fe9e2c00640 1 -- 172.21.15.100:0/1249489122 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe9dc06ec60 msgr2=0x7fe9dc06f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:26.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.149+0000 7fe9e2c00640 1 --2- 172.21.15.100:0/1249489122 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe9dc06ec60 0x7fe9dc06f0c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40097f0 tx=0x7fe9c401c370 comp rx=0 tx=0).stop 2024-04-23T14:33:26.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.149+0000 7fe9e2c00640 1 -- 172.21.15.100:0/1249489122 shutdown_connections 2024-04-23T14:33:26.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.149+0000 7fe9e2c00640 1 --2- 172.21.15.100:0/1249489122 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe9dc06ec60 0x7fe9dc06f0c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:26.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.149+0000 7fe9e2c00640 1 -- 172.21.15.100:0/1249489122 >> 172.21.15.100:0/1249489122 conn(0x7fe9dc0b3050 msgr2=0x7fe9dc0b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:26.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.149+0000 7fe9e2c00640 1 -- 172.21.15.100:0/1249489122 shutdown_connections 2024-04-23T14:33:26.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.149+0000 7fe9e2c00640 1 -- 172.21.15.100:0/1249489122 wait complete. 2024-04-23T14:33:26.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9e2c00640 1 Processor -- start 2024-04-23T14:33:26.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9e2c00640 1 -- start start 2024-04-23T14:33:26.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9e2c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9dc06ec60 0x7fe9dc1511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:26.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9e2c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe9dc072e80 0x7fe9dc138d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:26.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9e2c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9dc15c580 0x7fe9dc14f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:26.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9e2c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc06c7e0 con 0x7fe9dc15c580 2024-04-23T14:33:26.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9e2c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc06fdc0 con 0x7fe9dc06ec60 2024-04-23T14:33:26.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9e2c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe9dc16ad00 con 0x7fe9dc072e80 2024-04-23T14:33:26.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9e0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9dc15c580 0x7fe9dc14f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:26.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9e0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9dc15c580 0x7fe9dc14f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48276/0 (socket says 172.21.15.100:48276) 2024-04-23T14:33:26.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9e0c00640 1 -- 172.21.15.100:0/2210887394 learned_addr learned my addr 172.21.15.100:0/2210887394 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:26.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9dbe00640 1 --2- 172.21.15.100:0/2210887394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9dc06ec60 0x7fe9dc1511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:26.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9d3400640 1 --2- 172.21.15.100:0/2210887394 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe9dc072e80 0x7fe9dc138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:26.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9e0c00640 1 -- 172.21.15.100:0/2210887394 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe9dc072e80 msgr2=0x7fe9dc138d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:26.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9e0c00640 1 --2- 172.21.15.100:0/2210887394 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe9dc072e80 0x7fe9dc138d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:26.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9e0c00640 1 -- 172.21.15.100:0/2210887394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9dc06ec60 msgr2=0x7fe9dc1511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:26.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9e0c00640 1 --2- 172.21.15.100:0/2210887394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9dc06ec60 0x7fe9dc1511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:26.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9e0c00640 1 -- 172.21.15.100:0/2210887394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9dc0b5740 con 0x7fe9dc15c580 2024-04-23T14:33:26.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9dbe00640 1 --2- 172.21.15.100:0/2210887394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9dc06ec60 0x7fe9dc1511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:26.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9d3400640 1 --2- 172.21.15.100:0/2210887394 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe9dc072e80 0x7fe9dc138d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:26.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9e0c00640 1 --2- 172.21.15.100:0/2210887394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9dc15c580 0x7fe9dc14f1c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400cc40 tx=0x7fe9d400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:26.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9d8c00640 1 -- 172.21.15.100:0/2210887394 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d400ce50 con 0x7fe9dc15c580 2024-04-23T14:33:26.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9d8c00640 1 -- 172.21.15.100:0/2210887394 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9d4012400 con 0x7fe9dc15c580 2024-04-23T14:33:26.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9d8c00640 1 -- 172.21.15.100:0/2210887394 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d40127f0 con 0x7fe9dc15c580 2024-04-23T14:33:26.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9e2c00640 1 -- 172.21.15.100:0/2210887394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9dc0b5a20 con 0x7fe9dc15c580 2024-04-23T14:33:26.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.153+0000 7fe9e2c00640 1 -- 172.21.15.100:0/2210887394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9dc0b5fe0 con 0x7fe9dc15c580 2024-04-23T14:33:26.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.157+0000 7fe9e2c00640 1 -- 172.21.15.100:0/2210887394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9dc068ad0 con 0x7fe9dc15c580 2024-04-23T14:33:26.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.157+0000 7fe9d8c00640 1 -- 172.21.15.100:0/2210887394 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe9d4012990 con 0x7fe9dc15c580 2024-04-23T14:33:26.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.157+0000 7fe9d8c00640 1 --2- 172.21.15.100:0/2210887394 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe9b8068520 0x7fe9b806a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:26.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.157+0000 7fe9d8c00640 1 -- 172.21.15.100:0/2210887394 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe9d40876b0 con 0x7fe9dc15c580 2024-04-23T14:33:26.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.157+0000 7fe9dbe00640 1 --2- 172.21.15.100:0/2210887394 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe9b8068520 0x7fe9b806a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:26.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.157+0000 7fe9dbe00640 1 --2- 172.21.15.100:0/2210887394 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe9b8068520 0x7fe9b806a9e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc0011f0 tx=0x7fe9cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:26.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.161+0000 7fe9d8c00640 1 -- 172.21.15.100:0/2210887394 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe9d40577e0 con 0x7fe9dc15c580 2024-04-23T14:33:26.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.353+0000 7fe9e2c00640 1 -- 172.21.15.100:0/2210887394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9dc14efa0 con 0x7fe9dc15c580 2024-04-23T14:33:26.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.353+0000 7fe9d8c00640 1 -- 172.21.15.100:0/2210887394 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7fe9d40570e0 con 0x7fe9dc15c580 2024-04-23T14:33:26.356 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:26.356 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1456958549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1456958549},{"type":"v1","addr":"172.21.15.100:6836","nonce":1456958549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3655609551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3655609551},{"type":"v1","addr":"172.21.15.100:6837","nonce":3655609551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/4242180972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":4242180972},{"type":"v1","addr":"172.21.15.151:6834","nonce":4242180972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:32:08.976576+0000","modified":"2024-04-23T14:32:34.057740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1533322173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1533322173},{"type":"v1","addr":"172.21.15.151:6835","nonce":1533322173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-23T14:33:26.356 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 9 2024-04-23T14:33:26.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.353+0000 7fe9e2c00640 1 -- 172.21.15.100:0/2210887394 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe9b8068520 msgr2=0x7fe9b806a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:26.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.353+0000 7fe9e2c00640 1 --2- 172.21.15.100:0/2210887394 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe9b8068520 0x7fe9b806a9e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc0011f0 tx=0x7fe9cc009040 comp rx=0 tx=0).stop 2024-04-23T14:33:26.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.353+0000 7fe9e2c00640 1 -- 172.21.15.100:0/2210887394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9dc15c580 msgr2=0x7fe9dc14f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:26.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.353+0000 7fe9e2c00640 1 --2- 172.21.15.100:0/2210887394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9dc15c580 0x7fe9dc14f1c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400cc40 tx=0x7fe9d400a430 comp rx=0 tx=0).stop 2024-04-23T14:33:26.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.353+0000 7fe9e2c00640 1 -- 172.21.15.100:0/2210887394 shutdown_connections 2024-04-23T14:33:26.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.353+0000 7fe9e2c00640 1 --2- 172.21.15.100:0/2210887394 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe9b8068520 0x7fe9b806a9e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:26.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.353+0000 7fe9e2c00640 1 --2- 172.21.15.100:0/2210887394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9dc15c580 0x7fe9dc14f1c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:26.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.353+0000 7fe9e2c00640 1 --2- 172.21.15.100:0/2210887394 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe9dc072e80 0x7fe9dc138d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:26.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.353+0000 7fe9e2c00640 1 --2- 172.21.15.100:0/2210887394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9dc06ec60 0x7fe9dc1511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:26.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.353+0000 7fe9e2c00640 1 -- 172.21.15.100:0/2210887394 >> 172.21.15.100:0/2210887394 conn(0x7fe9dc0b3050 msgr2=0x7fe9dc0b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:26.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.353+0000 7fe9e2c00640 1 -- 172.21.15.100:0/2210887394 shutdown_connections 2024-04-23T14:33:26.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.353+0000 7fe9e2c00640 1 -- 172.21.15.100:0/2210887394 wait complete. 2024-04-23T14:33:26.378 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:33:26.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.509+0000 7fa8b9800640 1 Processor -- start 2024-04-23T14:33:26.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.509+0000 7fa8b9800640 1 -- start start 2024-04-23T14:33:26.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.509+0000 7fa8b9800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa8b415b640 0x7fa8b40adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:26.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.509+0000 7fa8b9800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fa8b40b2a10 0x7fa8b4072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:26.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.509+0000 7fa8b9800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8b40af080 con 0x7fa8b40cc750 2024-04-23T14:33:26.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.509+0000 7fa8b9800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fa8b40afbc0 con 0x7fa8b40b2a10 2024-04-23T14:33:26.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.509+0000 7fa8b9800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa8b415ba20 con 0x7fa8b415b640 2024-04-23T14:33:26.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.509+0000 7fa8b3400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fa8b40cc750 0x7fa8b4071ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58100/0 (socket says 172.21.15.100:58100) 2024-04-23T14:33:26.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.509+0000 7fa8b3400640 1 -- 172.21.15.100:0/3478367289 learned_addr learned my addr 172.21.15.100:0/3478367289 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:26.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.509+0000 7fa8b2000640 1 --2- 172.21.15.100:0/3478367289 >> v2:172.21.15.151:3300/0 conn(0x7fa8b40b2a10 0x7fa8b4072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:26.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.509+0000 7fa8b2a00640 1 --2- 172.21.15.100:0/3478367289 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa8b415b640 0x7fa8b40adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:26.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.513+0000 7fa8b1600640 1 -- 172.21.15.100:0/3478367289 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3309311716 0 0) 0x7fa8b40af080 con 0x7fa8b40cc750 2024-04-23T14:33:26.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.513+0000 7fa8b1600640 1 -- 172.21.15.100:0/3478367289 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa898003530 con 0x7fa8b40cc750 2024-04-23T14:33:26.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.513+0000 7fa8b2000640 1 -- 172.21.15.100:0/3478367289 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa8b415b640 msgr2=0x7fa8b40adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:26.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.513+0000 7fa8b2000640 1 --2- 172.21.15.100:0/3478367289 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa8b415b640 0x7fa8b40adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:26.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.513+0000 7fa8b2000640 1 -- 172.21.15.100:0/3478367289 >> v1:172.21.15.100:6789/0 conn(0x7fa8b40cc750 legacy=0x7fa8b4071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:26.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.513+0000 7fa8b2000640 1 -- 172.21.15.100:0/3478367289 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b40af320 con 0x7fa8b40b2a10 2024-04-23T14:33:26.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.513+0000 7fa8b2000640 1 --2- 172.21.15.100:0/3478367289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8b40b2a10 0x7fa8b4072f00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00097c0 tx=0x7fa8a001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e394fe3d07552a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.513+0000 7fa8b1600640 1 -- 172.21.15.100:0/3478367289 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a00a7070 con 0x7fa8b40b2a10 2024-04-23T14:33:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.513+0000 7fa8b1600640 1 -- 172.21.15.100:0/3478367289 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8a00b0070 con 0x7fa8b40b2a10 2024-04-23T14:33:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.513+0000 7fa8b1600640 1 -- 172.21.15.100:0/3478367289 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a00ab870 con 0x7fa8b40b2a10 2024-04-23T14:33:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.513+0000 7fa8b9800640 1 -- 172.21.15.100:0/3478367289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8b40b2a10 msgr2=0x7fa8b4072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.513+0000 7fa8b9800640 1 --2- 172.21.15.100:0/3478367289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8b40b2a10 0x7fa8b4072f00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00097c0 tx=0x7fa8a001c470 comp rx=0 tx=0).stop 2024-04-23T14:33:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b9800640 1 -- 172.21.15.100:0/3478367289 shutdown_connections 2024-04-23T14:33:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b9800640 1 --2- 172.21.15.100:0/3478367289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8b40b2a10 0x7fa8b4072f00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b9800640 1 --2- 172.21.15.100:0/3478367289 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa8b415b640 0x7fa8b40adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b9800640 1 -- 172.21.15.100:0/3478367289 >> 172.21.15.100:0/3478367289 conn(0x7fa8b40b1e80 msgr2=0x7fa8b40b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b9800640 1 -- 172.21.15.100:0/3478367289 shutdown_connections 2024-04-23T14:33:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b9800640 1 -- 172.21.15.100:0/3478367289 wait complete. 2024-04-23T14:33:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b9800640 1 Processor -- start 2024-04-23T14:33:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b9800640 1 -- start start 2024-04-23T14:33:26.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b9800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa8b40b2a10 0x7fa8b4099800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b40cc750 0x7fa8b4148700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b9800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8b415b640 0x7fa8b4148c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b9800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b41683c0 con 0x7fa8b40cc750 2024-04-23T14:33:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b9800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b4073850 con 0x7fa8b415b640 2024-04-23T14:33:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b9800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa8b4168540 con 0x7fa8b40b2a10 2024-04-23T14:33:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b40cc750 0x7fa8b4148700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b40cc750 0x7fa8b4148700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48286/0 (socket says 172.21.15.100:48286) 2024-04-23T14:33:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b2000640 1 -- 172.21.15.100:0/3878034887 learned_addr learned my addr 172.21.15.100:0/3878034887 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b2a00640 1 --2- 172.21.15.100:0/3878034887 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa8b40b2a10 0x7fa8b4099800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b3400640 1 --2- 172.21.15.100:0/3878034887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8b415b640 0x7fa8b4148c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b2000640 1 -- 172.21.15.100:0/3878034887 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa8b40b2a10 msgr2=0x7fa8b4099800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b2000640 1 --2- 172.21.15.100:0/3878034887 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa8b40b2a10 0x7fa8b4099800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b2000640 1 -- 172.21.15.100:0/3878034887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8b415b640 msgr2=0x7fa8b4148c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:26.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b2000640 1 --2- 172.21.15.100:0/3878034887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8b415b640 0x7fa8b4148c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:26.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b2000640 1 -- 172.21.15.100:0/3878034887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b40b1a90 con 0x7fa8b40cc750 2024-04-23T14:33:26.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b2a00640 1 --2- 172.21.15.100:0/3878034887 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa8b40b2a10 0x7fa8b4099800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:26.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b3400640 1 --2- 172.21.15.100:0/3878034887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8b415b640 0x7fa8b4148c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:26.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8b2000640 1 --2- 172.21.15.100:0/3878034887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b40cc750 0x7fa8b4148700 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00ad040 tx=0x7fa8a001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:26.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.517+0000 7fa8aaa00640 1 -- 172.21.15.100:0/3878034887 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a00bf610 con 0x7fa8b40cc750 2024-04-23T14:33:26.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.521+0000 7fa8aaa00640 1 -- 172.21.15.100:0/3878034887 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8a00abe50 con 0x7fa8b40cc750 2024-04-23T14:33:26.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.521+0000 7fa8aaa00640 1 -- 172.21.15.100:0/3878034887 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a00bf9b0 con 0x7fa8b40cc750 2024-04-23T14:33:26.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.521+0000 7fa8b9800640 1 -- 172.21.15.100:0/3878034887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b40b14d0 con 0x7fa8b40cc750 2024-04-23T14:33:26.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.521+0000 7fa8b9800640 1 -- 172.21.15.100:0/3878034887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b414d120 con 0x7fa8b40cc750 2024-04-23T14:33:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.521+0000 7fa8b9800640 1 -- 172.21.15.100:0/3878034887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8b4068ad0 con 0x7fa8b40cc750 2024-04-23T14:33:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.521+0000 7fa8aaa00640 1 -- 172.21.15.100:0/3878034887 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa8a00bd020 con 0x7fa8b40cc750 2024-04-23T14:33:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.521+0000 7fa8aaa00640 1 --2- 172.21.15.100:0/3878034887 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa894068380 0x7fa89406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.521+0000 7fa8aaa00640 1 -- 172.21.15.100:0/3878034887 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa8a0121480 con 0x7fa8b40cc750 2024-04-23T14:33:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.521+0000 7fa8b2a00640 1 --2- 172.21.15.100:0/3878034887 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa894068380 0x7fa89406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:26.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.521+0000 7fa8b2a00640 1 --2- 172.21.15.100:0/3878034887 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa894068380 0x7fa89406a840 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40cc720 tx=0x7fa8a400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:26.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.525+0000 7fa8aaa00640 1 -- 172.21.15.100:0/3878034887 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa8a0016090 con 0x7fa8b40cc750 2024-04-23T14:33:26.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.717+0000 7fa8b9800640 1 -- 172.21.15.100:0/3878034887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8b4152d10 con 0x7fa8b40cc750 2024-04-23T14:33:26.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.717+0000 7fa8aaa00640 1 -- 172.21.15.100:0/3878034887 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7fa8a00f1750 con 0x7fa8b40cc750 2024-04-23T14:33:26.719 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:26.719 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1456958549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1456958549},{"type":"v1","addr":"172.21.15.100:6836","nonce":1456958549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3655609551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3655609551},{"type":"v1","addr":"172.21.15.100:6837","nonce":3655609551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/4242180972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":4242180972},{"type":"v1","addr":"172.21.15.151:6834","nonce":4242180972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:32:08.976576+0000","modified":"2024-04-23T14:32:34.057740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1533322173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1533322173},{"type":"v1","addr":"172.21.15.151:6835","nonce":1533322173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-23T14:33:26.720 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 9 2024-04-23T14:33:26.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.717+0000 7fa8b9800640 1 -- 172.21.15.100:0/3878034887 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa894068380 msgr2=0x7fa89406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:26.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.717+0000 7fa8b9800640 1 --2- 172.21.15.100:0/3878034887 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa894068380 0x7fa89406a840 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40cc720 tx=0x7fa8a400b040 comp rx=0 tx=0).stop 2024-04-23T14:33:26.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.717+0000 7fa8b9800640 1 -- 172.21.15.100:0/3878034887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b40cc750 msgr2=0x7fa8b4148700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:26.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.717+0000 7fa8b9800640 1 --2- 172.21.15.100:0/3878034887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b40cc750 0x7fa8b4148700 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00ad040 tx=0x7fa8a001acb0 comp rx=0 tx=0).stop 2024-04-23T14:33:26.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.717+0000 7fa8b9800640 1 -- 172.21.15.100:0/3878034887 shutdown_connections 2024-04-23T14:33:26.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.717+0000 7fa8b9800640 1 --2- 172.21.15.100:0/3878034887 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa894068380 0x7fa89406a840 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:26.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.717+0000 7fa8b9800640 1 --2- 172.21.15.100:0/3878034887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8b415b640 0x7fa8b4148c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:26.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.717+0000 7fa8b9800640 1 --2- 172.21.15.100:0/3878034887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b40cc750 0x7fa8b4148700 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:26.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.717+0000 7fa8b9800640 1 --2- 172.21.15.100:0/3878034887 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa8b40b2a10 0x7fa8b4099800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:26.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.717+0000 7fa8b9800640 1 -- 172.21.15.100:0/3878034887 >> 172.21.15.100:0/3878034887 conn(0x7fa8b40b1e80 msgr2=0x7fa8b415a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:26.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.717+0000 7fa8b9800640 1 -- 172.21.15.100:0/3878034887 shutdown_connections 2024-04-23T14:33:26.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.717+0000 7fa8b9800640 1 -- 172.21.15.100:0/3878034887 wait complete. 2024-04-23T14:33:26.740 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:26.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.873+0000 7f4a3ea00640 1 Processor -- start 2024-04-23T14:33:26.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.873+0000 7f4a3ea00640 1 -- start start 2024-04-23T14:33:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.873+0000 7f4a3ea00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4a380cfd30 0x7f4a38162ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.873+0000 7f4a3ea00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a3806e3a0 con 0x7f4a381479e0 2024-04-23T14:33:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.873+0000 7f4a3ea00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a3806bc50 con 0x7f4a38071ce0 2024-04-23T14:33:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.873+0000 7f4a3ea00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f4a3806c580 con 0x7f4a380cfd30 2024-04-23T14:33:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.873+0000 7f4a37e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4a380cfd30 0x7f4a38162ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.873+0000 7f4a2f400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f4a381479e0 0x7f4a3806dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:58106/0 (socket says 172.21.15.100:58106) 2024-04-23T14:33:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.873+0000 7f4a2f400640 1 -- 172.21.15.100:0/2355271173 learned_addr learned my addr 172.21.15.100:0/2355271173 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:26.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.877+0000 7f4a37e00640 1 -- 172.21.15.100:0/2355271173 >> v1:172.21.15.151:6789/0 conn(0x7f4a38071ce0 legacy=0x7f4a3806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:26.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.877+0000 7f4a37e00640 1 -- 172.21.15.100:0/2355271173 >> v1:172.21.15.100:6789/0 conn(0x7f4a381479e0 legacy=0x7f4a3806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:26.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.877+0000 7f4a37e00640 1 -- 172.21.15.100:0/2355271173 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3806bf30 con 0x7f4a380cfd30 2024-04-23T14:33:26.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.877+0000 7f4a37400640 1 -- 172.21.15.100:0/2355271173 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3138238558 0 0) 0x7f4a3806e3a0 con 0x7f4a381479e0 2024-04-23T14:33:26.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.877+0000 7f4a37e00640 1 --2- 172.21.15.100:0/2355271173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a380cfd30 0x7f4a38162ce0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009d90 tx=0x7f4a2801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7478dd7004421217 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:26.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.877+0000 7f4a37400640 1 -- 172.21.15.100:0/2355271173 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a280b2070 con 0x7f4a380cfd30 2024-04-23T14:33:26.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.877+0000 7f4a37400640 1 -- 172.21.15.100:0/2355271173 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a280a5830 con 0x7f4a380cfd30 2024-04-23T14:33:26.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.877+0000 7f4a37400640 1 -- 172.21.15.100:0/2355271173 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a280a5c00 con 0x7f4a380cfd30 2024-04-23T14:33:26.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.877+0000 7f4a3ea00640 1 -- 172.21.15.100:0/2355271173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a380cfd30 msgr2=0x7f4a38162ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:26.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.877+0000 7f4a3ea00640 1 --2- 172.21.15.100:0/2355271173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a380cfd30 0x7f4a38162ce0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009d90 tx=0x7f4a2801c970 comp rx=0 tx=0).stop 2024-04-23T14:33:26.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.877+0000 7f4a3ea00640 1 -- 172.21.15.100:0/2355271173 shutdown_connections 2024-04-23T14:33:26.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.877+0000 7f4a3ea00640 1 --2- 172.21.15.100:0/2355271173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a380cfd30 0x7f4a38162ce0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:26.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a3ea00640 1 -- 172.21.15.100:0/2355271173 >> 172.21.15.100:0/2355271173 conn(0x7f4a380b1e80 msgr2=0x7f4a380b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:26.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a3ea00640 1 -- 172.21.15.100:0/2355271173 shutdown_connections 2024-04-23T14:33:26.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a3ea00640 1 -- 172.21.15.100:0/2355271173 wait complete. 2024-04-23T14:33:26.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a3ea00640 1 Processor -- start 2024-04-23T14:33:26.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a3ea00640 1 -- start start 2024-04-23T14:33:26.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a3ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a38071ce0 0x7f4a38157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:26.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a3ea00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4a380cfd30 0x7f4a3809aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:26.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a3ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a381479e0 0x7f4a38160c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:26.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a3ea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3816d010 con 0x7f4a381479e0 2024-04-23T14:33:26.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a3ea00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3816ce90 con 0x7f4a38071ce0 2024-04-23T14:33:26.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a3ea00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4a3816d190 con 0x7f4a380cfd30 2024-04-23T14:33:26.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a3ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a381479e0 0x7f4a38160c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:26.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a37e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a38071ce0 0x7f4a38157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:26.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a3ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a381479e0 0x7f4a38160c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48300/0 (socket says 172.21.15.100:48300) 2024-04-23T14:33:26.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a3ca00640 1 -- 172.21.15.100:0/2145606449 learned_addr learned my addr 172.21.15.100:0/2145606449 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:26.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a2f400640 1 --2- 172.21.15.100:0/2145606449 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4a380cfd30 0x7f4a3809aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:26.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a3ca00640 1 -- 172.21.15.100:0/2145606449 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4a380cfd30 msgr2=0x7f4a3809aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:26.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a3ca00640 1 --2- 172.21.15.100:0/2145606449 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4a380cfd30 0x7f4a3809aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:26.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a3ca00640 1 -- 172.21.15.100:0/2145606449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a38071ce0 msgr2=0x7f4a38157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:26.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a3ca00640 1 --2- 172.21.15.100:0/2145606449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a38071ce0 0x7f4a38157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:26.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a3ca00640 1 -- 172.21.15.100:0/2145606449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a38157c50 con 0x7f4a381479e0 2024-04-23T14:33:26.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a3ca00640 1 --2- 172.21.15.100:0/2145606449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a381479e0 0x7f4a38160c80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000c370 tx=0x7f4a3000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:26.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a34c00640 1 -- 172.21.15.100:0/2145606449 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a30017070 con 0x7f4a381479e0 2024-04-23T14:33:26.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a34c00640 1 -- 172.21.15.100:0/2145606449 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a300052d0 con 0x7f4a381479e0 2024-04-23T14:33:26.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a34c00640 1 -- 172.21.15.100:0/2145606449 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a30003840 con 0x7f4a381479e0 2024-04-23T14:33:26.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.881+0000 7f4a3ea00640 1 -- 172.21.15.100:0/2145606449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a38157ef0 con 0x7f4a381479e0 2024-04-23T14:33:26.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.885+0000 7f4a3ea00640 1 -- 172.21.15.100:0/2145606449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a38158790 con 0x7f4a381479e0 2024-04-23T14:33:26.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.885+0000 7f4a3ea00640 1 -- 172.21.15.100:0/2145606449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a381580c0 con 0x7f4a381479e0 2024-04-23T14:33:26.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.885+0000 7f4a34c00640 1 -- 172.21.15.100:0/2145606449 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4a3001a070 con 0x7f4a381479e0 2024-04-23T14:33:26.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.885+0000 7f4a34c00640 1 --2- 172.21.15.100:0/2145606449 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4a14068380 0x7f4a1406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:26.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.885+0000 7f4a34c00640 1 -- 172.21.15.100:0/2145606449 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f4a30085fe0 con 0x7f4a381479e0 2024-04-23T14:33:26.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.885+0000 7f4a37e00640 1 --2- 172.21.15.100:0/2145606449 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4a14068380 0x7f4a1406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:26.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.885+0000 7f4a37e00640 1 --2- 172.21.15.100:0/2145606449 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4a14068380 0x7f4a1406a840 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4a280af000 tx=0x7f4a280a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:26.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:26.889+0000 7f4a34c00640 1 -- 172.21.15.100:0/2145606449 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4a30005480 con 0x7f4a381479e0 2024-04-23T14:33:27.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.049+0000 7f4a3ea00640 1 -- 172.21.15.100:0/2145606449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a38158990 con 0x7f4a381479e0 2024-04-23T14:33:27.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.049+0000 7f4a34c00640 1 -- 172.21.15.100:0/2145606449 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18859 (secure 0 0 0) 0x7f4a38158990 con 0x7f4a381479e0 2024-04-23T14:33:27.053 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:27.053 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:32:49.784608+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:06.835560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:07.930489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-23T14:32:11.636814+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:27.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.053+0000 7f4a3ea00640 1 -- 172.21.15.100:0/2145606449 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4a14068380 msgr2=0x7f4a1406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.053+0000 7f4a3ea00640 1 --2- 172.21.15.100:0/2145606449 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4a14068380 0x7f4a1406a840 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4a280af000 tx=0x7f4a280a7070 comp rx=0 tx=0).stop 2024-04-23T14:33:27.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.053+0000 7f4a3ea00640 1 -- 172.21.15.100:0/2145606449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a381479e0 msgr2=0x7f4a38160c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.053+0000 7f4a3ea00640 1 --2- 172.21.15.100:0/2145606449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a381479e0 0x7f4a38160c80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000c370 tx=0x7f4a3000c830 comp rx=0 tx=0).stop 2024-04-23T14:33:27.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.053+0000 7f4a3ea00640 1 -- 172.21.15.100:0/2145606449 shutdown_connections 2024-04-23T14:33:27.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.053+0000 7f4a3ea00640 1 --2- 172.21.15.100:0/2145606449 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4a14068380 0x7f4a1406a840 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.053+0000 7f4a3ea00640 1 --2- 172.21.15.100:0/2145606449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a381479e0 0x7f4a38160c80 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.053+0000 7f4a3ea00640 1 --2- 172.21.15.100:0/2145606449 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4a380cfd30 0x7f4a3809aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.053+0000 7f4a3ea00640 1 --2- 172.21.15.100:0/2145606449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a38071ce0 0x7f4a38157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.053+0000 7f4a3ea00640 1 -- 172.21.15.100:0/2145606449 >> 172.21.15.100:0/2145606449 conn(0x7f4a380b1e80 msgr2=0x7f4a3815e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:27.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.053+0000 7f4a3ea00640 1 -- 172.21.15.100:0/2145606449 shutdown_connections 2024-04-23T14:33:27.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.053+0000 7f4a3ea00640 1 -- 172.21.15.100:0/2145606449 wait complete. 2024-04-23T14:33:27.075 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T14:33:27.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.253+0000 7f2891c00640 1 Processor -- start 2024-04-23T14:33:27.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.253+0000 7f2891c00640 1 -- start start 2024-04-23T14:33:27.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.253+0000 7f2891c00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f288c161960 0x7f288c0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.253+0000 7f2891c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f288c0c77e0 0x7f288c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.253+0000 7f2891c00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f288c0d0390 con 0x7f288c0cfe20 2024-04-23T14:33:27.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.253+0000 7f2891c00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f288c0d0670 con 0x7f288c0c77e0 2024-04-23T14:33:27.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.253+0000 7f2891c00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f288c0d0950 con 0x7f288c161960 2024-04-23T14:33:27.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.253+0000 7f2891200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f288c0c77e0 0x7f288c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.253+0000 7f2891200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f288c0c77e0 0x7f288c06d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48314/0 (socket says 172.21.15.100:48314) 2024-04-23T14:33:27.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.253+0000 7f2891200640 1 -- 172.21.15.100:0/1997265857 learned_addr learned my addr 172.21.15.100:0/1997265857 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:27.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.253+0000 7f288be00640 1 --2- 172.21.15.100:0/1997265857 >> v2:172.21.15.151:3300/0 conn(0x7f288c161960 0x7f288c0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.253+0000 7f2891200640 1 -- 172.21.15.100:0/1997265857 >> v2:172.21.15.151:3300/0 conn(0x7f288c161960 msgr2=0x7f288c0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:27.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.253+0000 7f2891200640 1 --2- 172.21.15.100:0/1997265857 >> v2:172.21.15.151:3300/0 conn(0x7f288c161960 0x7f288c0d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.253+0000 7f2891200640 1 -- 172.21.15.100:0/1997265857 >> v1:172.21.15.151:6789/0 conn(0x7f288c0cfe20 legacy=0x7f288c0c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:27.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.253+0000 7f2891200640 1 -- 172.21.15.100:0/1997265857 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288c0d0c30 con 0x7f288c0c77e0 2024-04-23T14:33:27.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.257+0000 7f2891200640 1 --2- 172.21.15.100:0/1997265857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f288c0c77e0 0x7f288c06d760 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f288c0c92a0 tx=0x7f288401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2276f344d4df05f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:27.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.257+0000 7f288b400640 1 -- 172.21.15.100:0/1997265857 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28840a5780 con 0x7f288c0c77e0 2024-04-23T14:33:27.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.257+0000 7f288b400640 1 -- 172.21.15.100:0/1997265857 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28840a5920 con 0x7f288c0c77e0 2024-04-23T14:33:27.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.257+0000 7f288b400640 1 -- 172.21.15.100:0/1997265857 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28840a5cf0 con 0x7f288c0c77e0 2024-04-23T14:33:27.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.257+0000 7f2891c00640 1 -- 172.21.15.100:0/1997265857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f288c0c77e0 msgr2=0x7f288c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.257+0000 7f2891c00640 1 --2- 172.21.15.100:0/1997265857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f288c0c77e0 0x7f288c06d760 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f288c0c92a0 tx=0x7f288401c9b0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.257+0000 7f2891c00640 1 -- 172.21.15.100:0/1997265857 shutdown_connections 2024-04-23T14:33:27.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.257+0000 7f2891c00640 1 --2- 172.21.15.100:0/1997265857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f288c0c77e0 0x7f288c06d760 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.257+0000 7f2891c00640 1 --2- 172.21.15.100:0/1997265857 >> v2:172.21.15.151:3300/0 conn(0x7f288c161960 0x7f288c0d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.257+0000 7f2891c00640 1 -- 172.21.15.100:0/1997265857 >> 172.21.15.100:0/1997265857 conn(0x7f288c0b3e60 msgr2=0x7f288c0b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:27.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.257+0000 7f2891c00640 1 -- 172.21.15.100:0/1997265857 shutdown_connections 2024-04-23T14:33:27.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.257+0000 7f2891c00640 1 -- 172.21.15.100:0/1997265857 wait complete. 2024-04-23T14:33:27.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f2891c00640 1 Processor -- start 2024-04-23T14:33:27.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f2891c00640 1 -- start start 2024-04-23T14:33:27.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f2891c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f288c0c77e0 0x7f288c0ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f2891c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f288c0cfe20 0x7f288c153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f2891c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f288c161960 0x7f288c160c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f2891c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f288c06e840 con 0x7f288c0c77e0 2024-04-23T14:33:27.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f2891c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f288c06e6c0 con 0x7f288c0cfe20 2024-04-23T14:33:27.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f2891c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f288c06e9c0 con 0x7f288c161960 2024-04-23T14:33:27.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f288be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f288c0c77e0 0x7f288c0ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f288be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f288c0c77e0 0x7f288c0ca460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48326/0 (socket says 172.21.15.100:48326) 2024-04-23T14:33:27.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f288be00640 1 -- 172.21.15.100:0/1309100772 learned_addr learned my addr 172.21.15.100:0/1309100772 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:27.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f2883400640 1 --2- 172.21.15.100:0/1309100772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f288c0cfe20 0x7f288c153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f2891200640 1 --2- 172.21.15.100:0/1309100772 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f288c161960 0x7f288c160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f288be00640 1 -- 172.21.15.100:0/1309100772 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f288c161960 msgr2=0x7f288c160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f288be00640 1 --2- 172.21.15.100:0/1309100772 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f288c161960 0x7f288c160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f288be00640 1 -- 172.21.15.100:0/1309100772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f288c0cfe20 msgr2=0x7f288c153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f288be00640 1 --2- 172.21.15.100:0/1309100772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f288c0cfe20 0x7f288c153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f288be00640 1 -- 172.21.15.100:0/1309100772 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288c154100 con 0x7f288c0c77e0 2024-04-23T14:33:27.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f2883400640 1 --2- 172.21.15.100:0/1309100772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f288c0cfe20 0x7f288c153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:27.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f288be00640 1 --2- 172.21.15.100:0/1309100772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f288c0c77e0 0x7f288c0ca460 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f287c00d830 tx=0x7f287c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:27.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f2888c00640 1 -- 172.21.15.100:0/1309100772 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c00b6c0 con 0x7f288c0c77e0 2024-04-23T14:33:27.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f2888c00640 1 -- 172.21.15.100:0/1309100772 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f287c019070 con 0x7f288c0c77e0 2024-04-23T14:33:27.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f2888c00640 1 -- 172.21.15.100:0/1309100772 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c00ba60 con 0x7f288c0c77e0 2024-04-23T14:33:27.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f2891c00640 1 -- 172.21.15.100:0/1309100772 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f288c0b7cf0 con 0x7f288c0c77e0 2024-04-23T14:33:27.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f2891c00640 1 -- 172.21.15.100:0/1309100772 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f288c0b7450 con 0x7f288c0c77e0 2024-04-23T14:33:27.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.261+0000 7f2891c00640 1 -- 172.21.15.100:0/1309100772 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f288c075c20 con 0x7f288c0c77e0 2024-04-23T14:33:27.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.265+0000 7f2888c00640 1 -- 172.21.15.100:0/1309100772 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f287c01e460 con 0x7f288c0c77e0 2024-04-23T14:33:27.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.265+0000 7f2888c00640 1 --2- 172.21.15.100:0/1309100772 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2868068380 0x7f286806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.265+0000 7f2888c00640 1 -- 172.21.15.100:0/1309100772 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f287c0873f0 con 0x7f288c0c77e0 2024-04-23T14:33:27.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.265+0000 7f2883400640 1 --2- 172.21.15.100:0/1309100772 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2868068380 0x7f286806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.265+0000 7f2883400640 1 --2- 172.21.15.100:0/1309100772 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2868068380 0x7f286806a840 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f288c0cbee0 tx=0x7f2874009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:27.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.269+0000 7f2888c00640 1 -- 172.21.15.100:0/1309100772 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f288c154100 con 0x7f288c0c77e0 2024-04-23T14:33:27.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.457+0000 7f2891c00640 1 -- 172.21.15.100:0/1309100772 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f288c0b36a0 con 0x7f288c0c77e0 2024-04-23T14:33:27.460 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-04-23T14:33:27.457+0000 7f7a7e400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T14:33:27.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.457+0000 7f2888c00640 1 -- 172.21.15.100:0/1309100772 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f287c057640 con 0x7f288c0c77e0 2024-04-23T14:33:27.462 INFO:teuthology.orchestra.run.smithi100.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T14:33:27.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.461+0000 7f2891c00640 1 -- 172.21.15.100:0/1309100772 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2868068380 msgr2=0x7f286806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.461+0000 7f2891c00640 1 --2- 172.21.15.100:0/1309100772 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2868068380 0x7f286806a840 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f288c0cbee0 tx=0x7f2874009040 comp rx=0 tx=0).stop 2024-04-23T14:33:27.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.461+0000 7f2891c00640 1 -- 172.21.15.100:0/1309100772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f288c0c77e0 msgr2=0x7f288c0ca460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.461+0000 7f2891c00640 1 --2- 172.21.15.100:0/1309100772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f288c0c77e0 0x7f288c0ca460 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f287c00d830 tx=0x7f287c00dcf0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.461+0000 7f2891c00640 1 -- 172.21.15.100:0/1309100772 shutdown_connections 2024-04-23T14:33:27.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.461+0000 7f2891c00640 1 --2- 172.21.15.100:0/1309100772 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2868068380 0x7f286806a840 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.461+0000 7f2891c00640 1 --2- 172.21.15.100:0/1309100772 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f288c161960 0x7f288c160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.461+0000 7f2891c00640 1 --2- 172.21.15.100:0/1309100772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f288c0cfe20 0x7f288c153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.461+0000 7f2891c00640 1 --2- 172.21.15.100:0/1309100772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f288c0c77e0 0x7f288c0ca460 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.461+0000 7f2891c00640 1 -- 172.21.15.100:0/1309100772 >> 172.21.15.100:0/1309100772 conn(0x7f288c0b3e60 msgr2=0x7f288c0b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:27.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.465+0000 7f2891c00640 1 -- 172.21.15.100:0/1309100772 shutdown_connections 2024-04-23T14:33:27.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.465+0000 7f2891c00640 1 -- 172.21.15.100:0/1309100772 wait complete. 2024-04-23T14:33:27.476 INFO:tasks.ceph.mds.d.smithi151.stderr: -5117> 2024-04-23T14:32:13.211+0000 7f70b3000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:33:27.486 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-23T14:33:27.528 INFO:tasks.ceph.mds.d.smithi151.stdout:starting mds.d at 2024-04-23T14:33:27.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.621+0000 7f4ed3600640 1 Processor -- start 2024-04-23T14:33:27.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.621+0000 7f4ed3600640 1 -- start start 2024-04-23T14:33:27.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.621+0000 7f4ed3600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4ecc06f8c0 0x7f4ecc06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.621+0000 7f4ed3600640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f4ecc0ce7f0 0x7f4ecc0add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.621+0000 7f4ed3600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ecc06dd60 con 0x7f4ecc0d0170 2024-04-23T14:33:27.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.621+0000 7f4ed3600640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f4ecc0ad050 con 0x7f4ecc0ce7f0 2024-04-23T14:33:27.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.621+0000 7f4ed3600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4ecc0ad8f0 con 0x7f4ecc06f8c0 2024-04-23T14:33:27.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.621+0000 7f4ecbe00640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f4ecc0d0170 0x7f4ecc0ce0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:59062/0 (socket says 172.21.15.100:59062) 2024-04-23T14:33:27.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.621+0000 7f4ecbe00640 1 -- 172.21.15.100:0/890295418 learned_addr learned my addr 172.21.15.100:0/890295418 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:27.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.621+0000 7f4ed0c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4ecc06f8c0 0x7f4ecc06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.621+0000 7f4ed1600640 1 --2- 172.21.15.100:0/890295418 >> v2:172.21.15.151:3300/0 conn(0x7f4ecc0ce7f0 0x7f4ecc0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.621+0000 7f4ecb400640 1 -- 172.21.15.100:0/890295418 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4217823524 0 0) 0x7f4eb8001120 con 0x7f4ecc0d0170 2024-04-23T14:33:27.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.621+0000 7f4ecb400640 1 -- 172.21.15.100:0/890295418 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4eb0003530 con 0x7f4ecc0d0170 2024-04-23T14:33:27.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.621+0000 7f4ed0c00640 1 -- 172.21.15.100:0/890295418 >> v2:172.21.15.151:3300/0 conn(0x7f4ecc0ce7f0 msgr2=0x7f4ecc0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:27.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.621+0000 7f4ed0c00640 1 --2- 172.21.15.100:0/890295418 >> v2:172.21.15.151:3300/0 conn(0x7f4ecc0ce7f0 0x7f4ecc0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.621+0000 7f4ed0c00640 1 -- 172.21.15.100:0/890295418 >> v1:172.21.15.151:6789/0 conn(0x7f4ecc0d0170 legacy=0x7f4ecc0ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.621+0000 7f4ed0c00640 1 -- 172.21.15.100:0/890295418 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ecc0d09c0 con 0x7f4ecc06f8c0 2024-04-23T14:33:27.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.625+0000 7f4ed0c00640 1 --2- 172.21.15.100:0/890295418 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4ecc06f8c0 0x7f4ecc06c6c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009d90 tx=0x7f4ebc01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=573f1fd6858b5798 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:27.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.625+0000 7f4ecb400640 1 -- 172.21.15.100:0/890295418 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc0b2070 con 0x7f4ecc06f8c0 2024-04-23T14:33:27.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.625+0000 7f4ecb400640 1 -- 172.21.15.100:0/890295418 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ebc0a5a30 con 0x7f4ecc06f8c0 2024-04-23T14:33:27.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.625+0000 7f4ecb400640 1 -- 172.21.15.100:0/890295418 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc0a5e00 con 0x7f4ecc06f8c0 2024-04-23T14:33:27.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.625+0000 7f4ed3600640 1 -- 172.21.15.100:0/890295418 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4ecc06f8c0 msgr2=0x7f4ecc06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.625+0000 7f4ed3600640 1 --2- 172.21.15.100:0/890295418 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4ecc06f8c0 0x7f4ecc06c6c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009d90 tx=0x7f4ebc01c970 comp rx=0 tx=0).stop 2024-04-23T14:33:27.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.625+0000 7f4ed3600640 1 -- 172.21.15.100:0/890295418 shutdown_connections 2024-04-23T14:33:27.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.625+0000 7f4ed3600640 1 --2- 172.21.15.100:0/890295418 >> v2:172.21.15.151:3300/0 conn(0x7f4ecc0ce7f0 0x7f4ecc0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.625+0000 7f4ed3600640 1 --2- 172.21.15.100:0/890295418 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4ecc06f8c0 0x7f4ecc06c6c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.625+0000 7f4ed3600640 1 -- 172.21.15.100:0/890295418 >> 172.21.15.100:0/890295418 conn(0x7f4ecc0b5200 msgr2=0x7f4ecc0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:27.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.625+0000 7f4ed3600640 1 -- 172.21.15.100:0/890295418 shutdown_connections 2024-04-23T14:33:27.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.625+0000 7f4ed3600640 1 -- 172.21.15.100:0/890295418 wait complete. 2024-04-23T14:33:27.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ed3600640 1 Processor -- start 2024-04-23T14:33:27.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ed3600640 1 -- start start 2024-04-23T14:33:27.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ed3600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ecc06f8c0 0x7f4ecc072610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ed3600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4ecc0ce7f0 0x7f4ecc1f76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ed3600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ecc0d0170 0x7f4ecc15e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ed3600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc0aead0 con 0x7f4ecc0d0170 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ed3600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc0d0760 con 0x7f4ecc06f8c0 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ed3600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4ecc0aec50 con 0x7f4ecc0ce7f0 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ed1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ecc0d0170 0x7f4ecc15e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ed1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ecc0d0170 0x7f4ecc15e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48340/0 (socket says 172.21.15.100:48340) 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ed1600640 1 -- 172.21.15.100:0/1884233355 learned_addr learned my addr 172.21.15.100:0/1884233355 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ed0c00640 1 --2- 172.21.15.100:0/1884233355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ecc06f8c0 0x7f4ecc072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ecbe00640 1 --2- 172.21.15.100:0/1884233355 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4ecc0ce7f0 0x7f4ecc1f76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ed1600640 1 -- 172.21.15.100:0/1884233355 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4ecc0ce7f0 msgr2=0x7f4ecc1f76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ed1600640 1 --2- 172.21.15.100:0/1884233355 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4ecc0ce7f0 0x7f4ecc1f76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ed1600640 1 -- 172.21.15.100:0/1884233355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ecc06f8c0 msgr2=0x7f4ecc072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ed1600640 1 --2- 172.21.15.100:0/1884233355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ecc06f8c0 0x7f4ecc072610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ed1600640 1 -- 172.21.15.100:0/1884233355 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ecc072e40 con 0x7f4ecc0d0170 2024-04-23T14:33:27.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ed1600640 1 --2- 172.21.15.100:0/1884233355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ecc0d0170 0x7f4ecc15e6f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400e930 tx=0x7f4ec400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:27.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ec8c00640 1 -- 172.21.15.100:0/1884233355 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4017070 con 0x7f4ecc0d0170 2024-04-23T14:33:27.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ec8c00640 1 -- 172.21.15.100:0/1884233355 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec400c9a0 con 0x7f4ecc0d0170 2024-04-23T14:33:27.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ec8c00640 1 -- 172.21.15.100:0/1884233355 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec400cd70 con 0x7f4ecc0d0170 2024-04-23T14:33:27.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ed3600640 1 -- 172.21.15.100:0/1884233355 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ecc073120 con 0x7f4ecc0d0170 2024-04-23T14:33:27.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.629+0000 7f4ed3600640 1 -- 172.21.15.100:0/1884233355 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ecc156fa0 con 0x7f4ecc0d0170 2024-04-23T14:33:27.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.633+0000 7f4ed3600640 1 -- 172.21.15.100:0/1884233355 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ecc1571a0 con 0x7f4ecc0d0170 2024-04-23T14:33:27.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.633+0000 7f4ec8c00640 1 -- 172.21.15.100:0/1884233355 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4ec401a070 con 0x7f4ecc0d0170 2024-04-23T14:33:27.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.633+0000 7f4ec8c00640 1 --2- 172.21.15.100:0/1884233355 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4eac068380 0x7f4eac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.633+0000 7f4ec8c00640 1 -- 172.21.15.100:0/1884233355 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7f4ec4087140 con 0x7f4ecc0d0170 2024-04-23T14:33:27.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.633+0000 7f4ed0c00640 1 --2- 172.21.15.100:0/1884233355 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4eac068380 0x7f4eac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.633+0000 7f4ed0c00640 1 --2- 172.21.15.100:0/1884233355 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4eac068380 0x7f4eac06a840 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc0af000 tx=0x7f4ebc0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:27.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.637+0000 7f4ec8c00640 1 -- 172.21.15.100:0/1884233355 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4ecc1571a0 con 0x7f4ecc0d0170 2024-04-23T14:33:27.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:27.829+0000 7f4ed3600640 1 -- 172.21.15.100:0/1884233355 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4ecc068ad0 con 0x7f4ecc0d0170 2024-04-23T14:33:28.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.461+0000 7f4ec8c00640 1 -- 172.21.15.100:0/1884233355 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f4ec4056d00 con 0x7f4ecc0d0170 2024-04-23T14:33:28.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.465+0000 7f4ed3600640 1 -- 172.21.15.100:0/1884233355 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4eac068380 msgr2=0x7f4eac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:28.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.465+0000 7f4ed3600640 1 --2- 172.21.15.100:0/1884233355 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4eac068380 0x7f4eac06a840 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc0af000 tx=0x7f4ebc0a7070 comp rx=0 tx=0).stop 2024-04-23T14:33:28.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.465+0000 7f4ed3600640 1 -- 172.21.15.100:0/1884233355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ecc0d0170 msgr2=0x7f4ecc15e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:28.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.465+0000 7f4ed3600640 1 --2- 172.21.15.100:0/1884233355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ecc0d0170 0x7f4ecc15e6f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400e930 tx=0x7f4ec400edf0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.465+0000 7f4ed3600640 1 -- 172.21.15.100:0/1884233355 shutdown_connections 2024-04-23T14:33:28.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.465+0000 7f4ed3600640 1 --2- 172.21.15.100:0/1884233355 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4eac068380 0x7f4eac06a840 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.465+0000 7f4ed3600640 1 --2- 172.21.15.100:0/1884233355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ecc0d0170 0x7f4ecc15e6f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.465+0000 7f4ed3600640 1 --2- 172.21.15.100:0/1884233355 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4ecc0ce7f0 0x7f4ecc1f76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.465+0000 7f4ed3600640 1 --2- 172.21.15.100:0/1884233355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ecc06f8c0 0x7f4ecc072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.465+0000 7f4ed3600640 1 -- 172.21.15.100:0/1884233355 >> 172.21.15.100:0/1884233355 conn(0x7f4ecc0b5200 msgr2=0x7f4ecc06bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:28.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.465+0000 7f4ed3600640 1 -- 172.21.15.100:0/1884233355 shutdown_connections 2024-04-23T14:33:28.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.465+0000 7f4ed3600640 1 -- 172.21.15.100:0/1884233355 wait complete. 2024-04-23T14:33:28.487 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-23T14:33:28.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.625+0000 7fe6e0000640 1 Processor -- start 2024-04-23T14:33:28.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.625+0000 7fe6e0000640 1 -- start start 2024-04-23T14:33:28.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.625+0000 7fe6e0000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fe6d815c7a0 0x7fe6d80af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:28.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.625+0000 7fe6e0000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6d806bb20 0x7fe6d80ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:28.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.625+0000 7fe6e0000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6d806cbb0 con 0x7fe6d80cd130 2024-04-23T14:33:28.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.625+0000 7fe6e0000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fe6d80cd5a0 con 0x7fe6d815c7a0 2024-04-23T14:33:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.625+0000 7fe6e0000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe6d806d450 con 0x7fe6d806bb20 2024-04-23T14:33:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.625+0000 7fe6de000640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe6d80cd130 0x7fe6d806e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:60330/0 (socket says 172.21.15.100:60330) 2024-04-23T14:33:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.625+0000 7fe6de000640 1 -- 172.21.15.100:0/319402712 learned_addr learned my addr 172.21.15.100:0/319402712 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6dcc00640 1 --2- 172.21.15.100:0/319402712 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6d806bb20 0x7fe6d80ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6dd600640 1 --2- 172.21.15.100:0/319402712 >> v2:172.21.15.151:3300/0 conn(0x7fe6d815c7a0 0x7fe6d80af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6d7e00640 1 -- 172.21.15.100:0/319402712 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 874283546 0 0) 0x7fe6d806cbb0 con 0x7fe6d80cd130 2024-04-23T14:33:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6d7e00640 1 -- 172.21.15.100:0/319402712 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6c0003530 con 0x7fe6d80cd130 2024-04-23T14:33:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6d7e00640 1 -- 172.21.15.100:0/319402712 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3906523644 0 0) 0x7fe6c0003530 con 0x7fe6d80cd130 2024-04-23T14:33:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6d7e00640 1 -- 172.21.15.100:0/319402712 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6d806bb20 msgr2=0x7fe6d80ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6d7e00640 1 --2- 172.21.15.100:0/319402712 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6d806bb20 0x7fe6d80ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6d7e00640 1 -- 172.21.15.100:0/319402712 >> v2:172.21.15.151:3300/0 conn(0x7fe6d815c7a0 msgr2=0x7fe6d80af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6d7e00640 1 --2- 172.21.15.100:0/319402712 >> v2:172.21.15.151:3300/0 conn(0x7fe6d815c7a0 0x7fe6d80af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6d7e00640 1 -- 172.21.15.100:0/319402712 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d806ce90 con 0x7fe6d80cd130 2024-04-23T14:33:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6d7e00640 1 -- 172.21.15.100:0/319402712 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7fe6d0001a00 con 0x7fe6d80cd130 2024-04-23T14:33:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6d7e00640 1 -- 172.21.15.100:0/319402712 >> v1:172.21.15.100:6789/0 conn(0x7fe6d80cd130 legacy=0x7fe6d806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6d7e00640 1 --2- 172.21.15.100:0/319402712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6c00039c0 0x7fe6c0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6d7e00640 1 --2- 172.21.15.100:0/319402712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6c0004810 0x7fe6c0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6d7e00640 1 --2- 172.21.15.100:0/319402712 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6d806bb20 0x7fe6c00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6d7e00640 1 -- 172.21.15.100:0/319402712 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d80cd5a0 con 0x7fe6c00039c0 2024-04-23T14:33:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6d7e00640 1 -- 172.21.15.100:0/319402712 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d806d450 con 0x7fe6c0004810 2024-04-23T14:33:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6d7e00640 1 -- 172.21.15.100:0/319402712 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe6c000b9d0 con 0x7fe6d806bb20 2024-04-23T14:33:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6dd600640 1 --2- 172.21.15.100:0/319402712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6c00039c0 0x7fe6c0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6dcc00640 1 --2- 172.21.15.100:0/319402712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6c0004810 0x7fe6c0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:28.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6de000640 1 --2- 172.21.15.100:0/319402712 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6d806bb20 0x7fe6c00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:28.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6dd600640 1 -- 172.21.15.100:0/319402712 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6d806bb20 msgr2=0x7fe6c00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:28.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6dd600640 1 --2- 172.21.15.100:0/319402712 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6d806bb20 0x7fe6c00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6dd600640 1 -- 172.21.15.100:0/319402712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6c0004810 msgr2=0x7fe6c0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:28.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6dd600640 1 --2- 172.21.15.100:0/319402712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6c0004810 0x7fe6c0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6dd600640 1 -- 172.21.15.100:0/319402712 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c0003530 con 0x7fe6c00039c0 2024-04-23T14:33:28.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.629+0000 7fe6de000640 1 --2- 172.21.15.100:0/319402712 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6d806bb20 0x7fe6c00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:28.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.633+0000 7fe6dd600640 1 --2- 172.21.15.100:0/319402712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6c00039c0 0x7fe6c0003de0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009b10 tx=0x7fe6cc01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec369fbe3b8c2b5a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:28.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.633+0000 7fe6d7e00640 1 -- 172.21.15.100:0/319402712 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc0096b0 con 0x7fe6c00039c0 2024-04-23T14:33:28.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.633+0000 7fe6d7e00640 1 -- 172.21.15.100:0/319402712 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6cc0a8410 con 0x7fe6c00039c0 2024-04-23T14:33:28.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.633+0000 7fe6d7e00640 1 -- 172.21.15.100:0/319402712 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc0a8800 con 0x7fe6c00039c0 2024-04-23T14:33:28.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.633+0000 7fe6e0000640 1 -- 172.21.15.100:0/319402712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6c00039c0 msgr2=0x7fe6c0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:28.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.633+0000 7fe6e0000640 1 --2- 172.21.15.100:0/319402712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6c00039c0 0x7fe6c0003de0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009b10 tx=0x7fe6cc01cab0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.633+0000 7fe6de000640 1 -- 172.21.15.100:0/319402712 reap_dead start 2024-04-23T14:33:28.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.633+0000 7fe6e0000640 1 -- 172.21.15.100:0/319402712 shutdown_connections 2024-04-23T14:33:28.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.633+0000 7fe6e0000640 1 -- 172.21.15.100:0/319402712 >> 172.21.15.100:0/319402712 conn(0x7fe6d80b5200 msgr2=0x7fe6d80b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:28.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.633+0000 7fe6e0000640 1 -- 172.21.15.100:0/319402712 shutdown_connections 2024-04-23T14:33:28.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.633+0000 7fe6e0000640 1 -- 172.21.15.100:0/319402712 wait complete. 2024-04-23T14:33:28.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.633+0000 7fe6e0000640 1 Processor -- start 2024-04-23T14:33:28.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.633+0000 7fe6e0000640 1 -- start start 2024-04-23T14:33:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.633+0000 7fe6e0000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6c00039c0 0x7fe6d8161ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.633+0000 7fe6e0000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6d8155500 0x7fe6d815f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.633+0000 7fe6e0000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d80d0340 0x7fe6d80d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.633+0000 7fe6e0000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8167580 con 0x7fe6d80d0340 2024-04-23T14:33:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.633+0000 7fe6e0000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8167400 con 0x7fe6d8155500 2024-04-23T14:33:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.633+0000 7fe6e0000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe6d8167700 con 0x7fe6c00039c0 2024-04-23T14:33:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.637+0000 7fe6de000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d80d0340 0x7fe6d80d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.637+0000 7fe6de000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d80d0340 0x7fe6d80d07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39838/0 (socket says 172.21.15.100:39838) 2024-04-23T14:33:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.637+0000 7fe6de000640 1 -- 172.21.15.100:0/2765654464 learned_addr learned my addr 172.21.15.100:0/2765654464 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.637+0000 7fe6dcc00640 1 --2- 172.21.15.100:0/2765654464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6d8155500 0x7fe6d815f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:28.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.637+0000 7fe6dd600640 1 --2- 172.21.15.100:0/2765654464 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6c00039c0 0x7fe6d8161ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:28.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.637+0000 7fe6de000640 1 -- 172.21.15.100:0/2765654464 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6c00039c0 msgr2=0x7fe6d8161ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:28.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.637+0000 7fe6de000640 1 --2- 172.21.15.100:0/2765654464 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6c00039c0 0x7fe6d8161ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.637+0000 7fe6de000640 1 -- 172.21.15.100:0/2765654464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6d8155500 msgr2=0x7fe6d815f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:28.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.637+0000 7fe6de000640 1 --2- 172.21.15.100:0/2765654464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6d8155500 0x7fe6d815f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.637+0000 7fe6de000640 1 -- 172.21.15.100:0/2765654464 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d8160c40 con 0x7fe6d80d0340 2024-04-23T14:33:28.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.637+0000 7fe6de000640 1 --2- 172.21.15.100:0/2765654464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d80d0340 0x7fe6d80d07c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0007960 tx=0x7fe6d000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:28.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.637+0000 7fe6d5600640 1 -- 172.21.15.100:0/2765654464 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d000eb60 con 0x7fe6d80d0340 2024-04-23T14:33:28.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.637+0000 7fe6d5600640 1 -- 172.21.15.100:0/2765654464 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d000ed00 con 0x7fe6d80d0340 2024-04-23T14:33:28.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.637+0000 7fe6d5600640 1 -- 172.21.15.100:0/2765654464 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0014630 con 0x7fe6d80d0340 2024-04-23T14:33:28.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.637+0000 7fe6e0000640 1 -- 172.21.15.100:0/2765654464 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d8160f20 con 0x7fe6d80d0340 2024-04-23T14:33:28.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.637+0000 7fe6e0000640 1 -- 172.21.15.100:0/2765654464 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d81614e0 con 0x7fe6d80d0340 2024-04-23T14:33:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.637+0000 7fe6e0000640 1 -- 172.21.15.100:0/2765654464 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d8068ad0 con 0x7fe6d80d0340 2024-04-23T14:33:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.637+0000 7fe6d5600640 1 -- 172.21.15.100:0/2765654464 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe6d00147d0 con 0x7fe6d80d0340 2024-04-23T14:33:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.637+0000 7fe6d5600640 1 --2- 172.21.15.100:0/2765654464 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe6b8068380 0x7fe6b806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:28.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.641+0000 7fe6d5600640 1 -- 172.21.15.100:0/2765654464 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7fe6d0086c30 con 0x7fe6d80d0340 2024-04-23T14:33:28.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.641+0000 7fe6dd600640 1 --2- 172.21.15.100:0/2765654464 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe6b8068380 0x7fe6b806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:28.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.641+0000 7fe6dd600640 1 --2- 172.21.15.100:0/2765654464 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe6b8068380 0x7fe6b806a840 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0af000 tx=0x7fe6cc0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:28.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.645+0000 7fe6d5600640 1 -- 172.21.15.100:0/2765654464 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe6d0056610 con 0x7fe6d80d0340 2024-04-23T14:33:28.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:28.805+0000 7fe6e0000640 1 -- 172.21.15.100:0/2765654464 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe6d8159830 con 0x7fe6d80d0340 2024-04-23T14:33:29.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.469+0000 7fe6d5600640 1 -- 172.21.15.100:0/2765654464 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) ==== 181+0+0 (secure 0 0 0) 0x7fe6d005a910 con 0x7fe6d80d0340 2024-04-23T14:33:29.472 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' removed 2024-04-23T14:33:29.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.469+0000 7fe6e0000640 1 -- 172.21.15.100:0/2765654464 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe6b8068380 msgr2=0x7fe6b806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:29.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.469+0000 7fe6e0000640 1 --2- 172.21.15.100:0/2765654464 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe6b8068380 0x7fe6b806a840 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0af000 tx=0x7fe6cc0046d0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.469+0000 7fe6e0000640 1 -- 172.21.15.100:0/2765654464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d80d0340 msgr2=0x7fe6d80d07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:29.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.469+0000 7fe6e0000640 1 --2- 172.21.15.100:0/2765654464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d80d0340 0x7fe6d80d07c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0007960 tx=0x7fe6d000e530 comp rx=0 tx=0).stop 2024-04-23T14:33:29.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.469+0000 7fe6e0000640 1 -- 172.21.15.100:0/2765654464 shutdown_connections 2024-04-23T14:33:29.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.469+0000 7fe6e0000640 1 --2- 172.21.15.100:0/2765654464 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe6b8068380 0x7fe6b806a840 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.469+0000 7fe6e0000640 1 --2- 172.21.15.100:0/2765654464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d80d0340 0x7fe6d80d07c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.469+0000 7fe6e0000640 1 --2- 172.21.15.100:0/2765654464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6d8155500 0x7fe6d815f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.469+0000 7fe6e0000640 1 --2- 172.21.15.100:0/2765654464 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6c00039c0 0x7fe6d8161ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.473+0000 7fe6e0000640 1 -- 172.21.15.100:0/2765654464 >> 172.21.15.100:0/2765654464 conn(0x7fe6d80b5200 msgr2=0x7fe6d815af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:29.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.473+0000 7fe6e0000640 1 -- 172.21.15.100:0/2765654464 shutdown_connections 2024-04-23T14:33:29.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.473+0000 7fe6e0000640 1 -- 172.21.15.100:0/2765654464 wait complete. 2024-04-23T14:33:29.500 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-23T14:33:29.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.641+0000 7f20bec00640 1 Processor -- start 2024-04-23T14:33:29.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.641+0000 7f20bec00640 1 -- start start 2024-04-23T14:33:29.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.641+0000 7f20bec00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f20b815f890 0x7f20b815ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.641+0000 7f20bec00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20b80cefc0 con 0x7f20b80ce7e0 2024-04-23T14:33:29.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.641+0000 7f20bec00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20b80cf2a0 con 0x7f20b806d450 2024-04-23T14:33:29.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.641+0000 7f20bec00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f20b80cf580 con 0x7f20b815f890 2024-04-23T14:33:29.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.641+0000 7f20b7400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f20b80ce7e0 0x7f20b80d0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:60338/0 (socket says 172.21.15.100:60338) 2024-04-23T14:33:29.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.641+0000 7f20b7400640 1 -- 172.21.15.100:0/3002923901 learned_addr learned my addr 172.21.15.100:0/3002923901 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:29.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.641+0000 7f20b7e00640 1 --2- 172.21.15.100:0/3002923901 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f20b815f890 0x7f20b815ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.641+0000 7f20b6a00640 1 -- 172.21.15.100:0/3002923901 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2880548857 0 0) 0x7f20b80cefc0 con 0x7f20b80ce7e0 2024-04-23T14:33:29.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.641+0000 7f20b6a00640 1 -- 172.21.15.100:0/3002923901 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f209c003530 con 0x7f20b80ce7e0 2024-04-23T14:33:29.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.641+0000 7f20b6a00640 1 -- 172.21.15.100:0/3002923901 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4140525031 0 0) 0x7f20b80cf2a0 con 0x7f20b806d450 2024-04-23T14:33:29.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.641+0000 7f20b7e00640 1 -- 172.21.15.100:0/3002923901 >> v1:172.21.15.151:6789/0 conn(0x7f20b806d450 legacy=0x7f20b80cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:29.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.641+0000 7f20b7e00640 1 -- 172.21.15.100:0/3002923901 >> v1:172.21.15.100:6789/0 conn(0x7f20b80ce7e0 legacy=0x7f20b80d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:29.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.641+0000 7f20b7e00640 1 -- 172.21.15.100:0/3002923901 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b80ccbf0 con 0x7f20b815f890 2024-04-23T14:33:29.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.645+0000 7f20b7e00640 1 --2- 172.21.15.100:0/3002923901 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f20b815f890 0x7f20b815ed80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009dc0 tx=0x7f20a801c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=823c1001618f746b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:29.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.645+0000 7f20b6a00640 1 -- 172.21.15.100:0/3002923901 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a80b2070 con 0x7f20b815f890 2024-04-23T14:33:29.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.645+0000 7f20b6a00640 1 -- 172.21.15.100:0/3002923901 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20a80a5850 con 0x7f20b815f890 2024-04-23T14:33:29.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.645+0000 7f20b6a00640 1 -- 172.21.15.100:0/3002923901 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a80a5c20 con 0x7f20b815f890 2024-04-23T14:33:29.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.645+0000 7f20bec00640 1 -- 172.21.15.100:0/3002923901 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f20b815f890 msgr2=0x7f20b815ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:29.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.645+0000 7f20bec00640 1 --2- 172.21.15.100:0/3002923901 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f20b815f890 0x7f20b815ed80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009dc0 tx=0x7f20a801c990 comp rx=0 tx=0).stop 2024-04-23T14:33:29.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.645+0000 7f20bec00640 1 -- 172.21.15.100:0/3002923901 shutdown_connections 2024-04-23T14:33:29.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.645+0000 7f20bec00640 1 --2- 172.21.15.100:0/3002923901 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f20b815f890 0x7f20b815ed80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.645+0000 7f20bec00640 1 -- 172.21.15.100:0/3002923901 >> 172.21.15.100:0/3002923901 conn(0x7f20b80719d0 msgr2=0x7f20b8071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:29.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.645+0000 7f20bec00640 1 -- 172.21.15.100:0/3002923901 shutdown_connections 2024-04-23T14:33:29.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.645+0000 7f20bec00640 1 -- 172.21.15.100:0/3002923901 wait complete. 2024-04-23T14:33:29.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20bec00640 1 Processor -- start 2024-04-23T14:33:29.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20bec00640 1 -- start start 2024-04-23T14:33:29.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20bec00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20b806d450 0x7f20b80c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20bec00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f20b80ce7e0 0x7f20b80c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20bec00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20b815f890 0x7f20b80cb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20bec00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f20b816d0b0 con 0x7f20b815f890 2024-04-23T14:33:29.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20bec00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f20b816cf30 con 0x7f20b806d450 2024-04-23T14:33:29.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20bec00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f20b816d230 con 0x7f20b80ce7e0 2024-04-23T14:33:29.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20bcc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20b815f890 0x7f20b80cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20bcc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20b815f890 0x7f20b80cb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39844/0 (socket says 172.21.15.100:39844) 2024-04-23T14:33:29.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20bcc00640 1 -- 172.21.15.100:0/3072217001 learned_addr learned my addr 172.21.15.100:0/3072217001 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:29.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20b7e00640 1 --2- 172.21.15.100:0/3072217001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20b806d450 0x7f20b80c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20b7400640 1 --2- 172.21.15.100:0/3072217001 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f20b80ce7e0 0x7f20b80c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20bcc00640 1 -- 172.21.15.100:0/3072217001 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f20b80ce7e0 msgr2=0x7f20b80c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:29.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20bcc00640 1 --2- 172.21.15.100:0/3072217001 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f20b80ce7e0 0x7f20b80c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20bcc00640 1 -- 172.21.15.100:0/3072217001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20b806d450 msgr2=0x7f20b80c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:29.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20bcc00640 1 --2- 172.21.15.100:0/3072217001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20b806d450 0x7f20b80c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20bcc00640 1 -- 172.21.15.100:0/3072217001 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b80d2270 con 0x7f20b815f890 2024-04-23T14:33:29.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20bcc00640 1 --2- 172.21.15.100:0/3072217001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20b815f890 0x7f20b80cb480 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f20b000c410 tx=0x7f20b000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:29.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20afe00640 1 -- 172.21.15.100:0/3072217001 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0017070 con 0x7f20b815f890 2024-04-23T14:33:29.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20afe00640 1 -- 172.21.15.100:0/3072217001 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20b0003600 con 0x7f20b815f890 2024-04-23T14:33:29.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20afe00640 1 -- 172.21.15.100:0/3072217001 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b00039f0 con 0x7f20b815f890 2024-04-23T14:33:29.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20bec00640 1 -- 172.21.15.100:0/3072217001 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b80d2550 con 0x7f20b815f890 2024-04-23T14:33:29.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.649+0000 7f20bec00640 1 -- 172.21.15.100:0/3072217001 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b814f9f0 con 0x7f20b815f890 2024-04-23T14:33:29.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.653+0000 7f20bec00640 1 -- 172.21.15.100:0/3072217001 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b814fbf0 con 0x7f20b815f890 2024-04-23T14:33:29.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.653+0000 7f20afe00640 1 -- 172.21.15.100:0/3072217001 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f20b001a070 con 0x7f20b815f890 2024-04-23T14:33:29.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.653+0000 7f20afe00640 1 --2- 172.21.15.100:0/3072217001 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2094068380 0x7f209406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.653+0000 7f20afe00640 1 -- 172.21.15.100:0/3072217001 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6383+0+0 (secure 0 0 0) 0x7f20b0085fa0 con 0x7f20b815f890 2024-04-23T14:33:29.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.653+0000 7f20b7e00640 1 --2- 172.21.15.100:0/3072217001 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2094068380 0x7f209406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.653+0000 7f20b7e00640 1 --2- 172.21.15.100:0/3072217001 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2094068380 0x7f209406a840 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f20a80af000 tx=0x7f20a80a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:29.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.657+0000 7f20afe00640 1 -- 172.21.15.100:0/3072217001 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f20b814fbf0 con 0x7f20b815f890 2024-04-23T14:33:29.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:29.825+0000 7f20bec00640 1 -- 172.21.15.100:0/3072217001 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f20b80d2720 con 0x7f20b815f890 2024-04-23T14:33:30.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.469+0000 7f20afe00640 1 -- 172.21.15.100:0/3072217001 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) ==== 169+0+0 (secure 0 0 0) 0x7f20b0055bd0 con 0x7f20b815f890 2024-04-23T14:33:30.474 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' removed 2024-04-23T14:33:30.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.473+0000 7f20bec00640 1 -- 172.21.15.100:0/3072217001 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2094068380 msgr2=0x7f209406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:30.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.473+0000 7f20bec00640 1 --2- 172.21.15.100:0/3072217001 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2094068380 0x7f209406a840 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f20a80af000 tx=0x7f20a80a7070 comp rx=0 tx=0).stop 2024-04-23T14:33:30.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.473+0000 7f20bec00640 1 -- 172.21.15.100:0/3072217001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20b815f890 msgr2=0x7f20b80cb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:30.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.473+0000 7f20bec00640 1 --2- 172.21.15.100:0/3072217001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20b815f890 0x7f20b80cb480 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f20b000c410 tx=0x7f20b000c810 comp rx=0 tx=0).stop 2024-04-23T14:33:30.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.473+0000 7f20bec00640 1 -- 172.21.15.100:0/3072217001 shutdown_connections 2024-04-23T14:33:30.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.473+0000 7f20bec00640 1 --2- 172.21.15.100:0/3072217001 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2094068380 0x7f209406a840 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.473+0000 7f20bec00640 1 --2- 172.21.15.100:0/3072217001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20b815f890 0x7f20b80cb480 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.473+0000 7f20bec00640 1 --2- 172.21.15.100:0/3072217001 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f20b80ce7e0 0x7f20b80c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.473+0000 7f20bec00640 1 --2- 172.21.15.100:0/3072217001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20b806d450 0x7f20b80c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.473+0000 7f20bec00640 1 -- 172.21.15.100:0/3072217001 >> 172.21.15.100:0/3072217001 conn(0x7f20b80719d0 msgr2=0x7f20b80d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:30.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.473+0000 7f20bec00640 1 -- 172.21.15.100:0/3072217001 shutdown_connections 2024-04-23T14:33:30.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.473+0000 7f20bec00640 1 -- 172.21.15.100:0/3072217001 wait complete. 2024-04-23T14:33:30.505 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-23T14:33:30.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.649+0000 7fd731600640 1 Processor -- start 2024-04-23T14:33:30.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.649+0000 7fd731600640 1 -- start start 2024-04-23T14:33:30.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.649+0000 7fd731600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd72c15c7a0 0x7fd72c0af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:30.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.649+0000 7fd731600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd72c0cd300 0x7fd72c0ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:30.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.649+0000 7fd731600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd72c06cbb0 con 0x7fd72c06bb20 2024-04-23T14:33:30.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.649+0000 7fd731600640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fd72c06d450 con 0x7fd72c0cd300 2024-04-23T14:33:30.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.649+0000 7fd731600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd72c06ce90 con 0x7fd72c15c7a0 2024-04-23T14:33:30.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.649+0000 7fd72b400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd72c0cd300 0x7fd72c0ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:30.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.649+0000 7fd72b400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd72c0cd300 0x7fd72c0ad430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39852/0 (socket says 172.21.15.100:39852) 2024-04-23T14:33:30.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.649+0000 7fd72b400640 1 -- 172.21.15.100:0/4062856405 learned_addr learned my addr 172.21.15.100:0/4062856405 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:30.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.649+0000 7fd72aa00640 1 --2- 172.21.15.100:0/4062856405 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd72c15c7a0 0x7fd72c0af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:30.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.649+0000 7fd72b400640 1 -- 172.21.15.100:0/4062856405 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd72c15c7a0 msgr2=0x7fd72c0af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:30.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.649+0000 7fd72b400640 1 --2- 172.21.15.100:0/4062856405 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd72c15c7a0 0x7fd72c0af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.649+0000 7fd72b400640 1 -- 172.21.15.100:0/4062856405 >> v1:172.21.15.151:6789/0 conn(0x7fd72c06bb20 legacy=0x7fd72c0ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:30.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.649+0000 7fd72b400640 1 -- 172.21.15.100:0/4062856405 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72c06d170 con 0x7fd72c0cd300 2024-04-23T14:33:30.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.653+0000 7fd72b400640 1 --2- 172.21.15.100:0/4062856405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd72c0cd300 0x7fd72c0ad430 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd724009870 tx=0x7fd72401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929adaed1eccf16e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:30.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.653+0000 7fd72a000640 1 -- 172.21.15.100:0/4062856405 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7240b0070 con 0x7fd72c0cd300 2024-04-23T14:33:30.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.653+0000 7fd72a000640 1 -- 172.21.15.100:0/4062856405 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7240ab600 con 0x7fd72c0cd300 2024-04-23T14:33:30.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.653+0000 7fd72a000640 1 -- 172.21.15.100:0/4062856405 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7240ab9d0 con 0x7fd72c0cd300 2024-04-23T14:33:30.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.653+0000 7fd731600640 1 -- 172.21.15.100:0/4062856405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd72c0cd300 msgr2=0x7fd72c0ad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:30.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.653+0000 7fd731600640 1 --2- 172.21.15.100:0/4062856405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd72c0cd300 0x7fd72c0ad430 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd724009870 tx=0x7fd72401c4d0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.653+0000 7fd731600640 1 -- 172.21.15.100:0/4062856405 shutdown_connections 2024-04-23T14:33:30.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.653+0000 7fd731600640 1 --2- 172.21.15.100:0/4062856405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd72c0cd300 0x7fd72c0ad430 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.653+0000 7fd731600640 1 --2- 172.21.15.100:0/4062856405 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd72c15c7a0 0x7fd72c0af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.653+0000 7fd731600640 1 -- 172.21.15.100:0/4062856405 >> 172.21.15.100:0/4062856405 conn(0x7fd72c0b5200 msgr2=0x7fd72c0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:30.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.653+0000 7fd731600640 1 -- 172.21.15.100:0/4062856405 shutdown_connections 2024-04-23T14:33:30.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.653+0000 7fd731600640 1 -- 172.21.15.100:0/4062856405 wait complete. 2024-04-23T14:33:30.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd731600640 1 Processor -- start 2024-04-23T14:33:30.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd731600640 1 -- start start 2024-04-23T14:33:30.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd731600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd72c06bb20 0x7fd72c072c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:30.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd731600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd72c0cd300 0x7fd72c099770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:30.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd731600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd72c15c7a0 0x7fd72c151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:30.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd731600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c0ae550 con 0x7fd72c15c7a0 2024-04-23T14:33:30.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd731600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c0ae3d0 con 0x7fd72c06bb20 2024-04-23T14:33:30.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd731600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd72c0ae6d0 con 0x7fd72c0cd300 2024-04-23T14:33:30.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd72b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd72c15c7a0 0x7fd72c151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:30.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd72b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd72c15c7a0 0x7fd72c151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39858/0 (socket says 172.21.15.100:39858) 2024-04-23T14:33:30.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd72b400640 1 -- 172.21.15.100:0/1284409067 learned_addr learned my addr 172.21.15.100:0/1284409067 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:30.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd72aa00640 1 --2- 172.21.15.100:0/1284409067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd72c06bb20 0x7fd72c072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:30.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd722000640 1 --2- 172.21.15.100:0/1284409067 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd72c0cd300 0x7fd72c099770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:30.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd72b400640 1 -- 172.21.15.100:0/1284409067 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd72c0cd300 msgr2=0x7fd72c099770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:30.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd72b400640 1 --2- 172.21.15.100:0/1284409067 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd72c0cd300 0x7fd72c099770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd72b400640 1 -- 172.21.15.100:0/1284409067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd72c06bb20 msgr2=0x7fd72c072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:30.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd72b400640 1 --2- 172.21.15.100:0/1284409067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd72c06bb20 0x7fd72c072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd72b400640 1 -- 172.21.15.100:0/1284409067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72c0c3f20 con 0x7fd72c15c7a0 2024-04-23T14:33:30.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd72b400640 1 --2- 172.21.15.100:0/1284409067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd72c15c7a0 0x7fd72c151a10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd7240ae040 tx=0x7fd724019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:30.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd723400640 1 -- 172.21.15.100:0/1284409067 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7240b0040 con 0x7fd72c15c7a0 2024-04-23T14:33:30.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd723400640 1 -- 172.21.15.100:0/1284409067 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7240b4610 con 0x7fd72c15c7a0 2024-04-23T14:33:30.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd723400640 1 -- 172.21.15.100:0/1284409067 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7240b4a00 con 0x7fd72c15c7a0 2024-04-23T14:33:30.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd731600640 1 -- 172.21.15.100:0/1284409067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72c0c4200 con 0x7fd72c15c7a0 2024-04-23T14:33:30.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd731600640 1 -- 172.21.15.100:0/1284409067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd72c099cb0 con 0x7fd72c15c7a0 2024-04-23T14:33:30.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.657+0000 7fd731600640 1 -- 172.21.15.100:0/1284409067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd72c0c72b0 con 0x7fd72c15c7a0 2024-04-23T14:33:30.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.661+0000 7fd723400640 1 -- 172.21.15.100:0/1284409067 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd7240b4ba0 con 0x7fd72c15c7a0 2024-04-23T14:33:30.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.661+0000 7fd723400640 1 --2- 172.21.15.100:0/1284409067 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd708068380 0x7fd70806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:30.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.661+0000 7fd723400640 1 -- 172.21.15.100:0/1284409067 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5853+0+0 (secure 0 0 0) 0x7fd724120790 con 0x7fd72c15c7a0 2024-04-23T14:33:30.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.661+0000 7fd72aa00640 1 --2- 172.21.15.100:0/1284409067 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd708068380 0x7fd70806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:30.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.661+0000 7fd72aa00640 1 --2- 172.21.15.100:0/1284409067 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd708068380 0x7fd70806a840 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd71c003b90 tx=0x7fd71c0098d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:30.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.665+0000 7fd723400640 1 -- 172.21.15.100:0/1284409067 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd72c0c72b0 con 0x7fd72c15c7a0 2024-04-23T14:33:30.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:30.825+0000 7fd731600640 1 -- 172.21.15.100:0/1284409067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd72c0af4e0 con 0x7fd72c15c7a0 2024-04-23T14:33:31.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.489+0000 7fd723400640 1 -- 172.21.15.100:0/1284409067 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) ==== 178+0+0 (secure 0 0 0) 0x7fd724016150 con 0x7fd72c15c7a0 2024-04-23T14:33:31.492 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data_ec' removed 2024-04-23T14:33:31.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.489+0000 7fd731600640 1 -- 172.21.15.100:0/1284409067 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd708068380 msgr2=0x7fd70806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:31.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.489+0000 7fd731600640 1 --2- 172.21.15.100:0/1284409067 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd708068380 0x7fd70806a840 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd71c003b90 tx=0x7fd71c0098d0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.489+0000 7fd731600640 1 -- 172.21.15.100:0/1284409067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd72c15c7a0 msgr2=0x7fd72c151a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:31.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.489+0000 7fd731600640 1 --2- 172.21.15.100:0/1284409067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd72c15c7a0 0x7fd72c151a10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd7240ae040 tx=0x7fd724019270 comp rx=0 tx=0).stop 2024-04-23T14:33:31.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.489+0000 7fd731600640 1 -- 172.21.15.100:0/1284409067 shutdown_connections 2024-04-23T14:33:31.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.489+0000 7fd731600640 1 --2- 172.21.15.100:0/1284409067 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd708068380 0x7fd70806a840 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.489+0000 7fd731600640 1 --2- 172.21.15.100:0/1284409067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd72c15c7a0 0x7fd72c151a10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.489+0000 7fd731600640 1 --2- 172.21.15.100:0/1284409067 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd72c0cd300 0x7fd72c099770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.489+0000 7fd731600640 1 --2- 172.21.15.100:0/1284409067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd72c06bb20 0x7fd72c072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.489+0000 7fd731600640 1 -- 172.21.15.100:0/1284409067 >> 172.21.15.100:0/1284409067 conn(0x7fd72c0b5200 msgr2=0x7fd72c152e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:31.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.489+0000 7fd731600640 1 -- 172.21.15.100:0/1284409067 shutdown_connections 2024-04-23T14:33:31.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.493+0000 7fd731600640 1 -- 172.21.15.100:0/1284409067 wait complete. 2024-04-23T14:33:31.523 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-23T14:33:31.523 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-23T14:33:31.523 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-23T14:33:31.523 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:33:31.523 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-23T14:33:31.523 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-23T14:33:31.524 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-23T14:33:31.524 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:33:31.524 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-23T14:33:31.524 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-23T14:33:31.524 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-23T14:33:31.524 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:33:31.524 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-23T14:33:31.525 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-23T14:33:31.525 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-23T14:33:31.525 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:33:31.538 INFO:tasks.ceph.mds.a:Stopped 2024-04-23T14:33:31.538 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-23T14:33:31.539 INFO:tasks.ceph.mds.c:Stopped 2024-04-23T14:33:31.539 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-23T14:33:31.540 INFO:tasks.ceph.mds.b:Stopped 2024-04-23T14:33:31.540 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-23T14:33:31.540 INFO:tasks.ceph.mds.d:Stopped 2024-04-23T14:33:31.541 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-23T14:33:31.585 INFO:tasks.ceph.mds.a:Started 2024-04-23T14:33:31.585 DEBUG:teuthology.parallel:result is None 2024-04-23T14:33:31.586 INFO:tasks.ceph.mds.b:Started 2024-04-23T14:33:31.586 DEBUG:teuthology.parallel:result is None 2024-04-23T14:33:31.587 INFO:tasks.ceph.mds.c:Started 2024-04-23T14:33:31.587 DEBUG:teuthology.parallel:result is None 2024-04-23T14:33:31.588 INFO:tasks.ceph.mds.d:Started 2024-04-23T14:33:31.588 DEBUG:teuthology.parallel:result is None 2024-04-23T14:33:31.588 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-23T14:33:31.673 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-04-23T14:33:31.673 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-04-23T14:33:31.676 INFO:tasks.ceph.mds.b.smithi151.stdout:starting mds.b at 2024-04-23T14:33:31.677 INFO:tasks.ceph.mds.d.smithi151.stdout:starting mds.d at 2024-04-23T14:33:31.680 INFO:tasks.ceph.mds.d.smithi151.stderr:2024-04-23T14:33:31.674+0000 7f6480800640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6833/0: (98) Address already in use 2024-04-23T14:33:31.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.729+0000 7f8900000640 1 Processor -- start 2024-04-23T14:33:31.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.729+0000 7f8900000640 1 -- start start 2024-04-23T14:33:31.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.729+0000 7f8900000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f88f815d0c0 0x7f88f80cde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:31.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.729+0000 7f8900000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88f80ce500 con 0x7f88f80cf160 2024-04-23T14:33:31.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.729+0000 7f8900000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88f80ce7e0 con 0x7f88f815d4a0 2024-04-23T14:33:31.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.729+0000 7f8900000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f88f80cff50 con 0x7f88f815d0c0 2024-04-23T14:33:31.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.729+0000 7f88fe000640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f88f80cf160 0x7f88f80cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:60358/0 (socket says 172.21.15.100:60358) 2024-04-23T14:33:31.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.729+0000 7f88fe000640 1 -- 172.21.15.100:0/2159683820 learned_addr learned my addr 172.21.15.100:0/2159683820 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:31.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.729+0000 7f88fd600640 1 --2- 172.21.15.100:0/2159683820 >> v2:172.21.15.151:3300/0 conn(0x7f88f815d0c0 0x7f88f80cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:31.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.729+0000 7f88f7e00640 1 -- 172.21.15.100:0/2159683820 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2454531651 0 0) 0x7f88f80ce500 con 0x7f88f80cf160 2024-04-23T14:33:31.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.729+0000 7f88f7e00640 1 -- 172.21.15.100:0/2159683820 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88e0003530 con 0x7f88f80cf160 2024-04-23T14:33:31.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.729+0000 7f88f7e00640 1 -- 172.21.15.100:0/2159683820 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3629912640 0 0) 0x7f88f80ce7e0 con 0x7f88f815d4a0 2024-04-23T14:33:31.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.729+0000 7f88f7e00640 1 -- 172.21.15.100:0/2159683820 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88f80ce500 con 0x7f88f815d4a0 2024-04-23T14:33:31.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.729+0000 7f88fd600640 1 -- 172.21.15.100:0/2159683820 >> v1:172.21.15.151:6789/0 conn(0x7f88f815d4a0 legacy=0x7f88f80cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:31.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.729+0000 7f88fd600640 1 -- 172.21.15.100:0/2159683820 >> v1:172.21.15.100:6789/0 conn(0x7f88f80cf160 legacy=0x7f88f80cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:31.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.729+0000 7f88fd600640 1 -- 172.21.15.100:0/2159683820 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f80ceac0 con 0x7f88f815d0c0 2024-04-23T14:33:31.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.733+0000 7f88fd600640 1 --2- 172.21.15.100:0/2159683820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88f815d0c0 0x7f88f80cde20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f88ec009d90 tx=0x7f88ec01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=107fdd5e333d69bf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:31.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.733+0000 7f88f7e00640 1 -- 172.21.15.100:0/2159683820 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec0b2070 con 0x7f88f815d0c0 2024-04-23T14:33:31.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.733+0000 7f88f7e00640 1 -- 172.21.15.100:0/2159683820 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88ec0a58b0 con 0x7f88f815d0c0 2024-04-23T14:33:31.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.733+0000 7f88f7e00640 1 -- 172.21.15.100:0/2159683820 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec0a5c80 con 0x7f88f815d0c0 2024-04-23T14:33:31.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.733+0000 7f8900000640 1 -- 172.21.15.100:0/2159683820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88f815d0c0 msgr2=0x7f88f80cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:31.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.733+0000 7f8900000640 1 --2- 172.21.15.100:0/2159683820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88f815d0c0 0x7f88f80cde20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f88ec009d90 tx=0x7f88ec01c8e0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.733+0000 7f8900000640 1 -- 172.21.15.100:0/2159683820 shutdown_connections 2024-04-23T14:33:31.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.733+0000 7f8900000640 1 --2- 172.21.15.100:0/2159683820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88f815d0c0 0x7f88f80cde20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.733+0000 7f8900000640 1 -- 172.21.15.100:0/2159683820 >> 172.21.15.100:0/2159683820 conn(0x7f88f80b5310 msgr2=0x7f88f80b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:31.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.733+0000 7f8900000640 1 -- 172.21.15.100:0/2159683820 shutdown_connections 2024-04-23T14:33:31.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.733+0000 7f8900000640 1 -- 172.21.15.100:0/2159683820 wait complete. 2024-04-23T14:33:31.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f8900000640 1 Processor -- start 2024-04-23T14:33:31.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f8900000640 1 -- start start 2024-04-23T14:33:31.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f8900000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f88f80cf160 0x7f88f81512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:31.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f8900000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88f815d0c0 0x7f88f80ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:31.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f8900000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88f815d4a0 0x7f88f81573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:31.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f8900000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f88f8153200 con 0x7f88f815d4a0 2024-04-23T14:33:31.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f8900000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f88f8153080 con 0x7f88f815d0c0 2024-04-23T14:33:31.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f8900000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f88f8153380 con 0x7f88f80cf160 2024-04-23T14:33:31.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f88fe000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88f815d4a0 0x7f88f81573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:31.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f88fe000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88f815d4a0 0x7f88f81573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39884/0 (socket says 172.21.15.100:39884) 2024-04-23T14:33:31.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f88fe000640 1 -- 172.21.15.100:0/727656799 learned_addr learned my addr 172.21.15.100:0/727656799 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:31.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f88fd600640 1 --2- 172.21.15.100:0/727656799 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f88f80cf160 0x7f88f81512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:31.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f88fcc00640 1 --2- 172.21.15.100:0/727656799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88f815d0c0 0x7f88f80ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:31.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f88fe000640 1 -- 172.21.15.100:0/727656799 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f88f80cf160 msgr2=0x7f88f81512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:31.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f88fe000640 1 --2- 172.21.15.100:0/727656799 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f88f80cf160 0x7f88f81512a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f88fe000640 1 -- 172.21.15.100:0/727656799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88f815d0c0 msgr2=0x7f88f80ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:31.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f88fe000640 1 --2- 172.21.15.100:0/727656799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88f815d0c0 0x7f88f80ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f88fe000640 1 -- 172.21.15.100:0/727656799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f80ad3b0 con 0x7f88f815d4a0 2024-04-23T14:33:31.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f88fe000640 1 --2- 172.21.15.100:0/727656799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88f815d4a0 0x7f88f81573c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f88f000c480 tx=0x7f88f000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:31.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f88f5600640 1 -- 172.21.15.100:0/727656799 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f0017070 con 0x7f88f815d4a0 2024-04-23T14:33:31.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f88f5600640 1 -- 172.21.15.100:0/727656799 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88f0005db0 con 0x7f88f815d4a0 2024-04-23T14:33:31.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f88f5600640 1 -- 172.21.15.100:0/727656799 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f0007290 con 0x7f88f815d4a0 2024-04-23T14:33:31.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f8900000640 1 -- 172.21.15.100:0/727656799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f80ad690 con 0x7f88f815d4a0 2024-04-23T14:33:31.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.737+0000 7f8900000640 1 -- 172.21.15.100:0/727656799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f80aeeb0 con 0x7f88f815d4a0 2024-04-23T14:33:31.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.741+0000 7f8900000640 1 -- 172.21.15.100:0/727656799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88f8068ad0 con 0x7f88f815d4a0 2024-04-23T14:33:31.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.741+0000 7f88f5600640 1 -- 172.21.15.100:0/727656799 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f88f001a070 con 0x7f88f815d4a0 2024-04-23T14:33:31.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.741+0000 7f88f5600640 1 --2- 172.21.15.100:0/727656799 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f88d80683b0 0x7f88d806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:31.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.741+0000 7f88f5600640 1 -- 172.21.15.100:0/727656799 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5335+0+0 (secure 0 0 0) 0x7f88f0085b50 con 0x7f88f815d4a0 2024-04-23T14:33:31.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.741+0000 7f88fd600640 1 --2- 172.21.15.100:0/727656799 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f88d80683b0 0x7f88d806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:31.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.741+0000 7f88fd600640 1 --2- 172.21.15.100:0/727656799 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f88d80683b0 0x7f88d806a870 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0af000 tx=0x7f88ec0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:31.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.745+0000 7f88f5600640 1 -- 172.21.15.100:0/727656799 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f88f0055c20 con 0x7f88f815d4a0 2024-04-23T14:33:31.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:31.917+0000 7f8900000640 1 -- 172.21.15.100:0/727656799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f88f806c730 con 0x7f88f815d4a0 2024-04-23T14:33:32.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.505+0000 7f88f5600640 1 -- 172.21.15.100:0/727656799 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) ==== 97+0+0 (secure 0 0 0) 0x7f88f0059f20 con 0x7f88f815d4a0 2024-04-23T14:33:32.508 INFO:teuthology.orchestra.run.smithi100.stderr: removed all blocklist entries 2024-04-23T14:33:32.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.509+0000 7f8900000640 1 -- 172.21.15.100:0/727656799 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f88d80683b0 msgr2=0x7f88d806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:32.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.509+0000 7f8900000640 1 --2- 172.21.15.100:0/727656799 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f88d80683b0 0x7f88d806a870 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0af000 tx=0x7f88ec0a7070 comp rx=0 tx=0).stop 2024-04-23T14:33:32.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.509+0000 7f8900000640 1 -- 172.21.15.100:0/727656799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88f815d4a0 msgr2=0x7f88f81573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:32.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.509+0000 7f8900000640 1 --2- 172.21.15.100:0/727656799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88f815d4a0 0x7f88f81573c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f88f000c480 tx=0x7f88f000c940 comp rx=0 tx=0).stop 2024-04-23T14:33:32.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.509+0000 7f8900000640 1 -- 172.21.15.100:0/727656799 shutdown_connections 2024-04-23T14:33:32.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.509+0000 7f8900000640 1 --2- 172.21.15.100:0/727656799 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f88d80683b0 0x7f88d806a870 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.509+0000 7f8900000640 1 --2- 172.21.15.100:0/727656799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88f815d4a0 0x7f88f81573c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.509+0000 7f8900000640 1 --2- 172.21.15.100:0/727656799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88f815d0c0 0x7f88f80ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.509+0000 7f8900000640 1 --2- 172.21.15.100:0/727656799 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f88f80cf160 0x7f88f81512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.509+0000 7f8900000640 1 -- 172.21.15.100:0/727656799 >> 172.21.15.100:0/727656799 conn(0x7f88f80b5310 msgr2=0x7f88f815ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:32.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.509+0000 7f8900000640 1 -- 172.21.15.100:0/727656799 shutdown_connections 2024-04-23T14:33:32.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.509+0000 7f8900000640 1 -- 172.21.15.100:0/727656799 wait complete. 2024-04-23T14:33:32.537 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-23T14:33:32.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.669+0000 7eff40c00640 1 Processor -- start 2024-04-23T14:33:32.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.669+0000 7eff40c00640 1 -- start start 2024-04-23T14:33:32.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.669+0000 7eff40c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7eff3c153460 0x7eff3c0d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.669+0000 7eff40c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7eff3c15c940 0x7eff3c0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.669+0000 7eff40c00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff3c153840 con 0x7eff3c0d0620 2024-04-23T14:33:32.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.669+0000 7eff40c00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7eff3c15cfe0 con 0x7eff3c153460 2024-04-23T14:33:32.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.669+0000 7eff40c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7eff3c15d2c0 con 0x7eff3c15c940 2024-04-23T14:33:32.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.669+0000 7eff3b400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7eff3c153460 0x7eff3c0d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.669+0000 7eff3be00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7eff3c0d0620 0x7eff3c0cc940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:60370/0 (socket says 172.21.15.100:60370) 2024-04-23T14:33:32.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.669+0000 7eff3b400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7eff3c153460 0x7eff3c0d1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39886/0 (socket says 172.21.15.100:39886) 2024-04-23T14:33:32.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.669+0000 7eff3be00640 1 -- 172.21.15.100:0/3274197000 learned_addr learned my addr 172.21.15.100:0/3274197000 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:32.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.669+0000 7eff32a00640 1 --2- 172.21.15.100:0/3274197000 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7eff3c15c940 0x7eff3c0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.669+0000 7eff3b400640 1 -- 172.21.15.100:0/3274197000 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7eff3c15c940 msgr2=0x7eff3c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:32.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.669+0000 7eff3b400640 1 --2- 172.21.15.100:0/3274197000 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7eff3c15c940 0x7eff3c0d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.669+0000 7eff3b400640 1 -- 172.21.15.100:0/3274197000 >> v1:172.21.15.100:6789/0 conn(0x7eff3c0d0620 legacy=0x7eff3c0cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:32.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.669+0000 7eff3b400640 1 -- 172.21.15.100:0/3274197000 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff3c15d5a0 con 0x7eff3c153460 2024-04-23T14:33:32.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.669+0000 7eff3aa00640 1 -- 172.21.15.100:0/3274197000 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2505380012 0 0) 0x7eff340014a0 con 0x7eff3c0d0620 2024-04-23T14:33:32.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.673+0000 7eff3b400640 1 --2- 172.21.15.100:0/3274197000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff3c153460 0x7eff3c0d1d60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0097c0 tx=0x7eff2c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e59ebd1e9ac5a24 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:32.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.673+0000 7eff3aa00640 1 -- 172.21.15.100:0/3274197000 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c0b0070 con 0x7eff3c153460 2024-04-23T14:33:32.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.673+0000 7eff3aa00640 1 -- 172.21.15.100:0/3274197000 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff2c0ab600 con 0x7eff3c153460 2024-04-23T14:33:32.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.673+0000 7eff3aa00640 1 -- 172.21.15.100:0/3274197000 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c0ab9d0 con 0x7eff3c153460 2024-04-23T14:33:32.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.673+0000 7eff40c00640 1 -- 172.21.15.100:0/3274197000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff3c153460 msgr2=0x7eff3c0d1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:32.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.673+0000 7eff40c00640 1 --2- 172.21.15.100:0/3274197000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff3c153460 0x7eff3c0d1d60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0097c0 tx=0x7eff2c01c410 comp rx=0 tx=0).stop 2024-04-23T14:33:32.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.673+0000 7eff40c00640 1 -- 172.21.15.100:0/3274197000 shutdown_connections 2024-04-23T14:33:32.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.673+0000 7eff40c00640 1 --2- 172.21.15.100:0/3274197000 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7eff3c15c940 0x7eff3c0d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.673+0000 7eff40c00640 1 --2- 172.21.15.100:0/3274197000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff3c153460 0x7eff3c0d1d60 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.673+0000 7eff40c00640 1 -- 172.21.15.100:0/3274197000 >> 172.21.15.100:0/3274197000 conn(0x7eff3c0b3e60 msgr2=0x7eff3c0b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:32.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.673+0000 7eff40c00640 1 -- 172.21.15.100:0/3274197000 shutdown_connections 2024-04-23T14:33:32.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.673+0000 7eff40c00640 1 -- 172.21.15.100:0/3274197000 wait complete. 2024-04-23T14:33:32.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff40c00640 1 Processor -- start 2024-04-23T14:33:32.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff40c00640 1 -- start start 2024-04-23T14:33:32.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff40c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff3c0d0620 0x7eff3c14c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff40c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff3c153460 0x7eff3c14bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff40c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7eff3c15c940 0x7eff3c0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff40c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c0cd1d0 con 0x7eff3c153460 2024-04-23T14:33:32.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff40c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c0cd050 con 0x7eff3c0d0620 2024-04-23T14:33:32.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff40c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7eff3c0ce850 con 0x7eff3c15c940 2024-04-23T14:33:32.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff32a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff3c153460 0x7eff3c14bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff32a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff3c153460 0x7eff3c14bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39898/0 (socket says 172.21.15.100:39898) 2024-04-23T14:33:32.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff32a00640 1 -- 172.21.15.100:0/3299687445 learned_addr learned my addr 172.21.15.100:0/3299687445 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:32.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff3be00640 1 --2- 172.21.15.100:0/3299687445 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7eff3c15c940 0x7eff3c0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff3b400640 1 --2- 172.21.15.100:0/3299687445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff3c0d0620 0x7eff3c14c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff32a00640 1 -- 172.21.15.100:0/3299687445 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7eff3c15c940 msgr2=0x7eff3c0caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:32.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff32a00640 1 --2- 172.21.15.100:0/3299687445 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7eff3c15c940 0x7eff3c0caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff32a00640 1 -- 172.21.15.100:0/3299687445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff3c0d0620 msgr2=0x7eff3c14c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:32.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff32a00640 1 --2- 172.21.15.100:0/3299687445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff3c0d0620 0x7eff3c14c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff32a00640 1 -- 172.21.15.100:0/3299687445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff3c0cc230 con 0x7eff3c153460 2024-04-23T14:33:32.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff3be00640 1 --2- 172.21.15.100:0/3299687445 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7eff3c15c940 0x7eff3c0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:32.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff32a00640 1 --2- 172.21.15.100:0/3299687445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff3c153460 0x7eff3c14bbd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7eff2400d5f0 tx=0x7eff2400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:32.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff33e00640 1 -- 172.21.15.100:0/3299687445 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff240125f0 con 0x7eff3c153460 2024-04-23T14:33:32.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff33e00640 1 -- 172.21.15.100:0/3299687445 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff24019070 con 0x7eff3c153460 2024-04-23T14:33:32.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff33e00640 1 -- 172.21.15.100:0/3299687445 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff240129b0 con 0x7eff3c153460 2024-04-23T14:33:32.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff40c00640 1 -- 172.21.15.100:0/3299687445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff3c0c8c40 con 0x7eff3c153460 2024-04-23T14:33:32.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.677+0000 7eff40c00640 1 -- 172.21.15.100:0/3299687445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff3c0cb750 con 0x7eff3c153460 2024-04-23T14:33:32.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.681+0000 7eff40c00640 1 -- 172.21.15.100:0/3299687445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff3c075c20 con 0x7eff3c153460 2024-04-23T14:33:32.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.681+0000 7eff33e00640 1 -- 172.21.15.100:0/3299687445 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7eff24012b50 con 0x7eff3c153460 2024-04-23T14:33:32.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.681+0000 7eff33e00640 1 --2- 172.21.15.100:0/3299687445 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7eff18068380 0x7eff1806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.681+0000 7eff33e00640 1 -- 172.21.15.100:0/3299687445 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7eff24086be0 con 0x7eff3c153460 2024-04-23T14:33:32.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.681+0000 7eff3b400640 1 --2- 172.21.15.100:0/3299687445 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7eff18068380 0x7eff1806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.681+0000 7eff3b400640 1 --2- 172.21.15.100:0/3299687445 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7eff18068380 0x7eff1806a840 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0ad000 tx=0x7eff2c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:32.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.685+0000 7eff33e00640 1 -- 172.21.15.100:0/3299687445 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7eff240572c0 con 0x7eff3c153460 2024-04-23T14:33:32.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.897+0000 7eff40c00640 1 -- 172.21.15.100:0/3299687445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7eff3c0cfec0 con 0x7eff3c153460 2024-04-23T14:33:32.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.897+0000 7eff33e00640 1 -- 172.21.15.100:0/3299687445 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7eff24056bc0 con 0x7eff3c153460 2024-04-23T14:33:32.902 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:32.902 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:33:32.902 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_dump": [ 2024-04-23T14:33:32.902 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.902 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.a", 2024-04-23T14:33:32.902 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtxidmcgr8GBAA944tuhsKzhnsmZrdxOKbAA==", 2024-04-23T14:33:32.902 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.903 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-04-23T14:33:32.903 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-04-23T14:33:32.903 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-04-23T14:33:32.903 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:33:32.903 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.903 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.903 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.903 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.b", 2024-04-23T14:33:32.903 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtxidmJcqnJBAA95zJpqTJW92I+2Nnh7SpXg==", 2024-04-23T14:33:32.903 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.903 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-04-23T14:33:32.903 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-04-23T14:33:32.903 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-04-23T14:33:32.903 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:33:32.903 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.904 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.904 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.904 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.c", 2024-04-23T14:33:32.904 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtxidmrYjvHxAABsf7ouy0ULqAFxwPrLCMsg==", 2024-04-23T14:33:32.904 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.904 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-04-23T14:33:32.904 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-04-23T14:33:32.904 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-04-23T14:33:32.904 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:33:32.904 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.904 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.904 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.904 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.d", 2024-04-23T14:33:32.905 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtxidmmN4JLBAANawA9RtKrB31GqCz/wuz3Q==", 2024-04-23T14:33:32.905 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.905 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-04-23T14:33:32.905 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-04-23T14:33:32.905 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-04-23T14:33:32.905 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:33:32.905 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.905 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.905 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.905 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.0", 2024-04-23T14:33:32.905 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAwxidmtJJBMBAAktb3nILtB6ywDU5lca43OQ==", 2024-04-23T14:33:32.905 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.905 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-04-23T14:33:32.906 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-04-23T14:33:32.906 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:33:32.906 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.906 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.906 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.1", 2024-04-23T14:33:32.906 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAzxidmUgz/ORAASX3vikkabjnWpzY1efthig==", 2024-04-23T14:33:32.906 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.906 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-04-23T14:33:32.906 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-04-23T14:33:32.906 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:33:32.906 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.907 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.907 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.2", 2024-04-23T14:33:32.907 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA3xidmUzrNBhAAI8ymZvKTD5dm+gLYh42WMA==", 2024-04-23T14:33:32.907 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.907 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-04-23T14:33:32.907 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-04-23T14:33:32.907 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:33:32.907 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.907 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.907 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.907 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.3", 2024-04-23T14:33:32.907 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA6xidmfEC8EBAAiv84aaTlpMGaG6ZFxKp2ng==", 2024-04-23T14:33:32.907 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.907 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-04-23T14:33:32.907 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-04-23T14:33:32.908 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:33:32.908 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.908 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.908 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.4", 2024-04-23T14:33:32.908 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBAxidmWoujCBAAclplSDUC/Hwhh4a4NWgH/Q==", 2024-04-23T14:33:32.908 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.908 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-04-23T14:33:32.908 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-04-23T14:33:32.908 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:33:32.908 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.908 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.908 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.5", 2024-04-23T14:33:32.908 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBDxidmRO+eEhAADfwa1qucvA1FGwuIY0A62w==", 2024-04-23T14:33:32.908 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.909 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-04-23T14:33:32.909 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-04-23T14:33:32.909 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:33:32.909 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.909 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.909 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.909 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.6", 2024-04-23T14:33:32.909 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBGxidmMujsHBAAKNjjM4nlyaUVesLu1CixhA==", 2024-04-23T14:33:32.909 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.909 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-04-23T14:33:32.909 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-04-23T14:33:32.909 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:33:32.909 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.909 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.909 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.910 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.7", 2024-04-23T14:33:32.910 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBJxidmHxlnJxAAk69hiCwS8GrIPMcLV+iVpg==", 2024-04-23T14:33:32.910 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.910 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-04-23T14:33:32.910 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-04-23T14:33:32.910 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:33:32.910 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.910 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.910 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.910 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.0", 2024-04-23T14:33:32.910 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtxidmyhImMBAAa49ApQEF2Lng6rY94F1HAQ==", 2024-04-23T14:33:32.910 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.910 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-04-23T14:33:32.910 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-04-23T14:33:32.911 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-04-23T14:33:32.911 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-04-23T14:33:32.911 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.911 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.911 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.911 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.1", 2024-04-23T14:33:32.911 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtxidmpLhsORAAFLucAGu/GbEJTmkxy0LY8Q==", 2024-04-23T14:33:32.911 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.911 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-04-23T14:33:32.911 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-04-23T14:33:32.911 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-04-23T14:33:32.911 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-04-23T14:33:32.911 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.911 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.911 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.911 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.2", 2024-04-23T14:33:32.912 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtxidm34okNhAAkagR1bwlKj7wxsY/E/nRGg==", 2024-04-23T14:33:32.912 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.912 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-04-23T14:33:32.912 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-04-23T14:33:32.912 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-04-23T14:33:32.912 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-04-23T14:33:32.912 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.912 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.912 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.912 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.3", 2024-04-23T14:33:32.912 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAuxidmtJWrAxAAYBGSngF85+O1YO5on41OMw==", 2024-04-23T14:33:32.912 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.912 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-04-23T14:33:32.912 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-04-23T14:33:32.913 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-04-23T14:33:32.913 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-04-23T14:33:32.913 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.913 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.913 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.913 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.admin", 2024-04-23T14:33:32.913 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsxidmumdhOBAAa9JaVIetaPBZmY7B4sayLQ==", 2024-04-23T14:33:32.913 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.913 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-04-23T14:33:32.913 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-04-23T14:33:32.913 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-04-23T14:33:32.913 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:33:32.913 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.913 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.913 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.914 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.bootstrap-mds", 2024-04-23T14:33:32.914 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBPxidmWr12KBAAAXUWyu6hu4sOdPpm78kjlQ==", 2024-04-23T14:33:32.914 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.914 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile bootstrap-mds" 2024-04-23T14:33:32.914 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.914 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.914 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.914 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.bootstrap-mgr", 2024-04-23T14:33:32.914 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBPxidmQMd2KBAApupWUL2qwL9C8viNCU/wPw==", 2024-04-23T14:33:32.914 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.914 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-23T14:33:32.914 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.914 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.914 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.915 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.bootstrap-osd", 2024-04-23T14:33:32.915 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBPxidm4892KBAAdTGI18Mpz7L+Zj2rx4JYGQ==", 2024-04-23T14:33:32.915 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.915 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile bootstrap-osd" 2024-04-23T14:33:32.915 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.915 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.915 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.bootstrap-rbd", 2024-04-23T14:33:32.915 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBPxidmMNh2KBAAk0BD6Minm/inNPJcj8iyEA==", 2024-04-23T14:33:32.915 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.915 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-23T14:33:32.915 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.915 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.915 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-23T14:33:32.915 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBPxidmpuB2KBAA+iP5yn1fGVaNDc2M3vSkoQ==", 2024-04-23T14:33:32.916 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.916 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-23T14:33:32.916 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.916 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.916 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.916 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.bootstrap-rgw", 2024-04-23T14:33:32.916 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBPxidmTOl2KBAAlkbwUWI5k0eqZW57RmYGTQ==", 2024-04-23T14:33:32.916 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.916 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-23T14:33:32.916 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.916 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.916 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.916 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.x", 2024-04-23T14:33:32.916 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtxidmKdCRFBAA4h8/t4aNmcR58fKgXGUB4Q==", 2024-04-23T14:33:32.916 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.916 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-04-23T14:33:32.917 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-04-23T14:33:32.917 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:33:32.917 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.917 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:32.917 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:32.917 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.y", 2024-04-23T14:33:32.917 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtxidmNTL4EBAAj88L1GzandAkaqg9AxEQqg==", 2024-04-23T14:33:32.917 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:33:32.917 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-04-23T14:33:32.917 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-04-23T14:33:32.917 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:33:32.917 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.917 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:32.917 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-04-23T14:33:32.917 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:33:32.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.901+0000 7eff40c00640 1 -- 172.21.15.100:0/3299687445 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7eff18068380 msgr2=0x7eff1806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:32.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.901+0000 7eff40c00640 1 --2- 172.21.15.100:0/3299687445 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7eff18068380 0x7eff1806a840 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0ad000 tx=0x7eff2c0a6000 comp rx=0 tx=0).stop 2024-04-23T14:33:32.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.901+0000 7eff40c00640 1 -- 172.21.15.100:0/3299687445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff3c153460 msgr2=0x7eff3c14bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:32.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.901+0000 7eff40c00640 1 --2- 172.21.15.100:0/3299687445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff3c153460 0x7eff3c14bbd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7eff2400d5f0 tx=0x7eff2400dab0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.901+0000 7eff40c00640 1 -- 172.21.15.100:0/3299687445 shutdown_connections 2024-04-23T14:33:32.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.901+0000 7eff40c00640 1 --2- 172.21.15.100:0/3299687445 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7eff18068380 0x7eff1806a840 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.901+0000 7eff40c00640 1 --2- 172.21.15.100:0/3299687445 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7eff3c15c940 0x7eff3c0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.901+0000 7eff40c00640 1 --2- 172.21.15.100:0/3299687445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff3c153460 0x7eff3c14bbd0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.901+0000 7eff40c00640 1 --2- 172.21.15.100:0/3299687445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff3c0d0620 0x7eff3c14c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.901+0000 7eff40c00640 1 -- 172.21.15.100:0/3299687445 >> 172.21.15.100:0/3299687445 conn(0x7eff3c0b3e60 msgr2=0x7eff3c14d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:32.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.901+0000 7eff40c00640 1 -- 172.21.15.100:0/3299687445 shutdown_connections 2024-04-23T14:33:32.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:32.901+0000 7eff40c00640 1 -- 172.21.15.100:0/3299687445 wait complete. 2024-04-23T14:33:32.923 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-23T14:33:33.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.057+0000 7f22a4000640 1 Processor -- start 2024-04-23T14:33:33.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.057+0000 7f22a4000640 1 -- start start 2024-04-23T14:33:33.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.057+0000 7f22a4000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f229c15d0c0 0x7f229c0cde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.057+0000 7f22a4000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f229c15d4a0 0x7f229c0cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.057+0000 7f22a4000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f229c0ce500 con 0x7f229c15d900 2024-04-23T14:33:33.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.057+0000 7f22a4000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f229c0cff50 con 0x7f229c15d4a0 2024-04-23T14:33:33.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.057+0000 7f22a4000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f229c0ceda0 con 0x7f229c15d0c0 2024-04-23T14:33:33.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.057+0000 7f22a0c00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f229c15d4a0 0x7f229c0cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.057+0000 7f22a1600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f229c15d0c0 0x7f229c0cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.057+0000 7f22a2000640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f229c15d900 0x7f229c0cf160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:33452/0 (socket says 172.21.15.100:33452) 2024-04-23T14:33:33.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.057+0000 7f22a0c00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f229c15d4a0 0x7f229c0cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:58046/0 (socket says 172.21.15.100:58046) 2024-04-23T14:33:33.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.057+0000 7f22a2000640 1 -- 172.21.15.100:0/1457925516 learned_addr learned my addr 172.21.15.100:0/1457925516 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:33.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.057+0000 7f22a1600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f229c15d0c0 0x7f229c0cde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:49856/0 (socket says 172.21.15.100:49856) 2024-04-23T14:33:33.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.057+0000 7f22a0c00640 1 -- 172.21.15.100:0/1457925516 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f229c15d0c0 msgr2=0x7f229c0cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:33.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.057+0000 7f22a0c00640 1 --2- 172.21.15.100:0/1457925516 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f229c15d0c0 0x7f229c0cde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.057+0000 7f22a0c00640 1 -- 172.21.15.100:0/1457925516 >> v1:172.21.15.151:6789/0 conn(0x7f229c15d900 legacy=0x7f229c0cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.057+0000 7f22a0c00640 1 -- 172.21.15.100:0/1457925516 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229c0ce7e0 con 0x7f229c15d4a0 2024-04-23T14:33:33.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.057+0000 7f229be00640 1 -- 172.21.15.100:0/1457925516 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2387705128 0 0) 0x7f229c0ce500 con 0x7f229c15d900 2024-04-23T14:33:33.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.061+0000 7f22a0c00640 1 --2- 172.21.15.100:0/1457925516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f229c15d4a0 0x7f229c0cb9a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f22880097c0 tx=0x7f228801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adda9dd59ff89fbc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:33.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.061+0000 7f229be00640 1 -- 172.21.15.100:0/1457925516 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22880b1070 con 0x7f229c15d4a0 2024-04-23T14:33:33.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.061+0000 7f229be00640 1 -- 172.21.15.100:0/1457925516 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22880a8050 con 0x7f229c15d4a0 2024-04-23T14:33:33.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.061+0000 7f229be00640 1 -- 172.21.15.100:0/1457925516 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22880ac860 con 0x7f229c15d4a0 2024-04-23T14:33:33.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.061+0000 7f22a4000640 1 -- 172.21.15.100:0/1457925516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f229c15d4a0 msgr2=0x7f229c0cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.061+0000 7f22a4000640 1 --2- 172.21.15.100:0/1457925516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f229c15d4a0 0x7f229c0cb9a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f22880097c0 tx=0x7f228801c3e0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.061+0000 7f22a4000640 1 -- 172.21.15.100:0/1457925516 shutdown_connections 2024-04-23T14:33:33.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.061+0000 7f22a4000640 1 --2- 172.21.15.100:0/1457925516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f229c15d4a0 0x7f229c0cb9a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.061+0000 7f22a4000640 1 --2- 172.21.15.100:0/1457925516 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f229c15d0c0 0x7f229c0cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.061+0000 7f22a4000640 1 -- 172.21.15.100:0/1457925516 >> 172.21.15.100:0/1457925516 conn(0x7f229c0b5310 msgr2=0x7f229c0b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:33.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.061+0000 7f22a4000640 1 -- 172.21.15.100:0/1457925516 shutdown_connections 2024-04-23T14:33:33.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.061+0000 7f22a4000640 1 -- 172.21.15.100:0/1457925516 wait complete. 2024-04-23T14:33:33.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f22a4000640 1 Processor -- start 2024-04-23T14:33:33.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f22a4000640 1 -- start start 2024-04-23T14:33:33.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f22a4000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f229c15d0c0 0x7f229c14f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f22a4000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f229c15d4a0 0x7f229c15f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f22a4000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f229c149510 0x7f229c1499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f22a4000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f229c0cf9f0 con 0x7f229c15d0c0 2024-04-23T14:33:33.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f22a4000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f229c0cf870 con 0x7f229c149510 2024-04-23T14:33:33.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f22a4000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f229c0cbee0 con 0x7f229c15d4a0 2024-04-23T14:33:33.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f22a1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f229c15d0c0 0x7f229c14f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f22a1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f229c15d0c0 0x7f229c14f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39914/0 (socket says 172.21.15.100:39914) 2024-04-23T14:33:33.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f22a1600640 1 -- 172.21.15.100:0/2738981009 learned_addr learned my addr 172.21.15.100:0/2738981009 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:33.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f22a2000640 1 --2- 172.21.15.100:0/2738981009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f229c149510 0x7f229c1499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f22a0c00640 1 --2- 172.21.15.100:0/2738981009 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f229c15d4a0 0x7f229c15f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f22a1600640 1 -- 172.21.15.100:0/2738981009 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f229c15d4a0 msgr2=0x7f229c15f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f22a1600640 1 --2- 172.21.15.100:0/2738981009 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f229c15d4a0 0x7f229c15f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f22a1600640 1 -- 172.21.15.100:0/2738981009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f229c149510 msgr2=0x7f229c1499c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f22a1600640 1 --2- 172.21.15.100:0/2738981009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f229c149510 0x7f229c1499c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f22a1600640 1 -- 172.21.15.100:0/2738981009 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229c151460 con 0x7f229c15d0c0 2024-04-23T14:33:33.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f22a0c00640 1 --2- 172.21.15.100:0/2738981009 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f229c15d4a0 0x7f229c15f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:33.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f22a1600640 1 --2- 172.21.15.100:0/2738981009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f229c15d0c0 0x7f229c14f210 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f229000e930 tx=0x7f229000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:33.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f2299600640 1 -- 172.21.15.100:0/2738981009 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2290017070 con 0x7f229c15d0c0 2024-04-23T14:33:33.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f2299600640 1 -- 172.21.15.100:0/2738981009 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f229000ca00 con 0x7f229c15d0c0 2024-04-23T14:33:33.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f22a4000640 1 -- 172.21.15.100:0/2738981009 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f229c151740 con 0x7f229c15d0c0 2024-04-23T14:33:33.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f2299600640 1 -- 172.21.15.100:0/2738981009 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229000cdb0 con 0x7f229c15d0c0 2024-04-23T14:33:33.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.065+0000 7f22a4000640 1 -- 172.21.15.100:0/2738981009 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f229c151d00 con 0x7f229c15d0c0 2024-04-23T14:33:33.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.069+0000 7f22a4000640 1 -- 172.21.15.100:0/2738981009 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f229c068ad0 con 0x7f229c15d0c0 2024-04-23T14:33:33.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.069+0000 7f2299600640 1 -- 172.21.15.100:0/2738981009 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f229001a070 con 0x7f229c15d0c0 2024-04-23T14:33:33.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.069+0000 7f2299600640 1 --2- 172.21.15.100:0/2738981009 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f227c068380 0x7f227c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.069+0000 7f2299600640 1 -- 172.21.15.100:0/2738981009 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f2290086c50 con 0x7f229c15d0c0 2024-04-23T14:33:33.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.069+0000 7f22a0c00640 1 --2- 172.21.15.100:0/2738981009 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f227c068380 0x7f227c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.069+0000 7f22a0c00640 1 --2- 172.21.15.100:0/2738981009 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f227c068380 0x7f227c06a840 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f22880ae000 tx=0x7f2288008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:33.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.073+0000 7f2299600640 1 -- 172.21.15.100:0/2738981009 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f22900573f0 con 0x7f229c15d0c0 2024-04-23T14:33:33.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.285+0000 7f22a4000640 1 -- 172.21.15.100:0/2738981009 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f229c06c730 con 0x7f229c15d0c0 2024-04-23T14:33:33.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.289+0000 7f2299600640 1 -- 172.21.15.100:0/2738981009 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f2290056cf0 con 0x7f229c15d0c0 2024-04-23T14:33:33.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.289+0000 7f22a4000640 1 -- 172.21.15.100:0/2738981009 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f227c068380 msgr2=0x7f227c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.289+0000 7f22a4000640 1 --2- 172.21.15.100:0/2738981009 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f227c068380 0x7f227c06a840 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f22880ae000 tx=0x7f2288008040 comp rx=0 tx=0).stop 2024-04-23T14:33:33.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.289+0000 7f22a4000640 1 -- 172.21.15.100:0/2738981009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f229c15d0c0 msgr2=0x7f229c14f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.289+0000 7f22a4000640 1 --2- 172.21.15.100:0/2738981009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f229c15d0c0 0x7f229c14f210 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f229000e930 tx=0x7f229000edf0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.289+0000 7f22a4000640 1 -- 172.21.15.100:0/2738981009 shutdown_connections 2024-04-23T14:33:33.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.289+0000 7f22a4000640 1 --2- 172.21.15.100:0/2738981009 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f227c068380 0x7f227c06a840 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.289+0000 7f22a4000640 1 --2- 172.21.15.100:0/2738981009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f229c149510 0x7f229c1499c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.289+0000 7f22a4000640 1 --2- 172.21.15.100:0/2738981009 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f229c15d4a0 0x7f229c15f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.289+0000 7f22a4000640 1 --2- 172.21.15.100:0/2738981009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f229c15d0c0 0x7f229c14f210 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.289+0000 7f22a4000640 1 -- 172.21.15.100:0/2738981009 >> 172.21.15.100:0/2738981009 conn(0x7f229c0b5310 msgr2=0x7f229c0cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:33.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.293+0000 7f22a4000640 1 -- 172.21.15.100:0/2738981009 shutdown_connections 2024-04-23T14:33:33.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.293+0000 7f22a4000640 1 -- 172.21.15.100:0/2738981009 wait complete. 2024-04-23T14:33:33.313 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-23T14:33:33.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.445+0000 7f5586e00640 1 Processor -- start 2024-04-23T14:33:33.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.445+0000 7f5586e00640 1 -- start start 2024-04-23T14:33:33.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.445+0000 7f5586e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f558015b660 0x7f55800c6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.445+0000 7f5586e00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55800cee00 con 0x7f55800d08a0 2024-04-23T14:33:33.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.445+0000 7f5586e00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f558006bc50 con 0x7f55800c6d60 2024-04-23T14:33:33.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.445+0000 7f5586e00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f558015ba40 con 0x7f558015b660 2024-04-23T14:33:33.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.445+0000 7f557fe00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f558015b660 0x7f55800c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.445+0000 7f5584e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f55800d08a0 0x7f55800c5160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:60372/0 (socket says 172.21.15.100:60372) 2024-04-23T14:33:33.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.445+0000 7f5584e00640 1 -- 172.21.15.100:0/1918510143 learned_addr learned my addr 172.21.15.100:0/1918510143 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:33.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.449+0000 7f557f400640 1 -- 172.21.15.100:0/1918510143 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 927139431 0 0) 0x7f5578001120 con 0x7f55800d08a0 2024-04-23T14:33:33.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.449+0000 7f557f400640 1 -- 172.21.15.100:0/1918510143 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5564003530 con 0x7f55800d08a0 2024-04-23T14:33:33.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.449+0000 7f557fe00640 1 -- 172.21.15.100:0/1918510143 >> v1:172.21.15.151:6789/0 conn(0x7f55800c6d60 legacy=0x7f55800d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.449+0000 7f557fe00640 1 -- 172.21.15.100:0/1918510143 >> v1:172.21.15.100:6789/0 conn(0x7f55800d08a0 legacy=0x7f55800c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.449+0000 7f557fe00640 1 -- 172.21.15.100:0/1918510143 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558006bf30 con 0x7f558015b660 2024-04-23T14:33:33.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.449+0000 7f557fe00640 1 --2- 172.21.15.100:0/1918510143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558015b660 0x7f55800c6820 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f55700097c0 tx=0x7f557001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160b3b169e7bf044 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:33.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.449+0000 7f557f400640 1 -- 172.21.15.100:0/1918510143 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55700a7070 con 0x7f558015b660 2024-04-23T14:33:33.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.449+0000 7f557f400640 1 -- 172.21.15.100:0/1918510143 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55700b0070 con 0x7f558015b660 2024-04-23T14:33:33.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.449+0000 7f557f400640 1 -- 172.21.15.100:0/1918510143 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55700ab890 con 0x7f558015b660 2024-04-23T14:33:33.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.449+0000 7f5586e00640 1 -- 172.21.15.100:0/1918510143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558015b660 msgr2=0x7f55800c6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.449+0000 7f5586e00640 1 --2- 172.21.15.100:0/1918510143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558015b660 0x7f55800c6820 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f55700097c0 tx=0x7f557001c470 comp rx=0 tx=0).stop 2024-04-23T14:33:33.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.449+0000 7f5586e00640 1 -- 172.21.15.100:0/1918510143 shutdown_connections 2024-04-23T14:33:33.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.449+0000 7f5586e00640 1 --2- 172.21.15.100:0/1918510143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558015b660 0x7f55800c6820 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.449+0000 7f5586e00640 1 -- 172.21.15.100:0/1918510143 >> 172.21.15.100:0/1918510143 conn(0x7f55800b5310 msgr2=0x7f55800b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:33.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.449+0000 7f5586e00640 1 -- 172.21.15.100:0/1918510143 shutdown_connections 2024-04-23T14:33:33.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.449+0000 7f5586e00640 1 -- 172.21.15.100:0/1918510143 wait complete. 2024-04-23T14:33:33.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f5586e00640 1 Processor -- start 2024-04-23T14:33:33.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f5586e00640 1 -- start start 2024-04-23T14:33:33.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f5586e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55800c6d60 0x7f558014c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f5586e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f55800d08a0 0x7f55800aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f5586e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558015b660 0x7f55801572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f5586e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f55800c6200 con 0x7f558015b660 2024-04-23T14:33:33.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f5586e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f55800c6080 con 0x7f55800c6d60 2024-04-23T14:33:33.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f5586e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f55800c6380 con 0x7f55800d08a0 2024-04-23T14:33:33.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f5584e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558015b660 0x7f55801572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f5584e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558015b660 0x7f55801572a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39932/0 (socket says 172.21.15.100:39932) 2024-04-23T14:33:33.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f5584e00640 1 -- 172.21.15.100:0/959317491 learned_addr learned my addr 172.21.15.100:0/959317491 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:33.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f557fe00640 1 --2- 172.21.15.100:0/959317491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55800c6d60 0x7f558014c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f5584e00640 1 -- 172.21.15.100:0/959317491 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f55800d08a0 msgr2=0x7f55800aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f5577400640 1 --2- 172.21.15.100:0/959317491 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f55800d08a0 0x7f55800aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f5584e00640 1 --2- 172.21.15.100:0/959317491 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f55800d08a0 0x7f55800aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f5584e00640 1 -- 172.21.15.100:0/959317491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55800c6d60 msgr2=0x7f558014c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f5584e00640 1 --2- 172.21.15.100:0/959317491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55800c6d60 0x7f558014c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f5584e00640 1 -- 172.21.15.100:0/959317491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558015a110 con 0x7f558015b660 2024-04-23T14:33:33.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f557fe00640 1 --2- 172.21.15.100:0/959317491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55800c6d60 0x7f558014c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:33.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f5577400640 1 --2- 172.21.15.100:0/959317491 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f55800d08a0 0x7f55800aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:33.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f5584e00640 1 --2- 172.21.15.100:0/959317491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558015b660 0x7f55801572a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f557800c3c0 tx=0x7f557800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:33.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f557cc00640 1 -- 172.21.15.100:0/959317491 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578017070 con 0x7f558015b660 2024-04-23T14:33:33.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f557cc00640 1 -- 172.21.15.100:0/959317491 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5578005cf0 con 0x7f558015b660 2024-04-23T14:33:33.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f557cc00640 1 -- 172.21.15.100:0/959317491 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55780072f0 con 0x7f558015b660 2024-04-23T14:33:33.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f5586e00640 1 -- 172.21.15.100:0/959317491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558015a3f0 con 0x7f558015b660 2024-04-23T14:33:33.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.453+0000 7f5586e00640 1 -- 172.21.15.100:0/959317491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f558015a9b0 con 0x7f558015b660 2024-04-23T14:33:33.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.457+0000 7f5586e00640 1 -- 172.21.15.100:0/959317491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5580068ad0 con 0x7f558015b660 2024-04-23T14:33:33.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.457+0000 7f557cc00640 1 -- 172.21.15.100:0/959317491 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f557801a070 con 0x7f558015b660 2024-04-23T14:33:33.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.457+0000 7f557cc00640 1 --2- 172.21.15.100:0/959317491 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f555c068380 0x7f555c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.457+0000 7f557cc00640 1 -- 172.21.15.100:0/959317491 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f5578085bf0 con 0x7f558015b660 2024-04-23T14:33:33.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.457+0000 7f557fe00640 1 --2- 172.21.15.100:0/959317491 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f555c068380 0x7f555c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.457+0000 7f557fe00640 1 --2- 172.21.15.100:0/959317491 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f555c068380 0x7f555c06a840 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f55700af000 tx=0x7f5570008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:33.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.461+0000 7f557cc00640 1 -- 172.21.15.100:0/959317491 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f55800cee00 con 0x7f558015b660 2024-04-23T14:33:33.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.681+0000 7f5586e00640 1 -- 172.21.15.100:0/959317491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f558006d3a0 con 0x7f558015b660 2024-04-23T14:33:33.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.681+0000 7f557cc00640 1 -- 172.21.15.100:0/959317491 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f5578056360 con 0x7f558015b660 2024-04-23T14:33:33.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.685+0000 7f5586e00640 1 -- 172.21.15.100:0/959317491 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f555c068380 msgr2=0x7f555c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.685+0000 7f5586e00640 1 --2- 172.21.15.100:0/959317491 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f555c068380 0x7f555c06a840 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f55700af000 tx=0x7f5570008040 comp rx=0 tx=0).stop 2024-04-23T14:33:33.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.685+0000 7f5586e00640 1 -- 172.21.15.100:0/959317491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558015b660 msgr2=0x7f55801572a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.685+0000 7f5586e00640 1 --2- 172.21.15.100:0/959317491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558015b660 0x7f55801572a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f557800c3c0 tx=0x7f557800c880 comp rx=0 tx=0).stop 2024-04-23T14:33:33.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.685+0000 7f5586e00640 1 -- 172.21.15.100:0/959317491 shutdown_connections 2024-04-23T14:33:33.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.685+0000 7f5586e00640 1 --2- 172.21.15.100:0/959317491 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f555c068380 0x7f555c06a840 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.685+0000 7f5586e00640 1 --2- 172.21.15.100:0/959317491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558015b660 0x7f55801572a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.685+0000 7f5586e00640 1 --2- 172.21.15.100:0/959317491 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f55800d08a0 0x7f55800aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.685+0000 7f5586e00640 1 --2- 172.21.15.100:0/959317491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55800c6d60 0x7f558014c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.685+0000 7f5586e00640 1 -- 172.21.15.100:0/959317491 >> 172.21.15.100:0/959317491 conn(0x7f55800b5310 msgr2=0x7f55800b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:33.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.685+0000 7f5586e00640 1 -- 172.21.15.100:0/959317491 shutdown_connections 2024-04-23T14:33:33.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.685+0000 7f5586e00640 1 -- 172.21.15.100:0/959317491 wait complete. 2024-04-23T14:33:33.707 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-23T14:33:33.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.841+0000 7fed57200640 1 Processor -- start 2024-04-23T14:33:33.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.841+0000 7fed57200640 1 -- start start 2024-04-23T14:33:33.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.841+0000 7fed57200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed58160030 0x7fed58160470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.841+0000 7fed57200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fed5806cd50 0x7fed5806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.841+0000 7fed57200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed58160e10 con 0x7fed58145c30 2024-04-23T14:33:33.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.841+0000 7fed57200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fed580cf930 con 0x7fed5806cd50 2024-04-23T14:33:33.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.841+0000 7fed57200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fed581609b0 con 0x7fed58160030 2024-04-23T14:33:33.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.841+0000 7fed56800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fed5806cd50 0x7fed5806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.841+0000 7fed55e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fed58145c30 0x7fed580d0920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:60382/0 (socket says 172.21.15.100:60382) 2024-04-23T14:33:33.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.841+0000 7fed55e00640 1 -- 172.21.15.100:0/3743212537 learned_addr learned my addr 172.21.15.100:0/3743212537 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:33.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.841+0000 7fed56800640 1 -- 172.21.15.100:0/3743212537 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed58160030 msgr2=0x7fed58160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:33.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.841+0000 7fed55400640 1 --2- 172.21.15.100:0/3743212537 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed58160030 0x7fed58160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.841+0000 7fed56800640 1 --2- 172.21.15.100:0/3743212537 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed58160030 0x7fed58160470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.841+0000 7fed56800640 1 -- 172.21.15.100:0/3743212537 >> v1:172.21.15.100:6789/0 conn(0x7fed58145c30 legacy=0x7fed580d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.841+0000 7fed56800640 1 -- 172.21.15.100:0/3743212537 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed580cfb80 con 0x7fed5806cd50 2024-04-23T14:33:33.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.841+0000 7fed55400640 1 --2- 172.21.15.100:0/3743212537 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed58160030 0x7fed58160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:33.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.841+0000 7fed54a00640 1 -- 172.21.15.100:0/3743212537 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1407917536 0 0) 0x7fed58160e10 con 0x7fed58145c30 2024-04-23T14:33:33.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.845+0000 7fed56800640 1 --2- 172.21.15.100:0/3743212537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed5806cd50 0x7fed5806d1a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fed50009d90 tx=0x7fed5001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b16259c09a706b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:33.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.845+0000 7fed54a00640 1 -- 172.21.15.100:0/3743212537 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed500b2070 con 0x7fed5806cd50 2024-04-23T14:33:33.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.845+0000 7fed54a00640 1 -- 172.21.15.100:0/3743212537 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed500a58b0 con 0x7fed5806cd50 2024-04-23T14:33:33.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.845+0000 7fed54a00640 1 -- 172.21.15.100:0/3743212537 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed500a5c80 con 0x7fed5806cd50 2024-04-23T14:33:33.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.845+0000 7fed57200640 1 -- 172.21.15.100:0/3743212537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed5806cd50 msgr2=0x7fed5806d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.845+0000 7fed57200640 1 --2- 172.21.15.100:0/3743212537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed5806cd50 0x7fed5806d1a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fed50009d90 tx=0x7fed5001c8e0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.845+0000 7fed57200640 1 -- 172.21.15.100:0/3743212537 shutdown_connections 2024-04-23T14:33:33.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.845+0000 7fed57200640 1 --2- 172.21.15.100:0/3743212537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed5806cd50 0x7fed5806d1a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.845+0000 7fed57200640 1 --2- 172.21.15.100:0/3743212537 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed58160030 0x7fed58160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.845+0000 7fed57200640 1 -- 172.21.15.100:0/3743212537 >> 172.21.15.100:0/3743212537 conn(0x7fed58072bd0 msgr2=0x7fed58072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:33.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.845+0000 7fed57200640 1 -- 172.21.15.100:0/3743212537 shutdown_connections 2024-04-23T14:33:33.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.845+0000 7fed57200640 1 -- 172.21.15.100:0/3743212537 wait complete. 2024-04-23T14:33:33.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed57200640 1 Processor -- start 2024-04-23T14:33:33.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed57200640 1 -- start start 2024-04-23T14:33:33.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed57200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed5806cd50 0x7fed580b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed57200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed58145c30 0x7fed580b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed57200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed58160030 0x7fed580b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed57200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fed5806d6e0 con 0x7fed58145c30 2024-04-23T14:33:33.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed57200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fed58146220 con 0x7fed5806cd50 2024-04-23T14:33:33.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed57200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fed580ce260 con 0x7fed58160030 2024-04-23T14:33:33.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed55400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed58145c30 0x7fed580b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed55400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed58145c30 0x7fed580b4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39962/0 (socket says 172.21.15.100:39962) 2024-04-23T14:33:33.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed55400640 1 -- 172.21.15.100:0/2199537555 learned_addr learned my addr 172.21.15.100:0/2199537555 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:33.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed55e00640 1 --2- 172.21.15.100:0/2199537555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed5806cd50 0x7fed580b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed56800640 1 --2- 172.21.15.100:0/2199537555 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed58160030 0x7fed580b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed55400640 1 -- 172.21.15.100:0/2199537555 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed58160030 msgr2=0x7fed580b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed55400640 1 --2- 172.21.15.100:0/2199537555 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed58160030 0x7fed580b5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed55400640 1 -- 172.21.15.100:0/2199537555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed5806cd50 msgr2=0x7fed580b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed55400640 1 --2- 172.21.15.100:0/2199537555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed5806cd50 0x7fed580b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed55400640 1 -- 172.21.15.100:0/2199537555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed581fae10 con 0x7fed58145c30 2024-04-23T14:33:33.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed55400640 1 --2- 172.21.15.100:0/2199537555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed58145c30 0x7fed580b4d10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fed4400c820 tx=0x7fed4400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:33.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed4e000640 1 -- 172.21.15.100:0/2199537555 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44015070 con 0x7fed58145c30 2024-04-23T14:33:33.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed4e000640 1 -- 172.21.15.100:0/2199537555 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed44007a30 con 0x7fed58145c30 2024-04-23T14:33:33.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed57200640 1 -- 172.21.15.100:0/2199537555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed580b5950 con 0x7fed58145c30 2024-04-23T14:33:33.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed4e000640 1 -- 172.21.15.100:0/2199537555 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44007e40 con 0x7fed58145c30 2024-04-23T14:33:33.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.849+0000 7fed57200640 1 -- 172.21.15.100:0/2199537555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed58142cd0 con 0x7fed58145c30 2024-04-23T14:33:33.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.853+0000 7fed57200640 1 -- 172.21.15.100:0/2199537555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed58142ed0 con 0x7fed58145c30 2024-04-23T14:33:33.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.853+0000 7fed4e000640 1 -- 172.21.15.100:0/2199537555 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fed44018070 con 0x7fed58145c30 2024-04-23T14:33:33.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.853+0000 7fed4e000640 1 --2- 172.21.15.100:0/2199537555 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fed38068380 0x7fed3806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.853+0000 7fed4e000640 1 -- 172.21.15.100:0/2199537555 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fed440866d0 con 0x7fed58145c30 2024-04-23T14:33:33.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.853+0000 7fed55e00640 1 --2- 172.21.15.100:0/2199537555 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fed38068380 0x7fed3806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.853+0000 7fed55e00640 1 --2- 172.21.15.100:0/2199537555 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fed38068380 0x7fed3806a840 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fed48007fd0 tx=0x7fed48007d60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:33.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:33.857+0000 7fed4e000640 1 -- 172.21.15.100:0/2199537555 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fed58142ed0 con 0x7fed58145c30 2024-04-23T14:33:34.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.069+0000 7fed57200640 1 -- 172.21.15.100:0/2199537555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fed5813e0e0 con 0x7fed58145c30 2024-04-23T14:33:34.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.073+0000 7fed4e000640 1 -- 172.21.15.100:0/2199537555 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7fed580b5950 con 0x7fed58145c30 2024-04-23T14:33:34.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.073+0000 7fed57200640 1 -- 172.21.15.100:0/2199537555 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fed38068380 msgr2=0x7fed3806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:34.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.073+0000 7fed57200640 1 --2- 172.21.15.100:0/2199537555 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fed38068380 0x7fed3806a840 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fed48007fd0 tx=0x7fed48007d60 comp rx=0 tx=0).stop 2024-04-23T14:33:34.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.073+0000 7fed57200640 1 -- 172.21.15.100:0/2199537555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed58145c30 msgr2=0x7fed580b4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:34.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.073+0000 7fed57200640 1 --2- 172.21.15.100:0/2199537555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed58145c30 0x7fed580b4d10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fed4400c820 tx=0x7fed4400cce0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.073+0000 7fed57200640 1 -- 172.21.15.100:0/2199537555 shutdown_connections 2024-04-23T14:33:34.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.073+0000 7fed57200640 1 --2- 172.21.15.100:0/2199537555 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fed38068380 0x7fed3806a840 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.073+0000 7fed57200640 1 --2- 172.21.15.100:0/2199537555 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed58160030 0x7fed580b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.073+0000 7fed57200640 1 --2- 172.21.15.100:0/2199537555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed58145c30 0x7fed580b4d10 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.073+0000 7fed57200640 1 --2- 172.21.15.100:0/2199537555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed5806cd50 0x7fed580b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.073+0000 7fed57200640 1 -- 172.21.15.100:0/2199537555 >> 172.21.15.100:0/2199537555 conn(0x7fed58072bd0 msgr2=0x7fed5813e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:34.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.073+0000 7fed57200640 1 -- 172.21.15.100:0/2199537555 shutdown_connections 2024-04-23T14:33:34.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.077+0000 7fed57200640 1 -- 172.21.15.100:0/2199537555 wait complete. 2024-04-23T14:33:34.098 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-23T14:33:34.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.229+0000 7f4b68800640 1 Processor -- start 2024-04-23T14:33:34.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.229+0000 7f4b68800640 1 -- start start 2024-04-23T14:33:34.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.229+0000 7f4b68800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b600cde20 0x7f4b600cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:34.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.229+0000 7f4b68800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b600cbee0 0x7f4b600cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:34.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.229+0000 7f4b68800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b600ce280 con 0x7f4b6015d0c0 2024-04-23T14:33:34.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.229+0000 7f4b68800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f4b600cff50 con 0x7f4b600cbee0 2024-04-23T14:33:34.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.229+0000 7f4b68800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4b600ceda0 con 0x7f4b600cde20 2024-04-23T14:33:34.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.233+0000 7f4b65e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f4b6015d0c0 0x7f4b6015d4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:60384/0 (socket says 172.21.15.100:60384) 2024-04-23T14:33:34.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.233+0000 7f4b65e00640 1 -- 172.21.15.100:0/821850667 learned_addr learned my addr 172.21.15.100:0/821850667 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:34.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.233+0000 7f4b66800640 1 --2- 172.21.15.100:0/821850667 >> v2:172.21.15.100:3300/0 conn(0x7f4b600cbee0 0x7f4b600cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:34.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.233+0000 7f4b65400640 1 --2- 172.21.15.100:0/821850667 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b600cde20 0x7f4b600cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:34.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.233+0000 7f4b66800640 1 -- 172.21.15.100:0/821850667 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b600cde20 msgr2=0x7f4b600cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:34.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.233+0000 7f4b66800640 1 --2- 172.21.15.100:0/821850667 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b600cde20 0x7f4b600cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.233+0000 7f4b66800640 1 -- 172.21.15.100:0/821850667 >> v1:172.21.15.100:6789/0 conn(0x7f4b6015d0c0 legacy=0x7f4b6015d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:34.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.233+0000 7f4b66800640 1 -- 172.21.15.100:0/821850667 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b600ce500 con 0x7f4b600cbee0 2024-04-23T14:33:34.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.233+0000 7f4b64a00640 1 -- 172.21.15.100:0/821850667 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3261888920 0 0) 0x7f4b54001120 con 0x7f4b6015d0c0 2024-04-23T14:33:34.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.233+0000 7f4b66800640 1 --2- 172.21.15.100:0/821850667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b600cbee0 0x7f4b600cf160 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009d90 tx=0x7f4b5c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cdd63896202615 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:34.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.233+0000 7f4b64a00640 1 -- 172.21.15.100:0/821850667 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c0b2070 con 0x7f4b600cbee0 2024-04-23T14:33:34.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.233+0000 7f4b64a00640 1 -- 172.21.15.100:0/821850667 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b5c0a57a0 con 0x7f4b600cbee0 2024-04-23T14:33:34.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.233+0000 7f4b64a00640 1 -- 172.21.15.100:0/821850667 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c0a5b70 con 0x7f4b600cbee0 2024-04-23T14:33:34.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b68800640 1 -- 172.21.15.100:0/821850667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b600cbee0 msgr2=0x7f4b600cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:34.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b68800640 1 --2- 172.21.15.100:0/821850667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b600cbee0 0x7f4b600cf160 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009d90 tx=0x7f4b5c01c8e0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b68800640 1 -- 172.21.15.100:0/821850667 shutdown_connections 2024-04-23T14:33:34.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b68800640 1 --2- 172.21.15.100:0/821850667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b600cbee0 0x7f4b600cf160 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b68800640 1 --2- 172.21.15.100:0/821850667 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b600cde20 0x7f4b600cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b68800640 1 -- 172.21.15.100:0/821850667 >> 172.21.15.100:0/821850667 conn(0x7f4b600b5310 msgr2=0x7f4b600b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:34.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b68800640 1 -- 172.21.15.100:0/821850667 shutdown_connections 2024-04-23T14:33:34.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b68800640 1 -- 172.21.15.100:0/821850667 wait complete. 2024-04-23T14:33:34.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b68800640 1 Processor -- start 2024-04-23T14:33:34.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b68800640 1 -- start start 2024-04-23T14:33:34.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b68800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b600cbee0 0x7f4b60073830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:34.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b68800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b600cde20 0x7f4b60165fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:34.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b68800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b6015d0c0 0x7f4b600b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:34.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b68800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4b60152fb0 con 0x7f4b600cde20 2024-04-23T14:33:34.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b68800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4b60152e30 con 0x7f4b600cbee0 2024-04-23T14:33:34.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b68800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4b60153130 con 0x7f4b6015d0c0 2024-04-23T14:33:34.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b65400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b600cde20 0x7f4b60165fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:34.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b65400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b600cde20 0x7f4b60165fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39986/0 (socket says 172.21.15.100:39986) 2024-04-23T14:33:34.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b65400640 1 -- 172.21.15.100:0/454984727 learned_addr learned my addr 172.21.15.100:0/454984727 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:34.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b66800640 1 --2- 172.21.15.100:0/454984727 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b6015d0c0 0x7f4b600b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:34.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b65e00640 1 --2- 172.21.15.100:0/454984727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b600cbee0 0x7f4b60073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:34.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b65400640 1 -- 172.21.15.100:0/454984727 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b6015d0c0 msgr2=0x7f4b600b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:34.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b65400640 1 --2- 172.21.15.100:0/454984727 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b6015d0c0 0x7f4b600b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b65400640 1 -- 172.21.15.100:0/454984727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b600cbee0 msgr2=0x7f4b60073830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:34.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b65400640 1 --2- 172.21.15.100:0/454984727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b600cbee0 0x7f4b60073830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.237+0000 7f4b65400640 1 -- 172.21.15.100:0/454984727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b60072880 con 0x7f4b600cde20 2024-04-23T14:33:34.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.241+0000 7f4b65400640 1 --2- 172.21.15.100:0/454984727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b600cde20 0x7f4b60165fc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000d800 tx=0x7f4b5000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:34.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.241+0000 7f4b5a000640 1 -- 172.21.15.100:0/454984727 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5000b6c0 con 0x7f4b600cde20 2024-04-23T14:33:34.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.241+0000 7f4b5a000640 1 -- 172.21.15.100:0/454984727 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b50019070 con 0x7f4b600cde20 2024-04-23T14:33:34.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.241+0000 7f4b5a000640 1 -- 172.21.15.100:0/454984727 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5000ba60 con 0x7f4b600cde20 2024-04-23T14:33:34.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.241+0000 7f4b68800640 1 -- 172.21.15.100:0/454984727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b60072b60 con 0x7f4b600cde20 2024-04-23T14:33:34.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.241+0000 7f4b68800640 1 -- 172.21.15.100:0/454984727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b60073120 con 0x7f4b600cde20 2024-04-23T14:33:34.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.241+0000 7f4b68800640 1 -- 172.21.15.100:0/454984727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b60072610 con 0x7f4b600cde20 2024-04-23T14:33:34.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.241+0000 7f4b5a000640 1 -- 172.21.15.100:0/454984727 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4b5000bc00 con 0x7f4b600cde20 2024-04-23T14:33:34.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.241+0000 7f4b5a000640 1 --2- 172.21.15.100:0/454984727 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4b44068380 0x7f4b4406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:34.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.241+0000 7f4b5a000640 1 -- 172.21.15.100:0/454984727 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f4b50086c30 con 0x7f4b600cde20 2024-04-23T14:33:34.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.241+0000 7f4b65e00640 1 --2- 172.21.15.100:0/454984727 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4b44068380 0x7f4b4406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:34.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.245+0000 7f4b65e00640 1 --2- 172.21.15.100:0/454984727 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4b44068380 0x7f4b4406a840 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4b54000f70 tx=0x7f4b54007c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:34.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.245+0000 7f4b5a000640 1 -- 172.21.15.100:0/454984727 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4b5008d080 con 0x7f4b600cde20 2024-04-23T14:33:34.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.457+0000 7f4b68800640 1 -- 172.21.15.100:0/454984727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f4b60068ad0 con 0x7f4b600cde20 2024-04-23T14:33:34.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.461+0000 7f4b5a000640 1 -- 172.21.15.100:0/454984727 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f4b50057420 con 0x7f4b600cde20 2024-04-23T14:33:34.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.461+0000 7f4b68800640 1 -- 172.21.15.100:0/454984727 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4b44068380 msgr2=0x7f4b4406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:34.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.461+0000 7f4b68800640 1 --2- 172.21.15.100:0/454984727 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4b44068380 0x7f4b4406a840 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4b54000f70 tx=0x7f4b54007c10 comp rx=0 tx=0).stop 2024-04-23T14:33:34.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.461+0000 7f4b68800640 1 -- 172.21.15.100:0/454984727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b600cde20 msgr2=0x7f4b60165fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:34.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.461+0000 7f4b68800640 1 --2- 172.21.15.100:0/454984727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b600cde20 0x7f4b60165fc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000d800 tx=0x7f4b5000dcc0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.461+0000 7f4b68800640 1 -- 172.21.15.100:0/454984727 shutdown_connections 2024-04-23T14:33:34.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.461+0000 7f4b68800640 1 --2- 172.21.15.100:0/454984727 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4b44068380 0x7f4b4406a840 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.461+0000 7f4b68800640 1 --2- 172.21.15.100:0/454984727 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b6015d0c0 0x7f4b600b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.461+0000 7f4b68800640 1 --2- 172.21.15.100:0/454984727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b600cde20 0x7f4b60165fc0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.461+0000 7f4b68800640 1 --2- 172.21.15.100:0/454984727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b600cbee0 0x7f4b60073830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.461+0000 7f4b68800640 1 -- 172.21.15.100:0/454984727 >> 172.21.15.100:0/454984727 conn(0x7f4b600b5310 msgr2=0x7f4b6015d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:34.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.461+0000 7f4b68800640 1 -- 172.21.15.100:0/454984727 shutdown_connections 2024-04-23T14:33:34.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.461+0000 7f4b68800640 1 -- 172.21.15.100:0/454984727 wait complete. 2024-04-23T14:33:34.485 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-23T14:33:34.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.617+0000 7f9371e00640 1 Processor -- start 2024-04-23T14:33:34.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.617+0000 7f9371e00640 1 -- start start 2024-04-23T14:33:34.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.621+0000 7f9371e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f936c0cde20 0x7f936c0cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:34.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.621+0000 7f9371e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f936c0cbee0 0x7f936c0cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:34.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.621+0000 7f9371e00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f936c0ce500 con 0x7f936c15d0c0 2024-04-23T14:33:34.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.621+0000 7f9371e00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f936c0cff50 con 0x7f936c0cde20 2024-04-23T14:33:34.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.621+0000 7f9371e00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f936c0ceda0 con 0x7f936c0cbee0 2024-04-23T14:33:34.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.621+0000 7f936a000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f936c0cde20 0x7f936c0cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:34.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.621+0000 7f936a000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f936c0cde20 0x7f936c0cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40002/0 (socket says 172.21.15.100:40002) 2024-04-23T14:33:34.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.621+0000 7f936a000640 1 -- 172.21.15.100:0/1828210723 learned_addr learned my addr 172.21.15.100:0/1828210723 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:34.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.621+0000 7f936b400640 1 --2- 172.21.15.100:0/1828210723 >> v2:172.21.15.151:3300/0 conn(0x7f936c0cbee0 0x7f936c0cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:34.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.621+0000 7f936a000640 1 -- 172.21.15.100:0/1828210723 >> v2:172.21.15.151:3300/0 conn(0x7f936c0cbee0 msgr2=0x7f936c0cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:34.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.621+0000 7f936a000640 1 --2- 172.21.15.100:0/1828210723 >> v2:172.21.15.151:3300/0 conn(0x7f936c0cbee0 0x7f936c0cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.621+0000 7f936a000640 1 -- 172.21.15.100:0/1828210723 >> v1:172.21.15.151:6789/0 conn(0x7f936c15d0c0 legacy=0x7f936c15d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:34.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.621+0000 7f936a000640 1 -- 172.21.15.100:0/1828210723 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936c0ce7e0 con 0x7f936c0cde20 2024-04-23T14:33:34.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.621+0000 7f936b400640 1 --2- 172.21.15.100:0/1828210723 >> v2:172.21.15.151:3300/0 conn(0x7f936c0cbee0 0x7f936c0cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:34.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.625+0000 7f936a000640 1 --2- 172.21.15.100:0/1828210723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f936c0cde20 0x7f936c0cb9a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f936c0af7d0 tx=0x7f935801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95322643ca16403d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:34.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.625+0000 7f9369600640 1 -- 172.21.15.100:0/1828210723 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93580b0070 con 0x7f936c0cde20 2024-04-23T14:33:34.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.625+0000 7f9369600640 1 -- 172.21.15.100:0/1828210723 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93580ab600 con 0x7f936c0cde20 2024-04-23T14:33:34.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.625+0000 7f9369600640 1 -- 172.21.15.100:0/1828210723 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93580ab9d0 con 0x7f936c0cde20 2024-04-23T14:33:34.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.625+0000 7f9371e00640 1 -- 172.21.15.100:0/1828210723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f936c0cde20 msgr2=0x7f936c0cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:34.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.625+0000 7f9371e00640 1 --2- 172.21.15.100:0/1828210723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f936c0cde20 0x7f936c0cb9a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f936c0af7d0 tx=0x7f935801c450 comp rx=0 tx=0).stop 2024-04-23T14:33:34.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.625+0000 7f9371e00640 1 -- 172.21.15.100:0/1828210723 shutdown_connections 2024-04-23T14:33:34.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.625+0000 7f9371e00640 1 --2- 172.21.15.100:0/1828210723 >> v2:172.21.15.151:3300/0 conn(0x7f936c0cbee0 0x7f936c0cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.625+0000 7f9371e00640 1 --2- 172.21.15.100:0/1828210723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f936c0cde20 0x7f936c0cb9a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.625+0000 7f9371e00640 1 -- 172.21.15.100:0/1828210723 >> 172.21.15.100:0/1828210723 conn(0x7f936c0b5310 msgr2=0x7f936c0b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:34.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.625+0000 7f9371e00640 1 -- 172.21.15.100:0/1828210723 shutdown_connections 2024-04-23T14:33:34.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.625+0000 7f9371e00640 1 -- 172.21.15.100:0/1828210723 wait complete. 2024-04-23T14:33:34.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.625+0000 7f9371e00640 1 Processor -- start 2024-04-23T14:33:34.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.625+0000 7f9371e00640 1 -- start start 2024-04-23T14:33:34.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f9371e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f936c0cbee0 0x7f936c0aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:34.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f9371e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f936c0cde20 0x7f936c0adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:34.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f9371e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f936c15d0c0 0x7f936c1f7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:34.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f9371e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f936c1531c0 con 0x7f936c0cbee0 2024-04-23T14:33:34.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f9371e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f936c153040 con 0x7f936c15d0c0 2024-04-23T14:33:34.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f9371e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f936c153340 con 0x7f936c0cde20 2024-04-23T14:33:34.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f936aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f936c0cbee0 0x7f936c0aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:34.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f936aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f936c0cbee0 0x7f936c0aec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40018/0 (socket says 172.21.15.100:40018) 2024-04-23T14:33:34.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f936aa00640 1 -- 172.21.15.100:0/174313622 learned_addr learned my addr 172.21.15.100:0/174313622 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:34.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f936b400640 1 --2- 172.21.15.100:0/174313622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f936c15d0c0 0x7f936c1f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:34.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f936a000640 1 --2- 172.21.15.100:0/174313622 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f936c0cde20 0x7f936c0adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:34.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f936aa00640 1 -- 172.21.15.100:0/174313622 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f936c0cde20 msgr2=0x7f936c0adb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:34.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f936aa00640 1 --2- 172.21.15.100:0/174313622 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f936c0cde20 0x7f936c0adb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f936aa00640 1 -- 172.21.15.100:0/174313622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f936c15d0c0 msgr2=0x7f936c1f7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:34.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f936aa00640 1 --2- 172.21.15.100:0/174313622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f936c15d0c0 0x7f936c1f7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f936aa00640 1 -- 172.21.15.100:0/174313622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936c071cc0 con 0x7f936c0cbee0 2024-04-23T14:33:34.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f936aa00640 1 --2- 172.21.15.100:0/174313622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f936c0cbee0 0x7f936c0aec50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f935c00ba80 tx=0x7f935c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:34.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f9363400640 1 -- 172.21.15.100:0/174313622 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c00cc00 con 0x7f936c0cbee0 2024-04-23T14:33:34.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f9363400640 1 -- 172.21.15.100:0/174313622 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f935c00cda0 con 0x7f936c0cbee0 2024-04-23T14:33:34.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f9371e00640 1 -- 172.21.15.100:0/174313622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f936c071fa0 con 0x7f936c0cbee0 2024-04-23T14:33:34.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f9363400640 1 -- 172.21.15.100:0/174313622 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c003bc0 con 0x7f936c0cbee0 2024-04-23T14:33:34.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f9371e00640 1 -- 172.21.15.100:0/174313622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f936c165920 con 0x7f936c0cbee0 2024-04-23T14:33:34.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f9371e00640 1 -- 172.21.15.100:0/174313622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f936c068ad0 con 0x7f936c0cbee0 2024-04-23T14:33:34.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.629+0000 7f9363400640 1 -- 172.21.15.100:0/174313622 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f935c006d10 con 0x7f936c0cbee0 2024-04-23T14:33:34.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.633+0000 7f9363400640 1 --2- 172.21.15.100:0/174313622 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f934c0683b0 0x7f934c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:34.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.633+0000 7f9363400640 1 -- 172.21.15.100:0/174313622 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f935c086350 con 0x7f936c0cbee0 2024-04-23T14:33:34.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.633+0000 7f936a000640 1 --2- 172.21.15.100:0/174313622 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f934c0683b0 0x7f934c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:34.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.633+0000 7f936a000640 1 --2- 172.21.15.100:0/174313622 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f934c0683b0 0x7f934c06a870 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f93580ad000 tx=0x7f9358008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:34.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.637+0000 7f9363400640 1 -- 172.21.15.100:0/174313622 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f935c056470 con 0x7f936c0cbee0 2024-04-23T14:33:34.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.849+0000 7f9371e00640 1 -- 172.21.15.100:0/174313622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f936c0af190 con 0x7f936c0cbee0 2024-04-23T14:33:34.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.853+0000 7f9363400640 1 -- 172.21.15.100:0/174313622 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f935c05a770 con 0x7f936c0cbee0 2024-04-23T14:33:34.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.853+0000 7f9371e00640 1 -- 172.21.15.100:0/174313622 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f934c0683b0 msgr2=0x7f934c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:34.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.853+0000 7f9371e00640 1 --2- 172.21.15.100:0/174313622 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f934c0683b0 0x7f934c06a870 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f93580ad000 tx=0x7f9358008040 comp rx=0 tx=0).stop 2024-04-23T14:33:34.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.853+0000 7f9371e00640 1 -- 172.21.15.100:0/174313622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f936c0cbee0 msgr2=0x7f936c0aec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:34.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.853+0000 7f9371e00640 1 --2- 172.21.15.100:0/174313622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f936c0cbee0 0x7f936c0aec50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f935c00ba80 tx=0x7f935c00bf40 comp rx=0 tx=0).stop 2024-04-23T14:33:34.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.853+0000 7f9371e00640 1 -- 172.21.15.100:0/174313622 shutdown_connections 2024-04-23T14:33:34.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.853+0000 7f9371e00640 1 --2- 172.21.15.100:0/174313622 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f934c0683b0 0x7f934c06a870 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.853+0000 7f9371e00640 1 --2- 172.21.15.100:0/174313622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f936c15d0c0 0x7f936c1f7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.853+0000 7f9371e00640 1 --2- 172.21.15.100:0/174313622 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f936c0cde20 0x7f936c0adb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.853+0000 7f9371e00640 1 --2- 172.21.15.100:0/174313622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f936c0cbee0 0x7f936c0aec50 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.853+0000 7f9371e00640 1 -- 172.21.15.100:0/174313622 >> 172.21.15.100:0/174313622 conn(0x7f936c0b5310 msgr2=0x7f936c15e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:34.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.853+0000 7f9371e00640 1 -- 172.21.15.100:0/174313622 shutdown_connections 2024-04-23T14:33:34.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:34.857+0000 7f9371e00640 1 -- 172.21.15.100:0/174313622 wait complete. 2024-04-23T14:33:34.878 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-23T14:33:35.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.009+0000 7f1d73600640 1 Processor -- start 2024-04-23T14:33:35.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.009+0000 7f1d73600640 1 -- start start 2024-04-23T14:33:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.013+0000 7f1d73600640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f1d6c0d0620 0x7f1d6c0c5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.013+0000 7f1d73600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d6c0cee00 con 0x7f1d6c0d0190 2024-04-23T14:33:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.013+0000 7f1d73600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d6c0d0ab0 con 0x7f1d6c15b660 2024-04-23T14:33:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.013+0000 7f1d73600640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f1d6c15ba40 con 0x7f1d6c0d0620 2024-04-23T14:33:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.013+0000 7f1d6be00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1d6c0d0190 0x7f1d6c0c5160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:60400/0 (socket says 172.21.15.100:60400) 2024-04-23T14:33:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.013+0000 7f1d6be00640 1 -- 172.21.15.100:0/4201217130 learned_addr learned my addr 172.21.15.100:0/4201217130 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.013+0000 7f1d71600640 1 --2- 172.21.15.100:0/4201217130 >> v2:172.21.15.151:3300/0 conn(0x7f1d6c0d0620 0x7f1d6c0c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.013+0000 7f1d6b400640 1 -- 172.21.15.100:0/4201217130 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1069451112 0 0) 0x7f1d58001120 con 0x7f1d6c0d0190 2024-04-23T14:33:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.013+0000 7f1d6b400640 1 -- 172.21.15.100:0/4201217130 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d50003530 con 0x7f1d6c0d0190 2024-04-23T14:33:35.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.013+0000 7f1d6b400640 1 -- 172.21.15.100:0/4201217130 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 443404860 0 0) 0x7f1d6c0d0ab0 con 0x7f1d6c15b660 2024-04-23T14:33:35.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.013+0000 7f1d6b400640 1 -- 172.21.15.100:0/4201217130 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d58001120 con 0x7f1d6c15b660 2024-04-23T14:33:35.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.013+0000 7f1d71600640 1 -- 172.21.15.100:0/4201217130 >> v1:172.21.15.151:6789/0 conn(0x7f1d6c15b660 legacy=0x7f1d6c0c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.013+0000 7f1d71600640 1 -- 172.21.15.100:0/4201217130 >> v1:172.21.15.100:6789/0 conn(0x7f1d6c0d0190 legacy=0x7f1d6c0c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.013+0000 7f1d71600640 1 -- 172.21.15.100:0/4201217130 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6c06bc50 con 0x7f1d6c0d0620 2024-04-23T14:33:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.017+0000 7f1d71600640 1 --2- 172.21.15.100:0/4201217130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d6c0d0620 0x7f1d6c0c5870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1d640097c0 tx=0x7f1d6401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b8a0a90795159f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:35.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.017+0000 7f1d6b400640 1 -- 172.21.15.100:0/4201217130 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d640a7070 con 0x7f1d6c0d0620 2024-04-23T14:33:35.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.017+0000 7f1d6b400640 1 -- 172.21.15.100:0/4201217130 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d640b0070 con 0x7f1d6c0d0620 2024-04-23T14:33:35.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.017+0000 7f1d6b400640 1 -- 172.21.15.100:0/4201217130 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d640ab890 con 0x7f1d6c0d0620 2024-04-23T14:33:35.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.017+0000 7f1d73600640 1 -- 172.21.15.100:0/4201217130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d6c0d0620 msgr2=0x7f1d6c0c5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.017+0000 7f1d73600640 1 --2- 172.21.15.100:0/4201217130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d6c0d0620 0x7f1d6c0c5870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1d640097c0 tx=0x7f1d6401c470 comp rx=0 tx=0).stop 2024-04-23T14:33:35.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.017+0000 7f1d73600640 1 -- 172.21.15.100:0/4201217130 shutdown_connections 2024-04-23T14:33:35.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.017+0000 7f1d73600640 1 --2- 172.21.15.100:0/4201217130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d6c0d0620 0x7f1d6c0c5870 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.017+0000 7f1d73600640 1 -- 172.21.15.100:0/4201217130 >> 172.21.15.100:0/4201217130 conn(0x7f1d6c0b5310 msgr2=0x7f1d6c0b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:35.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.017+0000 7f1d73600640 1 -- 172.21.15.100:0/4201217130 shutdown_connections 2024-04-23T14:33:35.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.017+0000 7f1d73600640 1 -- 172.21.15.100:0/4201217130 wait complete. 2024-04-23T14:33:35.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.017+0000 7f1d73600640 1 Processor -- start 2024-04-23T14:33:35.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.017+0000 7f1d73600640 1 -- start start 2024-04-23T14:33:35.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d73600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d6c0d0190 0x7f1d6c15e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d73600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d6c0d0620 0x7f1d6c141a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d73600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d6c15b660 0x7f1d6c072460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d73600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c0c6f30 con 0x7f1d6c0d0620 2024-04-23T14:33:35.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d73600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c06c7d0 con 0x7f1d6c15b660 2024-04-23T14:33:35.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d73600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1d6c0c70b0 con 0x7f1d6c0d0190 2024-04-23T14:33:35.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d6be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d6c0d0620 0x7f1d6c141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d6be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d6c0d0620 0x7f1d6c141a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40034/0 (socket says 172.21.15.100:40034) 2024-04-23T14:33:35.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d6be00640 1 -- 172.21.15.100:0/172013920 learned_addr learned my addr 172.21.15.100:0/172013920 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:35.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d71600640 1 --2- 172.21.15.100:0/172013920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d6c15b660 0x7f1d6c072460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d70c00640 1 --2- 172.21.15.100:0/172013920 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d6c0d0190 0x7f1d6c15e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d6be00640 1 -- 172.21.15.100:0/172013920 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d6c0d0190 msgr2=0x7f1d6c15e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d6be00640 1 --2- 172.21.15.100:0/172013920 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d6c0d0190 0x7f1d6c15e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d6be00640 1 -- 172.21.15.100:0/172013920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d6c15b660 msgr2=0x7f1d6c072460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d6be00640 1 --2- 172.21.15.100:0/172013920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d6c15b660 0x7f1d6c072460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d6be00640 1 -- 172.21.15.100:0/172013920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6c15db20 con 0x7f1d6c0d0620 2024-04-23T14:33:35.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d6be00640 1 --2- 172.21.15.100:0/172013920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d6c0d0620 0x7f1d6c141a90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800c690 tx=0x7f1d5800cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:35.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d68c00640 1 -- 172.21.15.100:0/172013920 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58017070 con 0x7f1d6c0d0620 2024-04-23T14:33:35.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d68c00640 1 -- 172.21.15.100:0/172013920 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d58005ae0 con 0x7f1d6c0d0620 2024-04-23T14:33:35.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d73600640 1 -- 172.21.15.100:0/172013920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6c141fd0 con 0x7f1d6c0d0620 2024-04-23T14:33:35.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d68c00640 1 -- 172.21.15.100:0/172013920 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58007070 con 0x7f1d6c0d0620 2024-04-23T14:33:35.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d73600640 1 -- 172.21.15.100:0/172013920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d6c15ddc0 con 0x7f1d6c0d0620 2024-04-23T14:33:35.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.021+0000 7f1d73600640 1 -- 172.21.15.100:0/172013920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d6c068ad0 con 0x7f1d6c0d0620 2024-04-23T14:33:35.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.025+0000 7f1d68c00640 1 -- 172.21.15.100:0/172013920 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1d5801a070 con 0x7f1d6c0d0620 2024-04-23T14:33:35.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.025+0000 7f1d68c00640 1 --2- 172.21.15.100:0/172013920 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1d4c068380 0x7f1d4c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.025+0000 7f1d68c00640 1 -- 172.21.15.100:0/172013920 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f1d58085d10 con 0x7f1d6c0d0620 2024-04-23T14:33:35.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.025+0000 7f1d70c00640 1 --2- 172.21.15.100:0/172013920 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1d4c068380 0x7f1d4c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.025+0000 7f1d70c00640 1 --2- 172.21.15.100:0/172013920 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1d4c068380 0x7f1d4c06a840 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c001570 tx=0x7f1d5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:35.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.029+0000 7f1d68c00640 1 -- 172.21.15.100:0/172013920 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1d6c0cee00 con 0x7f1d6c0d0620 2024-04-23T14:33:35.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.241+0000 7f1d73600640 1 -- 172.21.15.100:0/172013920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f1d6c06d3a0 con 0x7f1d6c0d0620 2024-04-23T14:33:35.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.245+0000 7f1d68c00640 1 -- 172.21.15.100:0/172013920 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f1d58055e40 con 0x7f1d6c0d0620 2024-04-23T14:33:35.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.245+0000 7f1d73600640 1 -- 172.21.15.100:0/172013920 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1d4c068380 msgr2=0x7f1d4c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.245+0000 7f1d73600640 1 --2- 172.21.15.100:0/172013920 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1d4c068380 0x7f1d4c06a840 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c001570 tx=0x7f1d5c009040 comp rx=0 tx=0).stop 2024-04-23T14:33:35.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.245+0000 7f1d73600640 1 -- 172.21.15.100:0/172013920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d6c0d0620 msgr2=0x7f1d6c141a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.245+0000 7f1d73600640 1 --2- 172.21.15.100:0/172013920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d6c0d0620 0x7f1d6c141a90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800c690 tx=0x7f1d5800cb50 comp rx=0 tx=0).stop 2024-04-23T14:33:35.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.245+0000 7f1d73600640 1 -- 172.21.15.100:0/172013920 shutdown_connections 2024-04-23T14:33:35.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.245+0000 7f1d73600640 1 --2- 172.21.15.100:0/172013920 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1d4c068380 0x7f1d4c06a840 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.245+0000 7f1d73600640 1 --2- 172.21.15.100:0/172013920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d6c15b660 0x7f1d6c072460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.245+0000 7f1d73600640 1 --2- 172.21.15.100:0/172013920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d6c0d0620 0x7f1d6c141a90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.245+0000 7f1d73600640 1 --2- 172.21.15.100:0/172013920 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d6c0d0190 0x7f1d6c15e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.245+0000 7f1d73600640 1 -- 172.21.15.100:0/172013920 >> 172.21.15.100:0/172013920 conn(0x7f1d6c0b5310 msgr2=0x7f1d6c0b56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:35.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.245+0000 7f1d73600640 1 -- 172.21.15.100:0/172013920 shutdown_connections 2024-04-23T14:33:35.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.245+0000 7f1d73600640 1 -- 172.21.15.100:0/172013920 wait complete. 2024-04-23T14:33:35.270 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:35.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.401+0000 7f8a55c00640 1 Processor -- start 2024-04-23T14:33:35.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.401+0000 7f8a55c00640 1 -- start start 2024-04-23T14:33:35.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.405+0000 7f8a55c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a5015ad40 0x7f8a500af390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.405+0000 7f8a55c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8a500af970 0x7f8a50072760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.405+0000 7f8a55c00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a5006d130 con 0x7f8a50072ca0 2024-04-23T14:33:35.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.405+0000 7f8a55c00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f8a500ad720 con 0x7f8a500af970 2024-04-23T14:33:35.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.405+0000 7f8a55c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8a5006d410 con 0x7f8a5015ad40 2024-04-23T14:33:35.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.405+0000 7f8a4e000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8a500af970 0x7f8a50072760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.405+0000 7f8a4e000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8a500af970 0x7f8a50072760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40040/0 (socket says 172.21.15.100:40040) 2024-04-23T14:33:35.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.405+0000 7f8a4e000640 1 -- 172.21.15.100:0/3784471756 learned_addr learned my addr 172.21.15.100:0/3784471756 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:35.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.405+0000 7f8a4ea00640 1 --2- 172.21.15.100:0/3784471756 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a5015ad40 0x7f8a500af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.405+0000 7f8a4e000640 1 -- 172.21.15.100:0/3784471756 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a5015ad40 msgr2=0x7f8a500af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:35.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.405+0000 7f8a4e000640 1 --2- 172.21.15.100:0/3784471756 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a5015ad40 0x7f8a500af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.405+0000 7f8a4e000640 1 -- 172.21.15.100:0/3784471756 >> v1:172.21.15.151:6789/0 conn(0x7f8a50072ca0 legacy=0x7f8a5015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:35.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.405+0000 7f8a4e000640 1 -- 172.21.15.100:0/3784471756 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a50071e10 con 0x7f8a500af970 2024-04-23T14:33:35.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.405+0000 7f8a4e000640 1 --2- 172.21.15.100:0/3784471756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a500af970 0x7f8a50072760 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c006fd0 tx=0x7f8a3c01e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2661596068e0e210 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:35.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.405+0000 7f8a4d600640 1 -- 172.21.15.100:0/3784471756 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0ad070 con 0x7f8a500af970 2024-04-23T14:33:35.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a4d600640 1 -- 172.21.15.100:0/3784471756 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a3c004230 con 0x7f8a500af970 2024-04-23T14:33:35.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a4d600640 1 -- 172.21.15.100:0/3784471756 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0a7640 con 0x7f8a500af970 2024-04-23T14:33:35.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a55c00640 1 -- 172.21.15.100:0/3784471756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a500af970 msgr2=0x7f8a50072760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a55c00640 1 --2- 172.21.15.100:0/3784471756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a500af970 0x7f8a50072760 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c006fd0 tx=0x7f8a3c01e920 comp rx=0 tx=0).stop 2024-04-23T14:33:35.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a55c00640 1 -- 172.21.15.100:0/3784471756 shutdown_connections 2024-04-23T14:33:35.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a55c00640 1 --2- 172.21.15.100:0/3784471756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a500af970 0x7f8a50072760 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a55c00640 1 --2- 172.21.15.100:0/3784471756 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a5015ad40 0x7f8a500af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a55c00640 1 -- 172.21.15.100:0/3784471756 >> 172.21.15.100:0/3784471756 conn(0x7f8a500b1e80 msgr2=0x7f8a500b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:35.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a55c00640 1 -- 172.21.15.100:0/3784471756 shutdown_connections 2024-04-23T14:33:35.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a55c00640 1 -- 172.21.15.100:0/3784471756 wait complete. 2024-04-23T14:33:35.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a55c00640 1 Processor -- start 2024-04-23T14:33:35.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a55c00640 1 -- start start 2024-04-23T14:33:35.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a55c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a50072ca0 0x7f8a50159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a55c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a500af970 0x7f8a5014d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a55c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a5015ad40 0x7f8a501556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a55c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5015f1f0 con 0x7f8a500af970 2024-04-23T14:33:35.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a55c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5015f070 con 0x7f8a5015ad40 2024-04-23T14:33:35.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a55c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8a5015f370 con 0x7f8a50072ca0 2024-04-23T14:33:35.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a4e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a500af970 0x7f8a5014d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a4e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a500af970 0x7f8a5014d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40048/0 (socket says 172.21.15.100:40048) 2024-04-23T14:33:35.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a4e000640 1 -- 172.21.15.100:0/2617191211 learned_addr learned my addr 172.21.15.100:0/2617191211 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:35.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a4ea00640 1 --2- 172.21.15.100:0/2617191211 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a50072ca0 0x7f8a50159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a4f400640 1 --2- 172.21.15.100:0/2617191211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a5015ad40 0x7f8a501556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a4e000640 1 -- 172.21.15.100:0/2617191211 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a50072ca0 msgr2=0x7f8a50159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a4e000640 1 --2- 172.21.15.100:0/2617191211 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a50072ca0 0x7f8a50159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a4e000640 1 -- 172.21.15.100:0/2617191211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a5015ad40 msgr2=0x7f8a501556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a4e000640 1 --2- 172.21.15.100:0/2617191211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a5015ad40 0x7f8a501556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.409+0000 7f8a4e000640 1 -- 172.21.15.100:0/2617191211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a500cc4b0 con 0x7f8a500af970 2024-04-23T14:33:35.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.413+0000 7f8a4e000640 1 --2- 172.21.15.100:0/2617191211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a500af970 0x7f8a5014d6b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c01cfd0 tx=0x7f8a3c01b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:35.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.413+0000 7f8a46a00640 1 -- 172.21.15.100:0/2617191211 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0c0070 con 0x7f8a500af970 2024-04-23T14:33:35.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.413+0000 7f8a46a00640 1 -- 172.21.15.100:0/2617191211 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a3c004030 con 0x7f8a500af970 2024-04-23T14:33:35.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.413+0000 7f8a46a00640 1 -- 172.21.15.100:0/2617191211 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0ad040 con 0x7f8a500af970 2024-04-23T14:33:35.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.413+0000 7f8a55c00640 1 -- 172.21.15.100:0/2617191211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a500cc790 con 0x7f8a500af970 2024-04-23T14:33:35.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.413+0000 7f8a55c00640 1 -- 172.21.15.100:0/2617191211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a500ccd50 con 0x7f8a500af970 2024-04-23T14:33:35.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.413+0000 7f8a55c00640 1 -- 172.21.15.100:0/2617191211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a50068ad0 con 0x7f8a500af970 2024-04-23T14:33:35.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.413+0000 7f8a46a00640 1 -- 172.21.15.100:0/2617191211 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8a3c003840 con 0x7f8a500af970 2024-04-23T14:33:35.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.413+0000 7f8a46a00640 1 --2- 172.21.15.100:0/2617191211 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8a300683b0 0x7f8a3006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.413+0000 7f8a46a00640 1 -- 172.21.15.100:0/2617191211 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f8a3c122840 con 0x7f8a500af970 2024-04-23T14:33:35.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.413+0000 7f8a4ea00640 1 --2- 172.21.15.100:0/2617191211 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8a300683b0 0x7f8a3006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.417+0000 7f8a4ea00640 1 --2- 172.21.15.100:0/2617191211 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8a300683b0 0x7f8a3006a870 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8a40003fd0 tx=0x7f8a400039c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:35.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.417+0000 7f8a46a00640 1 -- 172.21.15.100:0/2617191211 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8a3c0185a0 con 0x7f8a500af970 2024-04-23T14:33:35.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.577+0000 7f8a55c00640 1 -- 172.21.15.100:0/2617191211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a500b2a30 con 0x7f8a500af970 2024-04-23T14:33:35.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.577+0000 7f8a46a00640 1 -- 172.21.15.100:0/2617191211 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+13176 (secure 0 0 0) 0x7f8a3c0f3040 con 0x7f8a500af970 2024-04-23T14:33:35.581 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:35.582 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":34,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:33:32.516472+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:35.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.581+0000 7f8a55c00640 1 -- 172.21.15.100:0/2617191211 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8a300683b0 msgr2=0x7f8a3006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.581+0000 7f8a55c00640 1 --2- 172.21.15.100:0/2617191211 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8a300683b0 0x7f8a3006a870 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8a40003fd0 tx=0x7f8a400039c0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.581+0000 7f8a55c00640 1 -- 172.21.15.100:0/2617191211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a500af970 msgr2=0x7f8a5014d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.581+0000 7f8a55c00640 1 --2- 172.21.15.100:0/2617191211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a500af970 0x7f8a5014d6b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c01cfd0 tx=0x7f8a3c01b6c0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.581+0000 7f8a55c00640 1 -- 172.21.15.100:0/2617191211 shutdown_connections 2024-04-23T14:33:35.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.581+0000 7f8a55c00640 1 --2- 172.21.15.100:0/2617191211 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8a300683b0 0x7f8a3006a870 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.581+0000 7f8a55c00640 1 --2- 172.21.15.100:0/2617191211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a5015ad40 0x7f8a501556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.581+0000 7f8a55c00640 1 --2- 172.21.15.100:0/2617191211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a500af970 0x7f8a5014d6b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.581+0000 7f8a55c00640 1 --2- 172.21.15.100:0/2617191211 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a50072ca0 0x7f8a50159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.581+0000 7f8a55c00640 1 -- 172.21.15.100:0/2617191211 >> 172.21.15.100:0/2617191211 conn(0x7f8a500b1e80 msgr2=0x7f8a500d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:35.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.581+0000 7f8a55c00640 1 -- 172.21.15.100:0/2617191211 shutdown_connections 2024-04-23T14:33:35.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.581+0000 7f8a55c00640 1 -- 172.21.15.100:0/2617191211 wait complete. 2024-04-23T14:33:35.603 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:33:35.603 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:33:35.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.737+0000 7f99de400640 1 Processor -- start 2024-04-23T14:33:35.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.737+0000 7f99de400640 1 -- start start 2024-04-23T14:33:35.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.737+0000 7f99de400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f99d80cf410 0x7f99d80b5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.737+0000 7f99de400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f99d80b5c60 0x7f99d80d1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.737+0000 7f99de400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99d80b6450 con 0x7f99d8159d60 2024-04-23T14:33:35.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.737+0000 7f99de400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f99d80b6730 con 0x7f99d80b5c60 2024-04-23T14:33:35.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.737+0000 7f99de400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f99d80d1080 con 0x7f99d80cf410 2024-04-23T14:33:35.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.737+0000 7f99dda00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f99d80b5c60 0x7f99d80d1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.737+0000 7f99dd000640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f99d8159d60 0x7f99d80ced00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:60414/0 (socket says 172.21.15.100:60414) 2024-04-23T14:33:35.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.737+0000 7f99dd000640 1 -- 172.21.15.100:0/3062869050 learned_addr learned my addr 172.21.15.100:0/3062869050 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:35.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.737+0000 7f99dda00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f99d80b5c60 0x7f99d80d1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40052/0 (socket says 172.21.15.100:40052) 2024-04-23T14:33:35.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.741+0000 7f99cfe00640 1 --2- 172.21.15.100:0/3062869050 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f99d80cf410 0x7f99d80b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.741+0000 7f99dda00640 1 -- 172.21.15.100:0/3062869050 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f99d80cf410 msgr2=0x7f99d80b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:35.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.741+0000 7f99dda00640 1 --2- 172.21.15.100:0/3062869050 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f99d80cf410 0x7f99d80b5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.741+0000 7f99dda00640 1 -- 172.21.15.100:0/3062869050 >> v1:172.21.15.100:6789/0 conn(0x7f99d8159d60 legacy=0x7f99d80ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.741+0000 7f99dda00640 1 -- 172.21.15.100:0/3062869050 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d80d1360 con 0x7f99d80b5c60 2024-04-23T14:33:35.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.741+0000 7f99d7e00640 1 -- 172.21.15.100:0/3062869050 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2587223601 0 0) 0x7f99d80b6450 con 0x7f99d8159d60 2024-04-23T14:33:35.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.741+0000 7f99dda00640 1 --2- 172.21.15.100:0/3062869050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99d80b5c60 0x7f99d80d1880 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f99d0009d90 tx=0x7f99d001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f6ef1457baba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:35.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.741+0000 7f99d7e00640 1 -- 172.21.15.100:0/3062869050 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d00b2070 con 0x7f99d80b5c60 2024-04-23T14:33:35.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.741+0000 7f99d7e00640 1 -- 172.21.15.100:0/3062869050 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d00a5830 con 0x7f99d80b5c60 2024-04-23T14:33:35.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.741+0000 7f99d7e00640 1 -- 172.21.15.100:0/3062869050 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d00a5c00 con 0x7f99d80b5c60 2024-04-23T14:33:35.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.741+0000 7f99de400640 1 -- 172.21.15.100:0/3062869050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99d80b5c60 msgr2=0x7f99d80d1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.741+0000 7f99de400640 1 --2- 172.21.15.100:0/3062869050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99d80b5c60 0x7f99d80d1880 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f99d0009d90 tx=0x7f99d001c970 comp rx=0 tx=0).stop 2024-04-23T14:33:35.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99de400640 1 -- 172.21.15.100:0/3062869050 shutdown_connections 2024-04-23T14:33:35.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99de400640 1 --2- 172.21.15.100:0/3062869050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99d80b5c60 0x7f99d80d1880 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99de400640 1 --2- 172.21.15.100:0/3062869050 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f99d80cf410 0x7f99d80b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99de400640 1 -- 172.21.15.100:0/3062869050 >> 172.21.15.100:0/3062869050 conn(0x7f99d80b76f0 msgr2=0x7f99d80b7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:35.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99de400640 1 -- 172.21.15.100:0/3062869050 shutdown_connections 2024-04-23T14:33:35.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99de400640 1 -- 172.21.15.100:0/3062869050 wait complete. 2024-04-23T14:33:35.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99de400640 1 Processor -- start 2024-04-23T14:33:35.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99de400640 1 -- start start 2024-04-23T14:33:35.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99de400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f99d80b5c60 0x7f99d8145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99de400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99d80cf410 0x7f99d814fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99de400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99d8159d60 0x7f99d806dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99de400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f99d80d2960 con 0x7f99d8159d60 2024-04-23T14:33:35.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99de400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99d80d27e0 con 0x7f99d80cf410 2024-04-23T14:33:35.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99de400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f99d80d2ae0 con 0x7f99d80b5c60 2024-04-23T14:33:35.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99dda00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99d8159d60 0x7f99d806dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99dda00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99d8159d60 0x7f99d806dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40066/0 (socket says 172.21.15.100:40066) 2024-04-23T14:33:35.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99dda00640 1 -- 172.21.15.100:0/1692154462 learned_addr learned my addr 172.21.15.100:0/1692154462 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:35.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99cfe00640 1 --2- 172.21.15.100:0/1692154462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99d80cf410 0x7f99d814fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99dd000640 1 --2- 172.21.15.100:0/1692154462 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f99d80b5c60 0x7f99d8145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99dda00640 1 -- 172.21.15.100:0/1692154462 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f99d80b5c60 msgr2=0x7f99d8145fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99dda00640 1 --2- 172.21.15.100:0/1692154462 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f99d80b5c60 0x7f99d8145fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99dda00640 1 -- 172.21.15.100:0/1692154462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99d80cf410 msgr2=0x7f99d814fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99dda00640 1 --2- 172.21.15.100:0/1692154462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99d80cf410 0x7f99d814fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99dda00640 1 -- 172.21.15.100:0/1692154462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d814e960 con 0x7f99d8159d60 2024-04-23T14:33:35.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99dda00640 1 --2- 172.21.15.100:0/1692154462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99d8159d60 0x7f99d806dab0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f99d001b020 tx=0x7f99d0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:35.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.745+0000 7f99d5600640 1 -- 172.21.15.100:0/1692154462 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d00bd020 con 0x7f99d8159d60 2024-04-23T14:33:35.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.749+0000 7f99d5600640 1 -- 172.21.15.100:0/1692154462 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d00b8470 con 0x7f99d8159d60 2024-04-23T14:33:35.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.749+0000 7f99de400640 1 -- 172.21.15.100:0/1692154462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d814ec40 con 0x7f99d8159d60 2024-04-23T14:33:35.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.749+0000 7f99d5600640 1 -- 172.21.15.100:0/1692154462 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d00b2040 con 0x7f99d8159d60 2024-04-23T14:33:35.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.749+0000 7f99de400640 1 -- 172.21.15.100:0/1692154462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d814f200 con 0x7f99d8159d60 2024-04-23T14:33:35.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.749+0000 7f99de400640 1 -- 172.21.15.100:0/1692154462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99d8068ad0 con 0x7f99d8159d60 2024-04-23T14:33:35.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.749+0000 7f99d5600640 1 -- 172.21.15.100:0/1692154462 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f99d00c0020 con 0x7f99d8159d60 2024-04-23T14:33:35.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.749+0000 7f99d5600640 1 --2- 172.21.15.100:0/1692154462 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f99b40683b0 0x7f99b406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.749+0000 7f99d5600640 1 -- 172.21.15.100:0/1692154462 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f99d0121380 con 0x7f99d8159d60 2024-04-23T14:33:35.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.749+0000 7f99dd000640 1 --2- 172.21.15.100:0/1692154462 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f99b40683b0 0x7f99b406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.749+0000 7f99dd000640 1 --2- 172.21.15.100:0/1692154462 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f99b40683b0 0x7f99b406a870 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f99c8000f70 tx=0x7f99c8007b60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:35.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.753+0000 7f99d5600640 1 -- 172.21.15.100:0/1692154462 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f99d00c7070 con 0x7f99d8159d60 2024-04-23T14:33:35.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.913+0000 7f99de400640 1 -- 172.21.15.100:0/1692154462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f99d8146520 con 0x7f99d8159d60 2024-04-23T14:33:35.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.913+0000 7f99d5600640 1 -- 172.21.15.100:0/1692154462 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f99d0016560 con 0x7f99d8159d60 2024-04-23T14:33:35.917 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:33:35.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.917+0000 7f99de400640 1 -- 172.21.15.100:0/1692154462 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f99b40683b0 msgr2=0x7f99b406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.917+0000 7f99de400640 1 --2- 172.21.15.100:0/1692154462 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f99b40683b0 0x7f99b406a870 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f99c8000f70 tx=0x7f99c8007b60 comp rx=0 tx=0).stop 2024-04-23T14:33:35.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.917+0000 7f99de400640 1 -- 172.21.15.100:0/1692154462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99d8159d60 msgr2=0x7f99d806dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.917+0000 7f99de400640 1 --2- 172.21.15.100:0/1692154462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99d8159d60 0x7f99d806dab0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f99d001b020 tx=0x7f99d0019710 comp rx=0 tx=0).stop 2024-04-23T14:33:35.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.917+0000 7f99de400640 1 -- 172.21.15.100:0/1692154462 shutdown_connections 2024-04-23T14:33:35.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.917+0000 7f99de400640 1 --2- 172.21.15.100:0/1692154462 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f99b40683b0 0x7f99b406a870 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.917+0000 7f99de400640 1 --2- 172.21.15.100:0/1692154462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99d8159d60 0x7f99d806dab0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.917+0000 7f99de400640 1 --2- 172.21.15.100:0/1692154462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99d80cf410 0x7f99d814fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.917+0000 7f99de400640 1 --2- 172.21.15.100:0/1692154462 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f99d80b5c60 0x7f99d8145fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.917+0000 7f99de400640 1 -- 172.21.15.100:0/1692154462 >> 172.21.15.100:0/1692154462 conn(0x7f99d80b76f0 msgr2=0x7f99d80707c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:35.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.917+0000 7f99de400640 1 -- 172.21.15.100:0/1692154462 shutdown_connections 2024-04-23T14:33:35.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:35.917+0000 7f99de400640 1 -- 172.21.15.100:0/1692154462 wait complete. 2024-04-23T14:33:35.939 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-23T14:33:36.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.069+0000 7fe90d400640 1 Processor -- start 2024-04-23T14:33:36.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.069+0000 7fe90d400640 1 -- start start 2024-04-23T14:33:36.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.069+0000 7fe90d400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fe9080b2a10 0x7fe9080adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:36.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.069+0000 7fe90d400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe908073610 0x7fe90806df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:36.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.069+0000 7fe90d400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9080af080 con 0x7fe90815b640 2024-04-23T14:33:36.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.069+0000 7fe90d400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fe9080afbc0 con 0x7fe9080b2a10 2024-04-23T14:33:36.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.069+0000 7fe90d400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe90815ba20 con 0x7fe908073610 2024-04-23T14:33:36.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.073+0000 7fe906800640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe90815b640 0x7fe908072f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:60416/0 (socket says 172.21.15.100:60416) 2024-04-23T14:33:36.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.073+0000 7fe906800640 1 -- 172.21.15.100:0/1928429173 learned_addr learned my addr 172.21.15.100:0/1928429173 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:36.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.073+0000 7fe905e00640 1 --2- 172.21.15.100:0/1928429173 >> v2:172.21.15.151:3300/0 conn(0x7fe9080b2a10 0x7fe9080adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:36.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.073+0000 7fe907200640 1 --2- 172.21.15.100:0/1928429173 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe908073610 0x7fe90806df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:36.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.073+0000 7fe905400640 1 -- 172.21.15.100:0/1928429173 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2625026134 0 0) 0x7fe9080af080 con 0x7fe90815b640 2024-04-23T14:33:36.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.073+0000 7fe905400640 1 -- 172.21.15.100:0/1928429173 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8ec005740 con 0x7fe90815b640 2024-04-23T14:33:36.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.073+0000 7fe905e00640 1 -- 172.21.15.100:0/1928429173 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe908073610 msgr2=0x7fe90806df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:36.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.073+0000 7fe905e00640 1 --2- 172.21.15.100:0/1928429173 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe908073610 0x7fe90806df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.073+0000 7fe905400640 1 -- 172.21.15.100:0/1928429173 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2367721245 0 0) 0x7fe8ec005740 con 0x7fe90815b640 2024-04-23T14:33:36.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.073+0000 7fe905e00640 1 -- 172.21.15.100:0/1928429173 >> v1:172.21.15.100:6789/0 conn(0x7fe90815b640 legacy=0x7fe908072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:36.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.073+0000 7fe905e00640 1 -- 172.21.15.100:0/1928429173 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9080af320 con 0x7fe9080b2a10 2024-04-23T14:33:36.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.073+0000 7fe905e00640 1 --2- 172.21.15.100:0/1928429173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9080b2a10 0x7fe9080adfc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40097c0 tx=0x7fe8f401c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=185ac3967c52e7a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:36.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe905400640 1 -- 172.21.15.100:0/1928429173 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f40b0070 con 0x7fe9080b2a10 2024-04-23T14:33:36.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe905400640 1 -- 172.21.15.100:0/1928429173 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f40ab400 con 0x7fe9080b2a10 2024-04-23T14:33:36.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe905400640 1 -- 172.21.15.100:0/1928429173 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8001a00 con 0x7fe9080b2a10 2024-04-23T14:33:36.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe90d400640 1 -- 172.21.15.100:0/1928429173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9080b2a10 msgr2=0x7fe9080adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:36.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe90d400640 1 --2- 172.21.15.100:0/1928429173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9080b2a10 0x7fe9080adfc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40097c0 tx=0x7fe8f401c390 comp rx=0 tx=0).stop 2024-04-23T14:33:36.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe90d400640 1 -- 172.21.15.100:0/1928429173 shutdown_connections 2024-04-23T14:33:36.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe90d400640 1 --2- 172.21.15.100:0/1928429173 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe908073610 0x7fe90806df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe90d400640 1 --2- 172.21.15.100:0/1928429173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9080b2a10 0x7fe9080adfc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe90d400640 1 -- 172.21.15.100:0/1928429173 >> 172.21.15.100:0/1928429173 conn(0x7fe9080b1e80 msgr2=0x7fe9080b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:36.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe90d400640 1 -- 172.21.15.100:0/1928429173 shutdown_connections 2024-04-23T14:33:36.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe90d400640 1 -- 172.21.15.100:0/1928429173 wait complete. 2024-04-23T14:33:36.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe90d400640 1 Processor -- start 2024-04-23T14:33:36.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe90d400640 1 -- start start 2024-04-23T14:33:36.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe90d400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe908073610 0x7fe90815c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:36.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe90d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9080b2a10 0x7fe908158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:36.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe90d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe90815b640 0x7fe9080b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:36.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe90d400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe90806f0d0 con 0x7fe9080b2a10 2024-04-23T14:33:36.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe90d400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe90806ef50 con 0x7fe90815b640 2024-04-23T14:33:36.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe90d400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe90806f250 con 0x7fe908073610 2024-04-23T14:33:36.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe905e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9080b2a10 0x7fe908158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:36.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe905e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9080b2a10 0x7fe908158310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40080/0 (socket says 172.21.15.100:40080) 2024-04-23T14:33:36.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe905e00640 1 -- 172.21.15.100:0/2260732468 learned_addr learned my addr 172.21.15.100:0/2260732468 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:36.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe906800640 1 --2- 172.21.15.100:0/2260732468 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe908073610 0x7fe90815c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:36.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.077+0000 7fe907200640 1 --2- 172.21.15.100:0/2260732468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe90815b640 0x7fe9080b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:36.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.081+0000 7fe905e00640 1 -- 172.21.15.100:0/2260732468 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe908073610 msgr2=0x7fe90815c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:36.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.081+0000 7fe905e00640 1 --2- 172.21.15.100:0/2260732468 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe908073610 0x7fe90815c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.081+0000 7fe905e00640 1 -- 172.21.15.100:0/2260732468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe90815b640 msgr2=0x7fe9080b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:36.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.081+0000 7fe905e00640 1 --2- 172.21.15.100:0/2260732468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe90815b640 0x7fe9080b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.081+0000 7fe905e00640 1 -- 172.21.15.100:0/2260732468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9080b3080 con 0x7fe9080b2a10 2024-04-23T14:33:36.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.081+0000 7fe907200640 1 --2- 172.21.15.100:0/2260732468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe90815b640 0x7fe9080b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:36.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.081+0000 7fe905e00640 1 --2- 172.21.15.100:0/2260732468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9080b2a10 0x7fe908158310 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40ae040 tx=0x7fe8f401abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:36.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.081+0000 7fe8fea00640 1 -- 172.21.15.100:0/2260732468 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f40b0050 con 0x7fe9080b2a10 2024-04-23T14:33:36.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.081+0000 7fe8fea00640 1 -- 172.21.15.100:0/2260732468 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f40b1040 con 0x7fe9080b2a10 2024-04-23T14:33:36.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.081+0000 7fe90d400640 1 -- 172.21.15.100:0/2260732468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe90815c600 con 0x7fe9080b2a10 2024-04-23T14:33:36.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.081+0000 7fe8fea00640 1 -- 172.21.15.100:0/2260732468 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f40b7650 con 0x7fe9080b2a10 2024-04-23T14:33:36.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.081+0000 7fe90d400640 1 -- 172.21.15.100:0/2260732468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe90815cbc0 con 0x7fe9080b2a10 2024-04-23T14:33:36.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.081+0000 7fe90d400640 1 -- 172.21.15.100:0/2260732468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe908068ad0 con 0x7fe9080b2a10 2024-04-23T14:33:36.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.081+0000 7fe8fea00640 1 -- 172.21.15.100:0/2260732468 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe8f40be030 con 0x7fe9080b2a10 2024-04-23T14:33:36.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.081+0000 7fe8fea00640 1 --2- 172.21.15.100:0/2260732468 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe8e8068380 0x7fe8e806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:36.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.081+0000 7fe8fea00640 1 -- 172.21.15.100:0/2260732468 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fe8f4120600 con 0x7fe9080b2a10 2024-04-23T14:33:36.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.081+0000 7fe906800640 1 --2- 172.21.15.100:0/2260732468 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe8e8068380 0x7fe8e806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:36.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.085+0000 7fe906800640 1 --2- 172.21.15.100:0/2260732468 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe8e8068380 0x7fe8e806a840 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe90814bc10 tx=0x7fe8f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:36.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.085+0000 7fe8fea00640 1 -- 172.21.15.100:0/2260732468 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe8f40c5020 con 0x7fe9080b2a10 2024-04-23T14:33:36.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.245+0000 7fe90d400640 1 -- 172.21.15.100:0/2260732468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe9080b04d0 con 0x7fe9080b2a10 2024-04-23T14:33:36.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.245+0000 7fe8fea00640 1 -- 172.21.15.100:0/2260732468 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) ==== 84+0+65 (secure 0 0 0) 0x7fe8f4015fb0 con 0x7fe9080b2a10 2024-04-23T14:33:36.247 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:36.248 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-04-23T14:33:36.248 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:33:36.248 INFO:teuthology.orchestra.run.smithi100.stdout: "poolnum": 1, 2024-04-23T14:33:36.248 INFO:teuthology.orchestra.run.smithi100.stdout: "poolname": ".mgr" 2024-04-23T14:33:36.248 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:36.248 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-04-23T14:33:36.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.245+0000 7fe90d400640 1 -- 172.21.15.100:0/2260732468 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe8e8068380 msgr2=0x7fe8e806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:36.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.245+0000 7fe90d400640 1 --2- 172.21.15.100:0/2260732468 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe8e8068380 0x7fe8e806a840 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe90814bc10 tx=0x7fe8f8009040 comp rx=0 tx=0).stop 2024-04-23T14:33:36.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.245+0000 7fe90d400640 1 -- 172.21.15.100:0/2260732468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9080b2a10 msgr2=0x7fe908158310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:36.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.245+0000 7fe90d400640 1 --2- 172.21.15.100:0/2260732468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9080b2a10 0x7fe908158310 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40ae040 tx=0x7fe8f401abd0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.245+0000 7fe90d400640 1 -- 172.21.15.100:0/2260732468 shutdown_connections 2024-04-23T14:33:36.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.245+0000 7fe90d400640 1 --2- 172.21.15.100:0/2260732468 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe8e8068380 0x7fe8e806a840 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.245+0000 7fe90d400640 1 --2- 172.21.15.100:0/2260732468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe90815b640 0x7fe9080b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.245+0000 7fe90d400640 1 --2- 172.21.15.100:0/2260732468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9080b2a10 0x7fe908158310 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.245+0000 7fe90d400640 1 --2- 172.21.15.100:0/2260732468 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe908073610 0x7fe90815c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.245+0000 7fe90d400640 1 -- 172.21.15.100:0/2260732468 >> 172.21.15.100:0/2260732468 conn(0x7fe9080b1e80 msgr2=0x7fe908072eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:36.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.245+0000 7fe90d400640 1 -- 172.21.15.100:0/2260732468 shutdown_connections 2024-04-23T14:33:36.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.245+0000 7fe90d400640 1 -- 172.21.15.100:0/2260732468 wait complete. 2024-04-23T14:33:36.268 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-23T14:33:36.268 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-23T14:33:36.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.405+0000 7f6740000640 1 Processor -- start 2024-04-23T14:33:36.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.405+0000 7f6740000640 1 -- start start 2024-04-23T14:33:36.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.405+0000 7f6740000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f67380b5900 0x7f673815f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:36.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.405+0000 7f6740000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f67380ae290 0x7f67380ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:36.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.405+0000 7f6740000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67380b4860 con 0x7f673815f410 2024-04-23T14:33:36.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.405+0000 7f6740000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f67380aec20 con 0x7f67380b5900 2024-04-23T14:33:36.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.405+0000 7f6740000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f67380b4b40 con 0x7f67380ae290 2024-04-23T14:33:36.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.405+0000 7f673cc00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f67380b5900 0x7f673815f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:36.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.405+0000 7f673cc00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f67380b5900 0x7f673815f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40088/0 (socket says 172.21.15.100:40088) 2024-04-23T14:33:36.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.405+0000 7f673cc00640 1 -- 172.21.15.100:0/2046133531 learned_addr learned my addr 172.21.15.100:0/2046133531 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:36.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.405+0000 7f673d600640 1 --1- 172.21.15.100:0/2046133531 >> v1:172.21.15.100:6789/0 conn(0x7f673815f410 0x7f67380ad860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:60422/0 (socket says 172.21.15.100:60422) 2024-04-23T14:33:36.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.405+0000 7f673e000640 1 --2- 172.21.15.100:0/2046133531 >> v2:172.21.15.151:3300/0 conn(0x7f67380ae290 0x7f67380ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:36.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.405+0000 7f673cc00640 1 -- 172.21.15.100:0/2046133531 >> v2:172.21.15.151:3300/0 conn(0x7f67380ae290 msgr2=0x7f67380ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:36.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.405+0000 7f673cc00640 1 --2- 172.21.15.100:0/2046133531 >> v2:172.21.15.151:3300/0 conn(0x7f67380ae290 0x7f67380ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.405+0000 7f673cc00640 1 -- 172.21.15.100:0/2046133531 >> v1:172.21.15.100:6789/0 conn(0x7f673815f410 legacy=0x7f67380ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:36.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.405+0000 7f673cc00640 1 -- 172.21.15.100:0/2046133531 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67380b4e20 con 0x7f67380b5900 2024-04-23T14:33:36.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.405+0000 7f6737e00640 1 -- 172.21.15.100:0/2046133531 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3724306120 0 0) 0x7f67380b4860 con 0x7f673815f410 2024-04-23T14:33:36.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.409+0000 7f673cc00640 1 --2- 172.21.15.100:0/2046133531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67380b5900 0x7f673815f7f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f673815fd60 tx=0x7f672401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb6f7be21ad9deb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:36.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.409+0000 7f6737e00640 1 -- 172.21.15.100:0/2046133531 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67240aa650 con 0x7f67380b5900 2024-04-23T14:33:36.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.409+0000 7f6737e00640 1 -- 172.21.15.100:0/2046133531 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67240af070 con 0x7f67380b5900 2024-04-23T14:33:36.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.409+0000 7f6737e00640 1 -- 172.21.15.100:0/2046133531 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67240aaa10 con 0x7f67380b5900 2024-04-23T14:33:36.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.409+0000 7f6740000640 1 -- 172.21.15.100:0/2046133531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67380b5900 msgr2=0x7f673815f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:36.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.409+0000 7f6740000640 1 --2- 172.21.15.100:0/2046133531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67380b5900 0x7f673815f7f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f673815fd60 tx=0x7f672401c4f0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.409+0000 7f6740000640 1 -- 172.21.15.100:0/2046133531 shutdown_connections 2024-04-23T14:33:36.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.409+0000 7f6740000640 1 --2- 172.21.15.100:0/2046133531 >> v2:172.21.15.151:3300/0 conn(0x7f67380ae290 0x7f67380ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.409+0000 7f6740000640 1 --2- 172.21.15.100:0/2046133531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67380b5900 0x7f673815f7f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.409+0000 7f6740000640 1 -- 172.21.15.100:0/2046133531 >> 172.21.15.100:0/2046133531 conn(0x7f67380d0860 msgr2=0x7f67380d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:36.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.409+0000 7f6740000640 1 -- 172.21.15.100:0/2046133531 shutdown_connections 2024-04-23T14:33:36.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.409+0000 7f6740000640 1 -- 172.21.15.100:0/2046133531 wait complete. 2024-04-23T14:33:36.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f6740000640 1 Processor -- start 2024-04-23T14:33:36.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f6740000640 1 -- start start 2024-04-23T14:33:36.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f6740000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67380ae290 0x7f6738152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:36.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f6740000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f67380b5900 0x7f673815b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:36.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f6740000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f673815f410 0x7f67381fa240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:36.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f6740000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f673806c440 con 0x7f673815f410 2024-04-23T14:33:36.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f6740000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f673806c2c0 con 0x7f67380ae290 2024-04-23T14:33:36.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f6740000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f673806c5c0 con 0x7f67380b5900 2024-04-23T14:33:36.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f673e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f673815f410 0x7f67381fa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:36.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f673e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f673815f410 0x7f67381fa240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40090/0 (socket says 172.21.15.100:40090) 2024-04-23T14:33:36.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f673e000640 1 -- 172.21.15.100:0/846748178 learned_addr learned my addr 172.21.15.100:0/846748178 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:36.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f673d600640 1 --2- 172.21.15.100:0/846748178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67380ae290 0x7f6738152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:36.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f673cc00640 1 --2- 172.21.15.100:0/846748178 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f67380b5900 0x7f673815b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:36.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f673e000640 1 -- 172.21.15.100:0/846748178 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f67380b5900 msgr2=0x7f673815b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:36.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f673e000640 1 --2- 172.21.15.100:0/846748178 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f67380b5900 0x7f673815b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f673e000640 1 -- 172.21.15.100:0/846748178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67380ae290 msgr2=0x7f6738152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:36.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f673e000640 1 --2- 172.21.15.100:0/846748178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67380ae290 0x7f6738152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f673e000640 1 -- 172.21.15.100:0/846748178 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67381fac20 con 0x7f673815f410 2024-04-23T14:33:36.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f673d600640 1 --2- 172.21.15.100:0/846748178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67380ae290 0x7f6738152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:36.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f673cc00640 1 --2- 172.21.15.100:0/846748178 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f67380b5900 0x7f673815b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:36.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f673e000640 1 --2- 172.21.15.100:0/846748178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f673815f410 0x7f67381fa240 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f673000e890 tx=0x7f673000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:36.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f6735600640 1 -- 172.21.15.100:0/846748178 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6730017070 con 0x7f673815f410 2024-04-23T14:33:36.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f6735600640 1 -- 172.21.15.100:0/846748178 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f673000c940 con 0x7f673815f410 2024-04-23T14:33:36.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f6740000640 1 -- 172.21.15.100:0/846748178 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67381faf00 con 0x7f673815f410 2024-04-23T14:33:36.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f6735600640 1 -- 172.21.15.100:0/846748178 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673000cd10 con 0x7f673815f410 2024-04-23T14:33:36.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f6740000640 1 -- 172.21.15.100:0/846748178 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f673815b030 con 0x7f673815f410 2024-04-23T14:33:36.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.413+0000 7f6740000640 1 -- 172.21.15.100:0/846748178 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6738068ad0 con 0x7f673815f410 2024-04-23T14:33:36.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.417+0000 7f6735600640 1 -- 172.21.15.100:0/846748178 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f673001a070 con 0x7f673815f410 2024-04-23T14:33:36.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.417+0000 7f6735600640 1 --2- 172.21.15.100:0/846748178 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6718068380 0x7f671806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:36.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.417+0000 7f6735600640 1 -- 172.21.15.100:0/846748178 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f6730086a50 con 0x7f673815f410 2024-04-23T14:33:36.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.417+0000 7f673d600640 1 --2- 172.21.15.100:0/846748178 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6718068380 0x7f671806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:36.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.417+0000 7f673d600640 1 --2- 172.21.15.100:0/846748178 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6718068380 0x7f671806a840 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f672c007fd0 tx=0x7f672c005b80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:36.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.421+0000 7f6735600640 1 -- 172.21.15.100:0/846748178 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6730057260 con 0x7f673815f410 2024-04-23T14:33:36.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:36.577+0000 7f6740000640 1 -- 172.21.15.100:0/846748178 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f67380b4050 con 0x7f673815f410 2024-04-23T14:33:37.142 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:33:37.146 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:33:37.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.525+0000 7f6735600640 1 -- 172.21.15.100:0/846748178 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) ==== 138+0+0 (secure 0 0 0) 0x7f6730056b60 con 0x7f673815f410 2024-04-23T14:33:37.528 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' created 2024-04-23T14:33:37.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.525+0000 7f6740000640 1 -- 172.21.15.100:0/846748178 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6718068380 msgr2=0x7f671806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:37.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.525+0000 7f6740000640 1 --2- 172.21.15.100:0/846748178 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6718068380 0x7f671806a840 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f672c007fd0 tx=0x7f672c005b80 comp rx=0 tx=0).stop 2024-04-23T14:33:37.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.525+0000 7f6740000640 1 -- 172.21.15.100:0/846748178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f673815f410 msgr2=0x7f67381fa240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:37.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.525+0000 7f6740000640 1 --2- 172.21.15.100:0/846748178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f673815f410 0x7f67381fa240 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f673000e890 tx=0x7f673000ed50 comp rx=0 tx=0).stop 2024-04-23T14:33:37.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.525+0000 7f6740000640 1 -- 172.21.15.100:0/846748178 shutdown_connections 2024-04-23T14:33:37.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.525+0000 7f6740000640 1 --2- 172.21.15.100:0/846748178 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6718068380 0x7f671806a840 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.525+0000 7f6740000640 1 --2- 172.21.15.100:0/846748178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f673815f410 0x7f67381fa240 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.525+0000 7f6740000640 1 --2- 172.21.15.100:0/846748178 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f67380b5900 0x7f673815b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.525+0000 7f6740000640 1 --2- 172.21.15.100:0/846748178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67380ae290 0x7f6738152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.525+0000 7f6740000640 1 -- 172.21.15.100:0/846748178 >> 172.21.15.100:0/846748178 conn(0x7f67380d0860 msgr2=0x7f67380d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:37.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.525+0000 7f6740000640 1 -- 172.21.15.100:0/846748178 shutdown_connections 2024-04-23T14:33:37.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.525+0000 7f6740000640 1 -- 172.21.15.100:0/846748178 wait complete. 2024-04-23T14:33:37.557 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-23T14:33:37.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.697+0000 7ff757e00640 1 Processor -- start 2024-04-23T14:33:37.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.697+0000 7ff757e00640 1 -- start start 2024-04-23T14:33:37.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.697+0000 7ff757e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff75006bb20 0x7ff7500ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:37.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.697+0000 7ff757e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff7500cd130 0x7ff7500ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:37.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.697+0000 7ff757e00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff75006cbb0 con 0x7ff75015c7a0 2024-04-23T14:33:37.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.697+0000 7ff757e00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7ff7500cd5a0 con 0x7ff7500cd130 2024-04-23T14:33:37.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.697+0000 7ff757e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff75006d450 con 0x7ff75006bb20 2024-04-23T14:33:37.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.697+0000 7ff755400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff75015c7a0 0x7ff7500af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:60424/0 (socket says 172.21.15.100:60424) 2024-04-23T14:33:37.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.697+0000 7ff755400640 1 -- 172.21.15.100:0/4208838383 learned_addr learned my addr 172.21.15.100:0/4208838383 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:37.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.697+0000 7ff754a00640 1 --2- 172.21.15.100:0/4208838383 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff75006bb20 0x7ff7500ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:37.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.697+0000 7ff755e00640 1 --2- 172.21.15.100:0/4208838383 >> v2:172.21.15.151:3300/0 conn(0x7ff7500cd130 0x7ff7500ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:37.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.697+0000 7ff74fe00640 1 -- 172.21.15.100:0/4208838383 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 646966381 0 0) 0x7ff75006cbb0 con 0x7ff75015c7a0 2024-04-23T14:33:37.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.697+0000 7ff74fe00640 1 -- 172.21.15.100:0/4208838383 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff738003530 con 0x7ff75015c7a0 2024-04-23T14:33:37.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.697+0000 7ff754a00640 1 -- 172.21.15.100:0/4208838383 >> v2:172.21.15.151:3300/0 conn(0x7ff7500cd130 msgr2=0x7ff7500ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:37.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.697+0000 7ff754a00640 1 --2- 172.21.15.100:0/4208838383 >> v2:172.21.15.151:3300/0 conn(0x7ff7500cd130 0x7ff7500ad430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.697+0000 7ff754a00640 1 -- 172.21.15.100:0/4208838383 >> v1:172.21.15.100:6789/0 conn(0x7ff75015c7a0 legacy=0x7ff7500af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:37.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.697+0000 7ff754a00640 1 -- 172.21.15.100:0/4208838383 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75006ce90 con 0x7ff75006bb20 2024-04-23T14:33:37.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.697+0000 7ff755e00640 1 --2- 172.21.15.100:0/4208838383 >> v2:172.21.15.151:3300/0 conn(0x7ff7500cd130 0x7ff7500ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:33:37.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.701+0000 7ff754a00640 1 --2- 172.21.15.100:0/4208838383 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff75006bb20 0x7ff7500ccbf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0097c0 tx=0x7ff73c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20d67fefc1792eb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:37.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.701+0000 7ff74fe00640 1 -- 172.21.15.100:0/4208838383 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0b0070 con 0x7ff75006bb20 2024-04-23T14:33:37.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.701+0000 7ff74fe00640 1 -- 172.21.15.100:0/4208838383 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff73c0ab600 con 0x7ff75006bb20 2024-04-23T14:33:37.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.701+0000 7ff74fe00640 1 -- 172.21.15.100:0/4208838383 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0ab9b0 con 0x7ff75006bb20 2024-04-23T14:33:37.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.701+0000 7ff757e00640 1 -- 172.21.15.100:0/4208838383 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff75006bb20 msgr2=0x7ff7500ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:37.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.701+0000 7ff757e00640 1 --2- 172.21.15.100:0/4208838383 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff75006bb20 0x7ff7500ccbf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0097c0 tx=0x7ff73c01c410 comp rx=0 tx=0).stop 2024-04-23T14:33:37.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.701+0000 7ff757e00640 1 -- 172.21.15.100:0/4208838383 shutdown_connections 2024-04-23T14:33:37.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.701+0000 7ff757e00640 1 --2- 172.21.15.100:0/4208838383 >> v2:172.21.15.151:3300/0 conn(0x7ff7500cd130 0x7ff7500ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.701+0000 7ff757e00640 1 --2- 172.21.15.100:0/4208838383 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff75006bb20 0x7ff7500ccbf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.701+0000 7ff757e00640 1 -- 172.21.15.100:0/4208838383 >> 172.21.15.100:0/4208838383 conn(0x7ff7500b5200 msgr2=0x7ff7500b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:37.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.701+0000 7ff757e00640 1 -- 172.21.15.100:0/4208838383 shutdown_connections 2024-04-23T14:33:37.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.701+0000 7ff757e00640 1 -- 172.21.15.100:0/4208838383 wait complete. 2024-04-23T14:33:37.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff757e00640 1 Processor -- start 2024-04-23T14:33:37.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff757e00640 1 -- start start 2024-04-23T14:33:37.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff757e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff75006bb20 0x7ff750148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:37.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff757e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7500cd130 0x7ff7500b2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:37.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff757e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff75015c7a0 0x7ff7501516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:37.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff757e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff7500ae510 con 0x7ff75015c7a0 2024-04-23T14:33:37.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff757e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff7500ae390 con 0x7ff7500cd130 2024-04-23T14:33:37.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff757e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff7500ae690 con 0x7ff75006bb20 2024-04-23T14:33:37.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff755e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff75015c7a0 0x7ff7501516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:37.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff755e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff75015c7a0 0x7ff7501516f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40094/0 (socket says 172.21.15.100:40094) 2024-04-23T14:33:37.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff755e00640 1 -- 172.21.15.100:0/2501328397 learned_addr learned my addr 172.21.15.100:0/2501328397 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:37.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff755400640 1 --2- 172.21.15.100:0/2501328397 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff75006bb20 0x7ff750148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:37.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff754a00640 1 --2- 172.21.15.100:0/2501328397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7500cd130 0x7ff7500b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:37.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff755e00640 1 -- 172.21.15.100:0/2501328397 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff75006bb20 msgr2=0x7ff750148fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:37.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff755e00640 1 --2- 172.21.15.100:0/2501328397 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff75006bb20 0x7ff750148fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff755e00640 1 -- 172.21.15.100:0/2501328397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7500cd130 msgr2=0x7ff7500b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:37.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff755e00640 1 --2- 172.21.15.100:0/2501328397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7500cd130 0x7ff7500b2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff755e00640 1 -- 172.21.15.100:0/2501328397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75009d0c0 con 0x7ff75015c7a0 2024-04-23T14:33:37.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff755e00640 1 --2- 172.21.15.100:0/2501328397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff75015c7a0 0x7ff7501516f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff74800efc0 tx=0x7ff74800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:37.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff74d600640 1 -- 172.21.15.100:0/2501328397 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748018030 con 0x7ff75015c7a0 2024-04-23T14:33:37.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff74d600640 1 -- 172.21.15.100:0/2501328397 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff74800cdf0 con 0x7ff75015c7a0 2024-04-23T14:33:37.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff74d600640 1 -- 172.21.15.100:0/2501328397 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748003bc0 con 0x7ff75015c7a0 2024-04-23T14:33:37.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff757e00640 1 -- 172.21.15.100:0/2501328397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75009d3a0 con 0x7ff75015c7a0 2024-04-23T14:33:37.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff757e00640 1 -- 172.21.15.100:0/2501328397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff750149500 con 0x7ff75015c7a0 2024-04-23T14:33:37.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.705+0000 7ff757e00640 1 -- 172.21.15.100:0/2501328397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff750068ad0 con 0x7ff75015c7a0 2024-04-23T14:33:37.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.709+0000 7ff74d600640 1 -- 172.21.15.100:0/2501328397 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff74801b070 con 0x7ff75015c7a0 2024-04-23T14:33:37.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.709+0000 7ff74d600640 1 --2- 172.21.15.100:0/2501328397 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff730068380 0x7ff73006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:37.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.709+0000 7ff74d600640 1 -- 172.21.15.100:0/2501328397 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5619+0+0 (secure 0 0 0) 0x7ff748087200 con 0x7ff75015c7a0 2024-04-23T14:33:37.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.709+0000 7ff755400640 1 --2- 172.21.15.100:0/2501328397 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff730068380 0x7ff73006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:37.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.709+0000 7ff755400640 1 --2- 172.21.15.100:0/2501328397 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff730068380 0x7ff73006a840 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff744007fd0 tx=0x7ff744009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:37.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.713+0000 7ff74d600640 1 -- 172.21.15.100:0/2501328397 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff748010040 con 0x7ff75015c7a0 2024-04-23T14:33:37.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:37.869+0000 7ff757e00640 1 -- 172.21.15.100:0/2501328397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff750071da0 con 0x7ff75015c7a0 2024-04-23T14:33:38.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.524+0000 7ff74d600640 1 -- 172.21.15.100:0/2501328397 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) ==== 170+0+0 (secure 0 0 0) 0x7ff7480571b0 con 0x7ff75015c7a0 2024-04-23T14:33:38.531 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' created 2024-04-23T14:33:38.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.532+0000 7ff757e00640 1 -- 172.21.15.100:0/2501328397 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff730068380 msgr2=0x7ff73006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:38.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.532+0000 7ff757e00640 1 --2- 172.21.15.100:0/2501328397 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff730068380 0x7ff73006a840 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff744007fd0 tx=0x7ff744009040 comp rx=0 tx=0).stop 2024-04-23T14:33:38.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.532+0000 7ff757e00640 1 -- 172.21.15.100:0/2501328397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff75015c7a0 msgr2=0x7ff7501516f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:38.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.532+0000 7ff757e00640 1 --2- 172.21.15.100:0/2501328397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff75015c7a0 0x7ff7501516f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff74800efc0 tx=0x7ff74800c490 comp rx=0 tx=0).stop 2024-04-23T14:33:38.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.532+0000 7ff757e00640 1 -- 172.21.15.100:0/2501328397 shutdown_connections 2024-04-23T14:33:38.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.532+0000 7ff757e00640 1 --2- 172.21.15.100:0/2501328397 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff730068380 0x7ff73006a840 secure :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff744007fd0 tx=0x7ff744009040 comp rx=0 tx=0).stop 2024-04-23T14:33:38.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.532+0000 7ff757e00640 1 --2- 172.21.15.100:0/2501328397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff75015c7a0 0x7ff7501516f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:38.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.532+0000 7ff757e00640 1 --2- 172.21.15.100:0/2501328397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7500cd130 0x7ff7500b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:38.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.532+0000 7ff757e00640 1 --2- 172.21.15.100:0/2501328397 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff75006bb20 0x7ff750148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:38.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.532+0000 7ff757e00640 1 -- 172.21.15.100:0/2501328397 >> 172.21.15.100:0/2501328397 conn(0x7ff7500b5200 msgr2=0x7ff7500b55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:38.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.532+0000 7ff757e00640 1 -- 172.21.15.100:0/2501328397 shutdown_connections 2024-04-23T14:33:38.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.532+0000 7ff757e00640 1 -- 172.21.15.100:0/2501328397 wait complete. 2024-04-23T14:33:38.579 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-23T14:33:38.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.716+0000 7f3112a00640 1 Processor -- start 2024-04-23T14:33:38.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.716+0000 7f3112a00640 1 -- start start 2024-04-23T14:33:38.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.716+0000 7f3112a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f310c0cfc20 0x7f310c15f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:38.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.716+0000 7f3112a00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f310c06f7c0 0x7f310c06cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:38.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.716+0000 7f3112a00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f310c15fc10 con 0x7f310c06d1c0 2024-04-23T14:33:38.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.716+0000 7f3112a00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f310c159180 con 0x7f310c06f7c0 2024-04-23T14:33:38.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.716+0000 7f3112a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f310c072a90 con 0x7f310c0cfc20 2024-04-23T14:33:38.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.716+0000 7f3112000640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f310c06d1c0 0x7f310c071c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:56962/0 (socket says 172.21.15.100:56962) 2024-04-23T14:33:38.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.716+0000 7f3112000640 1 -- 172.21.15.100:0/3761663193 learned_addr learned my addr 172.21.15.100:0/3761663193 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:38.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.716+0000 7f3110c00640 1 --2- 172.21.15.100:0/3761663193 >> v2:172.21.15.151:3300/0 conn(0x7f310c06f7c0 0x7f310c06cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:38.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.716+0000 7f3111600640 1 --2- 172.21.15.100:0/3761663193 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f310c0cfc20 0x7f310c15f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:38.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.716+0000 7f310be00640 1 -- 172.21.15.100:0/3761663193 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3059644281 0 0) 0x7f31040014a0 con 0x7f310c06d1c0 2024-04-23T14:33:38.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.716+0000 7f310be00640 1 -- 172.21.15.100:0/3761663193 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30f4003530 con 0x7f310c06d1c0 2024-04-23T14:33:38.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.716+0000 7f3110c00640 1 -- 172.21.15.100:0/3761663193 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f310c0cfc20 msgr2=0x7f310c15f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:38.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.716+0000 7f3110c00640 1 --2- 172.21.15.100:0/3761663193 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f310c0cfc20 0x7f310c15f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:38.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.716+0000 7f3110c00640 1 -- 172.21.15.100:0/3761663193 >> v1:172.21.15.100:6789/0 conn(0x7f310c06d1c0 legacy=0x7f310c071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:38.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.716+0000 7f3110c00640 1 -- 172.21.15.100:0/3761663193 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310c06bbe0 con 0x7f310c06f7c0 2024-04-23T14:33:38.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.716+0000 7f3111600640 1 --2- 172.21.15.100:0/3761663193 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f310c0cfc20 0x7f310c15f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:33:38.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.720+0000 7f3110c00640 1 --2- 172.21.15.100:0/3761663193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f310c06f7c0 0x7f310c06cc80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f30f80097f0 tx=0x7f30f801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7e0bdc46815d63a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:38.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.720+0000 7f310be00640 1 -- 172.21.15.100:0/3761663193 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f80b1070 con 0x7f310c06f7c0 2024-04-23T14:33:38.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.720+0000 7f310be00640 1 -- 172.21.15.100:0/3761663193 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30f80a8050 con 0x7f310c06f7c0 2024-04-23T14:33:38.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.720+0000 7f310be00640 1 -- 172.21.15.100:0/3761663193 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f80ac860 con 0x7f310c06f7c0 2024-04-23T14:33:38.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.720+0000 7f3112a00640 1 -- 172.21.15.100:0/3761663193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f310c06f7c0 msgr2=0x7f310c06cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:38.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.720+0000 7f3112a00640 1 --2- 172.21.15.100:0/3761663193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f310c06f7c0 0x7f310c06cc80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f30f80097f0 tx=0x7f30f801c3e0 comp rx=0 tx=0).stop 2024-04-23T14:33:38.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.720+0000 7f3112a00640 1 -- 172.21.15.100:0/3761663193 shutdown_connections 2024-04-23T14:33:38.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.720+0000 7f3112a00640 1 --2- 172.21.15.100:0/3761663193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f310c06f7c0 0x7f310c06cc80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:38.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.720+0000 7f3112a00640 1 --2- 172.21.15.100:0/3761663193 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f310c0cfc20 0x7f310c15f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:38.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.720+0000 7f3112a00640 1 -- 172.21.15.100:0/3761663193 >> 172.21.15.100:0/3761663193 conn(0x7f310c0b1d00 msgr2=0x7f310c0b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:38.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.720+0000 7f3112a00640 1 -- 172.21.15.100:0/3761663193 shutdown_connections 2024-04-23T14:33:38.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.720+0000 7f3112a00640 1 -- 172.21.15.100:0/3761663193 wait complete. 2024-04-23T14:33:38.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3112a00640 1 Processor -- start 2024-04-23T14:33:38.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3112a00640 1 -- start start 2024-04-23T14:33:38.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3112a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f310c06d1c0 0x7f310c09dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:38.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3112a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f310c06f7c0 0x7f310c09c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:38.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3112a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f310c0cfc20 0x7f310c150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:38.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3112a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f310c160620 con 0x7f310c06f7c0 2024-04-23T14:33:38.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3112a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f310c06c760 con 0x7f310c0cfc20 2024-04-23T14:33:38.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3112a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f310c1607a0 con 0x7f310c06d1c0 2024-04-23T14:33:38.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3110c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f310c06f7c0 0x7f310c09c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:38.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3110c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f310c06f7c0 0x7f310c09c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58110/0 (socket says 172.21.15.100:58110) 2024-04-23T14:33:38.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3110c00640 1 -- 172.21.15.100:0/3584464811 learned_addr learned my addr 172.21.15.100:0/3584464811 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:38.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3112000640 1 --2- 172.21.15.100:0/3584464811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f310c0cfc20 0x7f310c150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:38.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3111600640 1 --2- 172.21.15.100:0/3584464811 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f310c06d1c0 0x7f310c09dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:38.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3110c00640 1 -- 172.21.15.100:0/3584464811 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f310c06d1c0 msgr2=0x7f310c09dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:38.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3110c00640 1 --2- 172.21.15.100:0/3584464811 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f310c06d1c0 0x7f310c09dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:38.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3110c00640 1 -- 172.21.15.100:0/3584464811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f310c0cfc20 msgr2=0x7f310c150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:38.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3110c00640 1 --2- 172.21.15.100:0/3584464811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f310c0cfc20 0x7f310c150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:38.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3110c00640 1 -- 172.21.15.100:0/3584464811 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310c09cc30 con 0x7f310c06f7c0 2024-04-23T14:33:38.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3110c00640 1 --2- 172.21.15.100:0/3584464811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f310c06f7c0 0x7f310c09c350 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f30f801a010 tx=0x7f30f8019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:38.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3109600640 1 -- 172.21.15.100:0/3584464811 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f80b1040 con 0x7f310c06f7c0 2024-04-23T14:33:38.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3109600640 1 -- 172.21.15.100:0/3584464811 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30f80bb070 con 0x7f310c06f7c0 2024-04-23T14:33:38.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3109600640 1 -- 172.21.15.100:0/3584464811 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f80b5410 con 0x7f310c06f7c0 2024-04-23T14:33:38.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3112a00640 1 -- 172.21.15.100:0/3584464811 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f310c09cf10 con 0x7f310c06f7c0 2024-04-23T14:33:38.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.724+0000 7f3112a00640 1 -- 172.21.15.100:0/3584464811 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f310c09d4d0 con 0x7f310c06f7c0 2024-04-23T14:33:38.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.728+0000 7f3112a00640 1 -- 172.21.15.100:0/3584464811 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f310c068ad0 con 0x7f310c06f7c0 2024-04-23T14:33:38.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.728+0000 7f3109600640 1 -- 172.21.15.100:0/3584464811 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f30f80b55d0 con 0x7f310c06f7c0 2024-04-23T14:33:38.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.728+0000 7f3109600640 1 --2- 172.21.15.100:0/3584464811 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f30ec068380 0x7f30ec06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:38.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.728+0000 7f3109600640 1 -- 172.21.15.100:0/3584464811 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6001+0+0 (secure 0 0 0) 0x7f30f81212e0 con 0x7f310c06f7c0 2024-04-23T14:33:38.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.728+0000 7f3111600640 1 --2- 172.21.15.100:0/3584464811 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f30ec068380 0x7f30ec06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:38.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.728+0000 7f3111600640 1 --2- 172.21.15.100:0/3584464811 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f30ec068380 0x7f30ec06a840 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f310c15ead0 tx=0x7f3100009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:38.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.732+0000 7f3109600640 1 -- 172.21.15.100:0/3584464811 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f30f8016090 con 0x7f310c06f7c0 2024-04-23T14:33:38.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:38.924+0000 7f3112a00640 1 -- 172.21.15.100:0/3584464811 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f310c09c890 con 0x7f310c06f7c0 2024-04-23T14:33:39.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.544+0000 7f3109600640 1 -- 172.21.15.100:0/3584464811 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f30f80f1100 con 0x7f310c06f7c0 2024-04-23T14:33:39.548 INFO:teuthology.orchestra.run.smithi100.stderr:new fs with metadata pool 5 and data pool 6 2024-04-23T14:33:39.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.548+0000 7f3112a00640 1 -- 172.21.15.100:0/3584464811 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f30ec068380 msgr2=0x7f30ec06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:39.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.548+0000 7f3112a00640 1 --2- 172.21.15.100:0/3584464811 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f30ec068380 0x7f30ec06a840 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f310c15ead0 tx=0x7f3100009190 comp rx=0 tx=0).stop 2024-04-23T14:33:39.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.548+0000 7f3112a00640 1 -- 172.21.15.100:0/3584464811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f310c06f7c0 msgr2=0x7f310c09c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:39.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.548+0000 7f3112a00640 1 --2- 172.21.15.100:0/3584464811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f310c06f7c0 0x7f310c09c350 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f30f801a010 tx=0x7f30f8019180 comp rx=0 tx=0).stop 2024-04-23T14:33:39.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.548+0000 7f3112a00640 1 -- 172.21.15.100:0/3584464811 shutdown_connections 2024-04-23T14:33:39.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.548+0000 7f3112a00640 1 --2- 172.21.15.100:0/3584464811 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f30ec068380 0x7f30ec06a840 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.548+0000 7f3112a00640 1 --2- 172.21.15.100:0/3584464811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f310c0cfc20 0x7f310c150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.548+0000 7f3112a00640 1 --2- 172.21.15.100:0/3584464811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f310c06f7c0 0x7f310c09c350 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.548+0000 7f3112a00640 1 --2- 172.21.15.100:0/3584464811 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f310c06d1c0 0x7f310c09dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.548+0000 7f3112a00640 1 -- 172.21.15.100:0/3584464811 >> 172.21.15.100:0/3584464811 conn(0x7f310c0b1d00 msgr2=0x7f310c0b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:39.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.548+0000 7f3112a00640 1 -- 172.21.15.100:0/3584464811 shutdown_connections 2024-04-23T14:33:39.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.548+0000 7f3112a00640 1 -- 172.21.15.100:0/3584464811 wait complete. 2024-04-23T14:33:39.594 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:39.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.728+0000 7f6895400640 1 Processor -- start 2024-04-23T14:33:39.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.728+0000 7f6895400640 1 -- start start 2024-04-23T14:33:39.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.728+0000 7f6895400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f68900ad860 0x7f68900b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:39.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.728+0000 7f6895400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f689015f400 0x7f689006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:39.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.728+0000 7f6895400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68900b49c0 con 0x7f68900ae270 2024-04-23T14:33:39.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.728+0000 7f6895400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f68900b5590 con 0x7f689015f400 2024-04-23T14:33:39.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.728+0000 7f6895400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f68900b4c60 con 0x7f68900ad860 2024-04-23T14:33:39.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.728+0000 7f6894a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f689015f400 0x7f689006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:39.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.728+0000 7f6894a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f689015f400 0x7f689006bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58126/0 (socket says 172.21.15.100:58126) 2024-04-23T14:33:39.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.728+0000 7f6894a00640 1 -- 172.21.15.100:0/2371592371 learned_addr learned my addr 172.21.15.100:0/2371592371 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:39.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.728+0000 7f688f400640 1 --2- 172.21.15.100:0/2371592371 >> v2:172.21.15.151:3300/0 conn(0x7f68900ad860 0x7f68900b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:39.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.728+0000 7f6894a00640 1 -- 172.21.15.100:0/2371592371 >> v2:172.21.15.151:3300/0 conn(0x7f68900ad860 msgr2=0x7f68900b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:39.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.728+0000 7f6894a00640 1 --2- 172.21.15.100:0/2371592371 >> v2:172.21.15.151:3300/0 conn(0x7f68900ad860 0x7f68900b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.728+0000 7f6894a00640 1 -- 172.21.15.100:0/2371592371 >> v1:172.21.15.100:6789/0 conn(0x7f68900ae270 legacy=0x7f68900ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:39.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.728+0000 7f6894a00640 1 -- 172.21.15.100:0/2371592371 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68900b4f40 con 0x7f689015f400 2024-04-23T14:33:39.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.732+0000 7f6894a00640 1 --2- 172.21.15.100:0/2371592371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f689015f400 0x7f689006bab0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f68900b4b90 tx=0x7f688801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07262cc49a29aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:39.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.732+0000 7f688ea00640 1 -- 172.21.15.100:0/2371592371 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68880090d0 con 0x7f689015f400 2024-04-23T14:33:39.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.732+0000 7f688ea00640 1 -- 172.21.15.100:0/2371592371 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68880a57d0 con 0x7f689015f400 2024-04-23T14:33:39.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.732+0000 7f688ea00640 1 -- 172.21.15.100:0/2371592371 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888009270 con 0x7f689015f400 2024-04-23T14:33:39.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.732+0000 7f6895400640 1 -- 172.21.15.100:0/2371592371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f689015f400 msgr2=0x7f689006bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:39.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.732+0000 7f6895400640 1 --2- 172.21.15.100:0/2371592371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f689015f400 0x7f689006bab0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f68900b4b90 tx=0x7f688801c9f0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.732+0000 7f6895400640 1 -- 172.21.15.100:0/2371592371 shutdown_connections 2024-04-23T14:33:39.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.732+0000 7f6895400640 1 --2- 172.21.15.100:0/2371592371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f689015f400 0x7f689006bab0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.732+0000 7f6895400640 1 --2- 172.21.15.100:0/2371592371 >> v2:172.21.15.151:3300/0 conn(0x7f68900ad860 0x7f68900b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.732+0000 7f6895400640 1 -- 172.21.15.100:0/2371592371 >> 172.21.15.100:0/2371592371 conn(0x7f68900d0d10 msgr2=0x7f68900d1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:39.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.732+0000 7f6895400640 1 -- 172.21.15.100:0/2371592371 shutdown_connections 2024-04-23T14:33:39.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.732+0000 7f6895400640 1 -- 172.21.15.100:0/2371592371 wait complete. 2024-04-23T14:33:39.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.732+0000 7f6895400640 1 Processor -- start 2024-04-23T14:33:39.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.732+0000 7f6895400640 1 -- start start 2024-04-23T14:33:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f6895400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68900ad860 0x7f68901424d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f6895400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68900ae270 0x7f689006f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f6895400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f689015f400 0x7f689006fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f6895400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f689016b460 con 0x7f68900ae270 2024-04-23T14:33:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f6895400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f689015faa0 con 0x7f68900ad860 2024-04-23T14:33:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f6895400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f689016b5e0 con 0x7f689015f400 2024-04-23T14:33:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f688f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68900ae270 0x7f689006f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f688f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68900ae270 0x7f689006f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58140/0 (socket says 172.21.15.100:58140) 2024-04-23T14:33:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f688f400640 1 -- 172.21.15.100:0/2252032945 learned_addr learned my addr 172.21.15.100:0/2252032945 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f688fe00640 1 --2- 172.21.15.100:0/2252032945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68900ad860 0x7f68901424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f6894a00640 1 --2- 172.21.15.100:0/2252032945 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f689015f400 0x7f689006fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f688f400640 1 -- 172.21.15.100:0/2252032945 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f689015f400 msgr2=0x7f689006fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:39.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f688f400640 1 --2- 172.21.15.100:0/2252032945 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f689015f400 0x7f689006fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f688f400640 1 -- 172.21.15.100:0/2252032945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68900ad860 msgr2=0x7f68901424d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:39.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f688f400640 1 --2- 172.21.15.100:0/2252032945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68900ad860 0x7f68901424d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f688f400640 1 -- 172.21.15.100:0/2252032945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689015dbb0 con 0x7f68900ae270 2024-04-23T14:33:39.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f688f400640 1 --2- 172.21.15.100:0/2252032945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68900ae270 0x7f689006f500 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f687c00d620 tx=0x7f687c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:39.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f6887e00640 1 -- 172.21.15.100:0/2252032945 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c00b620 con 0x7f68900ae270 2024-04-23T14:33:39.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f6887e00640 1 -- 172.21.15.100:0/2252032945 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f687c019070 con 0x7f68900ae270 2024-04-23T14:33:39.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f6887e00640 1 -- 172.21.15.100:0/2252032945 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c00b9e0 con 0x7f68900ae270 2024-04-23T14:33:39.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f6895400640 1 -- 172.21.15.100:0/2252032945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689015de90 con 0x7f68900ae270 2024-04-23T14:33:39.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f6895400640 1 -- 172.21.15.100:0/2252032945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f689015e450 con 0x7f68900ae270 2024-04-23T14:33:39.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f6895400640 1 -- 172.21.15.100:0/2252032945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6890075c20 con 0x7f68900ae270 2024-04-23T14:33:39.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.736+0000 7f6887e00640 1 -- 172.21.15.100:0/2252032945 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f687c00bb80 con 0x7f68900ae270 2024-04-23T14:33:39.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.740+0000 7f6887e00640 1 --2- 172.21.15.100:0/2252032945 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f68700683b0 0x7f687006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:39.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.740+0000 7f6887e00640 1 -- 172.21.15.100:0/2252032945 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f687c086e70 con 0x7f68900ae270 2024-04-23T14:33:39.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.740+0000 7f688fe00640 1 --2- 172.21.15.100:0/2252032945 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f68700683b0 0x7f687006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:39.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.740+0000 7f688fe00640 1 --2- 172.21.15.100:0/2252032945 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f68700683b0 0x7f687006a870 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6880001810 tx=0x7f6880009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:39.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.744+0000 7f6887e00640 1 -- 172.21.15.100:0/2252032945 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f687c056bb0 con 0x7f68900ae270 2024-04-23T14:33:39.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.900+0000 7f6895400640 1 -- 172.21.15.100:0/2252032945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6890070a70 con 0x7f68900ae270 2024-04-23T14:33:39.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.904+0000 7f6887e00640 1 -- 172.21.15.100:0/2252032945 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16967 (secure 0 0 0) 0x7f687c05aeb0 con 0x7f68900ae270 2024-04-23T14:33:39.908 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:39.908 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":37,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:33:38.928278+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:33:36.592437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-23T14:33:37.882390+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:39.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.904+0000 7f6895400640 1 -- 172.21.15.100:0/2252032945 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f68700683b0 msgr2=0x7f687006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:39.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.904+0000 7f6895400640 1 --2- 172.21.15.100:0/2252032945 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f68700683b0 0x7f687006a870 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6880001810 tx=0x7f6880009040 comp rx=0 tx=0).stop 2024-04-23T14:33:39.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.904+0000 7f6895400640 1 -- 172.21.15.100:0/2252032945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68900ae270 msgr2=0x7f689006f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:39.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.904+0000 7f6895400640 1 --2- 172.21.15.100:0/2252032945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68900ae270 0x7f689006f500 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f687c00d620 tx=0x7f687c00dae0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.904+0000 7f6895400640 1 -- 172.21.15.100:0/2252032945 shutdown_connections 2024-04-23T14:33:39.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.904+0000 7f6895400640 1 --2- 172.21.15.100:0/2252032945 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f68700683b0 0x7f687006a870 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.904+0000 7f6895400640 1 --2- 172.21.15.100:0/2252032945 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f689015f400 0x7f689006fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.904+0000 7f6895400640 1 --2- 172.21.15.100:0/2252032945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68900ae270 0x7f689006f500 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.904+0000 7f6895400640 1 --2- 172.21.15.100:0/2252032945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68900ad860 0x7f68901424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.904+0000 7f6895400640 1 -- 172.21.15.100:0/2252032945 >> 172.21.15.100:0/2252032945 conn(0x7f68900d0d10 msgr2=0x7f689015fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:39.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.904+0000 7f6895400640 1 -- 172.21.15.100:0/2252032945 shutdown_connections 2024-04-23T14:33:39.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:39.904+0000 7f6895400640 1 -- 172.21.15.100:0/2252032945 wait complete. 2024-04-23T14:33:39.929 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:40.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.060+0000 7f0569400640 1 Processor -- start 2024-04-23T14:33:40.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.060+0000 7f0569400640 1 -- start start 2024-04-23T14:33:40.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.060+0000 7f0569400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f056415d600 0x7f05640d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:40.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.060+0000 7f0569400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05640d04a0 con 0x7f056415c7e0 2024-04-23T14:33:40.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.060+0000 7f0569400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05640d0780 con 0x7f05640cca50 2024-04-23T14:33:40.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.060+0000 7f0569400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f056415db00 con 0x7f056415d600 2024-04-23T14:33:40.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.060+0000 7f0563200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f056415d600 0x7f05640d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:40.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.060+0000 7f0561e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f056415c7e0 0x7f05640d1260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:56980/0 (socket says 172.21.15.100:56980) 2024-04-23T14:33:40.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.060+0000 7f0561e00640 1 -- 172.21.15.100:0/530119181 learned_addr learned my addr 172.21.15.100:0/530119181 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:40.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.060+0000 7f0563200640 1 -- 172.21.15.100:0/530119181 >> v1:172.21.15.151:6789/0 conn(0x7f05640cca50 legacy=0x7f05640cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:40.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.060+0000 7f0563200640 1 -- 172.21.15.100:0/530119181 >> v1:172.21.15.100:6789/0 conn(0x7f056415c7e0 legacy=0x7f05640d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:40.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.060+0000 7f0563200640 1 -- 172.21.15.100:0/530119181 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05640d0a60 con 0x7f056415d600 2024-04-23T14:33:40.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.060+0000 7f0561400640 1 -- 172.21.15.100:0/530119181 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1219334133 0 0) 0x7f05640d04a0 con 0x7f056415c7e0 2024-04-23T14:33:40.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.064+0000 7f0563200640 1 --2- 172.21.15.100:0/530119181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f056415d600 0x7f05640d1970 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f055c009d90 tx=0x7f055c01c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220d6f5db7dfafc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:40.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.064+0000 7f0561400640 1 -- 172.21.15.100:0/530119181 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c0b2070 con 0x7f056415d600 2024-04-23T14:33:40.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.064+0000 7f0561400640 1 -- 172.21.15.100:0/530119181 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f055c0a5990 con 0x7f056415d600 2024-04-23T14:33:40.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.064+0000 7f0561400640 1 -- 172.21.15.100:0/530119181 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c0a5d60 con 0x7f056415d600 2024-04-23T14:33:40.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.064+0000 7f0569400640 1 -- 172.21.15.100:0/530119181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f056415d600 msgr2=0x7f05640d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:40.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.064+0000 7f0569400640 1 --2- 172.21.15.100:0/530119181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f056415d600 0x7f05640d1970 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f055c009d90 tx=0x7f055c01c8b0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.064+0000 7f0569400640 1 -- 172.21.15.100:0/530119181 shutdown_connections 2024-04-23T14:33:40.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.064+0000 7f0569400640 1 --2- 172.21.15.100:0/530119181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f056415d600 0x7f05640d1970 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.064+0000 7f0569400640 1 -- 172.21.15.100:0/530119181 >> 172.21.15.100:0/530119181 conn(0x7f05640cee80 msgr2=0x7f05640cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:40.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.064+0000 7f0569400640 1 -- 172.21.15.100:0/530119181 shutdown_connections 2024-04-23T14:33:40.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.064+0000 7f0569400640 1 -- 172.21.15.100:0/530119181 wait complete. 2024-04-23T14:33:40.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f0569400640 1 Processor -- start 2024-04-23T14:33:40.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f0569400640 1 -- start start 2024-04-23T14:33:40.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f0569400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05640cca50 0x7f05641669f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:40.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f0569400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f056415c7e0 0x7f056406feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:40.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f0569400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f056415d600 0x7f056406d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:40.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f0569400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f056416ac60 con 0x7f05640cca50 2024-04-23T14:33:40.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f0569400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f056416aae0 con 0x7f056415c7e0 2024-04-23T14:33:40.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f0569400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f056416ade0 con 0x7f056415d600 2024-04-23T14:33:40.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f0562800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05640cca50 0x7f05641669f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:40.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f0562800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05640cca50 0x7f05641669f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58172/0 (socket says 172.21.15.100:58172) 2024-04-23T14:33:40.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f0562800640 1 -- 172.21.15.100:0/632339812 learned_addr learned my addr 172.21.15.100:0/632339812 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:40.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f0562800640 1 -- 172.21.15.100:0/632339812 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f056415d600 msgr2=0x7f056406d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:40.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f0562800640 1 --2- 172.21.15.100:0/632339812 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f056415d600 0x7f056406d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f0562800640 1 -- 172.21.15.100:0/632339812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f056415c7e0 msgr2=0x7f056406feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:40.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f0561e00640 1 --2- 172.21.15.100:0/632339812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f056415c7e0 0x7f056406feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:40.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f0562800640 1 --2- 172.21.15.100:0/632339812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f056415c7e0 0x7f056406feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f0562800640 1 -- 172.21.15.100:0/632339812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05640b5fd0 con 0x7f05640cca50 2024-04-23T14:33:40.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f0561e00640 1 --2- 172.21.15.100:0/632339812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f056415c7e0 0x7f056406feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:40.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f0562800640 1 --2- 172.21.15.100:0/632339812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05640cca50 0x7f05641669f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f055400c530 tx=0x7f055400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:40.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f055aa00640 1 -- 172.21.15.100:0/632339812 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0554016020 con 0x7f05640cca50 2024-04-23T14:33:40.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f055aa00640 1 -- 172.21.15.100:0/632339812 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0554005220 con 0x7f05640cca50 2024-04-23T14:33:40.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f0569400640 1 -- 172.21.15.100:0/632339812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0564165690 con 0x7f05640cca50 2024-04-23T14:33:40.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f055aa00640 1 -- 172.21.15.100:0/632339812 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0554007030 con 0x7f05640cca50 2024-04-23T14:33:40.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f0569400640 1 -- 172.21.15.100:0/632339812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0564165c50 con 0x7f05640cca50 2024-04-23T14:33:40.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.068+0000 7f0569400640 1 -- 172.21.15.100:0/632339812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0564068ad0 con 0x7f05640cca50 2024-04-23T14:33:40.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.072+0000 7f055aa00640 1 -- 172.21.15.100:0/632339812 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f05540071d0 con 0x7f05640cca50 2024-04-23T14:33:40.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.072+0000 7f055aa00640 1 --2- 172.21.15.100:0/632339812 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0544068380 0x7f054406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:40.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.072+0000 7f055aa00640 1 -- 172.21.15.100:0/632339812 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f0554086520 con 0x7f05640cca50 2024-04-23T14:33:40.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.072+0000 7f0561e00640 1 --2- 172.21.15.100:0/632339812 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0544068380 0x7f054406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:40.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.072+0000 7f0561e00640 1 --2- 172.21.15.100:0/632339812 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0544068380 0x7f054406a840 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f05640b6b00 tx=0x7f0550009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:40.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.076+0000 7f055aa00640 1 -- 172.21.15.100:0/632339812 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0554020020 con 0x7f05640cca50 2024-04-23T14:33:40.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.232+0000 7f0569400640 1 -- 172.21.15.100:0/632339812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05640b6b30 con 0x7f05640cca50 2024-04-23T14:33:40.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.232+0000 7f055aa00640 1 -- 172.21.15.100:0/632339812 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16967 (secure 0 0 0) 0x7f05540074c0 con 0x7f05640cca50 2024-04-23T14:33:40.238 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:40.238 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":37,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:33:38.928278+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:33:36.592437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-23T14:33:37.882390+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:40.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.236+0000 7f0569400640 1 -- 172.21.15.100:0/632339812 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0544068380 msgr2=0x7f054406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:40.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.236+0000 7f0569400640 1 --2- 172.21.15.100:0/632339812 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0544068380 0x7f054406a840 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f05640b6b00 tx=0x7f0550009040 comp rx=0 tx=0).stop 2024-04-23T14:33:40.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.236+0000 7f0569400640 1 -- 172.21.15.100:0/632339812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05640cca50 msgr2=0x7f05641669f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:40.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.236+0000 7f0569400640 1 --2- 172.21.15.100:0/632339812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05640cca50 0x7f05641669f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f055400c530 tx=0x7f055400c9f0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.236+0000 7f0569400640 1 -- 172.21.15.100:0/632339812 shutdown_connections 2024-04-23T14:33:40.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.236+0000 7f0569400640 1 --2- 172.21.15.100:0/632339812 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0544068380 0x7f054406a840 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.236+0000 7f0569400640 1 --2- 172.21.15.100:0/632339812 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f056415d600 0x7f056406d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.236+0000 7f0569400640 1 --2- 172.21.15.100:0/632339812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f056415c7e0 0x7f056406feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.236+0000 7f0569400640 1 --2- 172.21.15.100:0/632339812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05640cca50 0x7f05641669f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.236+0000 7f0569400640 1 -- 172.21.15.100:0/632339812 >> 172.21.15.100:0/632339812 conn(0x7f05640cee80 msgr2=0x7f05640cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:40.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.236+0000 7f0569400640 1 -- 172.21.15.100:0/632339812 shutdown_connections 2024-04-23T14:33:40.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.236+0000 7f0569400640 1 -- 172.21.15.100:0/632339812 wait complete. 2024-04-23T14:33:40.259 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-23T14:33:40.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.392+0000 7f036f000640 1 Processor -- start 2024-04-23T14:33:40.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.392+0000 7f036f000640 1 -- start start 2024-04-23T14:33:40.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.392+0000 7f036f000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f036815e860 0x7f03680b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:40.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.392+0000 7f036f000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f03680adb90 0x7f03680adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:40.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.392+0000 7f036f000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f036806e5f0 con 0x7f036806f950 2024-04-23T14:33:40.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.392+0000 7f036f000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f036806e8b0 con 0x7f036815e860 2024-04-23T14:33:40.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.392+0000 7f036f000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f036806eb90 con 0x7f03680adb90 2024-04-23T14:33:40.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.392+0000 7f0367e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f036815e860 0x7f03680b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:40.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.392+0000 7f0367e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f036815e860 0x7f03680b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58184/0 (socket says 172.21.15.100:58184) 2024-04-23T14:33:40.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.392+0000 7f036d000640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f036806f950 0x7f036806d350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:56994/0 (socket says 172.21.15.100:56994) 2024-04-23T14:33:40.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.392+0000 7f0367e00640 1 -- 172.21.15.100:0/2471148730 learned_addr learned my addr 172.21.15.100:0/2471148730 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:40.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.392+0000 7f0367400640 1 --2- 172.21.15.100:0/2471148730 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f03680adb90 0x7f03680adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:40.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.392+0000 7f0367e00640 1 -- 172.21.15.100:0/2471148730 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f03680adb90 msgr2=0x7f03680adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:40.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.392+0000 7f0367e00640 1 --2- 172.21.15.100:0/2471148730 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f03680adb90 0x7f03680adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.392+0000 7f0367e00640 1 -- 172.21.15.100:0/2471148730 >> v1:172.21.15.100:6789/0 conn(0x7f036806f950 legacy=0x7f036806d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:40.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.392+0000 7f0367e00640 1 -- 172.21.15.100:0/2471148730 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036806ee70 con 0x7f036815e860 2024-04-23T14:33:40.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.392+0000 7f0366a00640 1 -- 172.21.15.100:0/2471148730 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 840087736 0 0) 0x7f036806e5f0 con 0x7f036806f950 2024-04-23T14:33:40.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.396+0000 7f0367400640 1 --2- 172.21.15.100:0/2471148730 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f03680adb90 0x7f03680adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:40.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.396+0000 7f0367e00640 1 --2- 172.21.15.100:0/2471148730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f036815e860 0x7f03680b3ab0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0358009e50 tx=0x7f035801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7edb939109abb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:40.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.396+0000 7f0366a00640 1 -- 172.21.15.100:0/2471148730 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03580a58d0 con 0x7f036815e860 2024-04-23T14:33:40.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.396+0000 7f0366a00640 1 -- 172.21.15.100:0/2471148730 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03580a5a70 con 0x7f036815e860 2024-04-23T14:33:40.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.396+0000 7f0366a00640 1 -- 172.21.15.100:0/2471148730 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03580a5e40 con 0x7f036815e860 2024-04-23T14:33:40.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.396+0000 7f036f000640 1 -- 172.21.15.100:0/2471148730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f036815e860 msgr2=0x7f03680b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:40.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.396+0000 7f036f000640 1 --2- 172.21.15.100:0/2471148730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f036815e860 0x7f03680b3ab0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0358009e50 tx=0x7f035801ca00 comp rx=0 tx=0).stop 2024-04-23T14:33:40.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.396+0000 7f036f000640 1 -- 172.21.15.100:0/2471148730 shutdown_connections 2024-04-23T14:33:40.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.396+0000 7f036f000640 1 --2- 172.21.15.100:0/2471148730 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f03680adb90 0x7f03680adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.396+0000 7f036f000640 1 --2- 172.21.15.100:0/2471148730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f036815e860 0x7f03680b3ab0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.396+0000 7f036f000640 1 -- 172.21.15.100:0/2471148730 >> 172.21.15.100:0/2471148730 conn(0x7f0368072590 msgr2=0x7f03680729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:40.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.396+0000 7f036f000640 1 -- 172.21.15.100:0/2471148730 shutdown_connections 2024-04-23T14:33:40.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.396+0000 7f036f000640 1 -- 172.21.15.100:0/2471148730 wait complete. 2024-04-23T14:33:40.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f036f000640 1 Processor -- start 2024-04-23T14:33:40.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f036f000640 1 -- start start 2024-04-23T14:33:40.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f036f000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f036806f950 0x7f0368153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:40.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f036f000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f03680adb90 0x7f0368140f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:40.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f036f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f036815e860 0x7f0368160120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:40.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f036f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f036815ec40 con 0x7f036815e860 2024-04-23T14:33:40.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f036f000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0368073790 con 0x7f036806f950 2024-04-23T14:33:40.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f036f000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f036806da60 con 0x7f03680adb90 2024-04-23T14:33:40.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f036d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f036815e860 0x7f0368160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:40.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f036d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f036815e860 0x7f0368160120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58198/0 (socket says 172.21.15.100:58198) 2024-04-23T14:33:40.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f036d000640 1 -- 172.21.15.100:0/646555541 learned_addr learned my addr 172.21.15.100:0/646555541 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:40.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f0367e00640 1 --2- 172.21.15.100:0/646555541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f036806f950 0x7f0368153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:40.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f0367400640 1 --2- 172.21.15.100:0/646555541 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f03680adb90 0x7f0368140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:40.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f036d000640 1 -- 172.21.15.100:0/646555541 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f03680adb90 msgr2=0x7f0368140f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:40.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f036d000640 1 --2- 172.21.15.100:0/646555541 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f03680adb90 0x7f0368140f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f036d000640 1 -- 172.21.15.100:0/646555541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f036806f950 msgr2=0x7f0368153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:40.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f036d000640 1 --2- 172.21.15.100:0/646555541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f036806f950 0x7f0368153230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f036d000640 1 -- 172.21.15.100:0/646555541 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03681521d0 con 0x7f036815e860 2024-04-23T14:33:40.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f036d000640 1 --2- 172.21.15.100:0/646555541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f036815e860 0x7f0368160120 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f036000c480 tx=0x7f036000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:40.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f035fe00640 1 -- 172.21.15.100:0/646555541 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360018070 con 0x7f036815e860 2024-04-23T14:33:40.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f035fe00640 1 -- 172.21.15.100:0/646555541 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03600074e0 con 0x7f036815e860 2024-04-23T14:33:40.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f035fe00640 1 -- 172.21.15.100:0/646555541 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360004a30 con 0x7f036815e860 2024-04-23T14:33:40.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.400+0000 7f036f000640 1 -- 172.21.15.100:0/646555541 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0368152470 con 0x7f036815e860 2024-04-23T14:33:40.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.404+0000 7f036f000640 1 -- 172.21.15.100:0/646555541 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0368152a30 con 0x7f036815e860 2024-04-23T14:33:40.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.404+0000 7f036f000640 1 -- 172.21.15.100:0/646555541 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0368068ad0 con 0x7f036815e860 2024-04-23T14:33:40.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.404+0000 7f035fe00640 1 -- 172.21.15.100:0/646555541 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f036001b070 con 0x7f036815e860 2024-04-23T14:33:40.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.404+0000 7f035fe00640 1 --2- 172.21.15.100:0/646555541 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f03480683b0 0x7f034806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:40.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.404+0000 7f035fe00640 1 -- 172.21.15.100:0/646555541 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f0360085d40 con 0x7f036815e860 2024-04-23T14:33:40.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.404+0000 7f0367e00640 1 --2- 172.21.15.100:0/646555541 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f03480683b0 0x7f034806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:40.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.404+0000 7f0367e00640 1 --2- 172.21.15.100:0/646555541 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f03480683b0 0x7f034806a870 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f03580ae000 tx=0x7f03580a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:40.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.408+0000 7f035fe00640 1 -- 172.21.15.100:0/646555541 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0360055b10 con 0x7f036815e860 2024-04-23T14:33:40.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:40.608+0000 7f036f000640 1 -- 172.21.15.100:0/646555541 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0368072bb0 con 0x7f036815e860 2024-04-23T14:33:41.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.572+0000 7f035fe00640 1 -- 172.21.15.100:0/646555541 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f0360059e10 con 0x7f036815e860 2024-04-23T14:33:41.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.576+0000 7f036f000640 1 -- 172.21.15.100:0/646555541 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f03480683b0 msgr2=0x7f034806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:41.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.576+0000 7f036f000640 1 --2- 172.21.15.100:0/646555541 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f03480683b0 0x7f034806a870 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f03580ae000 tx=0x7f03580a7070 comp rx=0 tx=0).stop 2024-04-23T14:33:41.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.576+0000 7f036f000640 1 -- 172.21.15.100:0/646555541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f036815e860 msgr2=0x7f0368160120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:41.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.576+0000 7f036f000640 1 --2- 172.21.15.100:0/646555541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f036815e860 0x7f0368160120 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f036000c480 tx=0x7f036000c940 comp rx=0 tx=0).stop 2024-04-23T14:33:41.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.576+0000 7f036f000640 1 -- 172.21.15.100:0/646555541 shutdown_connections 2024-04-23T14:33:41.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.576+0000 7f036f000640 1 --2- 172.21.15.100:0/646555541 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f03480683b0 0x7f034806a870 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.576+0000 7f036f000640 1 --2- 172.21.15.100:0/646555541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f036815e860 0x7f0368160120 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.576+0000 7f036f000640 1 --2- 172.21.15.100:0/646555541 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f03680adb90 0x7f0368140f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.576+0000 7f036f000640 1 --2- 172.21.15.100:0/646555541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f036806f950 0x7f0368153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.576+0000 7f036f000640 1 -- 172.21.15.100:0/646555541 >> 172.21.15.100:0/646555541 conn(0x7f0368072590 msgr2=0x7f0368067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:41.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.576+0000 7f036f000640 1 -- 172.21.15.100:0/646555541 shutdown_connections 2024-04-23T14:33:41.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.576+0000 7f036f000640 1 -- 172.21.15.100:0/646555541 wait complete. 2024-04-23T14:33:41.604 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-23T14:33:41.604 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-23T14:33:41.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.744+0000 7fa1a9400640 1 Processor -- start 2024-04-23T14:33:41.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.744+0000 7fa1a9400640 1 -- start start 2024-04-23T14:33:41.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.744+0000 7fa1a9400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa1a406f8c0 0x7fa1a406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:41.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.744+0000 7fa1a9400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa1a40d0170 0x7fa1a40ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:41.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.744+0000 7fa1a9400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fa1a40ce620 0x7fa1a415f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:41.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.744+0000 7fa1a9400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a40ad050 con 0x7fa1a40d0170 2024-04-23T14:33:41.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.744+0000 7fa1a9400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a40ad8f0 con 0x7fa1a40ce620 2024-04-23T14:33:41.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.744+0000 7fa1a9400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a406dd60 con 0x7fa1a406f8c0 2024-04-23T14:33:41.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.744+0000 7fa1a1e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa1a40d0170 0x7fa1a40ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:41.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.744+0000 7fa1a1e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa1a40d0170 0x7fa1a40ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58202/0 (socket says 172.21.15.100:58202) 2024-04-23T14:33:41.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.744+0000 7fa1a1e00640 1 -- 172.21.15.100:0/1009673097 learned_addr learned my addr 172.21.15.100:0/1009673097 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:41.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.744+0000 7fa1a2800640 1 --2- 172.21.15.100:0/1009673097 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa1a406f8c0 0x7fa1a406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:41.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.744+0000 7fa1a3200640 1 --2- 172.21.15.100:0/1009673097 >> v2:172.21.15.151:3300/0 conn(0x7fa1a40ce620 0x7fa1a415f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:41.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.744+0000 7fa1a1e00640 1 -- 172.21.15.100:0/1009673097 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa1a406f8c0 msgr2=0x7fa1a406c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:41.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.744+0000 7fa1a1e00640 1 --2- 172.21.15.100:0/1009673097 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa1a406f8c0 0x7fa1a406c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.744+0000 7fa1a1e00640 1 -- 172.21.15.100:0/1009673097 >> v2:172.21.15.151:3300/0 conn(0x7fa1a40ce620 msgr2=0x7fa1a415f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:41.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.744+0000 7fa1a1e00640 1 --2- 172.21.15.100:0/1009673097 >> v2:172.21.15.151:3300/0 conn(0x7fa1a40ce620 0x7fa1a415f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.744+0000 7fa1a1e00640 1 -- 172.21.15.100:0/1009673097 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a40d09c0 con 0x7fa1a40d0170 2024-04-23T14:33:41.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.744+0000 7fa1a2800640 1 --2- 172.21.15.100:0/1009673097 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa1a406f8c0 0x7fa1a406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:41.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.744+0000 7fa1a3200640 1 --2- 172.21.15.100:0/1009673097 >> v2:172.21.15.151:3300/0 conn(0x7fa1a40ce620 0x7fa1a415f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:41.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.748+0000 7fa1a1e00640 1 --2- 172.21.15.100:0/1009673097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a40d0170 0x7fa1a40ce0e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40afcb0 tx=0x7fa19001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5654e2446453d474 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:41.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.748+0000 7fa1a1400640 1 -- 172.21.15.100:0/1009673097 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1900b0070 con 0x7fa1a40d0170 2024-04-23T14:33:41.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.748+0000 7fa1a1400640 1 -- 172.21.15.100:0/1009673097 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1900ab600 con 0x7fa1a40d0170 2024-04-23T14:33:41.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.748+0000 7fa1a1400640 1 -- 172.21.15.100:0/1009673097 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1900ab9d0 con 0x7fa1a40d0170 2024-04-23T14:33:41.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.748+0000 7fa1a9400640 1 -- 172.21.15.100:0/1009673097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a40d0170 msgr2=0x7fa1a40ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:41.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.748+0000 7fa1a9400640 1 --2- 172.21.15.100:0/1009673097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a40d0170 0x7fa1a40ce0e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40afcb0 tx=0x7fa19001c260 comp rx=0 tx=0).stop 2024-04-23T14:33:41.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.748+0000 7fa1a9400640 1 -- 172.21.15.100:0/1009673097 shutdown_connections 2024-04-23T14:33:41.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.748+0000 7fa1a9400640 1 --2- 172.21.15.100:0/1009673097 >> v2:172.21.15.151:3300/0 conn(0x7fa1a40ce620 0x7fa1a415f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.748+0000 7fa1a9400640 1 --2- 172.21.15.100:0/1009673097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a40d0170 0x7fa1a40ce0e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.748+0000 7fa1a9400640 1 --2- 172.21.15.100:0/1009673097 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa1a406f8c0 0x7fa1a406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.748+0000 7fa1a9400640 1 -- 172.21.15.100:0/1009673097 >> 172.21.15.100:0/1009673097 conn(0x7fa1a40b5200 msgr2=0x7fa1a40b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:41.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.748+0000 7fa1a9400640 1 -- 172.21.15.100:0/1009673097 shutdown_connections 2024-04-23T14:33:41.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.748+0000 7fa1a9400640 1 -- 172.21.15.100:0/1009673097 wait complete. 2024-04-23T14:33:41.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.748+0000 7fa1a9400640 1 Processor -- start 2024-04-23T14:33:41.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.748+0000 7fa1a9400640 1 -- start start 2024-04-23T14:33:41.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa1a9400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa1a406f8c0 0x7fa1a415c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:41.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa1a9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a40ce620 0x7fa1a40c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:41.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa1a9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a40d0170 0x7fa1a40c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:41.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa1a9400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a40adee0 con 0x7fa1a40d0170 2024-04-23T14:33:41.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa1a9400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a40add60 con 0x7fa1a40ce620 2024-04-23T14:33:41.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa1a9400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a40ae060 con 0x7fa1a406f8c0 2024-04-23T14:33:41.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa1a3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a40d0170 0x7fa1a40c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:41.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa1a3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a40d0170 0x7fa1a40c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58206/0 (socket says 172.21.15.100:58206) 2024-04-23T14:33:41.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa1a3200640 1 -- 172.21.15.100:0/4278464812 learned_addr learned my addr 172.21.15.100:0/4278464812 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:41.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa1a2800640 1 --2- 172.21.15.100:0/4278464812 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa1a406f8c0 0x7fa1a415c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:41.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa1a1e00640 1 --2- 172.21.15.100:0/4278464812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a40ce620 0x7fa1a40c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:41.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa1a3200640 1 -- 172.21.15.100:0/4278464812 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa1a406f8c0 msgr2=0x7fa1a415c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:41.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa1a3200640 1 --2- 172.21.15.100:0/4278464812 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa1a406f8c0 0x7fa1a415c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa1a3200640 1 -- 172.21.15.100:0/4278464812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a40ce620 msgr2=0x7fa1a40c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:41.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa1a3200640 1 --2- 172.21.15.100:0/4278464812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a40ce620 0x7fa1a40c9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa1a3200640 1 -- 172.21.15.100:0/4278464812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a40c63f0 con 0x7fa1a40d0170 2024-04-23T14:33:41.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa1a1e00640 1 --2- 172.21.15.100:0/4278464812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a40ce620 0x7fa1a40c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:41.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa1a2800640 1 --2- 172.21.15.100:0/4278464812 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa1a406f8c0 0x7fa1a415c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:41.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa1a3200640 1 --2- 172.21.15.100:0/4278464812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a40d0170 0x7fa1a40c89e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00cf20 tx=0x7fa19c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:41.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa19aa00640 1 -- 172.21.15.100:0/4278464812 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c01a070 con 0x7fa1a40d0170 2024-04-23T14:33:41.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa19aa00640 1 -- 172.21.15.100:0/4278464812 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa19c00d840 con 0x7fa1a40d0170 2024-04-23T14:33:41.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa1a9400640 1 -- 172.21.15.100:0/4278464812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a415c620 con 0x7fa1a40d0170 2024-04-23T14:33:41.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa19aa00640 1 -- 172.21.15.100:0/4278464812 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c00dc30 con 0x7fa1a40d0170 2024-04-23T14:33:41.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa1a9400640 1 -- 172.21.15.100:0/4278464812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a40c8430 con 0x7fa1a40d0170 2024-04-23T14:33:41.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.752+0000 7fa1a9400640 1 -- 172.21.15.100:0/4278464812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a4068ad0 con 0x7fa1a40d0170 2024-04-23T14:33:41.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.756+0000 7fa19aa00640 1 -- 172.21.15.100:0/4278464812 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa19c01d070 con 0x7fa1a40d0170 2024-04-23T14:33:41.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.756+0000 7fa19aa00640 1 --2- 172.21.15.100:0/4278464812 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa184068380 0x7fa18406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:41.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.756+0000 7fa19aa00640 1 -- 172.21.15.100:0/4278464812 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fa19c0873b0 con 0x7fa1a40d0170 2024-04-23T14:33:41.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.756+0000 7fa1a2800640 1 --2- 172.21.15.100:0/4278464812 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa184068380 0x7fa18406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:41.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.756+0000 7fa1a2800640 1 --2- 172.21.15.100:0/4278464812 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa184068380 0x7fa18406a840 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa194006fd0 tx=0x7fa194008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:41.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.760+0000 7fa19aa00640 1 -- 172.21.15.100:0/4278464812 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa1a40c63f0 con 0x7fa1a40d0170 2024-04-23T14:33:41.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:41.948+0000 7fa1a9400640 1 -- 172.21.15.100:0/4278464812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa1a406cc00 con 0x7fa1a40d0170 2024-04-23T14:33:42.504 INFO:tasks.ceph.mds.c.smithi100.stderr:2024-04-23T14:33:42.500+0000 7f3075600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:33:42.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.576+0000 7fa19aa00640 1 -- 172.21.15.100:0/4278464812 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7fa19c0577d0 con 0x7fa1a40d0170 2024-04-23T14:33:42.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.576+0000 7fa1a9400640 1 -- 172.21.15.100:0/4278464812 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa184068380 msgr2=0x7fa18406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:42.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.576+0000 7fa1a9400640 1 --2- 172.21.15.100:0/4278464812 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa184068380 0x7fa18406a840 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa194006fd0 tx=0x7fa194008040 comp rx=0 tx=0).stop 2024-04-23T14:33:42.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.576+0000 7fa1a9400640 1 -- 172.21.15.100:0/4278464812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a40d0170 msgr2=0x7fa1a40c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:42.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.576+0000 7fa1a9400640 1 --2- 172.21.15.100:0/4278464812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a40d0170 0x7fa1a40c89e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00cf20 tx=0x7fa19c00a7d0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.580+0000 7fa1a9400640 1 -- 172.21.15.100:0/4278464812 shutdown_connections 2024-04-23T14:33:42.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.580+0000 7fa1a9400640 1 --2- 172.21.15.100:0/4278464812 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa184068380 0x7fa18406a840 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.580+0000 7fa1a9400640 1 --2- 172.21.15.100:0/4278464812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a40d0170 0x7fa1a40c89e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.580+0000 7fa1a9400640 1 --2- 172.21.15.100:0/4278464812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a40ce620 0x7fa1a40c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.580+0000 7fa1a9400640 1 --2- 172.21.15.100:0/4278464812 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa1a406f8c0 0x7fa1a415c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.580+0000 7fa1a9400640 1 -- 172.21.15.100:0/4278464812 >> 172.21.15.100:0/4278464812 conn(0x7fa1a40b5200 msgr2=0x7fa1a40ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:42.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.580+0000 7fa1a9400640 1 -- 172.21.15.100:0/4278464812 shutdown_connections 2024-04-23T14:33:42.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.580+0000 7fa1a9400640 1 -- 172.21.15.100:0/4278464812 wait complete. 2024-04-23T14:33:42.606 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-23T14:33:42.606 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:33:42.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.740+0000 7f28d2400640 1 Processor -- start 2024-04-23T14:33:42.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.740+0000 7f28d2400640 1 -- start start 2024-04-23T14:33:42.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.744+0000 7f28d2400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28cc0ad870 0x7f28cc0b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:42.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.744+0000 7f28d2400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f28cc160de0 0x7f28cc06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:42.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.744+0000 7f28d2400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28cc160320 con 0x7f28cc15c470 2024-04-23T14:33:42.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.744+0000 7f28d2400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f28cc0cc7e0 con 0x7f28cc160de0 2024-04-23T14:33:42.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.744+0000 7f28d2400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f28cc1605e0 con 0x7f28cc0ad870 2024-04-23T14:33:42.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.744+0000 7f28cbe00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f28cc160de0 0x7f28cc06bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:42.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.744+0000 7f28cb400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f28cc15c470 0x7f28cc0ad160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57006/0 (socket says 172.21.15.100:57006) 2024-04-23T14:33:42.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.744+0000 7f28cb400640 1 -- 172.21.15.100:0/1060604668 learned_addr learned my addr 172.21.15.100:0/1060604668 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:42.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.744+0000 7f28c2a00640 1 --2- 172.21.15.100:0/1060604668 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28cc0ad870 0x7f28cc0b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:42.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.744+0000 7f28cbe00640 1 -- 172.21.15.100:0/1060604668 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28cc0ad870 msgr2=0x7f28cc0b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:42.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.744+0000 7f28cbe00640 1 --2- 172.21.15.100:0/1060604668 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28cc0ad870 0x7f28cc0b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.744+0000 7f28cbe00640 1 -- 172.21.15.100:0/1060604668 >> v1:172.21.15.100:6789/0 conn(0x7f28cc15c470 legacy=0x7f28cc0ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:42.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.744+0000 7f28cbe00640 1 -- 172.21.15.100:0/1060604668 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28cc1608c0 con 0x7f28cc160de0 2024-04-23T14:33:42.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.744+0000 7f28cbe00640 1 --2- 172.21.15.100:0/1060604668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28cc160de0 0x7f28cc06bab0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f28c40097c0 tx=0x7f28c401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484c4072eb06cb00 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:42.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.748+0000 7f28caa00640 1 -- 172.21.15.100:0/1060604668 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c40b0070 con 0x7f28cc160de0 2024-04-23T14:33:42.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.748+0000 7f28caa00640 1 -- 172.21.15.100:0/1060604668 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28c40ab600 con 0x7f28cc160de0 2024-04-23T14:33:42.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.748+0000 7f28caa00640 1 -- 172.21.15.100:0/1060604668 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c40ab9d0 con 0x7f28cc160de0 2024-04-23T14:33:42.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.748+0000 7f28d2400640 1 -- 172.21.15.100:0/1060604668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28cc160de0 msgr2=0x7f28cc06bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:42.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.748+0000 7f28d2400640 1 --2- 172.21.15.100:0/1060604668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28cc160de0 0x7f28cc06bab0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f28c40097c0 tx=0x7f28c401c410 comp rx=0 tx=0).stop 2024-04-23T14:33:42.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.748+0000 7f28d2400640 1 -- 172.21.15.100:0/1060604668 shutdown_connections 2024-04-23T14:33:42.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.748+0000 7f28d2400640 1 --2- 172.21.15.100:0/1060604668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28cc160de0 0x7f28cc06bab0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.748+0000 7f28d2400640 1 --2- 172.21.15.100:0/1060604668 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28cc0ad870 0x7f28cc0b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.748+0000 7f28d2400640 1 -- 172.21.15.100:0/1060604668 >> 172.21.15.100:0/1060604668 conn(0x7f28cc0b1e80 msgr2=0x7f28cc0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:42.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.748+0000 7f28d2400640 1 -- 172.21.15.100:0/1060604668 shutdown_connections 2024-04-23T14:33:42.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.748+0000 7f28d2400640 1 -- 172.21.15.100:0/1060604668 wait complete. 2024-04-23T14:33:42.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.748+0000 7f28d2400640 1 Processor -- start 2024-04-23T14:33:42.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.748+0000 7f28d2400640 1 -- start start 2024-04-23T14:33:42.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.748+0000 7f28d2400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28cc0ad870 0x7f28cc1550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:42.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.748+0000 7f28d2400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28cc15c470 0x7f28cc157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:42.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.748+0000 7f28d2400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28cc160de0 0x7f28cc157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:42.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.748+0000 7f28d2400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc15ca60 con 0x7f28cc0ad870 2024-04-23T14:33:42.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.748+0000 7f28d2400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc0cc580 con 0x7f28cc15c470 2024-04-23T14:33:42.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.748+0000 7f28d2400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f28cc169540 con 0x7f28cc160de0 2024-04-23T14:33:42.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.748+0000 7f28cb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28cc0ad870 0x7f28cc1550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.748+0000 7f28cb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28cc0ad870 0x7f28cc1550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58216/0 (socket says 172.21.15.100:58216) 2024-04-23T14:33:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.748+0000 7f28cb400640 1 -- 172.21.15.100:0/2903536007 learned_addr learned my addr 172.21.15.100:0/2903536007 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.752+0000 7f28cbe00640 1 --2- 172.21.15.100:0/2903536007 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28cc160de0 0x7f28cc157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.752+0000 7f28c2a00640 1 --2- 172.21.15.100:0/2903536007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28cc15c470 0x7f28cc157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.752+0000 7f28cb400640 1 -- 172.21.15.100:0/2903536007 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28cc160de0 msgr2=0x7f28cc157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.752+0000 7f28cb400640 1 --2- 172.21.15.100:0/2903536007 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28cc160de0 0x7f28cc157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.752+0000 7f28cb400640 1 -- 172.21.15.100:0/2903536007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28cc15c470 msgr2=0x7f28cc157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.752+0000 7f28cb400640 1 --2- 172.21.15.100:0/2903536007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28cc15c470 0x7f28cc157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.752+0000 7f28cb400640 1 -- 172.21.15.100:0/2903536007 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28cc06e2d0 con 0x7f28cc0ad870 2024-04-23T14:33:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.752+0000 7f28c2a00640 1 --2- 172.21.15.100:0/2903536007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28cc15c470 0x7f28cc157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:42.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.752+0000 7f28cb400640 1 --2- 172.21.15.100:0/2903536007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28cc0ad870 0x7f28cc1550a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00cb40 tx=0x7f28bc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:42.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.752+0000 7f28c3e00640 1 -- 172.21.15.100:0/2903536007 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc017070 con 0x7f28cc0ad870 2024-04-23T14:33:42.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.752+0000 7f28c3e00640 1 -- 172.21.15.100:0/2903536007 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28bc00aca0 con 0x7f28cc0ad870 2024-04-23T14:33:42.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.752+0000 7f28c3e00640 1 -- 172.21.15.100:0/2903536007 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc012620 con 0x7f28cc0ad870 2024-04-23T14:33:42.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.752+0000 7f28d2400640 1 -- 172.21.15.100:0/2903536007 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28cc1558d0 con 0x7f28cc0ad870 2024-04-23T14:33:42.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.752+0000 7f28d2400640 1 -- 172.21.15.100:0/2903536007 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28cc06e570 con 0x7f28cc0ad870 2024-04-23T14:33:42.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.752+0000 7f28d2400640 1 -- 172.21.15.100:0/2903536007 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28cc068ad0 con 0x7f28cc0ad870 2024-04-23T14:33:42.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.752+0000 7f28c3e00640 1 -- 172.21.15.100:0/2903536007 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f28bc01a070 con 0x7f28cc0ad870 2024-04-23T14:33:42.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.752+0000 7f28c3e00640 1 --2- 172.21.15.100:0/2903536007 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f28a8068380 0x7f28a806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:42.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.752+0000 7f28c3e00640 1 -- 172.21.15.100:0/2903536007 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f28bc086970 con 0x7f28cc0ad870 2024-04-23T14:33:42.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.752+0000 7f28c2a00640 1 --2- 172.21.15.100:0/2903536007 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f28a8068380 0x7f28a806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:42.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.756+0000 7f28c2a00640 1 --2- 172.21.15.100:0/2903536007 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f28a8068380 0x7f28a806a840 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f28cc06e870 tx=0x7f28b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:42.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.760+0000 7f28c3e00640 1 -- 172.21.15.100:0/2903536007 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f28bc056620 con 0x7f28cc0ad870 2024-04-23T14:33:42.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.948+0000 7f28d2400640 1 -- 172.21.15.100:0/2903536007 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28cc145da0 con 0x7f28cc0ad870 2024-04-23T14:33:42.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.948+0000 7f28c3e00640 1 -- 172.21.15.100:0/2903536007 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7f28bc05a920 con 0x7f28cc0ad870 2024-04-23T14:33:42.953 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:42.953 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2601071793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2601071793},{"type":"v1","addr":"172.21.15.100:6837","nonce":2601071793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2930424296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2930424296},{"type":"v1","addr":"172.21.15.151:6833","nonce":2930424296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1406976216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1406976216},{"type":"v1","addr":"172.21.15.151:6835","nonce":1406976216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:33:38.928245+0000","modified":"2024-04-23T14:33:41.955618+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5466":{"gid":5466,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/2062210445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2062210445},{"type":"v1","addr":"172.21.15.100:6836","nonce":2062210445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5466,"qdb_cluster":[5466]},"id":2}]} 2024-04-23T14:33:42.953 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 18 2024-04-23T14:33:42.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.952+0000 7f28d2400640 1 -- 172.21.15.100:0/2903536007 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f28a8068380 msgr2=0x7f28a806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:42.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.952+0000 7f28d2400640 1 --2- 172.21.15.100:0/2903536007 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f28a8068380 0x7f28a806a840 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f28cc06e870 tx=0x7f28b4008040 comp rx=0 tx=0).stop 2024-04-23T14:33:42.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.952+0000 7f28d2400640 1 -- 172.21.15.100:0/2903536007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28cc0ad870 msgr2=0x7f28cc1550a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:42.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.952+0000 7f28d2400640 1 --2- 172.21.15.100:0/2903536007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28cc0ad870 0x7f28cc1550a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00cb40 tx=0x7f28bc00a060 comp rx=0 tx=0).stop 2024-04-23T14:33:42.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.952+0000 7f28d2400640 1 -- 172.21.15.100:0/2903536007 shutdown_connections 2024-04-23T14:33:42.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.952+0000 7f28d2400640 1 --2- 172.21.15.100:0/2903536007 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f28a8068380 0x7f28a806a840 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.952+0000 7f28d2400640 1 --2- 172.21.15.100:0/2903536007 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28cc160de0 0x7f28cc157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.952+0000 7f28d2400640 1 --2- 172.21.15.100:0/2903536007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28cc15c470 0x7f28cc157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.952+0000 7f28d2400640 1 --2- 172.21.15.100:0/2903536007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28cc0ad870 0x7f28cc1550a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.952+0000 7f28d2400640 1 -- 172.21.15.100:0/2903536007 >> 172.21.15.100:0/2903536007 conn(0x7f28cc0b1e80 msgr2=0x7f28cc0afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:42.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.952+0000 7f28d2400640 1 -- 172.21.15.100:0/2903536007 shutdown_connections 2024-04-23T14:33:42.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:42.952+0000 7f28d2400640 1 -- 172.21.15.100:0/2903536007 wait complete. 2024-04-23T14:33:42.975 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:43.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.104+0000 7f2431400640 1 Processor -- start 2024-04-23T14:33:43.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.104+0000 7f2431400640 1 -- start start 2024-04-23T14:33:43.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.104+0000 7f2431400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f242c073610 0x7f242c0adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.104+0000 7f2431400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f242c0af080 con 0x7f242c0b2a10 2024-04-23T14:33:43.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.104+0000 7f2431400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f242c0afbc0 con 0x7f242c15b640 2024-04-23T14:33:43.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.104+0000 7f2431400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f242c15ba20 con 0x7f242c073610 2024-04-23T14:33:43.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.108+0000 7f2421e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f242c0b2a10 0x7f242c06df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57010/0 (socket says 172.21.15.100:57010) 2024-04-23T14:33:43.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.108+0000 7f2421e00640 1 -- 172.21.15.100:0/4239578577 learned_addr learned my addr 172.21.15.100:0/4239578577 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:43.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.108+0000 7f242b200640 1 --2- 172.21.15.100:0/4239578577 >> v2:172.21.15.151:3300/0 conn(0x7f242c073610 0x7f242c0adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.108+0000 7f2429e00640 1 -- 172.21.15.100:0/4239578577 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1899205667 0 0) 0x7f242c0af080 con 0x7f242c0b2a10 2024-04-23T14:33:43.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.108+0000 7f2429e00640 1 -- 172.21.15.100:0/4239578577 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2410003530 con 0x7f242c0b2a10 2024-04-23T14:33:43.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.108+0000 7f242b200640 1 -- 172.21.15.100:0/4239578577 >> v1:172.21.15.151:6789/0 conn(0x7f242c15b640 legacy=0x7f242c072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.108+0000 7f242b200640 1 -- 172.21.15.100:0/4239578577 >> v1:172.21.15.100:6789/0 conn(0x7f242c0b2a10 legacy=0x7f242c06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.108+0000 7f2429e00640 1 -- 172.21.15.100:0/4239578577 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3688321608 0 0) 0x7f2410003530 con 0x7f242c0b2a10 2024-04-23T14:33:43.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.108+0000 7f242b200640 1 -- 172.21.15.100:0/4239578577 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242c0af320 con 0x7f242c073610 2024-04-23T14:33:43.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.108+0000 7f242b200640 1 --2- 172.21.15.100:0/4239578577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f242c073610 0x7f242c0adfc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f24240097c0 tx=0x7f242401c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f9a8bf3984b6723 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:43.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f2429e00640 1 -- 172.21.15.100:0/4239578577 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24240b0070 con 0x7f242c073610 2024-04-23T14:33:43.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f2429e00640 1 -- 172.21.15.100:0/4239578577 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24240ab400 con 0x7f242c073610 2024-04-23T14:33:43.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f2429e00640 1 -- 172.21.15.100:0/4239578577 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414001680 con 0x7f242c073610 2024-04-23T14:33:43.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f2431400640 1 -- 172.21.15.100:0/4239578577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f242c073610 msgr2=0x7f242c0adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f2431400640 1 --2- 172.21.15.100:0/4239578577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f242c073610 0x7f242c0adfc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f24240097c0 tx=0x7f242401c300 comp rx=0 tx=0).stop 2024-04-23T14:33:43.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f2431400640 1 -- 172.21.15.100:0/4239578577 shutdown_connections 2024-04-23T14:33:43.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f2431400640 1 --2- 172.21.15.100:0/4239578577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f242c073610 0x7f242c0adfc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f2431400640 1 -- 172.21.15.100:0/4239578577 >> 172.21.15.100:0/4239578577 conn(0x7f242c0b1e80 msgr2=0x7f242c0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:43.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f2431400640 1 -- 172.21.15.100:0/4239578577 shutdown_connections 2024-04-23T14:33:43.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f2431400640 1 -- 172.21.15.100:0/4239578577 wait complete. 2024-04-23T14:33:43.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f2431400640 1 Processor -- start 2024-04-23T14:33:43.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f2431400640 1 -- start start 2024-04-23T14:33:43.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f2431400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f242c073610 0x7f242c148cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f2431400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f242c0b2a10 0x7f242c15ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f2431400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f242c15b640 0x7f242c15a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f2431400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f242c1691c0 con 0x7f242c0b2a10 2024-04-23T14:33:43.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f2431400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f242c06f160 con 0x7f242c15b640 2024-04-23T14:33:43.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f2431400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f242c169340 con 0x7f242c073610 2024-04-23T14:33:43.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f2421e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f242c0b2a10 0x7f242c15ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f2421e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f242c0b2a10 0x7f242c15ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58232/0 (socket says 172.21.15.100:58232) 2024-04-23T14:33:43.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f2421e00640 1 -- 172.21.15.100:0/1580182353 learned_addr learned my addr 172.21.15.100:0/1580182353 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:43.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f242b200640 1 --2- 172.21.15.100:0/1580182353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f242c15b640 0x7f242c15a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.112+0000 7f242a800640 1 --2- 172.21.15.100:0/1580182353 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f242c073610 0x7f242c148cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.116+0000 7f2421e00640 1 -- 172.21.15.100:0/1580182353 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f242c073610 msgr2=0x7f242c148cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.116+0000 7f2421e00640 1 --2- 172.21.15.100:0/1580182353 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f242c073610 0x7f242c148cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.116+0000 7f2421e00640 1 -- 172.21.15.100:0/1580182353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f242c15b640 msgr2=0x7f242c15a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.116+0000 7f2421e00640 1 --2- 172.21.15.100:0/1580182353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f242c15b640 0x7f242c15a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.116+0000 7f2421e00640 1 -- 172.21.15.100:0/1580182353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242c06c570 con 0x7f242c0b2a10 2024-04-23T14:33:43.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.116+0000 7f2421e00640 1 --2- 172.21.15.100:0/1580182353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f242c0b2a10 0x7f242c15ccb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f241400c700 tx=0x7f241400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:43.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.116+0000 7f2423400640 1 -- 172.21.15.100:0/1580182353 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414017070 con 0x7f242c0b2a10 2024-04-23T14:33:43.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.116+0000 7f2423400640 1 -- 172.21.15.100:0/1580182353 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2414005ae0 con 0x7f242c0b2a10 2024-04-23T14:33:43.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.116+0000 7f2423400640 1 -- 172.21.15.100:0/1580182353 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414007070 con 0x7f242c0b2a10 2024-04-23T14:33:43.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.116+0000 7f2431400640 1 -- 172.21.15.100:0/1580182353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f242c15df30 con 0x7f242c0b2a10 2024-04-23T14:33:43.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.116+0000 7f2431400640 1 -- 172.21.15.100:0/1580182353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f242c15d690 con 0x7f242c0b2a10 2024-04-23T14:33:43.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.116+0000 7f2431400640 1 -- 172.21.15.100:0/1580182353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f242c068ad0 con 0x7f242c0b2a10 2024-04-23T14:33:43.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.116+0000 7f2423400640 1 -- 172.21.15.100:0/1580182353 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f241401a070 con 0x7f242c0b2a10 2024-04-23T14:33:43.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.116+0000 7f2423400640 1 --2- 172.21.15.100:0/1580182353 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f24080683b0 0x7f240806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.116+0000 7f2423400640 1 -- 172.21.15.100:0/1580182353 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f2414085ee0 con 0x7f242c0b2a10 2024-04-23T14:33:43.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.116+0000 7f242a800640 1 --2- 172.21.15.100:0/1580182353 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f24080683b0 0x7f240806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.120+0000 7f242a800640 1 --2- 172.21.15.100:0/1580182353 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f24080683b0 0x7f240806a870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f241c0012f0 tx=0x7f241c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:43.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.120+0000 7f2423400640 1 -- 172.21.15.100:0/1580182353 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2414055cb0 con 0x7f242c0b2a10 2024-04-23T14:33:43.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.280+0000 7f2431400640 1 -- 172.21.15.100:0/1580182353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f242c166f20 con 0x7f242c0b2a10 2024-04-23T14:33:43.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.280+0000 7f2423400640 1 -- 172.21.15.100:0/1580182353 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16954 (secure 0 0 0) 0x7f2414059fb0 con 0x7f242c0b2a10 2024-04-23T14:33:43.285 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:43.286 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":38,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:33:40.548762+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:33:36.592437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-23T14:33:37.882390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:43.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.284+0000 7f2431400640 1 -- 172.21.15.100:0/1580182353 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f24080683b0 msgr2=0x7f240806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.284+0000 7f2431400640 1 --2- 172.21.15.100:0/1580182353 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f24080683b0 0x7f240806a870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f241c0012f0 tx=0x7f241c009040 comp rx=0 tx=0).stop 2024-04-23T14:33:43.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.284+0000 7f2431400640 1 -- 172.21.15.100:0/1580182353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f242c0b2a10 msgr2=0x7f242c15ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.284+0000 7f2431400640 1 --2- 172.21.15.100:0/1580182353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f242c0b2a10 0x7f242c15ccb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f241400c700 tx=0x7f241400cbc0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.284+0000 7f2431400640 1 -- 172.21.15.100:0/1580182353 shutdown_connections 2024-04-23T14:33:43.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.284+0000 7f2431400640 1 --2- 172.21.15.100:0/1580182353 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f24080683b0 0x7f240806a870 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.284+0000 7f2431400640 1 --2- 172.21.15.100:0/1580182353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f242c15b640 0x7f242c15a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.284+0000 7f2431400640 1 --2- 172.21.15.100:0/1580182353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f242c0b2a10 0x7f242c15ccb0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.284+0000 7f2431400640 1 --2- 172.21.15.100:0/1580182353 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f242c073610 0x7f242c148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.284+0000 7f2431400640 1 -- 172.21.15.100:0/1580182353 >> 172.21.15.100:0/1580182353 conn(0x7f242c0b1e80 msgr2=0x7f242c0ae590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:43.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.284+0000 7f2431400640 1 -- 172.21.15.100:0/1580182353 shutdown_connections 2024-04-23T14:33:43.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.284+0000 7f2431400640 1 -- 172.21.15.100:0/1580182353 wait complete. 2024-04-23T14:33:43.307 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-23T14:33:43.307 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:33:43.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.436+0000 7f03cd600640 1 Processor -- start 2024-04-23T14:33:43.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.436+0000 7f03cd600640 1 -- start start 2024-04-23T14:33:43.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.440+0000 7f03cd600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f03c815c470 0x7f03c80b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.440+0000 7f03cd600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03c8160320 con 0x7f03c80ad870 2024-04-23T14:33:43.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.440+0000 7f03cd600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03c80cc7e0 con 0x7f03c8160de0 2024-04-23T14:33:43.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.440+0000 7f03cd600640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f03c81605e0 con 0x7f03c815c470 2024-04-23T14:33:43.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.440+0000 7f03c6a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f03c815c470 0x7f03c80b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.440+0000 7f03c6a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f03c815c470 0x7f03c80b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58236/0 (socket says 172.21.15.100:58236) 2024-04-23T14:33:43.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.440+0000 7f03c6a00640 1 -- 172.21.15.100:0/4181731021 learned_addr learned my addr 172.21.15.100:0/4181731021 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:43.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.440+0000 7f03c7400640 1 --1- 172.21.15.100:0/4181731021 >> v1:172.21.15.100:6789/0 conn(0x7f03c80ad870 0x7f03c806bab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57014/0 (socket says 172.21.15.100:57014) 2024-04-23T14:33:43.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.440+0000 7f03c6a00640 1 -- 172.21.15.100:0/4181731021 >> v1:172.21.15.151:6789/0 conn(0x7f03c8160de0 legacy=0x7f03c80ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:43.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.440+0000 7f03c6a00640 1 -- 172.21.15.100:0/4181731021 >> v1:172.21.15.100:6789/0 conn(0x7f03c80ad870 legacy=0x7f03c806bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.440+0000 7f03c6a00640 1 -- 172.21.15.100:0/4181731021 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c81608c0 con 0x7f03c815c470 2024-04-23T14:33:43.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.440+0000 7f03c6a00640 1 --2- 172.21.15.100:0/4181731021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03c815c470 0x7f03c80b29a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009870 tx=0x7f03b801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04fa415f8001686 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:43.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.440+0000 7f03c6000640 1 -- 172.21.15.100:0/4181731021 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b80b1070 con 0x7f03c815c470 2024-04-23T14:33:43.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.440+0000 7f03c6000640 1 -- 172.21.15.100:0/4181731021 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03b80ac600 con 0x7f03c815c470 2024-04-23T14:33:43.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.440+0000 7f03c6000640 1 -- 172.21.15.100:0/4181731021 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b80ac9d0 con 0x7f03c815c470 2024-04-23T14:33:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.444+0000 7f03cd600640 1 -- 172.21.15.100:0/4181731021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03c815c470 msgr2=0x7f03c80b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.444+0000 7f03cd600640 1 --2- 172.21.15.100:0/4181731021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03c815c470 0x7f03c80b29a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009870 tx=0x7f03b801c490 comp rx=0 tx=0).stop 2024-04-23T14:33:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.444+0000 7f03cd600640 1 -- 172.21.15.100:0/4181731021 shutdown_connections 2024-04-23T14:33:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.444+0000 7f03cd600640 1 --2- 172.21.15.100:0/4181731021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03c815c470 0x7f03c80b29a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.444+0000 7f03cd600640 1 -- 172.21.15.100:0/4181731021 >> 172.21.15.100:0/4181731021 conn(0x7f03c80b1e80 msgr2=0x7f03c80b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.444+0000 7f03cd600640 1 -- 172.21.15.100:0/4181731021 shutdown_connections 2024-04-23T14:33:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.444+0000 7f03cd600640 1 -- 172.21.15.100:0/4181731021 wait complete. 2024-04-23T14:33:43.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.444+0000 7f03cd600640 1 Processor -- start 2024-04-23T14:33:43.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.444+0000 7f03cd600640 1 -- start start 2024-04-23T14:33:43.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.444+0000 7f03cd600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03c80ad870 0x7f03c80998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.444+0000 7f03cd600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f03c815c470 0x7f03c815cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.444+0000 7f03cd600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03c8160de0 0x7f03c815d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.444+0000 7f03cd600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f03c815ca60 con 0x7f03c8160de0 2024-04-23T14:33:43.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.444+0000 7f03cd600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f03c80cc5c0 con 0x7f03c80ad870 2024-04-23T14:33:43.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.444+0000 7f03cd600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f03c816a550 con 0x7f03c815c470 2024-04-23T14:33:43.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.444+0000 7f03c7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03c8160de0 0x7f03c815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.444+0000 7f03c7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03c8160de0 0x7f03c815d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58242/0 (socket says 172.21.15.100:58242) 2024-04-23T14:33:43.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.444+0000 7f03c7400640 1 -- 172.21.15.100:0/834096390 learned_addr learned my addr 172.21.15.100:0/834096390 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:43.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.444+0000 7f03c6a00640 1 --2- 172.21.15.100:0/834096390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03c80ad870 0x7f03c80998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.444+0000 7f03bfe00640 1 --2- 172.21.15.100:0/834096390 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f03c815c470 0x7f03c815cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.448+0000 7f03c7400640 1 -- 172.21.15.100:0/834096390 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f03c815c470 msgr2=0x7f03c815cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.448+0000 7f03c7400640 1 --2- 172.21.15.100:0/834096390 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f03c815c470 0x7f03c815cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.448+0000 7f03c7400640 1 -- 172.21.15.100:0/834096390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03c80ad870 msgr2=0x7f03c80998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.448+0000 7f03c7400640 1 --2- 172.21.15.100:0/834096390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03c80ad870 0x7f03c80998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.448+0000 7f03c7400640 1 -- 172.21.15.100:0/834096390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c8151ba0 con 0x7f03c8160de0 2024-04-23T14:33:43.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.448+0000 7f03c7400640 1 --2- 172.21.15.100:0/834096390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03c8160de0 0x7f03c815d120 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f03c000c760 tx=0x7f03c000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:43.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.448+0000 7f03bea00640 1 -- 172.21.15.100:0/834096390 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0017070 con 0x7f03c8160de0 2024-04-23T14:33:43.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.448+0000 7f03bea00640 1 -- 172.21.15.100:0/834096390 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c000a640 con 0x7f03c8160de0 2024-04-23T14:33:43.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.448+0000 7f03cd600640 1 -- 172.21.15.100:0/834096390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c8099de0 con 0x7f03c8160de0 2024-04-23T14:33:43.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.448+0000 7f03bea00640 1 -- 172.21.15.100:0/834096390 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c000a9f0 con 0x7f03c8160de0 2024-04-23T14:33:43.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.448+0000 7f03cd600640 1 -- 172.21.15.100:0/834096390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c815da30 con 0x7f03c8160de0 2024-04-23T14:33:43.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.448+0000 7f03cd600640 1 -- 172.21.15.100:0/834096390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03c8068ad0 con 0x7f03c8160de0 2024-04-23T14:33:43.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.448+0000 7f03bea00640 1 -- 172.21.15.100:0/834096390 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f03c001a070 con 0x7f03c8160de0 2024-04-23T14:33:43.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.448+0000 7f03bea00640 1 --2- 172.21.15.100:0/834096390 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f03a4068380 0x7f03a406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.448+0000 7f03bea00640 1 -- 172.21.15.100:0/834096390 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f03c00860d0 con 0x7f03c8160de0 2024-04-23T14:33:43.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.448+0000 7f03c6a00640 1 --2- 172.21.15.100:0/834096390 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f03a4068380 0x7f03a406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.452+0000 7f03c6a00640 1 --2- 172.21.15.100:0/834096390 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f03a4068380 0x7f03a406a840 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f03b80ae000 tx=0x7f03b80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:43.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.452+0000 7f03bea00640 1 -- 172.21.15.100:0/834096390 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f03c0055d80 con 0x7f03c8160de0 2024-04-23T14:33:43.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.612+0000 7f03cd600640 1 -- 172.21.15.100:0/834096390 --> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f03c806fea0 con 0x7f03a4068380 2024-04-23T14:33:43.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.616+0000 7f03bea00640 1 -- 172.21.15.100:0/834096390 <== mgr.4101 v2:172.21.15.100:6832/2859853957 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251957 (secure 0 0 0) 0x7f03c806fea0 con 0x7f03a4068380 2024-04-23T14:33:43.624 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:43.626 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-04-23T14:33:43.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.620+0000 7f03cd600640 1 -- 172.21.15.100:0/834096390 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f03a4068380 msgr2=0x7f03a406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.620+0000 7f03cd600640 1 --2- 172.21.15.100:0/834096390 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f03a4068380 0x7f03a406a840 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f03b80ae000 tx=0x7f03b80a7000 comp rx=0 tx=0).stop 2024-04-23T14:33:43.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.620+0000 7f03cd600640 1 -- 172.21.15.100:0/834096390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03c8160de0 msgr2=0x7f03c815d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.620+0000 7f03cd600640 1 --2- 172.21.15.100:0/834096390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03c8160de0 0x7f03c815d120 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f03c000c760 tx=0x7f03c000cc20 comp rx=0 tx=0).stop 2024-04-23T14:33:43.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.620+0000 7f03cd600640 1 -- 172.21.15.100:0/834096390 shutdown_connections 2024-04-23T14:33:43.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.620+0000 7f03cd600640 1 --2- 172.21.15.100:0/834096390 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f03a4068380 0x7f03a406a840 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.620+0000 7f03cd600640 1 --2- 172.21.15.100:0/834096390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03c8160de0 0x7f03c815d120 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.620+0000 7f03cd600640 1 --2- 172.21.15.100:0/834096390 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f03c815c470 0x7f03c815cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.620+0000 7f03cd600640 1 --2- 172.21.15.100:0/834096390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03c80ad870 0x7f03c80998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.620+0000 7f03cd600640 1 -- 172.21.15.100:0/834096390 >> 172.21.15.100:0/834096390 conn(0x7f03c80b1e80 msgr2=0x7f03c806e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:43.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.620+0000 7f03cd600640 1 -- 172.21.15.100:0/834096390 shutdown_connections 2024-04-23T14:33:43.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.620+0000 7f03cd600640 1 -- 172.21.15.100:0/834096390 wait complete. 2024-04-23T14:33:43.645 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-04-23T14:33:42.754353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224936,"kb_used_data":4600,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749784,"statfs":{"total":773094113280,"available":772863778816,"internally_reserved":0,"allocated":4710400,"data_stored":2679012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.225576"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.367470+0000","last_change":"2024-04-23T14:33:39.580767+0000","last_active":"2024-04-23T14:33:41.367470+0000","last_peered":"2024-04-23T14:33:41.367470+0000","last_clean":"2024-04-23T14:33:41.367470+0000","last_became_active":"2024-04-23T14:33:39.580526+0000","last_became_peered":"2024-04-23T14:33:39.580526+0000","last_unstale":"2024-04-23T14:33:41.367470+0000","last_undegraded":"2024-04-23T14:33:41.367470+0000","last_fullsized":"2024-04-23T14:33:41.367470+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:38:48.128956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.367385+0000","last_change":"2024-04-23T14:33:39.583938+0000","last_active":"2024-04-23T14:33:41.367385+0000","last_peered":"2024-04-23T14:33:41.367385+0000","last_clean":"2024-04-23T14:33:41.367385+0000","last_became_active":"2024-04-23T14:33:39.583603+0000","last_became_peered":"2024-04-23T14:33:39.583603+0000","last_unstale":"2024-04-23T14:33:41.367385+0000","last_undegraded":"2024-04-23T14:33:41.367385+0000","last_fullsized":"2024-04-23T14:33:41.367385+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:17:24.890164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555363+0000","last_change":"2024-04-23T14:33:39.561485+0000","last_active":"2024-04-23T14:33:40.555363+0000","last_peered":"2024-04-23T14:33:40.555363+0000","last_clean":"2024-04-23T14:33:40.555363+0000","last_became_active":"2024-04-23T14:33:39.561215+0000","last_became_peered":"2024-04-23T14:33:39.561215+0000","last_unstale":"2024-04-23T14:33:40.555363+0000","last_undegraded":"2024-04-23T14:33:40.555363+0000","last_fullsized":"2024-04-23T14:33:40.555363+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:10:16.577796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.562555+0000","last_change":"2024-04-23T14:33:39.586107+0000","last_active":"2024-04-23T14:33:41.562555+0000","last_peered":"2024-04-23T14:33:41.562555+0000","last_clean":"2024-04-23T14:33:41.562555+0000","last_became_active":"2024-04-23T14:33:39.585880+0000","last_became_peered":"2024-04-23T14:33:39.585880+0000","last_unstale":"2024-04-23T14:33:41.562555+0000","last_undegraded":"2024-04-23T14:33:41.562555+0000","last_fullsized":"2024-04-23T14:33:41.562555+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:32:42.672129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.562377+0000","last_change":"2024-04-23T14:33:39.574966+0000","last_active":"2024-04-23T14:33:41.562377+0000","last_peered":"2024-04-23T14:33:41.562377+0000","last_clean":"2024-04-23T14:33:41.562377+0000","last_became_active":"2024-04-23T14:33:39.574684+0000","last_became_peered":"2024-04-23T14:33:39.574684+0000","last_unstale":"2024-04-23T14:33:41.562377+0000","last_undegraded":"2024-04-23T14:33:41.562377+0000","last_fullsized":"2024-04-23T14:33:41.562377+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:30.908259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665078+0000","last_change":"2024-04-23T14:33:39.586533+0000","last_active":"2024-04-23T14:33:41.665078+0000","last_peered":"2024-04-23T14:33:41.665078+0000","last_clean":"2024-04-23T14:33:41.665078+0000","last_became_active":"2024-04-23T14:33:39.584448+0000","last_became_peered":"2024-04-23T14:33:39.584448+0000","last_unstale":"2024-04-23T14:33:41.665078+0000","last_undegraded":"2024-04-23T14:33:41.665078+0000","last_fullsized":"2024-04-23T14:33:41.665078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:10:16.577796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599394+0000","last_change":"2024-04-23T14:33:39.588547+0000","last_active":"2024-04-23T14:33:40.599394+0000","last_peered":"2024-04-23T14:33:40.599394+0000","last_clean":"2024-04-23T14:33:40.599394+0000","last_became_active":"2024-04-23T14:33:39.588220+0000","last_became_peered":"2024-04-23T14:33:39.588220+0000","last_unstale":"2024-04-23T14:33:40.599394+0000","last_undegraded":"2024-04-23T14:33:40.599394+0000","last_fullsized":"2024-04-23T14:33:40.599394+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:15:25.124076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.366929+0000","last_change":"2024-04-23T14:33:39.580516+0000","last_active":"2024-04-23T14:33:41.366929+0000","last_peered":"2024-04-23T14:33:41.366929+0000","last_clean":"2024-04-23T14:33:41.366929+0000","last_became_active":"2024-04-23T14:33:39.580215+0000","last_became_peered":"2024-04-23T14:33:39.580215+0000","last_unstale":"2024-04-23T14:33:41.366929+0000","last_undegraded":"2024-04-23T14:33:41.366929+0000","last_fullsized":"2024-04-23T14:33:41.366929+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:07:21.114361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555479+0000","last_change":"2024-04-23T14:33:39.570882+0000","last_active":"2024-04-23T14:33:40.555479+0000","last_peered":"2024-04-23T14:33:40.555479+0000","last_clean":"2024-04-23T14:33:40.555479+0000","last_became_active":"2024-04-23T14:33:39.570610+0000","last_became_peered":"2024-04-23T14:33:39.570610+0000","last_unstale":"2024-04-23T14:33:40.555479+0000","last_undegraded":"2024-04-23T14:33:40.555479+0000","last_fullsized":"2024-04-23T14:33:40.555479+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:11:49.792303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599663+0000","last_change":"2024-04-23T14:33:39.589859+0000","last_active":"2024-04-23T14:33:40.599663+0000","last_peered":"2024-04-23T14:33:40.599663+0000","last_clean":"2024-04-23T14:33:40.599663+0000","last_became_active":"2024-04-23T14:33:39.589603+0000","last_became_peered":"2024-04-23T14:33:39.589603+0000","last_unstale":"2024-04-23T14:33:40.599663+0000","last_undegraded":"2024-04-23T14:33:40.599663+0000","last_fullsized":"2024-04-23T14:33:40.599663+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:32:29.944091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599519+0000","last_change":"2024-04-23T14:33:39.582574+0000","last_active":"2024-04-23T14:33:40.599519+0000","last_peered":"2024-04-23T14:33:40.599519+0000","last_clean":"2024-04-23T14:33:40.599519+0000","last_became_active":"2024-04-23T14:33:39.576219+0000","last_became_peered":"2024-04-23T14:33:39.576219+0000","last_unstale":"2024-04-23T14:33:40.599519+0000","last_undegraded":"2024-04-23T14:33:40.599519+0000","last_fullsized":"2024-04-23T14:33:40.599519+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:49:09.617492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555191+0000","last_change":"2024-04-23T14:33:39.578969+0000","last_active":"2024-04-23T14:33:40.555191+0000","last_peered":"2024-04-23T14:33:40.555191+0000","last_clean":"2024-04-23T14:33:40.555191+0000","last_became_active":"2024-04-23T14:33:39.578666+0000","last_became_peered":"2024-04-23T14:33:39.578666+0000","last_unstale":"2024-04-23T14:33:40.555191+0000","last_undegraded":"2024-04-23T14:33:40.555191+0000","last_fullsized":"2024-04-23T14:33:40.555191+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:04:54.194423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599239+0000","last_change":"2024-04-23T14:33:39.590435+0000","last_active":"2024-04-23T14:33:40.599239+0000","last_peered":"2024-04-23T14:33:40.599239+0000","last_clean":"2024-04-23T14:33:40.599239+0000","last_became_active":"2024-04-23T14:33:39.590169+0000","last_became_peered":"2024-04-23T14:33:39.590169+0000","last_unstale":"2024-04-23T14:33:40.599239+0000","last_undegraded":"2024-04-23T14:33:40.599239+0000","last_fullsized":"2024-04-23T14:33:40.599239+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:35:49.748984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665105+0000","last_change":"2024-04-23T14:33:39.569496+0000","last_active":"2024-04-23T14:33:41.665105+0000","last_peered":"2024-04-23T14:33:41.665105+0000","last_clean":"2024-04-23T14:33:41.665105+0000","last_became_active":"2024-04-23T14:33:39.569197+0000","last_became_peered":"2024-04-23T14:33:39.569197+0000","last_unstale":"2024-04-23T14:33:41.665105+0000","last_undegraded":"2024-04-23T14:33:41.665105+0000","last_fullsized":"2024-04-23T14:33:41.665105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:05:24.295743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599237+0000","last_change":"2024-04-23T14:33:39.587019+0000","last_active":"2024-04-23T14:33:40.599237+0000","last_peered":"2024-04-23T14:33:40.599237+0000","last_clean":"2024-04-23T14:33:40.599237+0000","last_became_active":"2024-04-23T14:33:39.586701+0000","last_became_peered":"2024-04-23T14:33:39.586701+0000","last_unstale":"2024-04-23T14:33:40.599237+0000","last_undegraded":"2024-04-23T14:33:40.599237+0000","last_fullsized":"2024-04-23T14:33:40.599237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:04.758788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.554967+0000","last_change":"2024-04-23T14:33:39.588110+0000","last_active":"2024-04-23T14:33:40.554967+0000","last_peered":"2024-04-23T14:33:40.554967+0000","last_clean":"2024-04-23T14:33:40.554967+0000","last_became_active":"2024-04-23T14:33:39.587400+0000","last_became_peered":"2024-04-23T14:33:39.587400+0000","last_unstale":"2024-04-23T14:33:40.554967+0000","last_undegraded":"2024-04-23T14:33:40.554967+0000","last_fullsized":"2024-04-23T14:33:40.554967+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:10:16.577796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555946+0000","last_change":"2024-04-23T14:33:39.583576+0000","last_active":"2024-04-23T14:33:40.555946+0000","last_peered":"2024-04-23T14:33:40.555946+0000","last_clean":"2024-04-23T14:33:40.555946+0000","last_became_active":"2024-04-23T14:33:39.576639+0000","last_became_peered":"2024-04-23T14:33:39.576639+0000","last_unstale":"2024-04-23T14:33:40.555946+0000","last_undegraded":"2024-04-23T14:33:40.555946+0000","last_fullsized":"2024-04-23T14:33:40.555946+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:15:25.124076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555171+0000","last_change":"2024-04-23T14:33:39.577460+0000","last_active":"2024-04-23T14:33:40.555171+0000","last_peered":"2024-04-23T14:33:40.555171+0000","last_clean":"2024-04-23T14:33:40.555171+0000","last_became_active":"2024-04-23T14:33:39.576980+0000","last_became_peered":"2024-04-23T14:33:39.576980+0000","last_unstale":"2024-04-23T14:33:40.555171+0000","last_undegraded":"2024-04-23T14:33:40.555171+0000","last_fullsized":"2024-04-23T14:33:40.555171+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:21:29.390457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.562645+0000","last_change":"2024-04-23T14:33:39.576950+0000","last_active":"2024-04-23T14:33:41.562645+0000","last_peered":"2024-04-23T14:33:41.562645+0000","last_clean":"2024-04-23T14:33:41.562645+0000","last_became_active":"2024-04-23T14:33:39.576398+0000","last_became_peered":"2024-04-23T14:33:39.576398+0000","last_unstale":"2024-04-23T14:33:41.562645+0000","last_undegraded":"2024-04-23T14:33:41.562645+0000","last_fullsized":"2024-04-23T14:33:41.562645+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:11:42.727379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555028+0000","last_change":"2024-04-23T14:33:39.585038+0000","last_active":"2024-04-23T14:33:40.555028+0000","last_peered":"2024-04-23T14:33:40.555028+0000","last_clean":"2024-04-23T14:33:40.555028+0000","last_became_active":"2024-04-23T14:33:39.584726+0000","last_became_peered":"2024-04-23T14:33:39.584726+0000","last_unstale":"2024-04-23T14:33:40.555028+0000","last_undegraded":"2024-04-23T14:33:40.555028+0000","last_fullsized":"2024-04-23T14:33:40.555028+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:06:14.688979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599453+0000","last_change":"2024-04-23T14:33:39.565846+0000","last_active":"2024-04-23T14:33:40.599453+0000","last_peered":"2024-04-23T14:33:40.599453+0000","last_clean":"2024-04-23T14:33:40.599453+0000","last_became_active":"2024-04-23T14:33:39.565515+0000","last_became_peered":"2024-04-23T14:33:39.565515+0000","last_unstale":"2024-04-23T14:33:40.599453+0000","last_undegraded":"2024-04-23T14:33:40.599453+0000","last_fullsized":"2024-04-23T14:33:40.599453+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:35.592096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665416+0000","last_change":"2024-04-23T14:33:39.579363+0000","last_active":"2024-04-23T14:33:41.665416+0000","last_peered":"2024-04-23T14:33:41.665416+0000","last_clean":"2024-04-23T14:33:41.665416+0000","last_became_active":"2024-04-23T14:33:39.579078+0000","last_became_peered":"2024-04-23T14:33:39.579078+0000","last_unstale":"2024-04-23T14:33:41.665416+0000","last_undegraded":"2024-04-23T14:33:41.665416+0000","last_fullsized":"2024-04-23T14:33:41.665416+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:09:39.794441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555589+0000","last_change":"2024-04-23T14:33:39.588147+0000","last_active":"2024-04-23T14:33:40.555589+0000","last_peered":"2024-04-23T14:33:40.555589+0000","last_clean":"2024-04-23T14:33:40.555589+0000","last_became_active":"2024-04-23T14:33:39.587477+0000","last_became_peered":"2024-04-23T14:33:39.587477+0000","last_unstale":"2024-04-23T14:33:40.555589+0000","last_undegraded":"2024-04-23T14:33:40.555589+0000","last_fullsized":"2024-04-23T14:33:40.555589+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:11:49.792303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.562513+0000","last_change":"2024-04-23T14:33:39.590063+0000","last_active":"2024-04-23T14:33:41.562513+0000","last_peered":"2024-04-23T14:33:41.562513+0000","last_clean":"2024-04-23T14:33:41.562513+0000","last_became_active":"2024-04-23T14:33:39.589737+0000","last_became_peered":"2024-04-23T14:33:39.589737+0000","last_unstale":"2024-04-23T14:33:41.562513+0000","last_undegraded":"2024-04-23T14:33:41.562513+0000","last_fullsized":"2024-04-23T14:33:41.562513+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:29:38.570728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555063+0000","last_change":"2024-04-23T14:33:39.575003+0000","last_active":"2024-04-23T14:33:40.555063+0000","last_peered":"2024-04-23T14:33:40.555063+0000","last_clean":"2024-04-23T14:33:40.555063+0000","last_became_active":"2024-04-23T14:33:39.574698+0000","last_became_peered":"2024-04-23T14:33:39.574698+0000","last_unstale":"2024-04-23T14:33:40.555063+0000","last_undegraded":"2024-04-23T14:33:40.555063+0000","last_fullsized":"2024-04-23T14:33:40.555063+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:05:24.295743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665037+0000","last_change":"2024-04-23T14:33:39.576893+0000","last_active":"2024-04-23T14:33:41.665037+0000","last_peered":"2024-04-23T14:33:41.665037+0000","last_clean":"2024-04-23T14:33:41.665037+0000","last_became_active":"2024-04-23T14:33:39.576643+0000","last_became_peered":"2024-04-23T14:33:39.576643+0000","last_unstale":"2024-04-23T14:33:41.665037+0000","last_undegraded":"2024-04-23T14:33:41.665037+0000","last_fullsized":"2024-04-23T14:33:41.665037+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:04:54.194423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555002+0000","last_change":"2024-04-23T14:33:39.578017+0000","last_active":"2024-04-23T14:33:40.555002+0000","last_peered":"2024-04-23T14:33:40.555002+0000","last_clean":"2024-04-23T14:33:40.555002+0000","last_became_active":"2024-04-23T14:33:39.577080+0000","last_became_peered":"2024-04-23T14:33:39.577080+0000","last_unstale":"2024-04-23T14:33:40.555002+0000","last_undegraded":"2024-04-23T14:33:40.555002+0000","last_fullsized":"2024-04-23T14:33:40.555002+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:55:01.766955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599752+0000","last_change":"2024-04-23T14:33:39.588715+0000","last_active":"2024-04-23T14:33:40.599752+0000","last_peered":"2024-04-23T14:33:40.599752+0000","last_clean":"2024-04-23T14:33:40.599752+0000","last_became_active":"2024-04-23T14:33:39.588273+0000","last_became_peered":"2024-04-23T14:33:39.588273+0000","last_unstale":"2024-04-23T14:33:40.599752+0000","last_undegraded":"2024-04-23T14:33:40.599752+0000","last_fullsized":"2024-04-23T14:33:40.599752+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:07:21.114361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.554798+0000","last_change":"2024-04-23T14:33:39.562550+0000","last_active":"2024-04-23T14:33:40.554798+0000","last_peered":"2024-04-23T14:33:40.554798+0000","last_clean":"2024-04-23T14:33:40.554798+0000","last_became_active":"2024-04-23T14:33:39.562221+0000","last_became_peered":"2024-04-23T14:33:39.562221+0000","last_unstale":"2024-04-23T14:33:40.554798+0000","last_undegraded":"2024-04-23T14:33:40.554798+0000","last_fullsized":"2024-04-23T14:33:40.554798+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:06:42.070784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.554804+0000","last_change":"2024-04-23T14:33:39.577502+0000","last_active":"2024-04-23T14:33:40.554804+0000","last_peered":"2024-04-23T14:33:40.554804+0000","last_clean":"2024-04-23T14:33:40.554804+0000","last_became_active":"2024-04-23T14:33:39.577100+0000","last_became_peered":"2024-04-23T14:33:39.577100+0000","last_unstale":"2024-04-23T14:33:40.554804+0000","last_undegraded":"2024-04-23T14:33:40.554804+0000","last_fullsized":"2024-04-23T14:33:40.554804+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:42:41.880055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555703+0000","last_change":"2024-04-23T14:33:39.588390+0000","last_active":"2024-04-23T14:33:40.555703+0000","last_peered":"2024-04-23T14:33:40.555703+0000","last_clean":"2024-04-23T14:33:40.555703+0000","last_became_active":"2024-04-23T14:33:39.587840+0000","last_became_peered":"2024-04-23T14:33:39.587840+0000","last_unstale":"2024-04-23T14:33:40.555703+0000","last_undegraded":"2024-04-23T14:33:40.555703+0000","last_fullsized":"2024-04-23T14:33:40.555703+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:51:28.301256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.664679+0000","last_change":"2024-04-23T14:33:39.577383+0000","last_active":"2024-04-23T14:33:41.664679+0000","last_peered":"2024-04-23T14:33:41.664679+0000","last_clean":"2024-04-23T14:33:41.664679+0000","last_became_active":"2024-04-23T14:33:39.576990+0000","last_became_peered":"2024-04-23T14:33:39.576990+0000","last_unstale":"2024-04-23T14:33:41.664679+0000","last_undegraded":"2024-04-23T14:33:41.664679+0000","last_fullsized":"2024-04-23T14:33:41.664679+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:09:39.794441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.556149+0000","last_change":"2024-04-23T14:33:39.579412+0000","last_active":"2024-04-23T14:33:40.556149+0000","last_peered":"2024-04-23T14:33:40.556149+0000","last_clean":"2024-04-23T14:33:40.556149+0000","last_became_active":"2024-04-23T14:33:39.579163+0000","last_became_peered":"2024-04-23T14:33:39.579163+0000","last_unstale":"2024-04-23T14:33:40.556149+0000","last_undegraded":"2024-04-23T14:33:40.556149+0000","last_fullsized":"2024-04-23T14:33:40.556149+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:30:44.259502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555156+0000","last_change":"2024-04-23T14:33:38.564377+0000","last_active":"2024-04-23T14:33:40.555156+0000","last_peered":"2024-04-23T14:33:40.555156+0000","last_clean":"2024-04-23T14:33:40.555156+0000","last_became_active":"2024-04-23T14:33:38.564062+0000","last_became_peered":"2024-04-23T14:33:38.564062+0000","last_unstale":"2024-04-23T14:33:40.555156+0000","last_undegraded":"2024-04-23T14:33:40.555156+0000","last_fullsized":"2024-04-23T14:33:40.555156+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:31.873991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.664940+0000","last_change":"2024-04-23T14:33:39.578881+0000","last_active":"2024-04-23T14:33:41.664940+0000","last_peered":"2024-04-23T14:33:41.664940+0000","last_clean":"2024-04-23T14:33:41.664940+0000","last_became_active":"2024-04-23T14:33:39.578461+0000","last_became_peered":"2024-04-23T14:33:39.578461+0000","last_unstale":"2024-04-23T14:33:41.664940+0000","last_undegraded":"2024-04-23T14:33:41.664940+0000","last_fullsized":"2024-04-23T14:33:41.664940+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:56:14.508220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555459+0000","last_change":"2024-04-23T14:33:38.545282+0000","last_active":"2024-04-23T14:33:40.555459+0000","last_peered":"2024-04-23T14:33:40.555459+0000","last_clean":"2024-04-23T14:33:40.555459+0000","last_became_active":"2024-04-23T14:33:38.544710+0000","last_became_peered":"2024-04-23T14:33:38.544710+0000","last_unstale":"2024-04-23T14:33:40.555459+0000","last_undegraded":"2024-04-23T14:33:40.555459+0000","last_fullsized":"2024-04-23T14:33:40.555459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:11:41.721234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665583+0000","last_change":"2024-04-23T14:33:39.572263+0000","last_active":"2024-04-23T14:33:41.665583+0000","last_peered":"2024-04-23T14:33:41.665583+0000","last_clean":"2024-04-23T14:33:41.665583+0000","last_became_active":"2024-04-23T14:33:39.571823+0000","last_became_peered":"2024-04-23T14:33:39.571823+0000","last_unstale":"2024-04-23T14:33:41.665583+0000","last_undegraded":"2024-04-23T14:33:41.665583+0000","last_fullsized":"2024-04-23T14:33:41.665583+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:04:54.194423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665008+0000","last_change":"2024-04-23T14:33:38.561205+0000","last_active":"2024-04-23T14:33:41.665008+0000","last_peered":"2024-04-23T14:33:41.665008+0000","last_clean":"2024-04-23T14:33:41.665008+0000","last_became_active":"2024-04-23T14:33:38.560405+0000","last_became_peered":"2024-04-23T14:33:38.560405+0000","last_unstale":"2024-04-23T14:33:41.665008+0000","last_undegraded":"2024-04-23T14:33:41.665008+0000","last_fullsized":"2024-04-23T14:33:41.665008+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:01:45.964960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.367431+0000","last_change":"2024-04-23T14:33:39.580591+0000","last_active":"2024-04-23T14:33:41.367431+0000","last_peered":"2024-04-23T14:33:41.367431+0000","last_clean":"2024-04-23T14:33:41.367431+0000","last_became_active":"2024-04-23T14:33:39.580242+0000","last_became_peered":"2024-04-23T14:33:39.580242+0000","last_unstale":"2024-04-23T14:33:41.367431+0000","last_undegraded":"2024-04-23T14:33:41.367431+0000","last_fullsized":"2024-04-23T14:33:41.367431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:59:21.654571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665426+0000","last_change":"2024-04-23T14:33:38.560326+0000","last_active":"2024-04-23T14:33:41.665426+0000","last_peered":"2024-04-23T14:33:41.665426+0000","last_clean":"2024-04-23T14:33:41.665426+0000","last_became_active":"2024-04-23T14:33:38.560067+0000","last_became_peered":"2024-04-23T14:33:38.560067+0000","last_unstale":"2024-04-23T14:33:41.665426+0000","last_undegraded":"2024-04-23T14:33:41.665426+0000","last_fullsized":"2024-04-23T14:33:41.665426+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:01:45.964960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.554987+0000","last_change":"2024-04-23T14:33:39.563470+0000","last_active":"2024-04-23T14:33:40.554987+0000","last_peered":"2024-04-23T14:33:40.554987+0000","last_clean":"2024-04-23T14:33:40.554987+0000","last_became_active":"2024-04-23T14:33:39.562136+0000","last_became_peered":"2024-04-23T14:33:39.562136+0000","last_unstale":"2024-04-23T14:33:40.554987+0000","last_undegraded":"2024-04-23T14:33:40.554987+0000","last_fullsized":"2024-04-23T14:33:40.554987+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:09:39.794441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599385+0000","last_change":"2024-04-23T14:33:38.546642+0000","last_active":"2024-04-23T14:33:40.599385+0000","last_peered":"2024-04-23T14:33:40.599385+0000","last_clean":"2024-04-23T14:33:40.599385+0000","last_became_active":"2024-04-23T14:33:38.546361+0000","last_became_peered":"2024-04-23T14:33:38.546361+0000","last_unstale":"2024-04-23T14:33:40.599385+0000","last_undegraded":"2024-04-23T14:33:40.599385+0000","last_fullsized":"2024-04-23T14:33:40.599385+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:42:40.873909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599766+0000","last_change":"2024-04-23T14:33:39.563007+0000","last_active":"2024-04-23T14:33:40.599766+0000","last_peered":"2024-04-23T14:33:40.599766+0000","last_clean":"2024-04-23T14:33:40.599766+0000","last_became_active":"2024-04-23T14:33:39.561126+0000","last_became_peered":"2024-04-23T14:33:39.561126+0000","last_unstale":"2024-04-23T14:33:40.599766+0000","last_undegraded":"2024-04-23T14:33:40.599766+0000","last_fullsized":"2024-04-23T14:33:40.599766+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:34.088037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555826+0000","last_change":"2024-04-23T14:33:38.556496+0000","last_active":"2024-04-23T14:33:40.555826+0000","last_peered":"2024-04-23T14:33:40.555826+0000","last_clean":"2024-04-23T14:33:40.555826+0000","last_became_active":"2024-04-23T14:33:38.556228+0000","last_became_peered":"2024-04-23T14:33:38.556228+0000","last_unstale":"2024-04-23T14:33:40.555826+0000","last_undegraded":"2024-04-23T14:33:40.555826+0000","last_fullsized":"2024-04-23T14:33:40.555826+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:52.527966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665492+0000","last_change":"2024-04-23T14:33:39.592397+0000","last_active":"2024-04-23T14:33:41.665492+0000","last_peered":"2024-04-23T14:33:41.665492+0000","last_clean":"2024-04-23T14:33:41.665492+0000","last_became_active":"2024-04-23T14:33:39.592221+0000","last_became_peered":"2024-04-23T14:33:39.592221+0000","last_unstale":"2024-04-23T14:33:41.665492+0000","last_undegraded":"2024-04-23T14:33:41.665492+0000","last_fullsized":"2024-04-23T14:33:41.665492+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:56:14.508220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.664643+0000","last_change":"2024-04-23T14:33:38.558760+0000","last_active":"2024-04-23T14:33:41.664643+0000","last_peered":"2024-04-23T14:33:41.664643+0000","last_clean":"2024-04-23T14:33:41.664643+0000","last_became_active":"2024-04-23T14:33:38.558390+0000","last_became_peered":"2024-04-23T14:33:38.558390+0000","last_unstale":"2024-04-23T14:33:41.664643+0000","last_undegraded":"2024-04-23T14:33:41.664643+0000","last_fullsized":"2024-04-23T14:33:41.664643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:52.527966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.366930+0000","last_change":"2024-04-23T14:33:39.587661+0000","last_active":"2024-04-23T14:33:41.366930+0000","last_peered":"2024-04-23T14:33:41.366930+0000","last_clean":"2024-04-23T14:33:41.366930+0000","last_became_active":"2024-04-23T14:33:39.587332+0000","last_became_peered":"2024-04-23T14:33:39.587332+0000","last_unstale":"2024-04-23T14:33:41.366930+0000","last_undegraded":"2024-04-23T14:33:41.366930+0000","last_fullsized":"2024-04-23T14:33:41.366930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:34.088037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665062+0000","last_change":"2024-04-23T14:33:38.558584+0000","last_active":"2024-04-23T14:33:41.665062+0000","last_peered":"2024-04-23T14:33:41.665062+0000","last_clean":"2024-04-23T14:33:41.665062+0000","last_became_active":"2024-04-23T14:33:38.558265+0000","last_became_peered":"2024-04-23T14:33:38.558265+0000","last_unstale":"2024-04-23T14:33:41.665062+0000","last_undegraded":"2024-04-23T14:33:41.665062+0000","last_fullsized":"2024-04-23T14:33:41.665062+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:29.902113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.562594+0000","last_change":"2024-04-23T14:33:39.586168+0000","last_active":"2024-04-23T14:33:41.562594+0000","last_peered":"2024-04-23T14:33:41.562594+0000","last_clean":"2024-04-23T14:33:41.562594+0000","last_became_active":"2024-04-23T14:33:39.585305+0000","last_became_peered":"2024-04-23T14:33:39.585305+0000","last_unstale":"2024-04-23T14:33:41.562594+0000","last_undegraded":"2024-04-23T14:33:41.562594+0000","last_fullsized":"2024-04-23T14:33:41.562594+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:01:46.971105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.367487+0000","last_change":"2024-04-23T14:33:38.555154+0000","last_active":"2024-04-23T14:33:41.367487+0000","last_peered":"2024-04-23T14:33:41.367487+0000","last_clean":"2024-04-23T14:33:41.367487+0000","last_became_active":"2024-04-23T14:33:38.554887+0000","last_became_peered":"2024-04-23T14:33:38.554887+0000","last_unstale":"2024-04-23T14:33:41.367487+0000","last_undegraded":"2024-04-23T14:33:41.367487+0000","last_fullsized":"2024-04-23T14:33:41.367487+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:52:58.709384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.556460+0000","last_change":"2024-04-23T14:33:39.576139+0000","last_active":"2024-04-23T14:33:40.556460+0000","last_peered":"2024-04-23T14:33:40.556460+0000","last_clean":"2024-04-23T14:33:40.556460+0000","last_became_active":"2024-04-23T14:33:39.575854+0000","last_became_peered":"2024-04-23T14:33:39.575854+0000","last_unstale":"2024-04-23T14:33:40.556460+0000","last_undegraded":"2024-04-23T14:33:40.556460+0000","last_fullsized":"2024-04-23T14:33:40.556460+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:35.592096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599794+0000","last_change":"2024-04-23T14:33:38.555826+0000","last_active":"2024-04-23T14:33:40.599794+0000","last_peered":"2024-04-23T14:33:40.599794+0000","last_clean":"2024-04-23T14:33:40.599794+0000","last_became_active":"2024-04-23T14:33:38.555473+0000","last_became_peered":"2024-04-23T14:33:38.555473+0000","last_unstale":"2024-04-23T14:33:40.599794+0000","last_undegraded":"2024-04-23T14:33:40.599794+0000","last_fullsized":"2024-04-23T14:33:40.599794+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:55:00.760809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555384+0000","last_change":"2024-04-23T14:33:39.589371+0000","last_active":"2024-04-23T14:33:40.555384+0000","last_peered":"2024-04-23T14:33:40.555384+0000","last_clean":"2024-04-23T14:33:40.555384+0000","last_became_active":"2024-04-23T14:33:39.588894+0000","last_became_peered":"2024-04-23T14:33:39.588894+0000","last_unstale":"2024-04-23T14:33:40.555384+0000","last_undegraded":"2024-04-23T14:33:40.555384+0000","last_fullsized":"2024-04-23T14:33:40.555384+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:52:59.715529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599658+0000","last_change":"2024-04-23T14:33:38.560341+0000","last_active":"2024-04-23T14:33:40.599658+0000","last_peered":"2024-04-23T14:33:40.599658+0000","last_clean":"2024-04-23T14:33:40.599658+0000","last_became_active":"2024-04-23T14:33:38.560058+0000","last_became_peered":"2024-04-23T14:33:38.560058+0000","last_unstale":"2024-04-23T14:33:40.599658+0000","last_undegraded":"2024-04-23T14:33:40.599658+0000","last_fullsized":"2024-04-23T14:33:40.599658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:37.098622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665588+0000","last_change":"2024-04-23T14:33:39.580339+0000","last_active":"2024-04-23T14:33:41.665588+0000","last_peered":"2024-04-23T14:33:41.665588+0000","last_clean":"2024-04-23T14:33:41.665588+0000","last_became_active":"2024-04-23T14:33:39.580003+0000","last_became_peered":"2024-04-23T14:33:39.580003+0000","last_unstale":"2024-04-23T14:33:41.665588+0000","last_undegraded":"2024-04-23T14:33:41.665588+0000","last_fullsized":"2024-04-23T14:33:41.665588+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:10:16.577796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.367306+0000","last_change":"2024-04-23T14:33:38.554839+0000","last_active":"2024-04-23T14:33:41.367306+0000","last_peered":"2024-04-23T14:33:41.367306+0000","last_clean":"2024-04-23T14:33:41.367306+0000","last_became_active":"2024-04-23T14:33:38.554575+0000","last_became_peered":"2024-04-23T14:33:38.554575+0000","last_unstale":"2024-04-23T14:33:41.367306+0000","last_undegraded":"2024-04-23T14:33:41.367306+0000","last_fullsized":"2024-04-23T14:33:41.367306+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:10:15.571650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.554989+0000","last_change":"2024-04-23T14:33:39.566810+0000","last_active":"2024-04-23T14:33:40.554989+0000","last_peered":"2024-04-23T14:33:40.554989+0000","last_clean":"2024-04-23T14:33:40.554989+0000","last_became_active":"2024-04-23T14:33:39.566557+0000","last_became_peered":"2024-04-23T14:33:39.566557+0000","last_unstale":"2024-04-23T14:33:40.554989+0000","last_undegraded":"2024-04-23T14:33:40.554989+0000","last_fullsized":"2024-04-23T14:33:40.554989+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:21:29.390457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.367188+0000","last_change":"2024-04-23T14:33:38.556392+0000","last_active":"2024-04-23T14:33:41.367188+0000","last_peered":"2024-04-23T14:33:41.367188+0000","last_clean":"2024-04-23T14:33:41.367188+0000","last_became_active":"2024-04-23T14:33:38.555979+0000","last_became_peered":"2024-04-23T14:33:38.555979+0000","last_unstale":"2024-04-23T14:33:41.367188+0000","last_undegraded":"2024-04-23T14:33:41.367188+0000","last_fullsized":"2024-04-23T14:33:41.367188+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:04:53.188277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.366998+0000","last_change":"2024-04-23T14:33:39.578228+0000","last_active":"2024-04-23T14:33:41.366998+0000","last_peered":"2024-04-23T14:33:41.366998+0000","last_clean":"2024-04-23T14:33:41.366998+0000","last_became_active":"2024-04-23T14:33:39.577971+0000","last_became_peered":"2024-04-23T14:33:39.577971+0000","last_unstale":"2024-04-23T14:33:41.366998+0000","last_undegraded":"2024-04-23T14:33:41.366998+0000","last_fullsized":"2024-04-23T14:33:41.366998+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:45:10.440511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.664640+0000","last_change":"2024-04-23T14:33:38.561030+0000","last_active":"2024-04-23T14:33:41.664640+0000","last_peered":"2024-04-23T14:33:41.664640+0000","last_clean":"2024-04-23T14:33:41.664640+0000","last_became_active":"2024-04-23T14:33:38.560751+0000","last_became_peered":"2024-04-23T14:33:38.560751+0000","last_unstale":"2024-04-23T14:33:41.664640+0000","last_undegraded":"2024-04-23T14:33:41.664640+0000","last_fullsized":"2024-04-23T14:33:41.664640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:38:45.384204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555695+0000","last_change":"2024-04-23T14:33:39.588591+0000","last_active":"2024-04-23T14:33:40.555695+0000","last_peered":"2024-04-23T14:33:40.555695+0000","last_clean":"2024-04-23T14:33:40.555695+0000","last_became_active":"2024-04-23T14:33:39.588159+0000","last_became_peered":"2024-04-23T14:33:39.588159+0000","last_unstale":"2024-04-23T14:33:40.555695+0000","last_undegraded":"2024-04-23T14:33:40.555695+0000","last_fullsized":"2024-04-23T14:33:40.555695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:08:00.391713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.554668+0000","last_change":"2024-04-23T14:33:38.561064+0000","last_active":"2024-04-23T14:33:40.554668+0000","last_peered":"2024-04-23T14:33:40.554668+0000","last_clean":"2024-04-23T14:33:40.554668+0000","last_became_active":"2024-04-23T14:33:38.560797+0000","last_became_peered":"2024-04-23T14:33:38.560797+0000","last_unstale":"2024-04-23T14:33:40.554668+0000","last_undegraded":"2024-04-23T14:33:40.554668+0000","last_fullsized":"2024-04-23T14:33:40.554668+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:14:14.334310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.367111+0000","last_change":"2024-04-23T14:33:39.579653+0000","last_active":"2024-04-23T14:33:41.367111+0000","last_peered":"2024-04-23T14:33:41.367111+0000","last_clean":"2024-04-23T14:33:41.367111+0000","last_became_active":"2024-04-23T14:33:39.579360+0000","last_became_peered":"2024-04-23T14:33:39.579360+0000","last_unstale":"2024-04-23T14:33:41.367111+0000","last_undegraded":"2024-04-23T14:33:41.367111+0000","last_fullsized":"2024-04-23T14:33:41.367111+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:10:49.093987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555340+0000","last_change":"2024-04-23T14:33:38.555718+0000","last_active":"2024-04-23T14:33:40.555340+0000","last_peered":"2024-04-23T14:33:40.555340+0000","last_clean":"2024-04-23T14:33:40.555340+0000","last_became_active":"2024-04-23T14:33:38.555445+0000","last_became_peered":"2024-04-23T14:33:38.555445+0000","last_unstale":"2024-04-23T14:33:40.555340+0000","last_undegraded":"2024-04-23T14:33:40.555340+0000","last_fullsized":"2024-04-23T14:33:40.555340+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:11:41.721234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.367353+0000","last_change":"2024-04-23T14:33:39.581109+0000","last_active":"2024-04-23T14:33:41.367353+0000","last_peered":"2024-04-23T14:33:41.367353+0000","last_clean":"2024-04-23T14:33:41.367353+0000","last_became_active":"2024-04-23T14:33:39.580853+0000","last_became_peered":"2024-04-23T14:33:39.580853+0000","last_unstale":"2024-04-23T14:33:41.367353+0000","last_undegraded":"2024-04-23T14:33:41.367353+0000","last_fullsized":"2024-04-23T14:33:41.367353+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:50:18.302699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599647+0000","last_change":"2024-04-23T14:33:38.561267+0000","last_active":"2024-04-23T14:33:40.599647+0000","last_peered":"2024-04-23T14:33:40.599647+0000","last_clean":"2024-04-23T14:33:40.599647+0000","last_became_active":"2024-04-23T14:33:38.560607+0000","last_became_peered":"2024-04-23T14:33:38.560607+0000","last_unstale":"2024-04-23T14:33:40.599647+0000","last_undegraded":"2024-04-23T14:33:40.599647+0000","last_fullsized":"2024-04-23T14:33:40.599647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:38:45.384204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599685+0000","last_change":"2024-04-23T14:33:39.566783+0000","last_active":"2024-04-23T14:33:40.599685+0000","last_peered":"2024-04-23T14:33:40.599685+0000","last_clean":"2024-04-23T14:33:40.599685+0000","last_became_active":"2024-04-23T14:33:39.566487+0000","last_became_peered":"2024-04-23T14:33:39.566487+0000","last_unstale":"2024-04-23T14:33:40.599685+0000","last_undegraded":"2024-04-23T14:33:40.599685+0000","last_fullsized":"2024-04-23T14:33:40.599685+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:17:01.997568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665583+0000","last_change":"2024-04-23T14:33:38.557553+0000","last_active":"2024-04-23T14:33:41.665583+0000","last_peered":"2024-04-23T14:33:41.665583+0000","last_clean":"2024-04-23T14:33:41.665583+0000","last_became_active":"2024-04-23T14:33:38.557269+0000","last_became_peered":"2024-04-23T14:33:38.557269+0000","last_unstale":"2024-04-23T14:33:41.665583+0000","last_undegraded":"2024-04-23T14:33:41.665583+0000","last_fullsized":"2024-04-23T14:33:41.665583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:29:37.564582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665487+0000","last_change":"2024-04-23T14:33:39.573689+0000","last_active":"2024-04-23T14:33:41.665487+0000","last_peered":"2024-04-23T14:33:41.665487+0000","last_clean":"2024-04-23T14:33:41.665487+0000","last_became_active":"2024-04-23T14:33:39.573393+0000","last_became_peered":"2024-04-23T14:33:39.573393+0000","last_unstale":"2024-04-23T14:33:41.665487+0000","last_undegraded":"2024-04-23T14:33:41.665487+0000","last_fullsized":"2024-04-23T14:33:41.665487+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:39.146588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.562586+0000","last_change":"2024-04-23T14:33:38.545969+0000","last_active":"2024-04-23T14:33:41.562586+0000","last_peered":"2024-04-23T14:33:41.562586+0000","last_clean":"2024-04-23T14:33:41.562586+0000","last_became_active":"2024-04-23T14:33:38.545512+0000","last_became_peered":"2024-04-23T14:33:38.545512+0000","last_unstale":"2024-04-23T14:33:41.562586+0000","last_undegraded":"2024-04-23T14:33:41.562586+0000","last_fullsized":"2024-04-23T14:33:41.562586+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:57:34.344856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.562841+0000","last_change":"2024-04-23T14:33:39.590166+0000","last_active":"2024-04-23T14:33:41.562841+0000","last_peered":"2024-04-23T14:33:41.562841+0000","last_clean":"2024-04-23T14:33:41.562841+0000","last_became_active":"2024-04-23T14:33:39.589722+0000","last_became_peered":"2024-04-23T14:33:39.589722+0000","last_unstale":"2024-04-23T14:33:41.562841+0000","last_undegraded":"2024-04-23T14:33:41.562841+0000","last_fullsized":"2024-04-23T14:33:41.562841+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:53.534112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.664863+0000","last_change":"2024-04-23T14:33:38.551345+0000","last_active":"2024-04-23T14:33:41.664863+0000","last_peered":"2024-04-23T14:33:41.664863+0000","last_clean":"2024-04-23T14:33:41.664863+0000","last_became_active":"2024-04-23T14:33:38.551084+0000","last_became_peered":"2024-04-23T14:33:38.551084+0000","last_unstale":"2024-04-23T14:33:41.664863+0000","last_undegraded":"2024-04-23T14:33:41.664863+0000","last_fullsized":"2024-04-23T14:33:41.664863+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:11:41.721234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555427+0000","last_change":"2024-04-23T14:33:39.588843+0000","last_active":"2024-04-23T14:33:40.555427+0000","last_peered":"2024-04-23T14:33:40.555427+0000","last_clean":"2024-04-23T14:33:40.555427+0000","last_became_active":"2024-04-23T14:33:39.587822+0000","last_became_peered":"2024-04-23T14:33:39.587822+0000","last_unstale":"2024-04-23T14:33:40.555427+0000","last_undegraded":"2024-04-23T14:33:40.555427+0000","last_fullsized":"2024-04-23T14:33:40.555427+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:52:59.715529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.562410+0000","last_change":"2024-04-23T14:33:38.539420+0000","last_active":"2024-04-23T14:33:41.562410+0000","last_peered":"2024-04-23T14:33:41.562410+0000","last_clean":"2024-04-23T14:33:41.562410+0000","last_became_active":"2024-04-23T14:33:38.538810+0000","last_became_peered":"2024-04-23T14:33:38.538810+0000","last_unstale":"2024-04-23T14:33:41.562410+0000","last_undegraded":"2024-04-23T14:33:41.562410+0000","last_fullsized":"2024-04-23T14:33:41.562410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:38.198200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555107+0000","last_change":"2024-04-23T14:33:39.582674+0000","last_active":"2024-04-23T14:33:40.555107+0000","last_peered":"2024-04-23T14:33:40.555107+0000","last_clean":"2024-04-23T14:33:40.555107+0000","last_became_active":"2024-04-23T14:33:39.582398+0000","last_became_peered":"2024-04-23T14:33:39.582398+0000","last_unstale":"2024-04-23T14:33:40.555107+0000","last_undegraded":"2024-04-23T14:33:40.555107+0000","last_fullsized":"2024-04-23T14:33:40.555107+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:39.146588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.562346+0000","last_change":"2024-04-23T14:33:38.551409+0000","last_active":"2024-04-23T14:33:41.562346+0000","last_peered":"2024-04-23T14:33:41.562346+0000","last_clean":"2024-04-23T14:33:41.562346+0000","last_became_active":"2024-04-23T14:33:38.551145+0000","last_became_peered":"2024-04-23T14:33:38.551145+0000","last_unstale":"2024-04-23T14:33:41.562346+0000","last_undegraded":"2024-04-23T14:33:41.562346+0000","last_fullsized":"2024-04-23T14:33:41.562346+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:33.119876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599329+0000","last_change":"2024-04-23T14:33:39.564292+0000","last_active":"2024-04-23T14:33:40.599329+0000","last_peered":"2024-04-23T14:33:40.599329+0000","last_clean":"2024-04-23T14:33:40.599329+0000","last_became_active":"2024-04-23T14:33:39.564025+0000","last_became_peered":"2024-04-23T14:33:39.564025+0000","last_unstale":"2024-04-23T14:33:40.599329+0000","last_undegraded":"2024-04-23T14:33:40.599329+0000","last_fullsized":"2024-04-23T14:33:40.599329+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:30:44.259502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555838+0000","last_change":"2024-04-23T14:33:38.554466+0000","last_active":"2024-04-23T14:33:40.555838+0000","last_peered":"2024-04-23T14:33:40.555838+0000","last_clean":"2024-04-23T14:33:40.555838+0000","last_became_active":"2024-04-23T14:33:38.554155+0000","last_became_peered":"2024-04-23T14:33:38.554155+0000","last_unstale":"2024-04-23T14:33:40.555838+0000","last_undegraded":"2024-04-23T14:33:40.555838+0000","last_fullsized":"2024-04-23T14:33:40.555838+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:38:45.384204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.367091+0000","last_change":"2024-04-23T14:33:39.587638+0000","last_active":"2024-04-23T14:33:41.367091+0000","last_peered":"2024-04-23T14:33:41.367091+0000","last_clean":"2024-04-23T14:33:41.367091+0000","last_became_active":"2024-04-23T14:33:39.587344+0000","last_became_peered":"2024-04-23T14:33:39.587344+0000","last_unstale":"2024-04-23T14:33:41.367091+0000","last_undegraded":"2024-04-23T14:33:41.367091+0000","last_fullsized":"2024-04-23T14:33:41.367091+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:45:25.313578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.562339+0000","last_change":"2024-04-23T14:33:38.539108+0000","last_active":"2024-04-23T14:33:41.562339+0000","last_peered":"2024-04-23T14:33:41.562339+0000","last_clean":"2024-04-23T14:33:41.562339+0000","last_became_active":"2024-04-23T14:33:38.538806+0000","last_became_peered":"2024-04-23T14:33:38.538806+0000","last_unstale":"2024-04-23T14:33:41.562339+0000","last_undegraded":"2024-04-23T14:33:41.562339+0000","last_fullsized":"2024-04-23T14:33:41.562339+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:14:49.245496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.664847+0000","last_change":"2024-04-23T14:33:39.579759+0000","last_active":"2024-04-23T14:33:41.664847+0000","last_peered":"2024-04-23T14:33:41.664847+0000","last_clean":"2024-04-23T14:33:41.664847+0000","last_became_active":"2024-04-23T14:33:39.579458+0000","last_became_peered":"2024-04-23T14:33:39.579458+0000","last_unstale":"2024-04-23T14:33:41.664847+0000","last_undegraded":"2024-04-23T14:33:41.664847+0000","last_fullsized":"2024-04-23T14:33:41.664847+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:39.146588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599759+0000","last_change":"2024-04-23T14:33:38.559480+0000","last_active":"2024-04-23T14:33:40.599759+0000","last_peered":"2024-04-23T14:33:40.599759+0000","last_clean":"2024-04-23T14:33:40.599759+0000","last_became_active":"2024-04-23T14:33:38.559223+0000","last_became_peered":"2024-04-23T14:33:38.559223+0000","last_unstale":"2024-04-23T14:33:40.599759+0000","last_undegraded":"2024-04-23T14:33:40.599759+0000","last_fullsized":"2024-04-23T14:33:40.599759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:06:41.064638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555920+0000","last_change":"2024-04-23T14:33:39.576658+0000","last_active":"2024-04-23T14:33:40.555920+0000","last_peered":"2024-04-23T14:33:40.555920+0000","last_clean":"2024-04-23T14:33:40.555920+0000","last_became_active":"2024-04-23T14:33:39.576402+0000","last_became_peered":"2024-04-23T14:33:39.576402+0000","last_unstale":"2024-04-23T14:33:40.555920+0000","last_undegraded":"2024-04-23T14:33:40.555920+0000","last_fullsized":"2024-04-23T14:33:40.555920+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:35:49.748984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.564757+0000","last_change":"2024-04-23T14:33:38.547438+0000","last_active":"2024-04-23T14:33:41.564757+0000","last_peered":"2024-04-23T14:33:41.564757+0000","last_clean":"2024-04-23T14:33:41.564757+0000","last_became_active":"2024-04-23T14:33:38.547054+0000","last_became_peered":"2024-04-23T14:33:38.547054+0000","last_unstale":"2024-04-23T14:33:41.564757+0000","last_undegraded":"2024-04-23T14:33:41.564757+0000","last_fullsized":"2024-04-23T14:33:41.564757+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:57:37.478718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.367385+0000","last_change":"2024-04-23T14:33:39.580248+0000","last_active":"2024-04-23T14:33:41.367385+0000","last_peered":"2024-04-23T14:33:41.367385+0000","last_clean":"2024-04-23T14:33:41.367385+0000","last_became_active":"2024-04-23T14:33:39.579977+0000","last_became_peered":"2024-04-23T14:33:39.579977+0000","last_unstale":"2024-04-23T14:33:41.367385+0000","last_undegraded":"2024-04-23T14:33:41.367385+0000","last_fullsized":"2024-04-23T14:33:41.367385+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:43:52.636588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555302+0000","last_change":"2024-04-23T14:33:38.551995+0000","last_active":"2024-04-23T14:33:40.555302+0000","last_peered":"2024-04-23T14:33:40.555302+0000","last_clean":"2024-04-23T14:33:40.555302+0000","last_became_active":"2024-04-23T14:33:38.551685+0000","last_became_peered":"2024-04-23T14:33:38.551685+0000","last_unstale":"2024-04-23T14:33:40.555302+0000","last_undegraded":"2024-04-23T14:33:40.555302+0000","last_fullsized":"2024-04-23T14:33:40.555302+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:01:45.964960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555136+0000","last_change":"2024-04-23T14:33:39.585259+0000","last_active":"2024-04-23T14:33:40.555136+0000","last_peered":"2024-04-23T14:33:40.555136+0000","last_clean":"2024-04-23T14:33:40.555136+0000","last_became_active":"2024-04-23T14:33:39.585011+0000","last_became_peered":"2024-04-23T14:33:39.585011+0000","last_unstale":"2024-04-23T14:33:40.555136+0000","last_undegraded":"2024-04-23T14:33:40.555136+0000","last_fullsized":"2024-04-23T14:33:40.555136+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:56:14.508220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555251+0000","last_change":"2024-04-23T14:33:38.553908+0000","last_active":"2024-04-23T14:33:40.555251+0000","last_peered":"2024-04-23T14:33:40.555251+0000","last_clean":"2024-04-23T14:33:40.555251+0000","last_became_active":"2024-04-23T14:33:38.553483+0000","last_became_peered":"2024-04-23T14:33:38.553483+0000","last_unstale":"2024-04-23T14:33:40.555251+0000","last_undegraded":"2024-04-23T14:33:40.555251+0000","last_fullsized":"2024-04-23T14:33:40.555251+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:32:41.665983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.664679+0000","last_change":"2024-04-23T14:33:39.565585+0000","last_active":"2024-04-23T14:33:41.664679+0000","last_peered":"2024-04-23T14:33:41.664679+0000","last_clean":"2024-04-23T14:33:41.664679+0000","last_became_active":"2024-04-23T14:33:39.565310+0000","last_became_peered":"2024-04-23T14:33:39.565310+0000","last_unstale":"2024-04-23T14:33:41.664679+0000","last_undegraded":"2024-04-23T14:33:41.664679+0000","last_fullsized":"2024-04-23T14:33:41.664679+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:21:29.390457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.554757+0000","last_change":"2024-04-23T14:33:38.559974+0000","last_active":"2024-04-23T14:33:40.554757+0000","last_peered":"2024-04-23T14:33:40.554757+0000","last_clean":"2024-04-23T14:33:40.554757+0000","last_became_active":"2024-04-23T14:33:38.557889+0000","last_became_peered":"2024-04-23T14:33:38.557889+0000","last_unstale":"2024-04-23T14:33:40.554757+0000","last_undegraded":"2024-04-23T14:33:40.554757+0000","last_fullsized":"2024-04-23T14:33:40.554757+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:29.902113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665201+0000","last_change":"2024-04-23T14:33:39.583378+0000","last_active":"2024-04-23T14:33:41.665201+0000","last_peered":"2024-04-23T14:33:41.665201+0000","last_clean":"2024-04-23T14:33:41.665201+0000","last_became_active":"2024-04-23T14:33:39.583100+0000","last_became_peered":"2024-04-23T14:33:39.583100+0000","last_unstale":"2024-04-23T14:33:41.665201+0000","last_undegraded":"2024-04-23T14:33:41.665201+0000","last_fullsized":"2024-04-23T14:33:41.665201+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:21:29.390457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665041+0000","last_change":"2024-04-23T14:33:38.560737+0000","last_active":"2024-04-23T14:33:41.665041+0000","last_peered":"2024-04-23T14:33:41.665041+0000","last_clean":"2024-04-23T14:33:41.665041+0000","last_became_active":"2024-04-23T14:33:38.559721+0000","last_became_peered":"2024-04-23T14:33:38.559721+0000","last_unstale":"2024-04-23T14:33:41.665041+0000","last_undegraded":"2024-04-23T14:33:41.665041+0000","last_fullsized":"2024-04-23T14:33:41.665041+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:32:41.665983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.556037+0000","last_change":"2024-04-23T14:33:39.586013+0000","last_active":"2024-04-23T14:33:40.556037+0000","last_peered":"2024-04-23T14:33:40.556037+0000","last_clean":"2024-04-23T14:33:40.556037+0000","last_became_active":"2024-04-23T14:33:39.583815+0000","last_became_peered":"2024-04-23T14:33:39.583815+0000","last_unstale":"2024-04-23T14:33:40.556037+0000","last_undegraded":"2024-04-23T14:33:40.556037+0000","last_fullsized":"2024-04-23T14:33:40.556037+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:04.758788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.366952+0000","last_change":"2024-04-23T14:33:38.558963+0000","last_active":"2024-04-23T14:33:41.366952+0000","last_peered":"2024-04-23T14:33:41.366952+0000","last_clean":"2024-04-23T14:33:41.366952+0000","last_became_active":"2024-04-23T14:33:38.558410+0000","last_became_peered":"2024-04-23T14:33:38.558410+0000","last_unstale":"2024-04-23T14:33:41.366952+0000","last_undegraded":"2024-04-23T14:33:41.366952+0000","last_fullsized":"2024-04-23T14:33:41.366952+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:56:13.502074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599591+0000","last_change":"2024-04-23T14:33:39.588638+0000","last_active":"2024-04-23T14:33:40.599591+0000","last_peered":"2024-04-23T14:33:40.599591+0000","last_clean":"2024-04-23T14:33:40.599591+0000","last_became_active":"2024-04-23T14:33:39.588262+0000","last_became_peered":"2024-04-23T14:33:39.588262+0000","last_unstale":"2024-04-23T14:33:40.599591+0000","last_undegraded":"2024-04-23T14:33:40.599591+0000","last_fullsized":"2024-04-23T14:33:40.599591+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:11:21.071775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.556159+0000","last_change":"2024-04-23T14:33:38.550575+0000","last_active":"2024-04-23T14:33:40.556159+0000","last_peered":"2024-04-23T14:33:40.556159+0000","last_clean":"2024-04-23T14:33:40.556159+0000","last_became_active":"2024-04-23T14:33:38.550160+0000","last_became_peered":"2024-04-23T14:33:38.550160+0000","last_unstale":"2024-04-23T14:33:40.556159+0000","last_undegraded":"2024-04-23T14:33:40.556159+0000","last_fullsized":"2024-04-23T14:33:40.556159+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:37.098622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555413+0000","last_change":"2024-04-23T14:31:58.314235+0000","last_active":"2024-04-23T14:33:40.555413+0000","last_peered":"2024-04-23T14:33:40.555413+0000","last_clean":"2024-04-23T14:33:40.555413+0000","last_became_active":"2024-04-23T14:31:58.313165+0000","last_became_peered":"2024-04-23T14:31:58.313165+0000","last_unstale":"2024-04-23T14:33:40.555413+0000","last_undegraded":"2024-04-23T14:33:40.555413+0000","last_fullsized":"2024-04-23T14:33:40.555413+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_clean_scrub_stamp":"2024-04-23T14:31:57.140712+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:44.170436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":782078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":190610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":190140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27444,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":434176,"data_stored":183940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":184982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":183406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":183426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1036288,"data_stored":780430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:33:43.649 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:33:43.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.816+0000 7f4b03600640 1 Processor -- start 2024-04-23T14:33:43.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.816+0000 7f4b03600640 1 -- start start 2024-04-23T14:33:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.816+0000 7f4b03600640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f4afc15e830 0x7f4afc15ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.816+0000 7f4b03600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4afc15f320 con 0x7f4afc0d00b0 2024-04-23T14:33:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.816+0000 7f4b03600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4afc15f5e0 con 0x7f4afc0cdd60 2024-04-23T14:33:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.816+0000 7f4b03600640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f4afc15f8c0 con 0x7f4afc15e830 2024-04-23T14:33:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.820+0000 7f4b01600640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f4afc0d00b0 0x7f4afc0b45c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57018/0 (socket says 172.21.15.100:57018) 2024-04-23T14:33:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.820+0000 7f4b01600640 1 -- 172.21.15.100:0/3342828223 learned_addr learned my addr 172.21.15.100:0/3342828223 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:43.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.820+0000 7f4b00c00640 1 --2- 172.21.15.100:0/3342828223 >> v2:172.21.15.151:3300/0 conn(0x7f4afc15e830 0x7f4afc15ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.820+0000 7f4afb400640 1 -- 172.21.15.100:0/3342828223 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1366858947 0 0) 0x7f4afc15f320 con 0x7f4afc0d00b0 2024-04-23T14:33:43.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.820+0000 7f4afb400640 1 -- 172.21.15.100:0/3342828223 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ae4003530 con 0x7f4afc0d00b0 2024-04-23T14:33:43.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.820+0000 7f4b00c00640 1 -- 172.21.15.100:0/3342828223 >> v1:172.21.15.151:6789/0 conn(0x7f4afc0cdd60 legacy=0x7f4afc0cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.820+0000 7f4b00c00640 1 -- 172.21.15.100:0/3342828223 >> v1:172.21.15.100:6789/0 conn(0x7f4afc0d00b0 legacy=0x7f4afc0b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.820+0000 7f4b00c00640 1 -- 172.21.15.100:0/3342828223 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4afc15fba0 con 0x7f4afc15e830 2024-04-23T14:33:43.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.820+0000 7f4b00c00640 1 --2- 172.21.15.100:0/3342828223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4afc15e830 0x7f4afc15ec10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4af00097c0 tx=0x7f4af001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a685d6a39f886dc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:43.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4afb400640 1 -- 172.21.15.100:0/3342828223 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af00b1070 con 0x7f4afc15e830 2024-04-23T14:33:43.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4afb400640 1 -- 172.21.15.100:0/3342828223 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4af00ac410 con 0x7f4afc15e830 2024-04-23T14:33:43.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4afb400640 1 -- 172.21.15.100:0/3342828223 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af00ac7e0 con 0x7f4afc15e830 2024-04-23T14:33:43.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b03600640 1 -- 172.21.15.100:0/3342828223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4afc15e830 msgr2=0x7f4afc15ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b03600640 1 --2- 172.21.15.100:0/3342828223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4afc15e830 0x7f4afc15ec10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4af00097c0 tx=0x7f4af001c4a0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b03600640 1 -- 172.21.15.100:0/3342828223 shutdown_connections 2024-04-23T14:33:43.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b03600640 1 --2- 172.21.15.100:0/3342828223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4afc15e830 0x7f4afc15ec10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b03600640 1 -- 172.21.15.100:0/3342828223 >> 172.21.15.100:0/3342828223 conn(0x7f4afc0704d0 msgr2=0x7f4afc0708e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:43.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b03600640 1 -- 172.21.15.100:0/3342828223 shutdown_connections 2024-04-23T14:33:43.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b03600640 1 -- 172.21.15.100:0/3342828223 wait complete. 2024-04-23T14:33:43.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b03600640 1 Processor -- start 2024-04-23T14:33:43.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b03600640 1 -- start start 2024-04-23T14:33:43.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b03600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4afc0cdd60 0x7f4afc15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b03600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4afc0d00b0 0x7f4afc140170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b03600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4afc15e830 0x7f4afc0b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b03600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc0b58f0 con 0x7f4afc0cdd60 2024-04-23T14:33:43.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b03600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc0b5770 con 0x7f4afc0d00b0 2024-04-23T14:33:43.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b03600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4afc16c4a0 con 0x7f4afc15e830 2024-04-23T14:33:43.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b00c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4afc0cdd60 0x7f4afc15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b00c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4afc0cdd60 0x7f4afc15aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58252/0 (socket says 172.21.15.100:58252) 2024-04-23T14:33:43.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b00c00640 1 -- 172.21.15.100:0/1462122404 learned_addr learned my addr 172.21.15.100:0/1462122404 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:43.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4afbe00640 1 --2- 172.21.15.100:0/1462122404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4afc0d00b0 0x7f4afc140170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b01600640 1 --2- 172.21.15.100:0/1462122404 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4afc15e830 0x7f4afc0b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b00c00640 1 -- 172.21.15.100:0/1462122404 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4afc15e830 msgr2=0x7f4afc0b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b00c00640 1 --2- 172.21.15.100:0/1462122404 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4afc15e830 0x7f4afc0b23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b00c00640 1 -- 172.21.15.100:0/1462122404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4afc0d00b0 msgr2=0x7f4afc140170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b00c00640 1 --2- 172.21.15.100:0/1462122404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4afc0d00b0 0x7f4afc140170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b00c00640 1 -- 172.21.15.100:0/1462122404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4afc161f70 con 0x7f4afc0cdd60 2024-04-23T14:33:43.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.824+0000 7f4b01600640 1 --2- 172.21.15.100:0/1462122404 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4afc15e830 0x7f4afc0b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:43.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.828+0000 7f4b00c00640 1 --2- 172.21.15.100:0/1462122404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4afc0cdd60 0x7f4afc15aaa0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4af00190b0 tx=0x7f4af0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:43.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.828+0000 7f4af8c00640 1 -- 172.21.15.100:0/1462122404 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af00b1040 con 0x7f4afc0cdd60 2024-04-23T14:33:43.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.828+0000 7f4af8c00640 1 -- 172.21.15.100:0/1462122404 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4af00bb070 con 0x7f4afc0cdd60 2024-04-23T14:33:43.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.828+0000 7f4af8c00640 1 -- 172.21.15.100:0/1462122404 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af00b5410 con 0x7f4afc0cdd60 2024-04-23T14:33:43.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.828+0000 7f4b03600640 1 -- 172.21.15.100:0/1462122404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4afc06de20 con 0x7f4afc0cdd60 2024-04-23T14:33:43.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.828+0000 7f4b03600640 1 -- 172.21.15.100:0/1462122404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4afc06e3e0 con 0x7f4afc0cdd60 2024-04-23T14:33:43.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.828+0000 7f4b03600640 1 -- 172.21.15.100:0/1462122404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4afc068ad0 con 0x7f4afc0cdd60 2024-04-23T14:33:43.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.828+0000 7f4af8c00640 1 -- 172.21.15.100:0/1462122404 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4af00b55b0 con 0x7f4afc0cdd60 2024-04-23T14:33:43.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.828+0000 7f4af8c00640 1 --2- 172.21.15.100:0/1462122404 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4adc0683b0 0x7f4adc06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.828+0000 7f4af8c00640 1 -- 172.21.15.100:0/1462122404 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f4af01205c0 con 0x7f4afc0cdd60 2024-04-23T14:33:43.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.828+0000 7f4afbe00640 1 --2- 172.21.15.100:0/1462122404 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4adc0683b0 0x7f4adc06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.832+0000 7f4afbe00640 1 --2- 172.21.15.100:0/1462122404 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4adc0683b0 0x7f4adc06a870 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4afc157b30 tx=0x7f4ae8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:43.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.832+0000 7f4af8c00640 1 -- 172.21.15.100:0/1462122404 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4af0016090 con 0x7f4afc0cdd60 2024-04-23T14:33:43.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.988+0000 7f4b03600640 1 -- 172.21.15.100:0/1462122404 --> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4afc0ce910 con 0x7f4adc0683b0 2024-04-23T14:33:43.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:43.996+0000 7f4af8c00640 1 -- 172.21.15.100:0/1462122404 <== mgr.4101 v2:172.21.15.100:6832/2859853957 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251957 (secure 0 0 0) 0x7f4afc0ce910 con 0x7f4adc0683b0 2024-04-23T14:33:44.002 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:44.003 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-04-23T14:33:44.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.000+0000 7f4b03600640 1 -- 172.21.15.100:0/1462122404 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4adc0683b0 msgr2=0x7f4adc06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.000+0000 7f4b03600640 1 --2- 172.21.15.100:0/1462122404 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4adc0683b0 0x7f4adc06a870 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4afc157b30 tx=0x7f4ae8009040 comp rx=0 tx=0).stop 2024-04-23T14:33:44.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.000+0000 7f4b03600640 1 -- 172.21.15.100:0/1462122404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4afc0cdd60 msgr2=0x7f4afc15aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.000+0000 7f4b03600640 1 --2- 172.21.15.100:0/1462122404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4afc0cdd60 0x7f4afc15aaa0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4af00190b0 tx=0x7f4af0019240 comp rx=0 tx=0).stop 2024-04-23T14:33:44.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.000+0000 7f4b03600640 1 -- 172.21.15.100:0/1462122404 shutdown_connections 2024-04-23T14:33:44.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.000+0000 7f4b03600640 1 --2- 172.21.15.100:0/1462122404 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4adc0683b0 0x7f4adc06a870 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.000+0000 7f4b03600640 1 --2- 172.21.15.100:0/1462122404 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4afc15e830 0x7f4afc0b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.000+0000 7f4b03600640 1 --2- 172.21.15.100:0/1462122404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4afc0d00b0 0x7f4afc140170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.000+0000 7f4b03600640 1 --2- 172.21.15.100:0/1462122404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4afc0cdd60 0x7f4afc15aaa0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.000+0000 7f4b03600640 1 -- 172.21.15.100:0/1462122404 >> 172.21.15.100:0/1462122404 conn(0x7f4afc0704d0 msgr2=0x7f4afc0cf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:44.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.000+0000 7f4b03600640 1 -- 172.21.15.100:0/1462122404 shutdown_connections 2024-04-23T14:33:44.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.000+0000 7f4b03600640 1 -- 172.21.15.100:0/1462122404 wait complete. 2024-04-23T14:33:44.023 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-04-23T14:33:42.754353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224936,"kb_used_data":4600,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749784,"statfs":{"total":773094113280,"available":772863778816,"internally_reserved":0,"allocated":4710400,"data_stored":2679012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.225576"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.367470+0000","last_change":"2024-04-23T14:33:39.580767+0000","last_active":"2024-04-23T14:33:41.367470+0000","last_peered":"2024-04-23T14:33:41.367470+0000","last_clean":"2024-04-23T14:33:41.367470+0000","last_became_active":"2024-04-23T14:33:39.580526+0000","last_became_peered":"2024-04-23T14:33:39.580526+0000","last_unstale":"2024-04-23T14:33:41.367470+0000","last_undegraded":"2024-04-23T14:33:41.367470+0000","last_fullsized":"2024-04-23T14:33:41.367470+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:38:48.128956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.367385+0000","last_change":"2024-04-23T14:33:39.583938+0000","last_active":"2024-04-23T14:33:41.367385+0000","last_peered":"2024-04-23T14:33:41.367385+0000","last_clean":"2024-04-23T14:33:41.367385+0000","last_became_active":"2024-04-23T14:33:39.583603+0000","last_became_peered":"2024-04-23T14:33:39.583603+0000","last_unstale":"2024-04-23T14:33:41.367385+0000","last_undegraded":"2024-04-23T14:33:41.367385+0000","last_fullsized":"2024-04-23T14:33:41.367385+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:17:24.890164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555363+0000","last_change":"2024-04-23T14:33:39.561485+0000","last_active":"2024-04-23T14:33:40.555363+0000","last_peered":"2024-04-23T14:33:40.555363+0000","last_clean":"2024-04-23T14:33:40.555363+0000","last_became_active":"2024-04-23T14:33:39.561215+0000","last_became_peered":"2024-04-23T14:33:39.561215+0000","last_unstale":"2024-04-23T14:33:40.555363+0000","last_undegraded":"2024-04-23T14:33:40.555363+0000","last_fullsized":"2024-04-23T14:33:40.555363+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:10:16.577796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.562555+0000","last_change":"2024-04-23T14:33:39.586107+0000","last_active":"2024-04-23T14:33:41.562555+0000","last_peered":"2024-04-23T14:33:41.562555+0000","last_clean":"2024-04-23T14:33:41.562555+0000","last_became_active":"2024-04-23T14:33:39.585880+0000","last_became_peered":"2024-04-23T14:33:39.585880+0000","last_unstale":"2024-04-23T14:33:41.562555+0000","last_undegraded":"2024-04-23T14:33:41.562555+0000","last_fullsized":"2024-04-23T14:33:41.562555+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:32:42.672129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.562377+0000","last_change":"2024-04-23T14:33:39.574966+0000","last_active":"2024-04-23T14:33:41.562377+0000","last_peered":"2024-04-23T14:33:41.562377+0000","last_clean":"2024-04-23T14:33:41.562377+0000","last_became_active":"2024-04-23T14:33:39.574684+0000","last_became_peered":"2024-04-23T14:33:39.574684+0000","last_unstale":"2024-04-23T14:33:41.562377+0000","last_undegraded":"2024-04-23T14:33:41.562377+0000","last_fullsized":"2024-04-23T14:33:41.562377+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:30.908259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665078+0000","last_change":"2024-04-23T14:33:39.586533+0000","last_active":"2024-04-23T14:33:41.665078+0000","last_peered":"2024-04-23T14:33:41.665078+0000","last_clean":"2024-04-23T14:33:41.665078+0000","last_became_active":"2024-04-23T14:33:39.584448+0000","last_became_peered":"2024-04-23T14:33:39.584448+0000","last_unstale":"2024-04-23T14:33:41.665078+0000","last_undegraded":"2024-04-23T14:33:41.665078+0000","last_fullsized":"2024-04-23T14:33:41.665078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:10:16.577796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599394+0000","last_change":"2024-04-23T14:33:39.588547+0000","last_active":"2024-04-23T14:33:40.599394+0000","last_peered":"2024-04-23T14:33:40.599394+0000","last_clean":"2024-04-23T14:33:40.599394+0000","last_became_active":"2024-04-23T14:33:39.588220+0000","last_became_peered":"2024-04-23T14:33:39.588220+0000","last_unstale":"2024-04-23T14:33:40.599394+0000","last_undegraded":"2024-04-23T14:33:40.599394+0000","last_fullsized":"2024-04-23T14:33:40.599394+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:15:25.124076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.366929+0000","last_change":"2024-04-23T14:33:39.580516+0000","last_active":"2024-04-23T14:33:41.366929+0000","last_peered":"2024-04-23T14:33:41.366929+0000","last_clean":"2024-04-23T14:33:41.366929+0000","last_became_active":"2024-04-23T14:33:39.580215+0000","last_became_peered":"2024-04-23T14:33:39.580215+0000","last_unstale":"2024-04-23T14:33:41.366929+0000","last_undegraded":"2024-04-23T14:33:41.366929+0000","last_fullsized":"2024-04-23T14:33:41.366929+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:07:21.114361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555479+0000","last_change":"2024-04-23T14:33:39.570882+0000","last_active":"2024-04-23T14:33:40.555479+0000","last_peered":"2024-04-23T14:33:40.555479+0000","last_clean":"2024-04-23T14:33:40.555479+0000","last_became_active":"2024-04-23T14:33:39.570610+0000","last_became_peered":"2024-04-23T14:33:39.570610+0000","last_unstale":"2024-04-23T14:33:40.555479+0000","last_undegraded":"2024-04-23T14:33:40.555479+0000","last_fullsized":"2024-04-23T14:33:40.555479+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:11:49.792303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599663+0000","last_change":"2024-04-23T14:33:39.589859+0000","last_active":"2024-04-23T14:33:40.599663+0000","last_peered":"2024-04-23T14:33:40.599663+0000","last_clean":"2024-04-23T14:33:40.599663+0000","last_became_active":"2024-04-23T14:33:39.589603+0000","last_became_peered":"2024-04-23T14:33:39.589603+0000","last_unstale":"2024-04-23T14:33:40.599663+0000","last_undegraded":"2024-04-23T14:33:40.599663+0000","last_fullsized":"2024-04-23T14:33:40.599663+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:32:29.944091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599519+0000","last_change":"2024-04-23T14:33:39.582574+0000","last_active":"2024-04-23T14:33:40.599519+0000","last_peered":"2024-04-23T14:33:40.599519+0000","last_clean":"2024-04-23T14:33:40.599519+0000","last_became_active":"2024-04-23T14:33:39.576219+0000","last_became_peered":"2024-04-23T14:33:39.576219+0000","last_unstale":"2024-04-23T14:33:40.599519+0000","last_undegraded":"2024-04-23T14:33:40.599519+0000","last_fullsized":"2024-04-23T14:33:40.599519+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:49:09.617492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555191+0000","last_change":"2024-04-23T14:33:39.578969+0000","last_active":"2024-04-23T14:33:40.555191+0000","last_peered":"2024-04-23T14:33:40.555191+0000","last_clean":"2024-04-23T14:33:40.555191+0000","last_became_active":"2024-04-23T14:33:39.578666+0000","last_became_peered":"2024-04-23T14:33:39.578666+0000","last_unstale":"2024-04-23T14:33:40.555191+0000","last_undegraded":"2024-04-23T14:33:40.555191+0000","last_fullsized":"2024-04-23T14:33:40.555191+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:04:54.194423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599239+0000","last_change":"2024-04-23T14:33:39.590435+0000","last_active":"2024-04-23T14:33:40.599239+0000","last_peered":"2024-04-23T14:33:40.599239+0000","last_clean":"2024-04-23T14:33:40.599239+0000","last_became_active":"2024-04-23T14:33:39.590169+0000","last_became_peered":"2024-04-23T14:33:39.590169+0000","last_unstale":"2024-04-23T14:33:40.599239+0000","last_undegraded":"2024-04-23T14:33:40.599239+0000","last_fullsized":"2024-04-23T14:33:40.599239+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:35:49.748984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665105+0000","last_change":"2024-04-23T14:33:39.569496+0000","last_active":"2024-04-23T14:33:41.665105+0000","last_peered":"2024-04-23T14:33:41.665105+0000","last_clean":"2024-04-23T14:33:41.665105+0000","last_became_active":"2024-04-23T14:33:39.569197+0000","last_became_peered":"2024-04-23T14:33:39.569197+0000","last_unstale":"2024-04-23T14:33:41.665105+0000","last_undegraded":"2024-04-23T14:33:41.665105+0000","last_fullsized":"2024-04-23T14:33:41.665105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:05:24.295743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599237+0000","last_change":"2024-04-23T14:33:39.587019+0000","last_active":"2024-04-23T14:33:40.599237+0000","last_peered":"2024-04-23T14:33:40.599237+0000","last_clean":"2024-04-23T14:33:40.599237+0000","last_became_active":"2024-04-23T14:33:39.586701+0000","last_became_peered":"2024-04-23T14:33:39.586701+0000","last_unstale":"2024-04-23T14:33:40.599237+0000","last_undegraded":"2024-04-23T14:33:40.599237+0000","last_fullsized":"2024-04-23T14:33:40.599237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:04.758788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.554967+0000","last_change":"2024-04-23T14:33:39.588110+0000","last_active":"2024-04-23T14:33:40.554967+0000","last_peered":"2024-04-23T14:33:40.554967+0000","last_clean":"2024-04-23T14:33:40.554967+0000","last_became_active":"2024-04-23T14:33:39.587400+0000","last_became_peered":"2024-04-23T14:33:39.587400+0000","last_unstale":"2024-04-23T14:33:40.554967+0000","last_undegraded":"2024-04-23T14:33:40.554967+0000","last_fullsized":"2024-04-23T14:33:40.554967+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:10:16.577796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555946+0000","last_change":"2024-04-23T14:33:39.583576+0000","last_active":"2024-04-23T14:33:40.555946+0000","last_peered":"2024-04-23T14:33:40.555946+0000","last_clean":"2024-04-23T14:33:40.555946+0000","last_became_active":"2024-04-23T14:33:39.576639+0000","last_became_peered":"2024-04-23T14:33:39.576639+0000","last_unstale":"2024-04-23T14:33:40.555946+0000","last_undegraded":"2024-04-23T14:33:40.555946+0000","last_fullsized":"2024-04-23T14:33:40.555946+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:15:25.124076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555171+0000","last_change":"2024-04-23T14:33:39.577460+0000","last_active":"2024-04-23T14:33:40.555171+0000","last_peered":"2024-04-23T14:33:40.555171+0000","last_clean":"2024-04-23T14:33:40.555171+0000","last_became_active":"2024-04-23T14:33:39.576980+0000","last_became_peered":"2024-04-23T14:33:39.576980+0000","last_unstale":"2024-04-23T14:33:40.555171+0000","last_undegraded":"2024-04-23T14:33:40.555171+0000","last_fullsized":"2024-04-23T14:33:40.555171+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:21:29.390457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.562645+0000","last_change":"2024-04-23T14:33:39.576950+0000","last_active":"2024-04-23T14:33:41.562645+0000","last_peered":"2024-04-23T14:33:41.562645+0000","last_clean":"2024-04-23T14:33:41.562645+0000","last_became_active":"2024-04-23T14:33:39.576398+0000","last_became_peered":"2024-04-23T14:33:39.576398+0000","last_unstale":"2024-04-23T14:33:41.562645+0000","last_undegraded":"2024-04-23T14:33:41.562645+0000","last_fullsized":"2024-04-23T14:33:41.562645+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:11:42.727379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555028+0000","last_change":"2024-04-23T14:33:39.585038+0000","last_active":"2024-04-23T14:33:40.555028+0000","last_peered":"2024-04-23T14:33:40.555028+0000","last_clean":"2024-04-23T14:33:40.555028+0000","last_became_active":"2024-04-23T14:33:39.584726+0000","last_became_peered":"2024-04-23T14:33:39.584726+0000","last_unstale":"2024-04-23T14:33:40.555028+0000","last_undegraded":"2024-04-23T14:33:40.555028+0000","last_fullsized":"2024-04-23T14:33:40.555028+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:06:14.688979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599453+0000","last_change":"2024-04-23T14:33:39.565846+0000","last_active":"2024-04-23T14:33:40.599453+0000","last_peered":"2024-04-23T14:33:40.599453+0000","last_clean":"2024-04-23T14:33:40.599453+0000","last_became_active":"2024-04-23T14:33:39.565515+0000","last_became_peered":"2024-04-23T14:33:39.565515+0000","last_unstale":"2024-04-23T14:33:40.599453+0000","last_undegraded":"2024-04-23T14:33:40.599453+0000","last_fullsized":"2024-04-23T14:33:40.599453+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:35.592096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665416+0000","last_change":"2024-04-23T14:33:39.579363+0000","last_active":"2024-04-23T14:33:41.665416+0000","last_peered":"2024-04-23T14:33:41.665416+0000","last_clean":"2024-04-23T14:33:41.665416+0000","last_became_active":"2024-04-23T14:33:39.579078+0000","last_became_peered":"2024-04-23T14:33:39.579078+0000","last_unstale":"2024-04-23T14:33:41.665416+0000","last_undegraded":"2024-04-23T14:33:41.665416+0000","last_fullsized":"2024-04-23T14:33:41.665416+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:09:39.794441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555589+0000","last_change":"2024-04-23T14:33:39.588147+0000","last_active":"2024-04-23T14:33:40.555589+0000","last_peered":"2024-04-23T14:33:40.555589+0000","last_clean":"2024-04-23T14:33:40.555589+0000","last_became_active":"2024-04-23T14:33:39.587477+0000","last_became_peered":"2024-04-23T14:33:39.587477+0000","last_unstale":"2024-04-23T14:33:40.555589+0000","last_undegraded":"2024-04-23T14:33:40.555589+0000","last_fullsized":"2024-04-23T14:33:40.555589+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:11:49.792303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.562513+0000","last_change":"2024-04-23T14:33:39.590063+0000","last_active":"2024-04-23T14:33:41.562513+0000","last_peered":"2024-04-23T14:33:41.562513+0000","last_clean":"2024-04-23T14:33:41.562513+0000","last_became_active":"2024-04-23T14:33:39.589737+0000","last_became_peered":"2024-04-23T14:33:39.589737+0000","last_unstale":"2024-04-23T14:33:41.562513+0000","last_undegraded":"2024-04-23T14:33:41.562513+0000","last_fullsized":"2024-04-23T14:33:41.562513+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:29:38.570728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555063+0000","last_change":"2024-04-23T14:33:39.575003+0000","last_active":"2024-04-23T14:33:40.555063+0000","last_peered":"2024-04-23T14:33:40.555063+0000","last_clean":"2024-04-23T14:33:40.555063+0000","last_became_active":"2024-04-23T14:33:39.574698+0000","last_became_peered":"2024-04-23T14:33:39.574698+0000","last_unstale":"2024-04-23T14:33:40.555063+0000","last_undegraded":"2024-04-23T14:33:40.555063+0000","last_fullsized":"2024-04-23T14:33:40.555063+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:05:24.295743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665037+0000","last_change":"2024-04-23T14:33:39.576893+0000","last_active":"2024-04-23T14:33:41.665037+0000","last_peered":"2024-04-23T14:33:41.665037+0000","last_clean":"2024-04-23T14:33:41.665037+0000","last_became_active":"2024-04-23T14:33:39.576643+0000","last_became_peered":"2024-04-23T14:33:39.576643+0000","last_unstale":"2024-04-23T14:33:41.665037+0000","last_undegraded":"2024-04-23T14:33:41.665037+0000","last_fullsized":"2024-04-23T14:33:41.665037+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:04:54.194423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555002+0000","last_change":"2024-04-23T14:33:39.578017+0000","last_active":"2024-04-23T14:33:40.555002+0000","last_peered":"2024-04-23T14:33:40.555002+0000","last_clean":"2024-04-23T14:33:40.555002+0000","last_became_active":"2024-04-23T14:33:39.577080+0000","last_became_peered":"2024-04-23T14:33:39.577080+0000","last_unstale":"2024-04-23T14:33:40.555002+0000","last_undegraded":"2024-04-23T14:33:40.555002+0000","last_fullsized":"2024-04-23T14:33:40.555002+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:55:01.766955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599752+0000","last_change":"2024-04-23T14:33:39.588715+0000","last_active":"2024-04-23T14:33:40.599752+0000","last_peered":"2024-04-23T14:33:40.599752+0000","last_clean":"2024-04-23T14:33:40.599752+0000","last_became_active":"2024-04-23T14:33:39.588273+0000","last_became_peered":"2024-04-23T14:33:39.588273+0000","last_unstale":"2024-04-23T14:33:40.599752+0000","last_undegraded":"2024-04-23T14:33:40.599752+0000","last_fullsized":"2024-04-23T14:33:40.599752+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:07:21.114361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.554798+0000","last_change":"2024-04-23T14:33:39.562550+0000","last_active":"2024-04-23T14:33:40.554798+0000","last_peered":"2024-04-23T14:33:40.554798+0000","last_clean":"2024-04-23T14:33:40.554798+0000","last_became_active":"2024-04-23T14:33:39.562221+0000","last_became_peered":"2024-04-23T14:33:39.562221+0000","last_unstale":"2024-04-23T14:33:40.554798+0000","last_undegraded":"2024-04-23T14:33:40.554798+0000","last_fullsized":"2024-04-23T14:33:40.554798+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:06:42.070784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.554804+0000","last_change":"2024-04-23T14:33:39.577502+0000","last_active":"2024-04-23T14:33:40.554804+0000","last_peered":"2024-04-23T14:33:40.554804+0000","last_clean":"2024-04-23T14:33:40.554804+0000","last_became_active":"2024-04-23T14:33:39.577100+0000","last_became_peered":"2024-04-23T14:33:39.577100+0000","last_unstale":"2024-04-23T14:33:40.554804+0000","last_undegraded":"2024-04-23T14:33:40.554804+0000","last_fullsized":"2024-04-23T14:33:40.554804+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:42:41.880055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555703+0000","last_change":"2024-04-23T14:33:39.588390+0000","last_active":"2024-04-23T14:33:40.555703+0000","last_peered":"2024-04-23T14:33:40.555703+0000","last_clean":"2024-04-23T14:33:40.555703+0000","last_became_active":"2024-04-23T14:33:39.587840+0000","last_became_peered":"2024-04-23T14:33:39.587840+0000","last_unstale":"2024-04-23T14:33:40.555703+0000","last_undegraded":"2024-04-23T14:33:40.555703+0000","last_fullsized":"2024-04-23T14:33:40.555703+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:51:28.301256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.664679+0000","last_change":"2024-04-23T14:33:39.577383+0000","last_active":"2024-04-23T14:33:41.664679+0000","last_peered":"2024-04-23T14:33:41.664679+0000","last_clean":"2024-04-23T14:33:41.664679+0000","last_became_active":"2024-04-23T14:33:39.576990+0000","last_became_peered":"2024-04-23T14:33:39.576990+0000","last_unstale":"2024-04-23T14:33:41.664679+0000","last_undegraded":"2024-04-23T14:33:41.664679+0000","last_fullsized":"2024-04-23T14:33:41.664679+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:09:39.794441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.556149+0000","last_change":"2024-04-23T14:33:39.579412+0000","last_active":"2024-04-23T14:33:40.556149+0000","last_peered":"2024-04-23T14:33:40.556149+0000","last_clean":"2024-04-23T14:33:40.556149+0000","last_became_active":"2024-04-23T14:33:39.579163+0000","last_became_peered":"2024-04-23T14:33:39.579163+0000","last_unstale":"2024-04-23T14:33:40.556149+0000","last_undegraded":"2024-04-23T14:33:40.556149+0000","last_fullsized":"2024-04-23T14:33:40.556149+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:30:44.259502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555156+0000","last_change":"2024-04-23T14:33:38.564377+0000","last_active":"2024-04-23T14:33:40.555156+0000","last_peered":"2024-04-23T14:33:40.555156+0000","last_clean":"2024-04-23T14:33:40.555156+0000","last_became_active":"2024-04-23T14:33:38.564062+0000","last_became_peered":"2024-04-23T14:33:38.564062+0000","last_unstale":"2024-04-23T14:33:40.555156+0000","last_undegraded":"2024-04-23T14:33:40.555156+0000","last_fullsized":"2024-04-23T14:33:40.555156+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:31.873991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.664940+0000","last_change":"2024-04-23T14:33:39.578881+0000","last_active":"2024-04-23T14:33:41.664940+0000","last_peered":"2024-04-23T14:33:41.664940+0000","last_clean":"2024-04-23T14:33:41.664940+0000","last_became_active":"2024-04-23T14:33:39.578461+0000","last_became_peered":"2024-04-23T14:33:39.578461+0000","last_unstale":"2024-04-23T14:33:41.664940+0000","last_undegraded":"2024-04-23T14:33:41.664940+0000","last_fullsized":"2024-04-23T14:33:41.664940+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:56:14.508220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555459+0000","last_change":"2024-04-23T14:33:38.545282+0000","last_active":"2024-04-23T14:33:40.555459+0000","last_peered":"2024-04-23T14:33:40.555459+0000","last_clean":"2024-04-23T14:33:40.555459+0000","last_became_active":"2024-04-23T14:33:38.544710+0000","last_became_peered":"2024-04-23T14:33:38.544710+0000","last_unstale":"2024-04-23T14:33:40.555459+0000","last_undegraded":"2024-04-23T14:33:40.555459+0000","last_fullsized":"2024-04-23T14:33:40.555459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:11:41.721234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665583+0000","last_change":"2024-04-23T14:33:39.572263+0000","last_active":"2024-04-23T14:33:41.665583+0000","last_peered":"2024-04-23T14:33:41.665583+0000","last_clean":"2024-04-23T14:33:41.665583+0000","last_became_active":"2024-04-23T14:33:39.571823+0000","last_became_peered":"2024-04-23T14:33:39.571823+0000","last_unstale":"2024-04-23T14:33:41.665583+0000","last_undegraded":"2024-04-23T14:33:41.665583+0000","last_fullsized":"2024-04-23T14:33:41.665583+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:04:54.194423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665008+0000","last_change":"2024-04-23T14:33:38.561205+0000","last_active":"2024-04-23T14:33:41.665008+0000","last_peered":"2024-04-23T14:33:41.665008+0000","last_clean":"2024-04-23T14:33:41.665008+0000","last_became_active":"2024-04-23T14:33:38.560405+0000","last_became_peered":"2024-04-23T14:33:38.560405+0000","last_unstale":"2024-04-23T14:33:41.665008+0000","last_undegraded":"2024-04-23T14:33:41.665008+0000","last_fullsized":"2024-04-23T14:33:41.665008+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:01:45.964960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.367431+0000","last_change":"2024-04-23T14:33:39.580591+0000","last_active":"2024-04-23T14:33:41.367431+0000","last_peered":"2024-04-23T14:33:41.367431+0000","last_clean":"2024-04-23T14:33:41.367431+0000","last_became_active":"2024-04-23T14:33:39.580242+0000","last_became_peered":"2024-04-23T14:33:39.580242+0000","last_unstale":"2024-04-23T14:33:41.367431+0000","last_undegraded":"2024-04-23T14:33:41.367431+0000","last_fullsized":"2024-04-23T14:33:41.367431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:59:21.654571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665426+0000","last_change":"2024-04-23T14:33:38.560326+0000","last_active":"2024-04-23T14:33:41.665426+0000","last_peered":"2024-04-23T14:33:41.665426+0000","last_clean":"2024-04-23T14:33:41.665426+0000","last_became_active":"2024-04-23T14:33:38.560067+0000","last_became_peered":"2024-04-23T14:33:38.560067+0000","last_unstale":"2024-04-23T14:33:41.665426+0000","last_undegraded":"2024-04-23T14:33:41.665426+0000","last_fullsized":"2024-04-23T14:33:41.665426+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:01:45.964960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.554987+0000","last_change":"2024-04-23T14:33:39.563470+0000","last_active":"2024-04-23T14:33:40.554987+0000","last_peered":"2024-04-23T14:33:40.554987+0000","last_clean":"2024-04-23T14:33:40.554987+0000","last_became_active":"2024-04-23T14:33:39.562136+0000","last_became_peered":"2024-04-23T14:33:39.562136+0000","last_unstale":"2024-04-23T14:33:40.554987+0000","last_undegraded":"2024-04-23T14:33:40.554987+0000","last_fullsized":"2024-04-23T14:33:40.554987+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:09:39.794441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599385+0000","last_change":"2024-04-23T14:33:38.546642+0000","last_active":"2024-04-23T14:33:40.599385+0000","last_peered":"2024-04-23T14:33:40.599385+0000","last_clean":"2024-04-23T14:33:40.599385+0000","last_became_active":"2024-04-23T14:33:38.546361+0000","last_became_peered":"2024-04-23T14:33:38.546361+0000","last_unstale":"2024-04-23T14:33:40.599385+0000","last_undegraded":"2024-04-23T14:33:40.599385+0000","last_fullsized":"2024-04-23T14:33:40.599385+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:42:40.873909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599766+0000","last_change":"2024-04-23T14:33:39.563007+0000","last_active":"2024-04-23T14:33:40.599766+0000","last_peered":"2024-04-23T14:33:40.599766+0000","last_clean":"2024-04-23T14:33:40.599766+0000","last_became_active":"2024-04-23T14:33:39.561126+0000","last_became_peered":"2024-04-23T14:33:39.561126+0000","last_unstale":"2024-04-23T14:33:40.599766+0000","last_undegraded":"2024-04-23T14:33:40.599766+0000","last_fullsized":"2024-04-23T14:33:40.599766+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:34.088037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555826+0000","last_change":"2024-04-23T14:33:38.556496+0000","last_active":"2024-04-23T14:33:40.555826+0000","last_peered":"2024-04-23T14:33:40.555826+0000","last_clean":"2024-04-23T14:33:40.555826+0000","last_became_active":"2024-04-23T14:33:38.556228+0000","last_became_peered":"2024-04-23T14:33:38.556228+0000","last_unstale":"2024-04-23T14:33:40.555826+0000","last_undegraded":"2024-04-23T14:33:40.555826+0000","last_fullsized":"2024-04-23T14:33:40.555826+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:52.527966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665492+0000","last_change":"2024-04-23T14:33:39.592397+0000","last_active":"2024-04-23T14:33:41.665492+0000","last_peered":"2024-04-23T14:33:41.665492+0000","last_clean":"2024-04-23T14:33:41.665492+0000","last_became_active":"2024-04-23T14:33:39.592221+0000","last_became_peered":"2024-04-23T14:33:39.592221+0000","last_unstale":"2024-04-23T14:33:41.665492+0000","last_undegraded":"2024-04-23T14:33:41.665492+0000","last_fullsized":"2024-04-23T14:33:41.665492+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:56:14.508220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.664643+0000","last_change":"2024-04-23T14:33:38.558760+0000","last_active":"2024-04-23T14:33:41.664643+0000","last_peered":"2024-04-23T14:33:41.664643+0000","last_clean":"2024-04-23T14:33:41.664643+0000","last_became_active":"2024-04-23T14:33:38.558390+0000","last_became_peered":"2024-04-23T14:33:38.558390+0000","last_unstale":"2024-04-23T14:33:41.664643+0000","last_undegraded":"2024-04-23T14:33:41.664643+0000","last_fullsized":"2024-04-23T14:33:41.664643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:52.527966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.366930+0000","last_change":"2024-04-23T14:33:39.587661+0000","last_active":"2024-04-23T14:33:41.366930+0000","last_peered":"2024-04-23T14:33:41.366930+0000","last_clean":"2024-04-23T14:33:41.366930+0000","last_became_active":"2024-04-23T14:33:39.587332+0000","last_became_peered":"2024-04-23T14:33:39.587332+0000","last_unstale":"2024-04-23T14:33:41.366930+0000","last_undegraded":"2024-04-23T14:33:41.366930+0000","last_fullsized":"2024-04-23T14:33:41.366930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:34.088037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665062+0000","last_change":"2024-04-23T14:33:38.558584+0000","last_active":"2024-04-23T14:33:41.665062+0000","last_peered":"2024-04-23T14:33:41.665062+0000","last_clean":"2024-04-23T14:33:41.665062+0000","last_became_active":"2024-04-23T14:33:38.558265+0000","last_became_peered":"2024-04-23T14:33:38.558265+0000","last_unstale":"2024-04-23T14:33:41.665062+0000","last_undegraded":"2024-04-23T14:33:41.665062+0000","last_fullsized":"2024-04-23T14:33:41.665062+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:29.902113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.562594+0000","last_change":"2024-04-23T14:33:39.586168+0000","last_active":"2024-04-23T14:33:41.562594+0000","last_peered":"2024-04-23T14:33:41.562594+0000","last_clean":"2024-04-23T14:33:41.562594+0000","last_became_active":"2024-04-23T14:33:39.585305+0000","last_became_peered":"2024-04-23T14:33:39.585305+0000","last_unstale":"2024-04-23T14:33:41.562594+0000","last_undegraded":"2024-04-23T14:33:41.562594+0000","last_fullsized":"2024-04-23T14:33:41.562594+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:01:46.971105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.367487+0000","last_change":"2024-04-23T14:33:38.555154+0000","last_active":"2024-04-23T14:33:41.367487+0000","last_peered":"2024-04-23T14:33:41.367487+0000","last_clean":"2024-04-23T14:33:41.367487+0000","last_became_active":"2024-04-23T14:33:38.554887+0000","last_became_peered":"2024-04-23T14:33:38.554887+0000","last_unstale":"2024-04-23T14:33:41.367487+0000","last_undegraded":"2024-04-23T14:33:41.367487+0000","last_fullsized":"2024-04-23T14:33:41.367487+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:52:58.709384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.556460+0000","last_change":"2024-04-23T14:33:39.576139+0000","last_active":"2024-04-23T14:33:40.556460+0000","last_peered":"2024-04-23T14:33:40.556460+0000","last_clean":"2024-04-23T14:33:40.556460+0000","last_became_active":"2024-04-23T14:33:39.575854+0000","last_became_peered":"2024-04-23T14:33:39.575854+0000","last_unstale":"2024-04-23T14:33:40.556460+0000","last_undegraded":"2024-04-23T14:33:40.556460+0000","last_fullsized":"2024-04-23T14:33:40.556460+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:38:35.592096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599794+0000","last_change":"2024-04-23T14:33:38.555826+0000","last_active":"2024-04-23T14:33:40.599794+0000","last_peered":"2024-04-23T14:33:40.599794+0000","last_clean":"2024-04-23T14:33:40.599794+0000","last_became_active":"2024-04-23T14:33:38.555473+0000","last_became_peered":"2024-04-23T14:33:38.555473+0000","last_unstale":"2024-04-23T14:33:40.599794+0000","last_undegraded":"2024-04-23T14:33:40.599794+0000","last_fullsized":"2024-04-23T14:33:40.599794+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:55:00.760809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555384+0000","last_change":"2024-04-23T14:33:39.589371+0000","last_active":"2024-04-23T14:33:40.555384+0000","last_peered":"2024-04-23T14:33:40.555384+0000","last_clean":"2024-04-23T14:33:40.555384+0000","last_became_active":"2024-04-23T14:33:39.588894+0000","last_became_peered":"2024-04-23T14:33:39.588894+0000","last_unstale":"2024-04-23T14:33:40.555384+0000","last_undegraded":"2024-04-23T14:33:40.555384+0000","last_fullsized":"2024-04-23T14:33:40.555384+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:52:59.715529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599658+0000","last_change":"2024-04-23T14:33:38.560341+0000","last_active":"2024-04-23T14:33:40.599658+0000","last_peered":"2024-04-23T14:33:40.599658+0000","last_clean":"2024-04-23T14:33:40.599658+0000","last_became_active":"2024-04-23T14:33:38.560058+0000","last_became_peered":"2024-04-23T14:33:38.560058+0000","last_unstale":"2024-04-23T14:33:40.599658+0000","last_undegraded":"2024-04-23T14:33:40.599658+0000","last_fullsized":"2024-04-23T14:33:40.599658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:37.098622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665588+0000","last_change":"2024-04-23T14:33:39.580339+0000","last_active":"2024-04-23T14:33:41.665588+0000","last_peered":"2024-04-23T14:33:41.665588+0000","last_clean":"2024-04-23T14:33:41.665588+0000","last_became_active":"2024-04-23T14:33:39.580003+0000","last_became_peered":"2024-04-23T14:33:39.580003+0000","last_unstale":"2024-04-23T14:33:41.665588+0000","last_undegraded":"2024-04-23T14:33:41.665588+0000","last_fullsized":"2024-04-23T14:33:41.665588+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:10:16.577796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.367306+0000","last_change":"2024-04-23T14:33:38.554839+0000","last_active":"2024-04-23T14:33:41.367306+0000","last_peered":"2024-04-23T14:33:41.367306+0000","last_clean":"2024-04-23T14:33:41.367306+0000","last_became_active":"2024-04-23T14:33:38.554575+0000","last_became_peered":"2024-04-23T14:33:38.554575+0000","last_unstale":"2024-04-23T14:33:41.367306+0000","last_undegraded":"2024-04-23T14:33:41.367306+0000","last_fullsized":"2024-04-23T14:33:41.367306+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:10:15.571650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.554989+0000","last_change":"2024-04-23T14:33:39.566810+0000","last_active":"2024-04-23T14:33:40.554989+0000","last_peered":"2024-04-23T14:33:40.554989+0000","last_clean":"2024-04-23T14:33:40.554989+0000","last_became_active":"2024-04-23T14:33:39.566557+0000","last_became_peered":"2024-04-23T14:33:39.566557+0000","last_unstale":"2024-04-23T14:33:40.554989+0000","last_undegraded":"2024-04-23T14:33:40.554989+0000","last_fullsized":"2024-04-23T14:33:40.554989+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:21:29.390457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.367188+0000","last_change":"2024-04-23T14:33:38.556392+0000","last_active":"2024-04-23T14:33:41.367188+0000","last_peered":"2024-04-23T14:33:41.367188+0000","last_clean":"2024-04-23T14:33:41.367188+0000","last_became_active":"2024-04-23T14:33:38.555979+0000","last_became_peered":"2024-04-23T14:33:38.555979+0000","last_unstale":"2024-04-23T14:33:41.367188+0000","last_undegraded":"2024-04-23T14:33:41.367188+0000","last_fullsized":"2024-04-23T14:33:41.367188+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:04:53.188277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.366998+0000","last_change":"2024-04-23T14:33:39.578228+0000","last_active":"2024-04-23T14:33:41.366998+0000","last_peered":"2024-04-23T14:33:41.366998+0000","last_clean":"2024-04-23T14:33:41.366998+0000","last_became_active":"2024-04-23T14:33:39.577971+0000","last_became_peered":"2024-04-23T14:33:39.577971+0000","last_unstale":"2024-04-23T14:33:41.366998+0000","last_undegraded":"2024-04-23T14:33:41.366998+0000","last_fullsized":"2024-04-23T14:33:41.366998+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:45:10.440511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.664640+0000","last_change":"2024-04-23T14:33:38.561030+0000","last_active":"2024-04-23T14:33:41.664640+0000","last_peered":"2024-04-23T14:33:41.664640+0000","last_clean":"2024-04-23T14:33:41.664640+0000","last_became_active":"2024-04-23T14:33:38.560751+0000","last_became_peered":"2024-04-23T14:33:38.560751+0000","last_unstale":"2024-04-23T14:33:41.664640+0000","last_undegraded":"2024-04-23T14:33:41.664640+0000","last_fullsized":"2024-04-23T14:33:41.664640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:38:45.384204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555695+0000","last_change":"2024-04-23T14:33:39.588591+0000","last_active":"2024-04-23T14:33:40.555695+0000","last_peered":"2024-04-23T14:33:40.555695+0000","last_clean":"2024-04-23T14:33:40.555695+0000","last_became_active":"2024-04-23T14:33:39.588159+0000","last_became_peered":"2024-04-23T14:33:39.588159+0000","last_unstale":"2024-04-23T14:33:40.555695+0000","last_undegraded":"2024-04-23T14:33:40.555695+0000","last_fullsized":"2024-04-23T14:33:40.555695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:08:00.391713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.554668+0000","last_change":"2024-04-23T14:33:38.561064+0000","last_active":"2024-04-23T14:33:40.554668+0000","last_peered":"2024-04-23T14:33:40.554668+0000","last_clean":"2024-04-23T14:33:40.554668+0000","last_became_active":"2024-04-23T14:33:38.560797+0000","last_became_peered":"2024-04-23T14:33:38.560797+0000","last_unstale":"2024-04-23T14:33:40.554668+0000","last_undegraded":"2024-04-23T14:33:40.554668+0000","last_fullsized":"2024-04-23T14:33:40.554668+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:14:14.334310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.367111+0000","last_change":"2024-04-23T14:33:39.579653+0000","last_active":"2024-04-23T14:33:41.367111+0000","last_peered":"2024-04-23T14:33:41.367111+0000","last_clean":"2024-04-23T14:33:41.367111+0000","last_became_active":"2024-04-23T14:33:39.579360+0000","last_became_peered":"2024-04-23T14:33:39.579360+0000","last_unstale":"2024-04-23T14:33:41.367111+0000","last_undegraded":"2024-04-23T14:33:41.367111+0000","last_fullsized":"2024-04-23T14:33:41.367111+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:10:49.093987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555340+0000","last_change":"2024-04-23T14:33:38.555718+0000","last_active":"2024-04-23T14:33:40.555340+0000","last_peered":"2024-04-23T14:33:40.555340+0000","last_clean":"2024-04-23T14:33:40.555340+0000","last_became_active":"2024-04-23T14:33:38.555445+0000","last_became_peered":"2024-04-23T14:33:38.555445+0000","last_unstale":"2024-04-23T14:33:40.555340+0000","last_undegraded":"2024-04-23T14:33:40.555340+0000","last_fullsized":"2024-04-23T14:33:40.555340+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:11:41.721234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.367353+0000","last_change":"2024-04-23T14:33:39.581109+0000","last_active":"2024-04-23T14:33:41.367353+0000","last_peered":"2024-04-23T14:33:41.367353+0000","last_clean":"2024-04-23T14:33:41.367353+0000","last_became_active":"2024-04-23T14:33:39.580853+0000","last_became_peered":"2024-04-23T14:33:39.580853+0000","last_unstale":"2024-04-23T14:33:41.367353+0000","last_undegraded":"2024-04-23T14:33:41.367353+0000","last_fullsized":"2024-04-23T14:33:41.367353+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:50:18.302699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599647+0000","last_change":"2024-04-23T14:33:38.561267+0000","last_active":"2024-04-23T14:33:40.599647+0000","last_peered":"2024-04-23T14:33:40.599647+0000","last_clean":"2024-04-23T14:33:40.599647+0000","last_became_active":"2024-04-23T14:33:38.560607+0000","last_became_peered":"2024-04-23T14:33:38.560607+0000","last_unstale":"2024-04-23T14:33:40.599647+0000","last_undegraded":"2024-04-23T14:33:40.599647+0000","last_fullsized":"2024-04-23T14:33:40.599647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:38:45.384204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599685+0000","last_change":"2024-04-23T14:33:39.566783+0000","last_active":"2024-04-23T14:33:40.599685+0000","last_peered":"2024-04-23T14:33:40.599685+0000","last_clean":"2024-04-23T14:33:40.599685+0000","last_became_active":"2024-04-23T14:33:39.566487+0000","last_became_peered":"2024-04-23T14:33:39.566487+0000","last_unstale":"2024-04-23T14:33:40.599685+0000","last_undegraded":"2024-04-23T14:33:40.599685+0000","last_fullsized":"2024-04-23T14:33:40.599685+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:17:01.997568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665583+0000","last_change":"2024-04-23T14:33:38.557553+0000","last_active":"2024-04-23T14:33:41.665583+0000","last_peered":"2024-04-23T14:33:41.665583+0000","last_clean":"2024-04-23T14:33:41.665583+0000","last_became_active":"2024-04-23T14:33:38.557269+0000","last_became_peered":"2024-04-23T14:33:38.557269+0000","last_unstale":"2024-04-23T14:33:41.665583+0000","last_undegraded":"2024-04-23T14:33:41.665583+0000","last_fullsized":"2024-04-23T14:33:41.665583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:29:37.564582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665487+0000","last_change":"2024-04-23T14:33:39.573689+0000","last_active":"2024-04-23T14:33:41.665487+0000","last_peered":"2024-04-23T14:33:41.665487+0000","last_clean":"2024-04-23T14:33:41.665487+0000","last_became_active":"2024-04-23T14:33:39.573393+0000","last_became_peered":"2024-04-23T14:33:39.573393+0000","last_unstale":"2024-04-23T14:33:41.665487+0000","last_undegraded":"2024-04-23T14:33:41.665487+0000","last_fullsized":"2024-04-23T14:33:41.665487+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:39.146588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.562586+0000","last_change":"2024-04-23T14:33:38.545969+0000","last_active":"2024-04-23T14:33:41.562586+0000","last_peered":"2024-04-23T14:33:41.562586+0000","last_clean":"2024-04-23T14:33:41.562586+0000","last_became_active":"2024-04-23T14:33:38.545512+0000","last_became_peered":"2024-04-23T14:33:38.545512+0000","last_unstale":"2024-04-23T14:33:41.562586+0000","last_undegraded":"2024-04-23T14:33:41.562586+0000","last_fullsized":"2024-04-23T14:33:41.562586+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:57:34.344856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.562841+0000","last_change":"2024-04-23T14:33:39.590166+0000","last_active":"2024-04-23T14:33:41.562841+0000","last_peered":"2024-04-23T14:33:41.562841+0000","last_clean":"2024-04-23T14:33:41.562841+0000","last_became_active":"2024-04-23T14:33:39.589722+0000","last_became_peered":"2024-04-23T14:33:39.589722+0000","last_unstale":"2024-04-23T14:33:41.562841+0000","last_undegraded":"2024-04-23T14:33:41.562841+0000","last_fullsized":"2024-04-23T14:33:41.562841+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:53.534112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.664863+0000","last_change":"2024-04-23T14:33:38.551345+0000","last_active":"2024-04-23T14:33:41.664863+0000","last_peered":"2024-04-23T14:33:41.664863+0000","last_clean":"2024-04-23T14:33:41.664863+0000","last_became_active":"2024-04-23T14:33:38.551084+0000","last_became_peered":"2024-04-23T14:33:38.551084+0000","last_unstale":"2024-04-23T14:33:41.664863+0000","last_undegraded":"2024-04-23T14:33:41.664863+0000","last_fullsized":"2024-04-23T14:33:41.664863+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:11:41.721234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555427+0000","last_change":"2024-04-23T14:33:39.588843+0000","last_active":"2024-04-23T14:33:40.555427+0000","last_peered":"2024-04-23T14:33:40.555427+0000","last_clean":"2024-04-23T14:33:40.555427+0000","last_became_active":"2024-04-23T14:33:39.587822+0000","last_became_peered":"2024-04-23T14:33:39.587822+0000","last_unstale":"2024-04-23T14:33:40.555427+0000","last_undegraded":"2024-04-23T14:33:40.555427+0000","last_fullsized":"2024-04-23T14:33:40.555427+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:52:59.715529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.562410+0000","last_change":"2024-04-23T14:33:38.539420+0000","last_active":"2024-04-23T14:33:41.562410+0000","last_peered":"2024-04-23T14:33:41.562410+0000","last_clean":"2024-04-23T14:33:41.562410+0000","last_became_active":"2024-04-23T14:33:38.538810+0000","last_became_peered":"2024-04-23T14:33:38.538810+0000","last_unstale":"2024-04-23T14:33:41.562410+0000","last_undegraded":"2024-04-23T14:33:41.562410+0000","last_fullsized":"2024-04-23T14:33:41.562410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:38.198200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555107+0000","last_change":"2024-04-23T14:33:39.582674+0000","last_active":"2024-04-23T14:33:40.555107+0000","last_peered":"2024-04-23T14:33:40.555107+0000","last_clean":"2024-04-23T14:33:40.555107+0000","last_became_active":"2024-04-23T14:33:39.582398+0000","last_became_peered":"2024-04-23T14:33:39.582398+0000","last_unstale":"2024-04-23T14:33:40.555107+0000","last_undegraded":"2024-04-23T14:33:40.555107+0000","last_fullsized":"2024-04-23T14:33:40.555107+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:39.146588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.562346+0000","last_change":"2024-04-23T14:33:38.551409+0000","last_active":"2024-04-23T14:33:41.562346+0000","last_peered":"2024-04-23T14:33:41.562346+0000","last_clean":"2024-04-23T14:33:41.562346+0000","last_became_active":"2024-04-23T14:33:38.551145+0000","last_became_peered":"2024-04-23T14:33:38.551145+0000","last_unstale":"2024-04-23T14:33:41.562346+0000","last_undegraded":"2024-04-23T14:33:41.562346+0000","last_fullsized":"2024-04-23T14:33:41.562346+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:33.119876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599329+0000","last_change":"2024-04-23T14:33:39.564292+0000","last_active":"2024-04-23T14:33:40.599329+0000","last_peered":"2024-04-23T14:33:40.599329+0000","last_clean":"2024-04-23T14:33:40.599329+0000","last_became_active":"2024-04-23T14:33:39.564025+0000","last_became_peered":"2024-04-23T14:33:39.564025+0000","last_unstale":"2024-04-23T14:33:40.599329+0000","last_undegraded":"2024-04-23T14:33:40.599329+0000","last_fullsized":"2024-04-23T14:33:40.599329+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:30:44.259502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555838+0000","last_change":"2024-04-23T14:33:38.554466+0000","last_active":"2024-04-23T14:33:40.555838+0000","last_peered":"2024-04-23T14:33:40.555838+0000","last_clean":"2024-04-23T14:33:40.555838+0000","last_became_active":"2024-04-23T14:33:38.554155+0000","last_became_peered":"2024-04-23T14:33:38.554155+0000","last_unstale":"2024-04-23T14:33:40.555838+0000","last_undegraded":"2024-04-23T14:33:40.555838+0000","last_fullsized":"2024-04-23T14:33:40.555838+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:38:45.384204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.367091+0000","last_change":"2024-04-23T14:33:39.587638+0000","last_active":"2024-04-23T14:33:41.367091+0000","last_peered":"2024-04-23T14:33:41.367091+0000","last_clean":"2024-04-23T14:33:41.367091+0000","last_became_active":"2024-04-23T14:33:39.587344+0000","last_became_peered":"2024-04-23T14:33:39.587344+0000","last_unstale":"2024-04-23T14:33:41.367091+0000","last_undegraded":"2024-04-23T14:33:41.367091+0000","last_fullsized":"2024-04-23T14:33:41.367091+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:45:25.313578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.562339+0000","last_change":"2024-04-23T14:33:38.539108+0000","last_active":"2024-04-23T14:33:41.562339+0000","last_peered":"2024-04-23T14:33:41.562339+0000","last_clean":"2024-04-23T14:33:41.562339+0000","last_became_active":"2024-04-23T14:33:38.538806+0000","last_became_peered":"2024-04-23T14:33:38.538806+0000","last_unstale":"2024-04-23T14:33:41.562339+0000","last_undegraded":"2024-04-23T14:33:41.562339+0000","last_fullsized":"2024-04-23T14:33:41.562339+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:14:49.245496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.664847+0000","last_change":"2024-04-23T14:33:39.579759+0000","last_active":"2024-04-23T14:33:41.664847+0000","last_peered":"2024-04-23T14:33:41.664847+0000","last_clean":"2024-04-23T14:33:41.664847+0000","last_became_active":"2024-04-23T14:33:39.579458+0000","last_became_peered":"2024-04-23T14:33:39.579458+0000","last_unstale":"2024-04-23T14:33:41.664847+0000","last_undegraded":"2024-04-23T14:33:41.664847+0000","last_fullsized":"2024-04-23T14:33:41.664847+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:39.146588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599759+0000","last_change":"2024-04-23T14:33:38.559480+0000","last_active":"2024-04-23T14:33:40.599759+0000","last_peered":"2024-04-23T14:33:40.599759+0000","last_clean":"2024-04-23T14:33:40.599759+0000","last_became_active":"2024-04-23T14:33:38.559223+0000","last_became_peered":"2024-04-23T14:33:38.559223+0000","last_unstale":"2024-04-23T14:33:40.599759+0000","last_undegraded":"2024-04-23T14:33:40.599759+0000","last_fullsized":"2024-04-23T14:33:40.599759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:06:41.064638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555920+0000","last_change":"2024-04-23T14:33:39.576658+0000","last_active":"2024-04-23T14:33:40.555920+0000","last_peered":"2024-04-23T14:33:40.555920+0000","last_clean":"2024-04-23T14:33:40.555920+0000","last_became_active":"2024-04-23T14:33:39.576402+0000","last_became_peered":"2024-04-23T14:33:39.576402+0000","last_unstale":"2024-04-23T14:33:40.555920+0000","last_undegraded":"2024-04-23T14:33:40.555920+0000","last_fullsized":"2024-04-23T14:33:40.555920+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:35:49.748984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.564757+0000","last_change":"2024-04-23T14:33:38.547438+0000","last_active":"2024-04-23T14:33:41.564757+0000","last_peered":"2024-04-23T14:33:41.564757+0000","last_clean":"2024-04-23T14:33:41.564757+0000","last_became_active":"2024-04-23T14:33:38.547054+0000","last_became_peered":"2024-04-23T14:33:38.547054+0000","last_unstale":"2024-04-23T14:33:41.564757+0000","last_undegraded":"2024-04-23T14:33:41.564757+0000","last_fullsized":"2024-04-23T14:33:41.564757+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:57:37.478718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.367385+0000","last_change":"2024-04-23T14:33:39.580248+0000","last_active":"2024-04-23T14:33:41.367385+0000","last_peered":"2024-04-23T14:33:41.367385+0000","last_clean":"2024-04-23T14:33:41.367385+0000","last_became_active":"2024-04-23T14:33:39.579977+0000","last_became_peered":"2024-04-23T14:33:39.579977+0000","last_unstale":"2024-04-23T14:33:41.367385+0000","last_undegraded":"2024-04-23T14:33:41.367385+0000","last_fullsized":"2024-04-23T14:33:41.367385+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:43:52.636588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555302+0000","last_change":"2024-04-23T14:33:38.551995+0000","last_active":"2024-04-23T14:33:40.555302+0000","last_peered":"2024-04-23T14:33:40.555302+0000","last_clean":"2024-04-23T14:33:40.555302+0000","last_became_active":"2024-04-23T14:33:38.551685+0000","last_became_peered":"2024-04-23T14:33:38.551685+0000","last_unstale":"2024-04-23T14:33:40.555302+0000","last_undegraded":"2024-04-23T14:33:40.555302+0000","last_fullsized":"2024-04-23T14:33:40.555302+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:01:45.964960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555136+0000","last_change":"2024-04-23T14:33:39.585259+0000","last_active":"2024-04-23T14:33:40.555136+0000","last_peered":"2024-04-23T14:33:40.555136+0000","last_clean":"2024-04-23T14:33:40.555136+0000","last_became_active":"2024-04-23T14:33:39.585011+0000","last_became_peered":"2024-04-23T14:33:39.585011+0000","last_unstale":"2024-04-23T14:33:40.555136+0000","last_undegraded":"2024-04-23T14:33:40.555136+0000","last_fullsized":"2024-04-23T14:33:40.555136+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:56:14.508220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555251+0000","last_change":"2024-04-23T14:33:38.553908+0000","last_active":"2024-04-23T14:33:40.555251+0000","last_peered":"2024-04-23T14:33:40.555251+0000","last_clean":"2024-04-23T14:33:40.555251+0000","last_became_active":"2024-04-23T14:33:38.553483+0000","last_became_peered":"2024-04-23T14:33:38.553483+0000","last_unstale":"2024-04-23T14:33:40.555251+0000","last_undegraded":"2024-04-23T14:33:40.555251+0000","last_fullsized":"2024-04-23T14:33:40.555251+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:32:41.665983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.664679+0000","last_change":"2024-04-23T14:33:39.565585+0000","last_active":"2024-04-23T14:33:41.664679+0000","last_peered":"2024-04-23T14:33:41.664679+0000","last_clean":"2024-04-23T14:33:41.664679+0000","last_became_active":"2024-04-23T14:33:39.565310+0000","last_became_peered":"2024-04-23T14:33:39.565310+0000","last_unstale":"2024-04-23T14:33:41.664679+0000","last_undegraded":"2024-04-23T14:33:41.664679+0000","last_fullsized":"2024-04-23T14:33:41.664679+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:21:29.390457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.554757+0000","last_change":"2024-04-23T14:33:38.559974+0000","last_active":"2024-04-23T14:33:40.554757+0000","last_peered":"2024-04-23T14:33:40.554757+0000","last_clean":"2024-04-23T14:33:40.554757+0000","last_became_active":"2024-04-23T14:33:38.557889+0000","last_became_peered":"2024-04-23T14:33:38.557889+0000","last_unstale":"2024-04-23T14:33:40.554757+0000","last_undegraded":"2024-04-23T14:33:40.554757+0000","last_fullsized":"2024-04-23T14:33:40.554757+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:29.902113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665201+0000","last_change":"2024-04-23T14:33:39.583378+0000","last_active":"2024-04-23T14:33:41.665201+0000","last_peered":"2024-04-23T14:33:41.665201+0000","last_clean":"2024-04-23T14:33:41.665201+0000","last_became_active":"2024-04-23T14:33:39.583100+0000","last_became_peered":"2024-04-23T14:33:39.583100+0000","last_unstale":"2024-04-23T14:33:41.665201+0000","last_undegraded":"2024-04-23T14:33:41.665201+0000","last_fullsized":"2024-04-23T14:33:41.665201+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:21:29.390457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.665041+0000","last_change":"2024-04-23T14:33:38.560737+0000","last_active":"2024-04-23T14:33:41.665041+0000","last_peered":"2024-04-23T14:33:41.665041+0000","last_clean":"2024-04-23T14:33:41.665041+0000","last_became_active":"2024-04-23T14:33:38.559721+0000","last_became_peered":"2024-04-23T14:33:38.559721+0000","last_unstale":"2024-04-23T14:33:41.665041+0000","last_undegraded":"2024-04-23T14:33:41.665041+0000","last_fullsized":"2024-04-23T14:33:41.665041+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:32:41.665983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.556037+0000","last_change":"2024-04-23T14:33:39.586013+0000","last_active":"2024-04-23T14:33:40.556037+0000","last_peered":"2024-04-23T14:33:40.556037+0000","last_clean":"2024-04-23T14:33:40.556037+0000","last_became_active":"2024-04-23T14:33:39.583815+0000","last_became_peered":"2024-04-23T14:33:39.583815+0000","last_unstale":"2024-04-23T14:33:40.556037+0000","last_undegraded":"2024-04-23T14:33:40.556037+0000","last_fullsized":"2024-04-23T14:33:40.556037+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:04.758788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:41.366952+0000","last_change":"2024-04-23T14:33:38.558963+0000","last_active":"2024-04-23T14:33:41.366952+0000","last_peered":"2024-04-23T14:33:41.366952+0000","last_clean":"2024-04-23T14:33:41.366952+0000","last_became_active":"2024-04-23T14:33:38.558410+0000","last_became_peered":"2024-04-23T14:33:38.558410+0000","last_unstale":"2024-04-23T14:33:41.366952+0000","last_undegraded":"2024-04-23T14:33:41.366952+0000","last_fullsized":"2024-04-23T14:33:41.366952+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:56:13.502074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.599591+0000","last_change":"2024-04-23T14:33:39.588638+0000","last_active":"2024-04-23T14:33:40.599591+0000","last_peered":"2024-04-23T14:33:40.599591+0000","last_clean":"2024-04-23T14:33:40.599591+0000","last_became_active":"2024-04-23T14:33:39.588262+0000","last_became_peered":"2024-04-23T14:33:39.588262+0000","last_unstale":"2024-04-23T14:33:40.599591+0000","last_undegraded":"2024-04-23T14:33:40.599591+0000","last_fullsized":"2024-04-23T14:33:40.599591+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:38.528162+0000","last_clean_scrub_stamp":"2024-04-23T14:33:38.528162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:11:21.071775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.556159+0000","last_change":"2024-04-23T14:33:38.550575+0000","last_active":"2024-04-23T14:33:40.556159+0000","last_peered":"2024-04-23T14:33:40.556159+0000","last_clean":"2024-04-23T14:33:40.556159+0000","last_became_active":"2024-04-23T14:33:38.550160+0000","last_became_peered":"2024-04-23T14:33:38.550160+0000","last_unstale":"2024-04-23T14:33:40.556159+0000","last_undegraded":"2024-04-23T14:33:40.556159+0000","last_fullsized":"2024-04-23T14:33:40.556159+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:37.522016+0000","last_clean_scrub_stamp":"2024-04-23T14:33:37.522016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:37.098622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-23T14:33:40.555413+0000","last_change":"2024-04-23T14:31:58.314235+0000","last_active":"2024-04-23T14:33:40.555413+0000","last_peered":"2024-04-23T14:33:40.555413+0000","last_clean":"2024-04-23T14:33:40.555413+0000","last_became_active":"2024-04-23T14:31:58.313165+0000","last_became_peered":"2024-04-23T14:31:58.313165+0000","last_unstale":"2024-04-23T14:33:40.555413+0000","last_undegraded":"2024-04-23T14:33:40.555413+0000","last_fullsized":"2024-04-23T14:33:40.555413+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_clean_scrub_stamp":"2024-04-23T14:31:57.140712+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:44.170436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":782078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":190610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":190140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27444,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":434176,"data_stored":183940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":184982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":183406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":183426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1036288,"data_stored":780430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:33:44.027 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-23T14:33:44.027 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-23T14:33:44.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f5526000640 1 Processor -- start 2024-04-23T14:33:44.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f5526000640 1 -- start start 2024-04-23T14:33:44.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f5526000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f552015e860 0x7f55200b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f5526000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55200ae3e0 con 0x7f55200adfb0 2024-04-23T14:33:44.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f5526000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f552006e5f0 con 0x7f55200adb90 2024-04-23T14:33:44.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f5526000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f552006e8b0 con 0x7f552015e860 2024-04-23T14:33:44.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551fe00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f55200adfb0 0x7f552006d350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57024/0 (socket says 172.21.15.100:57024) 2024-04-23T14:33:44.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551fe00640 1 -- 172.21.15.100:0/428395153 learned_addr learned my addr 172.21.15.100:0/428395153 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:44.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551f400640 1 --2- 172.21.15.100:0/428395153 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f552015e860 0x7f55200b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551e000640 1 -- 172.21.15.100:0/428395153 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4081529138 0 0) 0x7f55200ae3e0 con 0x7f55200adfb0 2024-04-23T14:33:44.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551e000640 1 -- 172.21.15.100:0/428395153 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5504003530 con 0x7f55200adfb0 2024-04-23T14:33:44.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551e000640 1 -- 172.21.15.100:0/428395153 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1904116032 0 0) 0x7f552006e5f0 con 0x7f55200adb90 2024-04-23T14:33:44.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551e000640 1 -- 172.21.15.100:0/428395153 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55200ae3e0 con 0x7f55200adb90 2024-04-23T14:33:44.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551e000640 1 -- 172.21.15.100:0/428395153 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 279137926 0 0) 0x7f5504003530 con 0x7f55200adfb0 2024-04-23T14:33:44.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551e000640 1 -- 172.21.15.100:0/428395153 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f552015e860 msgr2=0x7f55200b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:44.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551e000640 1 --2- 172.21.15.100:0/428395153 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f552015e860 0x7f55200b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551e000640 1 -- 172.21.15.100:0/428395153 >> v1:172.21.15.151:6789/0 conn(0x7f55200adb90 legacy=0x7f552006f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551e000640 1 -- 172.21.15.100:0/428395153 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552006eb90 con 0x7f55200adfb0 2024-04-23T14:33:44.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551e000640 1 -- 172.21.15.100:0/428395153 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f5518001a00 con 0x7f55200adfb0 2024-04-23T14:33:44.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551e000640 1 -- 172.21.15.100:0/428395153 >> v1:172.21.15.100:6789/0 conn(0x7f55200adfb0 legacy=0x7f552006d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551e000640 1 --2- 172.21.15.100:0/428395153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5504003b80 0x7f5504003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551e000640 1 --2- 172.21.15.100:0/428395153 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5504004810 0x7f5504004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551e000640 1 --2- 172.21.15.100:0/428395153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55040051d0 0x7f550400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551e000640 1 -- 172.21.15.100:0/428395153 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f552006e8b0 con 0x7f5504003b80 2024-04-23T14:33:44.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551e000640 1 -- 172.21.15.100:0/428395153 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f550400bfc0 con 0x7f55040051d0 2024-04-23T14:33:44.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551e000640 1 -- 172.21.15.100:0/428395153 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f550400c1c0 con 0x7f5504004810 2024-04-23T14:33:44.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551f400640 1 --2- 172.21.15.100:0/428395153 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f552015e860 0x7f55200b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:33:44.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551f400640 1 --2- 172.21.15.100:0/428395153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5504003b80 0x7f5504003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551ea00640 1 --2- 172.21.15.100:0/428395153 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5504004810 0x7f5504004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551fe00640 1 --2- 172.21.15.100:0/428395153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55040051d0 0x7f550400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551f400640 1 -- 172.21.15.100:0/428395153 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5504004810 msgr2=0x7f5504004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:44.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551f400640 1 --2- 172.21.15.100:0/428395153 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5504004810 0x7f5504004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551f400640 1 -- 172.21.15.100:0/428395153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55040051d0 msgr2=0x7f550400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:44.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551f400640 1 --2- 172.21.15.100:0/428395153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55040051d0 0x7f550400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.204+0000 7f551f400640 1 -- 172.21.15.100:0/428395153 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5504003530 con 0x7f5504003b80 2024-04-23T14:33:44.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.208+0000 7f551f400640 1 --2- 172.21.15.100:0/428395153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5504003b80 0x7f5504003f80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5520071cc0 tx=0x7f551001ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd00efd9bd050424 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:44.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.208+0000 7f551e000640 1 -- 172.21.15.100:0/428395153 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55100097f0 con 0x7f5504003b80 2024-04-23T14:33:44.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.208+0000 7f551e000640 1 -- 172.21.15.100:0/428395153 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5510004030 con 0x7f5504003b80 2024-04-23T14:33:44.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.208+0000 7f551e000640 1 -- 172.21.15.100:0/428395153 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5510004400 con 0x7f5504003b80 2024-04-23T14:33:44.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.208+0000 7f5526000640 1 -- 172.21.15.100:0/428395153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5504003b80 msgr2=0x7f5504003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.208+0000 7f5526000640 1 --2- 172.21.15.100:0/428395153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5504003b80 0x7f5504003f80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5520071cc0 tx=0x7f551001ca80 comp rx=0 tx=0).stop 2024-04-23T14:33:44.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.208+0000 7f551fe00640 1 -- 172.21.15.100:0/428395153 reap_dead start 2024-04-23T14:33:44.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.208+0000 7f5526000640 1 -- 172.21.15.100:0/428395153 shutdown_connections 2024-04-23T14:33:44.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.208+0000 7f5526000640 1 -- 172.21.15.100:0/428395153 >> 172.21.15.100:0/428395153 conn(0x7f5520072590 msgr2=0x7f55200729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:44.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.208+0000 7f5526000640 1 -- 172.21.15.100:0/428395153 shutdown_connections 2024-04-23T14:33:44.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.208+0000 7f5526000640 1 -- 172.21.15.100:0/428395153 wait complete. 2024-04-23T14:33:44.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.212+0000 7f5526000640 1 Processor -- start 2024-04-23T14:33:44.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.212+0000 7f5526000640 1 -- start start 2024-04-23T14:33:44.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.212+0000 7f5526000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5504003b80 0x7f552014f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.212+0000 7f5526000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5520140d80 0x7f55201612c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.212+0000 7f5526000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55201588e0 0x7f5520158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.212+0000 7f5526000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f552015ec40 con 0x7f55201588e0 2024-04-23T14:33:44.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.212+0000 7f5526000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5520073790 con 0x7f5504003b80 2024-04-23T14:33:44.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.212+0000 7f5526000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5520070060 con 0x7f5520140d80 2024-04-23T14:33:44.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.212+0000 7f551fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55201588e0 0x7f5520158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.212+0000 7f551fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55201588e0 0x7f5520158d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58262/0 (socket says 172.21.15.100:58262) 2024-04-23T14:33:44.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.212+0000 7f551fe00640 1 -- 172.21.15.100:0/114465891 learned_addr learned my addr 172.21.15.100:0/114465891 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:44.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.212+0000 7f551fe00640 1 -- 172.21.15.100:0/114465891 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5520140d80 msgr2=0x7f55201612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.212+0000 7f551fe00640 1 --2- 172.21.15.100:0/114465891 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5520140d80 0x7f55201612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.212+0000 7f551fe00640 1 -- 172.21.15.100:0/114465891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5504003b80 msgr2=0x7f552014f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.212+0000 7f551fe00640 1 --2- 172.21.15.100:0/114465891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5504003b80 0x7f552014f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.212+0000 7f551fe00640 1 -- 172.21.15.100:0/114465891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552013dfd0 con 0x7f55201588e0 2024-04-23T14:33:44.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.212+0000 7f551fe00640 1 --2- 172.21.15.100:0/114465891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55201588e0 0x7f5520158d40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f552006e4c0 tx=0x7f5518007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:44.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.212+0000 7f5517400640 1 -- 172.21.15.100:0/114465891 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551800abd0 con 0x7f55201588e0 2024-04-23T14:33:44.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.212+0000 7f5517400640 1 -- 172.21.15.100:0/114465891 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5518018070 con 0x7f55201588e0 2024-04-23T14:33:44.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.212+0000 7f5517400640 1 -- 172.21.15.100:0/114465891 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55180133f0 con 0x7f55201588e0 2024-04-23T14:33:44.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.212+0000 7f5526000640 1 -- 172.21.15.100:0/114465891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f552014fe30 con 0x7f55201588e0 2024-04-23T14:33:44.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.212+0000 7f5526000640 1 -- 172.21.15.100:0/114465891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f552013e570 con 0x7f55201588e0 2024-04-23T14:33:44.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.216+0000 7f5526000640 1 -- 172.21.15.100:0/114465891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5520068ad0 con 0x7f55201588e0 2024-04-23T14:33:44.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.216+0000 7f5517400640 1 -- 172.21.15.100:0/114465891 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5518013590 con 0x7f55201588e0 2024-04-23T14:33:44.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.216+0000 7f5517400640 1 --2- 172.21.15.100:0/114465891 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f55000683b0 0x7f550006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.216+0000 7f5517400640 1 -- 172.21.15.100:0/114465891 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f5518087100 con 0x7f55201588e0 2024-04-23T14:33:44.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.216+0000 7f551f400640 1 --2- 172.21.15.100:0/114465891 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f55000683b0 0x7f550006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.216+0000 7f551f400640 1 --2- 172.21.15.100:0/114465891 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f55000683b0 0x7f550006a870 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f55100ad000 tx=0x7f5510004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:44.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.220+0000 7f5517400640 1 -- 172.21.15.100:0/114465891 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5518057520 con 0x7f55201588e0 2024-04-23T14:33:44.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.428+0000 7f5526000640 1 -- 172.21.15.100:0/114465891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f55200b3da0 con 0x7f55201588e0 2024-04-23T14:33:44.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.432+0000 7f5517400640 1 -- 172.21.15.100:0/114465891 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f5518056e20 con 0x7f55201588e0 2024-04-23T14:33:44.437 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-04-23T14:33:44.437 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAtxidmyhImMBAAa49ApQEF2Lng6rY94F1HAQ== 2024-04-23T14:33:44.437 INFO:teuthology.orchestra.run.smithi100.stdout: caps mds = "allow" 2024-04-23T14:33:44.437 INFO:teuthology.orchestra.run.smithi100.stdout: caps mon = "allow r" 2024-04-23T14:33:44.438 INFO:teuthology.orchestra.run.smithi100.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-23T14:33:44.438 INFO:teuthology.orchestra.run.smithi100.stderr:updated caps for client.0 2024-04-23T14:33:44.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.436+0000 7f5526000640 1 -- 172.21.15.100:0/114465891 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f55000683b0 msgr2=0x7f550006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.436+0000 7f5526000640 1 --2- 172.21.15.100:0/114465891 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f55000683b0 0x7f550006a870 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f55100ad000 tx=0x7f5510004740 comp rx=0 tx=0).stop 2024-04-23T14:33:44.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.436+0000 7f5526000640 1 -- 172.21.15.100:0/114465891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55201588e0 msgr2=0x7f5520158d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.436+0000 7f5526000640 1 --2- 172.21.15.100:0/114465891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55201588e0 0x7f5520158d40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f552006e4c0 tx=0x7f5518007bd0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.436+0000 7f5526000640 1 -- 172.21.15.100:0/114465891 shutdown_connections 2024-04-23T14:33:44.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.436+0000 7f5526000640 1 --2- 172.21.15.100:0/114465891 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f55000683b0 0x7f550006a870 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.436+0000 7f5526000640 1 --2- 172.21.15.100:0/114465891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55201588e0 0x7f5520158d40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.436+0000 7f5526000640 1 --2- 172.21.15.100:0/114465891 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5520140d80 0x7f55201612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.436+0000 7f5526000640 1 --2- 172.21.15.100:0/114465891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5504003b80 0x7f552014f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.436+0000 7f5526000640 1 -- 172.21.15.100:0/114465891 >> 172.21.15.100:0/114465891 conn(0x7f5520072590 msgr2=0x7f552006d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:44.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.436+0000 7f5526000640 1 -- 172.21.15.100:0/114465891 shutdown_connections 2024-04-23T14:33:44.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.436+0000 7f5526000640 1 -- 172.21.15.100:0/114465891 wait complete. 2024-04-23T14:33:44.459 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:33:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.588+0000 7f09b7e00640 1 Processor -- start 2024-04-23T14:33:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.588+0000 7f09b7e00640 1 -- start start 2024-04-23T14:33:44.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.592+0000 7f09b7e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09b8158e60 0x7f09b8072f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.592+0000 7f09b7e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f09b8073480 0x7f09b80cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.592+0000 7f09b7e00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09b80b4200 con 0x7f09b8158a80 2024-04-23T14:33:44.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.592+0000 7f09b7e00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f09b80b4480 con 0x7f09b8073480 2024-04-23T14:33:44.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.592+0000 7f09b7e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f09b80b4760 con 0x7f09b8158e60 2024-04-23T14:33:44.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.592+0000 7f09b7400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f09b8073480 0x7f09b80cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.592+0000 7f09b6000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09b8158e60 0x7f09b8072f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.592+0000 7f09b6000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09b8158e60 0x7f09b8072f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:49110/0 (socket says 172.21.15.100:49110) 2024-04-23T14:33:44.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.592+0000 7f09b6000640 1 -- 172.21.15.100:0/1528099332 learned_addr learned my addr 172.21.15.100:0/1528099332 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:44.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.592+0000 7f09b7400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f09b8073480 0x7f09b80cf4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:48954/0 (socket says 172.21.15.100:48954) 2024-04-23T14:33:44.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.592+0000 7f09b6a00640 1 --1- 172.21.15.100:0/1528099332 >> v1:172.21.15.151:6789/0 conn(0x7f09b8158a80 0x7f09b815f5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:41924/0 (socket says 172.21.15.100:41924) 2024-04-23T14:33:44.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.592+0000 7f09b6000640 1 -- 172.21.15.100:0/1528099332 >> v2:172.21.15.151:3300/0 conn(0x7f09b8073480 msgr2=0x7f09b80cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:44.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.592+0000 7f09b6000640 1 --2- 172.21.15.100:0/1528099332 >> v2:172.21.15.151:3300/0 conn(0x7f09b8073480 0x7f09b80cf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.592+0000 7f09b6000640 1 -- 172.21.15.100:0/1528099332 >> v1:172.21.15.151:6789/0 conn(0x7f09b8158a80 legacy=0x7f09b815f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.592+0000 7f09b6000640 1 -- 172.21.15.100:0/1528099332 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b80b4a40 con 0x7f09b8158e60 2024-04-23T14:33:44.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.596+0000 7f09b6000640 1 --2- 172.21.15.100:0/1528099332 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09b8158e60 0x7f09b8072f40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f09a4009870 tx=0x7f09a401c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0c59b2297843193 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:44.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.596+0000 7f09b5600640 1 -- 172.21.15.100:0/1528099332 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a40b0070 con 0x7f09b8158e60 2024-04-23T14:33:44.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.596+0000 7f09b5600640 1 -- 172.21.15.100:0/1528099332 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a4004230 con 0x7f09b8158e60 2024-04-23T14:33:44.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.596+0000 7f09b5600640 1 -- 172.21.15.100:0/1528099332 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4003620 con 0x7f09b8158e60 2024-04-23T14:33:44.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.596+0000 7f09b7e00640 1 -- 172.21.15.100:0/1528099332 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09b8158e60 msgr2=0x7f09b8072f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.596+0000 7f09b7e00640 1 --2- 172.21.15.100:0/1528099332 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09b8158e60 0x7f09b8072f40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f09a4009870 tx=0x7f09a401c440 comp rx=0 tx=0).stop 2024-04-23T14:33:44.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.596+0000 7f09b7e00640 1 -- 172.21.15.100:0/1528099332 shutdown_connections 2024-04-23T14:33:44.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.596+0000 7f09b7e00640 1 --2- 172.21.15.100:0/1528099332 >> v2:172.21.15.151:3300/0 conn(0x7f09b8073480 0x7f09b80cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.596+0000 7f09b7e00640 1 --2- 172.21.15.100:0/1528099332 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09b8158e60 0x7f09b8072f40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.596+0000 7f09b7e00640 1 -- 172.21.15.100:0/1528099332 >> 172.21.15.100:0/1528099332 conn(0x7f09b80b3100 msgr2=0x7f09b80b3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:44.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.596+0000 7f09b7e00640 1 -- 172.21.15.100:0/1528099332 shutdown_connections 2024-04-23T14:33:44.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.596+0000 7f09b7e00640 1 -- 172.21.15.100:0/1528099332 wait complete. 2024-04-23T14:33:44.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.596+0000 7f09b7e00640 1 Processor -- start 2024-04-23T14:33:44.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.596+0000 7f09b7e00640 1 -- start start 2024-04-23T14:33:44.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09b7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09b8073480 0x7f09b813ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09b7e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09b8158a80 0x7f09b81537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09b7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09b8153ce0 0x7f09b814fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09b7e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f09b80d0390 con 0x7f09b8073480 2024-04-23T14:33:44.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09b7e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f09b80d0210 con 0x7f09b8153ce0 2024-04-23T14:33:44.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09b7e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f09b80d0510 con 0x7f09b8158a80 2024-04-23T14:33:44.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09b6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09b8073480 0x7f09b813ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09b6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09b8073480 0x7f09b813ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58268/0 (socket says 172.21.15.100:58268) 2024-04-23T14:33:44.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09b6a00640 1 -- 172.21.15.100:0/695459225 learned_addr learned my addr 172.21.15.100:0/695459225 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:44.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09b6000640 1 --2- 172.21.15.100:0/695459225 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09b8158a80 0x7f09b81537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09b7400640 1 --2- 172.21.15.100:0/695459225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09b8153ce0 0x7f09b814fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09b6a00640 1 -- 172.21.15.100:0/695459225 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09b8158a80 msgr2=0x7f09b81537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09b6a00640 1 --2- 172.21.15.100:0/695459225 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09b8158a80 0x7f09b81537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09b6a00640 1 -- 172.21.15.100:0/695459225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09b8153ce0 msgr2=0x7f09b814fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09b6a00640 1 --2- 172.21.15.100:0/695459225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09b8153ce0 0x7f09b814fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09b6a00640 1 -- 172.21.15.100:0/695459225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b8150ad0 con 0x7f09b8073480 2024-04-23T14:33:44.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09b6a00640 1 --2- 172.21.15.100:0/695459225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09b8073480 0x7f09b813ead0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f09a800c450 tx=0x7f09a800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:44.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09aea00640 1 -- 172.21.15.100:0/695459225 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a8018070 con 0x7f09b8073480 2024-04-23T14:33:44.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09aea00640 1 -- 172.21.15.100:0/695459225 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a8007480 con 0x7f09b8073480 2024-04-23T14:33:44.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09aea00640 1 -- 172.21.15.100:0/695459225 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a80047f0 con 0x7f09b8073480 2024-04-23T14:33:44.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09b7e00640 1 -- 172.21.15.100:0/695459225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b8150db0 con 0x7f09b8073480 2024-04-23T14:33:44.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09b7e00640 1 -- 172.21.15.100:0/695459225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b815f6c0 con 0x7f09b8073480 2024-04-23T14:33:44.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09b7e00640 1 -- 172.21.15.100:0/695459225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b8075c20 con 0x7f09b8073480 2024-04-23T14:33:44.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.600+0000 7f09aea00640 1 -- 172.21.15.100:0/695459225 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f09a8004990 con 0x7f09b8073480 2024-04-23T14:33:44.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.604+0000 7f09aea00640 1 --2- 172.21.15.100:0/695459225 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0998068380 0x7f099806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.604+0000 7f09aea00640 1 -- 172.21.15.100:0/695459225 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f09a8085c50 con 0x7f09b8073480 2024-04-23T14:33:44.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.604+0000 7f09b6000640 1 --2- 172.21.15.100:0/695459225 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0998068380 0x7f099806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.604+0000 7f09b6000640 1 --2- 172.21.15.100:0/695459225 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0998068380 0x7f099806a840 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f09a40ad000 tx=0x7f09a40a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:44.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.608+0000 7f09aea00640 1 -- 172.21.15.100:0/695459225 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f09b80b4200 con 0x7f09b8073480 2024-04-23T14:33:44.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.796+0000 7f09b7e00640 1 -- 172.21.15.100:0/695459225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09b81618f0 con 0x7f09b8073480 2024-04-23T14:33:44.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.796+0000 7f09aea00640 1 -- 172.21.15.100:0/695459225 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7f09a8055a20 con 0x7f09b8073480 2024-04-23T14:33:44.803 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:44.803 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2601071793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2601071793},{"type":"v1","addr":"172.21.15.100:6837","nonce":2601071793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2930424296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2930424296},{"type":"v1","addr":"172.21.15.151:6833","nonce":2930424296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1406976216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1406976216},{"type":"v1","addr":"172.21.15.151:6835","nonce":1406976216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:33:38.928245+0000","modified":"2024-04-23T14:33:41.955618+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5466":{"gid":5466,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/2062210445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2062210445},{"type":"v1","addr":"172.21.15.100:6836","nonce":2062210445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5466,"qdb_cluster":[5466]},"id":2}]} 2024-04-23T14:33:44.803 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 18 2024-04-23T14:33:44.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.800+0000 7f09b7e00640 1 -- 172.21.15.100:0/695459225 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0998068380 msgr2=0x7f099806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.800+0000 7f09b7e00640 1 --2- 172.21.15.100:0/695459225 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0998068380 0x7f099806a840 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f09a40ad000 tx=0x7f09a40a6000 comp rx=0 tx=0).stop 2024-04-23T14:33:44.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.800+0000 7f09b7e00640 1 -- 172.21.15.100:0/695459225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09b8073480 msgr2=0x7f09b813ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.800+0000 7f09b7e00640 1 --2- 172.21.15.100:0/695459225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09b8073480 0x7f09b813ead0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f09a800c450 tx=0x7f09a800c910 comp rx=0 tx=0).stop 2024-04-23T14:33:44.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.800+0000 7f09b7e00640 1 -- 172.21.15.100:0/695459225 shutdown_connections 2024-04-23T14:33:44.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.800+0000 7f09b7e00640 1 --2- 172.21.15.100:0/695459225 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0998068380 0x7f099806a840 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.800+0000 7f09b7e00640 1 --2- 172.21.15.100:0/695459225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09b8153ce0 0x7f09b814fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.800+0000 7f09b7e00640 1 --2- 172.21.15.100:0/695459225 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09b8158a80 0x7f09b81537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.800+0000 7f09b7e00640 1 --2- 172.21.15.100:0/695459225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09b8073480 0x7f09b813ead0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.800+0000 7f09b7e00640 1 -- 172.21.15.100:0/695459225 >> 172.21.15.100:0/695459225 conn(0x7f09b80b3100 msgr2=0x7f09b8169bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:44.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.800+0000 7f09b7e00640 1 -- 172.21.15.100:0/695459225 shutdown_connections 2024-04-23T14:33:44.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:44.800+0000 7f09b7e00640 1 -- 172.21.15.100:0/695459225 wait complete. 2024-04-23T14:33:45.226 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:33:38.928245+0000', 'modified': '2024-04-23T14:33:41.955618+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5466}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5466': {'gid': 5466, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6836/2062210445', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 2062210445}, {'type': 'v1', 'addr': '172.21.15.100:6836', 'nonce': 2062210445}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5466, 'qdb_cluster': [5466]} 2024-04-23T14:33:45.227 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-23T14:33:45.227 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-23T14:33:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.360+0000 7f8f77c00640 1 Processor -- start 2024-04-23T14:33:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.360+0000 7f8f77c00640 1 -- start start 2024-04-23T14:33:45.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.360+0000 7f8f77c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8f700cde20 0x7f8f700cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.360+0000 7f8f77c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8f700cbee0 0x7f8f700cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.360+0000 7f8f77c00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f700ce280 con 0x7f8f7015d0c0 2024-04-23T14:33:45.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.360+0000 7f8f77c00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f8f700cff50 con 0x7f8f700cbee0 2024-04-23T14:33:45.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.360+0000 7f8f77c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8f700ceda0 con 0x7f8f700cde20 2024-04-23T14:33:45.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.360+0000 7f8f75200640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f8f7015d0c0 0x7f8f7015d4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57038/0 (socket says 172.21.15.100:57038) 2024-04-23T14:33:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.360+0000 7f8f75200640 1 -- 172.21.15.100:0/1493827136 learned_addr learned my addr 172.21.15.100:0/1493827136 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.360+0000 7f8f75c00640 1 --2- 172.21.15.100:0/1493827136 >> v2:172.21.15.100:3300/0 conn(0x7f8f700cbee0 0x7f8f700cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.360+0000 7f8f6fe00640 1 --2- 172.21.15.100:0/1493827136 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8f700cde20 0x7f8f700cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.360+0000 7f8f6f400640 1 -- 172.21.15.100:0/1493827136 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 457454967 0 0) 0x7f8f600014a0 con 0x7f8f7015d0c0 2024-04-23T14:33:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.360+0000 7f8f6f400640 1 -- 172.21.15.100:0/1493827136 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f54003530 con 0x7f8f7015d0c0 2024-04-23T14:33:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.360+0000 7f8f75c00640 1 -- 172.21.15.100:0/1493827136 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8f700cde20 msgr2=0x7f8f700cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.360+0000 7f8f75c00640 1 --2- 172.21.15.100:0/1493827136 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8f700cde20 0x7f8f700cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.360+0000 7f8f75c00640 1 -- 172.21.15.100:0/1493827136 >> v1:172.21.15.100:6789/0 conn(0x7f8f7015d0c0 legacy=0x7f8f7015d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:45.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.360+0000 7f8f75c00640 1 -- 172.21.15.100:0/1493827136 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f700ce500 con 0x7f8f700cbee0 2024-04-23T14:33:45.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.364+0000 7f8f75c00640 1 --2- 172.21.15.100:0/1493827136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f700cbee0 0x7f8f700cf160 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8f680097c0 tx=0x7f8f6801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b995573910d7de8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:45.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.364+0000 7f8f6f400640 1 -- 172.21.15.100:0/1493827136 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f680b0070 con 0x7f8f700cbee0 2024-04-23T14:33:45.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.364+0000 7f8f6f400640 1 -- 172.21.15.100:0/1493827136 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f680ab600 con 0x7f8f700cbee0 2024-04-23T14:33:45.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.364+0000 7f8f6f400640 1 -- 172.21.15.100:0/1493827136 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f680ab9d0 con 0x7f8f700cbee0 2024-04-23T14:33:45.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.364+0000 7f8f77c00640 1 -- 172.21.15.100:0/1493827136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f700cbee0 msgr2=0x7f8f700cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:45.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.364+0000 7f8f77c00640 1 --2- 172.21.15.100:0/1493827136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f700cbee0 0x7f8f700cf160 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8f680097c0 tx=0x7f8f6801c410 comp rx=0 tx=0).stop 2024-04-23T14:33:45.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.364+0000 7f8f77c00640 1 -- 172.21.15.100:0/1493827136 shutdown_connections 2024-04-23T14:33:45.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.364+0000 7f8f77c00640 1 --2- 172.21.15.100:0/1493827136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f700cbee0 0x7f8f700cf160 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.364+0000 7f8f77c00640 1 --2- 172.21.15.100:0/1493827136 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8f700cde20 0x7f8f700cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.364+0000 7f8f77c00640 1 -- 172.21.15.100:0/1493827136 >> 172.21.15.100:0/1493827136 conn(0x7f8f700b5310 msgr2=0x7f8f700b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:45.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.364+0000 7f8f77c00640 1 -- 172.21.15.100:0/1493827136 shutdown_connections 2024-04-23T14:33:45.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.364+0000 7f8f77c00640 1 -- 172.21.15.100:0/1493827136 wait complete. 2024-04-23T14:33:45.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f77c00640 1 Processor -- start 2024-04-23T14:33:45.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f77c00640 1 -- start start 2024-04-23T14:33:45.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f77c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f700cbee0 0x7f8f70165b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f77c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8f700cde20 0x7f8f7006f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f77c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f7015d0c0 0x7f8f7006f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f77c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8f70152fb0 con 0x7f8f700cbee0 2024-04-23T14:33:45.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f77c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8f70152e30 con 0x7f8f7015d0c0 2024-04-23T14:33:45.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f77c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8f70153130 con 0x7f8f700cde20 2024-04-23T14:33:45.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f75200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f700cbee0 0x7f8f70165b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f75200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f700cbee0 0x7f8f70165b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58288/0 (socket says 172.21.15.100:58288) 2024-04-23T14:33:45.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f75200640 1 -- 172.21.15.100:0/4058213828 learned_addr learned my addr 172.21.15.100:0/4058213828 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:45.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f6fe00640 1 --2- 172.21.15.100:0/4058213828 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8f700cde20 0x7f8f7006f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f75c00640 1 --2- 172.21.15.100:0/4058213828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f7015d0c0 0x7f8f7006f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f75200640 1 -- 172.21.15.100:0/4058213828 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8f700cde20 msgr2=0x7f8f7006f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:45.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f75200640 1 --2- 172.21.15.100:0/4058213828 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8f700cde20 0x7f8f7006f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f75200640 1 -- 172.21.15.100:0/4058213828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f7015d0c0 msgr2=0x7f8f7006f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:45.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f75200640 1 --2- 172.21.15.100:0/4058213828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f7015d0c0 0x7f8f7006f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f75200640 1 -- 172.21.15.100:0/4058213828 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f70168340 con 0x7f8f700cbee0 2024-04-23T14:33:45.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f75200640 1 --2- 172.21.15.100:0/4058213828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f700cbee0 0x7f8f70165b60 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000cb10 tx=0x7f8f6000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:45.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f6cc00640 1 -- 172.21.15.100:0/4058213828 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f60017070 con 0x7f8f700cbee0 2024-04-23T14:33:45.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f6cc00640 1 -- 172.21.15.100:0/4058213828 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f6000ac10 con 0x7f8f700cbee0 2024-04-23T14:33:45.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f6cc00640 1 -- 172.21.15.100:0/4058213828 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f60012400 con 0x7f8f700cbee0 2024-04-23T14:33:45.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f77c00640 1 -- 172.21.15.100:0/4058213828 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f70168060 con 0x7f8f700cbee0 2024-04-23T14:33:45.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.368+0000 7f8f77c00640 1 -- 172.21.15.100:0/4058213828 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f70072030 con 0x7f8f700cbee0 2024-04-23T14:33:45.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.372+0000 7f8f6cc00640 1 -- 172.21.15.100:0/4058213828 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8f6001a070 con 0x7f8f700cbee0 2024-04-23T14:33:45.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.372+0000 7f8f6cc00640 1 --2- 172.21.15.100:0/4058213828 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8f500683b0 0x7f8f5006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.372+0000 7f8f6cc00640 1 -- 172.21.15.100:0/4058213828 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f8f60086940 con 0x7f8f700cbee0 2024-04-23T14:33:45.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.372+0000 7f8f6fe00640 1 --2- 172.21.15.100:0/4058213828 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8f500683b0 0x7f8f5006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.372+0000 7f8f7b11f000 1 Processor -- start 2024-04-23T14:33:45.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.372+0000 7f8f6fe00640 1 --2- 172.21.15.100:0/4058213828 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8f500683b0 0x7f8f5006a870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8f70152440 tx=0x7f8f5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:45.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.372+0000 7f8f7b11f000 1 -- start start 2024-04-23T14:33:45.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.372+0000 7f8f7b11f000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c040ab5b0 0x558c040ab990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.372+0000 7f8f7b11f000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c040abed0 0x558c0405ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.372+0000 7f8f7b11f000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x558c0405f140 0x558c0405f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.372+0000 7f8f7b11f000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558c03f486f0 con 0x558c040ab5b0 2024-04-23T14:33:45.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f7b11f000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558c03eaf4d0 con 0x558c040abed0 2024-04-23T14:33:45.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f7b11f000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x558c03e49bf0 con 0x558c0405f140 2024-04-23T14:33:45.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f75200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c040ab5b0 0x558c040ab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f75200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c040ab5b0 0x558c040ab990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58292/0 (socket says 172.21.15.100:58292) 2024-04-23T14:33:45.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f75200640 1 -- 172.21.15.100:0/899644062 learned_addr learned my addr 172.21.15.100:0/899644062 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:45.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f6fe00640 1 --2- 172.21.15.100:0/899644062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c040abed0 0x558c0405ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f75c00640 1 --2- 172.21.15.100:0/899644062 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x558c0405f140 0x558c0405f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f75200640 1 -- 172.21.15.100:0/899644062 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x558c0405f140 msgr2=0x558c0405f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:45.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f75200640 1 --2- 172.21.15.100:0/899644062 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x558c0405f140 0x558c0405f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f75200640 1 -- 172.21.15.100:0/899644062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c040abed0 msgr2=0x558c0405ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:45.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f75200640 1 --2- 172.21.15.100:0/899644062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c040abed0 0x558c0405ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f75200640 1 -- 172.21.15.100:0/899644062 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c03e49e40 con 0x558c040ab5b0 2024-04-23T14:33:45.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f75200640 1 --2- 172.21.15.100:0/899644062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c040ab5b0 0x558c040ab990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8f600565f0 tx=0x7f8f600567a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474bfae07d5b885 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:45.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f65600640 1 -- 172.21.15.100:0/899644062 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6006e070 con 0x558c040ab5b0 2024-04-23T14:33:45.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f65600640 1 -- 172.21.15.100:0/899644062 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f60054960 con 0x558c040ab5b0 2024-04-23T14:33:45.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f65600640 1 -- 172.21.15.100:0/899644062 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6005edb0 con 0x558c040ab5b0 2024-04-23T14:33:45.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f7b11f000 1 -- 172.21.15.100:0/899644062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c040ab5b0 msgr2=0x558c040ab990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:45.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f7b11f000 1 --2- 172.21.15.100:0/899644062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c040ab5b0 0x558c040ab990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8f600565f0 tx=0x7f8f600567a0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f7b11f000 1 -- 172.21.15.100:0/899644062 shutdown_connections 2024-04-23T14:33:45.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f7b11f000 1 --2- 172.21.15.100:0/899644062 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x558c0405f140 0x558c0405f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f7b11f000 1 --2- 172.21.15.100:0/899644062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c040abed0 0x558c0405ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f7b11f000 1 --2- 172.21.15.100:0/899644062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c040ab5b0 0x558c040ab990 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f7b11f000 1 -- 172.21.15.100:0/899644062 >> 172.21.15.100:0/899644062 conn(0x558c0408c350 msgr2=0x558c0408e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:45.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f7b11f000 1 -- 172.21.15.100:0/899644062 shutdown_connections 2024-04-23T14:33:45.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f7b11f000 1 -- 172.21.15.100:0/899644062 wait complete. 2024-04-23T14:33:45.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f7b11f000 1 -- start start 2024-04-23T14:33:45.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f7b11f000 1 Processor -- start 2024-04-23T14:33:45.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f7b11f000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x558c0405f140 0x558c04065250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f7b11f000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c040ab5b0 0x558c04065790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f7b11f000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c040abed0 0x558c04065cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f7b11f000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558c03f6be50 con 0x558c040abed0 2024-04-23T14:33:45.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f7b11f000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558c03e6ca60 con 0x558c040ab5b0 2024-04-23T14:33:45.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f7b11f000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x558c03e6acf0 con 0x558c0405f140 2024-04-23T14:33:45.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f7b11f000 10 client.0 _ll_register_callbacks cb 0x558c0404ca70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:33:45.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f75c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c040abed0 0x558c04065cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f75c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c040abed0 0x558c04065cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58306/0 (socket says 172.21.15.100:58306) 2024-04-23T14:33:45.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f75c00640 1 -- 172.21.15.100:0/3233013583 learned_addr learned my addr 172.21.15.100:0/3233013583 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:45.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.376+0000 7f8f6fe00640 1 --2- 172.21.15.100:0/3233013583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c040ab5b0 0x558c04065790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f75200640 1 --2- 172.21.15.100:0/3233013583 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x558c0405f140 0x558c04065250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f75c00640 1 -- 172.21.15.100:0/3233013583 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x558c0405f140 msgr2=0x558c04065250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:45.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f75c00640 1 --2- 172.21.15.100:0/3233013583 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x558c0405f140 0x558c04065250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f75c00640 1 -- 172.21.15.100:0/3233013583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c040ab5b0 msgr2=0x558c04065790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:45.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f75c00640 1 --2- 172.21.15.100:0/3233013583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c040ab5b0 0x558c04065790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f75c00640 1 -- 172.21.15.100:0/3233013583 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c03e7ae00 con 0x558c040abed0 2024-04-23T14:33:45.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f6fe00640 1 --2- 172.21.15.100:0/3233013583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c040ab5b0 0x558c04065790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:45.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f75200640 1 --2- 172.21.15.100:0/3233013583 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x558c0405f140 0x558c04065250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:45.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f75c00640 1 --2- 172.21.15.100:0/3233013583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c040abed0 0x558c04065cd0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8f680ae040 tx=0x7f8f6801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad6d865d2d93a44 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:45.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f5a000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-04-23T14:33:45.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f5a000640 1 -- 172.21.15.100:0/3233013583 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f680bb020 con 0x558c040abed0 2024-04-23T14:33:45.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f5a000640 1 -- 172.21.15.100:0/3233013583 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f8f680abb20 con 0x558c040abed0 2024-04-23T14:33:45.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f7b11f000 1 -- 172.21.15.100:0/3233013583 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f60054960 con 0x558c040abed0 2024-04-23T14:33:45.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f5a000640 1 -- 172.21.15.100:0/3233013583 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f8f680c5050 con 0x558c040abed0 2024-04-23T14:33:45.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f7b11f000 10 client.5661 fetch_fsmap learned FSMap version 18 2024-04-23T14:33:45.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f7b11f000 1 -- 172.21.15.100:0/3233013583 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x558c03f31670 con 0x558c040abed0 2024-04-23T14:33:45.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f5a000640 1 -- 172.21.15.100:0/3233013583 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f60054960 con 0x558c040abed0 2024-04-23T14:33:45.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f5a000640 1 -- 172.21.15.100:0/3233013583 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f8f680c8020 con 0x558c040abed0 2024-04-23T14:33:45.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f7b11f000 10 client.5661 fetch_fsmap finished waiting for FSMap version 18 2024-04-23T14:33:45.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f7b11f000 10 client.5661 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:33:45.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f7b11f000 20 client.5661 populate_metadata read hostname 'smithi100' 2024-04-23T14:33:45.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f7b11f000 1 --2- 172.21.15.100:0/3233013583 >> [v2:172.21.15.100:6834/2062210445,v1:172.21.15.100:6836/2062210445] conn(0x558c040665a0 0x558c040bcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f7b11f000 4 client.5661 mds_command: new command op to 5466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:33:45.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f7b11f000 1 -- 172.21.15.100:0/3233013583 --> [v2:172.21.15.100:6834/2062210445,v1:172.21.15.100:6836/2062210445] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c03e48bc0 con 0x558c040665a0 2024-04-23T14:33:45.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f75200640 1 --2- 172.21.15.100:0/3233013583 >> [v2:172.21.15.100:6834/2062210445,v1:172.21.15.100:6836/2062210445] conn(0x558c040665a0 0x558c040bcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f75200640 1 --2- 172.21.15.100:0/3233013583 >> [v2:172.21.15.100:6834/2062210445,v1:172.21.15.100:6836/2062210445] conn(0x558c040665a0 0x558c040bcd30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30db2d9a47b1f393 server_cookie=c16e02c3b53a2de9 in_seq=0 out_seq=0 2024-04-23T14:33:45.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f5a000640 10 client.5661 ms_handle_connect on v2:172.21.15.100:6834/2062210445 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f5a000640 1 -- 172.21.15.100:0/3233013583 <== mds.0 v2:172.21.15.100:6834/2062210445 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558c03e48bc0 con 0x558c040665a0 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f5a000640 10 client.5661 handle_command_reply: tid=0 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f7b11f000 1 client.5661 shutdown 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f7b11f000 1 -- 172.21.15.100:0/3233013583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c040abed0 msgr2=0x558c04065cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f7b11f000 1 --2- 172.21.15.100:0/3233013583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c040abed0 0x558c04065cd0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8f680ae040 tx=0x7f8f6801ac50 comp rx=0 tx=0).stop 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f7b11f000 1 -- 172.21.15.100:0/3233013583 shutdown_connections 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f7b11f000 1 --2- 172.21.15.100:0/3233013583 >> [v2:172.21.15.100:6834/2062210445,v1:172.21.15.100:6836/2062210445] conn(0x558c040665a0 0x558c040bcd30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f7b11f000 1 --2- 172.21.15.100:0/3233013583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c040abed0 0x558c04065cd0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f7b11f000 1 --2- 172.21.15.100:0/3233013583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c040ab5b0 0x558c04065790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f5a000640 0 client.5661 ms_handle_reset on v2:172.21.15.100:6834/2062210445 2024-04-23T14:33:45.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f7b11f000 1 --2- 172.21.15.100:0/3233013583 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x558c0405f140 0x558c04065250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f7b11f000 1 -- 172.21.15.100:0/3233013583 >> 172.21.15.100:0/3233013583 conn(0x558c0408c350 msgr2=0x558c0408e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:45.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f7b11f000 1 -- 172.21.15.100:0/3233013583 shutdown_connections 2024-04-23T14:33:45.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f7b11f000 1 -- 172.21.15.100:0/3233013583 wait complete. 2024-04-23T14:33:45.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.380+0000 7f8f7b11f000 20 client.5661 trim_cache size 0 max 16384 2024-04-23T14:33:45.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f7b11f000 1 Processor -- start 2024-04-23T14:33:45.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f7b11f000 1 -- start start 2024-04-23T14:33:45.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f7b11f000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x558c0408c350 0x558c040640e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f7b11f000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0405f140 0x558c04064620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f7b11f000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c040ab5b0 0x558c040aa880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f7b11f000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8f70203a10 con 0x558c0405f140 2024-04-23T14:33:45.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f7b11f000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8f700738d0 con 0x558c040ab5b0 2024-04-23T14:33:45.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f7b11f000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8f70203420 con 0x558c0408c350 2024-04-23T14:33:45.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f6fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0405f140 0x558c04064620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f6fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0405f140 0x558c04064620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58316/0 (socket says 172.21.15.100:58316) 2024-04-23T14:33:45.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f6fe00640 1 -- 172.21.15.100:0/2048589126 learned_addr learned my addr 172.21.15.100:0/2048589126 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:45.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f75200640 1 --2- 172.21.15.100:0/2048589126 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x558c0408c350 0x558c040640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f75c00640 1 --2- 172.21.15.100:0/2048589126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c040ab5b0 0x558c040aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f6fe00640 1 -- 172.21.15.100:0/2048589126 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x558c0408c350 msgr2=0x558c040640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:45.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f6fe00640 1 --2- 172.21.15.100:0/2048589126 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x558c0408c350 0x558c040640e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f6fe00640 1 -- 172.21.15.100:0/2048589126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c040ab5b0 msgr2=0x558c040aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:45.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f6fe00640 1 --2- 172.21.15.100:0/2048589126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c040ab5b0 0x558c040aa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f6fe00640 1 -- 172.21.15.100:0/2048589126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c0408ee90 con 0x558c0405f140 2024-04-23T14:33:45.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f75c00640 1 --2- 172.21.15.100:0/2048589126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c040ab5b0 0x558c040aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:45.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f6fe00640 1 --2- 172.21.15.100:0/2048589126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0405f140 0x558c04064620 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00fad0 tx=0x7f8f5c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b986fa7b333465b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:45.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f64c00640 1 -- 172.21.15.100:0/2048589126 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c01b040 con 0x558c0405f140 2024-04-23T14:33:45.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f64c00640 1 -- 172.21.15.100:0/2048589126 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f5c01c040 con 0x558c0405f140 2024-04-23T14:33:45.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f64c00640 1 -- 172.21.15.100:0/2048589126 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c015410 con 0x558c0405f140 2024-04-23T14:33:45.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f7b11f000 1 -- 172.21.15.100:0/2048589126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0405f140 msgr2=0x558c04064620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:45.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f7b11f000 1 --2- 172.21.15.100:0/2048589126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0405f140 0x558c04064620 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00fad0 tx=0x7f8f5c015000 comp rx=0 tx=0).stop 2024-04-23T14:33:45.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f7b11f000 1 -- 172.21.15.100:0/2048589126 shutdown_connections 2024-04-23T14:33:45.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f7b11f000 1 --2- 172.21.15.100:0/2048589126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c040ab5b0 0x558c040aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f7b11f000 1 --2- 172.21.15.100:0/2048589126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c0405f140 0x558c04064620 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f7b11f000 1 --2- 172.21.15.100:0/2048589126 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x558c0408c350 0x558c040640e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f7b11f000 1 -- 172.21.15.100:0/2048589126 >> 172.21.15.100:0/2048589126 conn(0x558c0408eab0 msgr2=0x558c040639d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:45.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f7b11f000 1 -- 172.21.15.100:0/2048589126 shutdown_connections 2024-04-23T14:33:45.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.400+0000 7f8f7b11f000 1 -- 172.21.15.100:0/2048589126 wait complete. 2024-04-23T14:33:45.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f7b11f000 1 -- start start 2024-04-23T14:33:45.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f7b11f000 1 Processor -- start 2024-04-23T14:33:45.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f7b11f000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x558c0405f140 0x558c040bdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f7b11f000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c0408c350 0x558c040be2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f7b11f000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c040ab5b0 0x558c040be800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f7b11f000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558c03eb3b00 con 0x558c040ab5b0 2024-04-23T14:33:45.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f7b11f000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558c03eae7e0 con 0x558c0408c350 2024-04-23T14:33:45.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f7b11f000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x558c03ebc6e0 con 0x558c0405f140 2024-04-23T14:33:45.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f7b11f000 10 client.0 _ll_register_callbacks cb 0x7f8f680ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:33:45.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f75c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c040ab5b0 0x558c040be800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f75c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c040ab5b0 0x558c040be800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58332/0 (socket says 172.21.15.100:58332) 2024-04-23T14:33:45.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f75c00640 1 -- 172.21.15.100:0/2186131946 learned_addr learned my addr 172.21.15.100:0/2186131946 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:45.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f75200640 1 --2- 172.21.15.100:0/2186131946 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x558c0405f140 0x558c040bdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f6fe00640 1 --2- 172.21.15.100:0/2186131946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c0408c350 0x558c040be2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f75c00640 1 -- 172.21.15.100:0/2186131946 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x558c0405f140 msgr2=0x558c040bdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:45.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f75c00640 1 --2- 172.21.15.100:0/2186131946 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x558c0405f140 0x558c040bdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f75c00640 1 -- 172.21.15.100:0/2186131946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c0408c350 msgr2=0x558c040be2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:45.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f75c00640 1 --2- 172.21.15.100:0/2186131946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c0408c350 0x558c040be2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f75c00640 1 -- 172.21.15.100:0/2186131946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c03f08340 con 0x558c040ab5b0 2024-04-23T14:33:45.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f6fe00640 1 --2- 172.21.15.100:0/2186131946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c0408c350 0x558c040be2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:45.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f75c00640 1 --2- 172.21.15.100:0/2186131946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c040ab5b0 0x558c040be800 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8f6801c980 tx=0x7f8f680a8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae040ed0e0c3fb1a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:45.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f5a000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-04-23T14:33:45.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f5a000640 1 -- 172.21.15.100:0/2186131946 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f680bb040 con 0x558c040ab5b0 2024-04-23T14:33:45.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f5a000640 1 -- 172.21.15.100:0/2186131946 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f8f680abbb0 con 0x558c040ab5b0 2024-04-23T14:33:45.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f7b11f000 1 -- 172.21.15.100:0/2186131946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f5c01c040 con 0x558c040ab5b0 2024-04-23T14:33:45.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f5a000640 1 -- 172.21.15.100:0/2186131946 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f8f680b0040 con 0x558c040ab5b0 2024-04-23T14:33:45.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f7b11f000 10 client.5667 fetch_fsmap learned FSMap version 18 2024-04-23T14:33:45.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f7b11f000 1 -- 172.21.15.100:0/2186131946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x558c03e417c0 con 0x558c040ab5b0 2024-04-23T14:33:45.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f5a000640 1 -- 172.21.15.100:0/2186131946 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f5c01c040 con 0x558c040ab5b0 2024-04-23T14:33:45.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f5a000640 1 -- 172.21.15.100:0/2186131946 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f8f680c8070 con 0x558c040ab5b0 2024-04-23T14:33:45.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f7b11f000 10 client.5667 fetch_fsmap finished waiting for FSMap version 18 2024-04-23T14:33:45.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f7b11f000 10 client.5667 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:33:45.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f7b11f000 20 client.5667 populate_metadata read hostname 'smithi100' 2024-04-23T14:33:45.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f7b11f000 1 --2- 172.21.15.100:0/2186131946 >> [v2:172.21.15.100:6834/2062210445,v1:172.21.15.100:6836/2062210445] conn(0x558c040abed0 0x558c040bf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f7b11f000 4 client.5667 mds_command: new command op to 5466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:33:45.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f7b11f000 1 -- 172.21.15.100:0/2186131946 --> [v2:172.21.15.100:6834/2062210445,v1:172.21.15.100:6836/2062210445] -- command(tid 0: {"prefix": "status"}) -- 0x558c03e844e0 con 0x558c040abed0 2024-04-23T14:33:45.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f75200640 1 --2- 172.21.15.100:0/2186131946 >> [v2:172.21.15.100:6834/2062210445,v1:172.21.15.100:6836/2062210445] conn(0x558c040abed0 0x558c040bf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.404+0000 7f8f75200640 1 --2- 172.21.15.100:0/2186131946 >> [v2:172.21.15.100:6834/2062210445,v1:172.21.15.100:6836/2062210445] conn(0x558c040abed0 0x558c040bf190 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae699c38757ad1ed server_cookie=8f48d034ad996b31 in_seq=0 out_seq=0 2024-04-23T14:33:45.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f5a000640 10 client.5667 ms_handle_connect on v2:172.21.15.100:6834/2062210445 2024-04-23T14:33:45.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f5a000640 1 -- 172.21.15.100:0/2186131946 <== mds.0 v2:172.21.15.100:6834/2062210445 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x558c03e844e0 con 0x558c040abed0 2024-04-23T14:33:45.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f5a000640 10 client.5667 handle_command_reply: tid=0 2024-04-23T14:33:45.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f7b11f000 1 client.5667 shutdown 2024-04-23T14:33:45.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f7b11f000 1 -- 172.21.15.100:0/2186131946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c040ab5b0 msgr2=0x558c040be800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:45.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f7b11f000 1 --2- 172.21.15.100:0/2186131946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c040ab5b0 0x558c040be800 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8f6801c980 tx=0x7f8f680a8000 comp rx=0 tx=0).stop 2024-04-23T14:33:45.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f7b11f000 1 -- 172.21.15.100:0/2186131946 shutdown_connections 2024-04-23T14:33:45.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f7b11f000 1 --2- 172.21.15.100:0/2186131946 >> [v2:172.21.15.100:6834/2062210445,v1:172.21.15.100:6836/2062210445] conn(0x558c040abed0 0x558c040bf190 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f7b11f000 1 --2- 172.21.15.100:0/2186131946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c040ab5b0 0x558c040be800 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f7b11f000 1 --2- 172.21.15.100:0/2186131946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c0408c350 0x558c040be2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f7b11f000 1 --2- 172.21.15.100:0/2186131946 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x558c0405f140 0x558c040bdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f5a000640 0 client.5667 ms_handle_reset on v2:172.21.15.100:6834/2062210445 2024-04-23T14:33:45.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f7b11f000 1 -- 172.21.15.100:0/2186131946 >> 172.21.15.100:0/2186131946 conn(0x558c0408eab0 msgr2=0x558c04064260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:45.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f7b11f000 1 -- 172.21.15.100:0/2186131946 shutdown_connections 2024-04-23T14:33:45.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f7b11f000 1 -- 172.21.15.100:0/2186131946 wait complete. 2024-04-23T14:33:45.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f7b11f000 20 client.5667 trim_cache size 0 max 16384 2024-04-23T14:33:45.415 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:33:45.415 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "167d2af1-01c4-4f87-8bce-0de22791f823", 2024-04-23T14:33:45.415 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-04-23T14:33:45.415 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 5466, 2024-04-23T14:33:45.416 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-04-23T14:33:45.416 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-04-23T14:33:45.416 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-04-23T14:33:45.416 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 5.8621101089999996, 2024-04-23T14:33:45.416 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 18, 2024-04-23T14:33:45.416 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 38, 2024-04-23T14:33:45.416 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 38, 2024-04-23T14:33:45.416 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 13.734494651 2024-04-23T14:33:45.416 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:33:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f77c00640 1 -- 172.21.15.100:0/4058213828 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8f500683b0 msgr2=0x7f8f5006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f77c00640 1 --2- 172.21.15.100:0/4058213828 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8f500683b0 0x7f8f5006a870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8f70152440 tx=0x7f8f5c008040 comp rx=0 tx=0).stop 2024-04-23T14:33:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f77c00640 1 -- 172.21.15.100:0/4058213828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f700cbee0 msgr2=0x7f8f70165b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f77c00640 1 --2- 172.21.15.100:0/4058213828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f700cbee0 0x7f8f70165b60 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000cb10 tx=0x7f8f6000a060 comp rx=0 tx=0).stop 2024-04-23T14:33:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f77c00640 1 -- 172.21.15.100:0/4058213828 shutdown_connections 2024-04-23T14:33:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f77c00640 1 --2- 172.21.15.100:0/4058213828 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8f500683b0 0x7f8f5006a870 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f77c00640 1 --2- 172.21.15.100:0/4058213828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f7015d0c0 0x7f8f7006f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f77c00640 1 --2- 172.21.15.100:0/4058213828 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8f700cde20 0x7f8f7006f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f77c00640 1 --2- 172.21.15.100:0/4058213828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f700cbee0 0x7f8f70165b60 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f77c00640 1 -- 172.21.15.100:0/4058213828 >> 172.21.15.100:0/4058213828 conn(0x7f8f700b5310 msgr2=0x7f8f7015d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f77c00640 1 -- 172.21.15.100:0/4058213828 shutdown_connections 2024-04-23T14:33:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.408+0000 7f8f77c00640 1 -- 172.21.15.100:0/4058213828 wait complete. 2024-04-23T14:33:45.431 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:33:45.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.560+0000 7f5900200640 1 Processor -- start 2024-04-23T14:33:45.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.560+0000 7f5900200640 1 -- start start 2024-04-23T14:33:45.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f5900200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f8073610 0x7f58f80adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f5900200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58f80af080 con 0x7f58f80b2a10 2024-04-23T14:33:45.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f5900200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58f80afbc0 con 0x7f58f815b640 2024-04-23T14:33:45.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f5900200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f58f815ba20 con 0x7f58f8073610 2024-04-23T14:33:45.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58fce00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f58f80b2a10 0x7f58f806df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57052/0 (socket says 172.21.15.100:57052) 2024-04-23T14:33:45.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58fce00640 1 -- 172.21.15.100:0/2244156698 learned_addr learned my addr 172.21.15.100:0/2244156698 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:45.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58fe200640 1 --2- 172.21.15.100:0/2244156698 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f8073610 0x7f58f80adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58f7e00640 1 -- 172.21.15.100:0/2244156698 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3421251736 0 0) 0x7f58f80af080 con 0x7f58f80b2a10 2024-04-23T14:33:45.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58f7e00640 1 -- 172.21.15.100:0/2244156698 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58e0003530 con 0x7f58f80b2a10 2024-04-23T14:33:45.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58f7e00640 1 -- 172.21.15.100:0/2244156698 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2841576539 0 0) 0x7f58e0003530 con 0x7f58f80b2a10 2024-04-23T14:33:45.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58f7e00640 1 -- 172.21.15.100:0/2244156698 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f8073610 msgr2=0x7f58f80adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:45.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58f7e00640 1 --2- 172.21.15.100:0/2244156698 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f8073610 0x7f58f80adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58f7e00640 1 -- 172.21.15.100:0/2244156698 >> v1:172.21.15.151:6789/0 conn(0x7f58f815b640 legacy=0x7f58f8072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:45.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58f7e00640 1 -- 172.21.15.100:0/2244156698 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f80af320 con 0x7f58f80b2a10 2024-04-23T14:33:45.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58f7e00640 1 -- 172.21.15.100:0/2244156698 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f58e4001680 con 0x7f58f80b2a10 2024-04-23T14:33:45.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58f7e00640 1 -- 172.21.15.100:0/2244156698 >> v1:172.21.15.100:6789/0 conn(0x7f58f80b2a10 legacy=0x7f58f806df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:45.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58f7e00640 1 --2- 172.21.15.100:0/2244156698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58e00039f0 0x7f58e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58f7e00640 1 --2- 172.21.15.100:0/2244156698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e00043a0 0x7f58e0004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58f7e00640 1 --2- 172.21.15.100:0/2244156698 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f8073610 0x7f58e00050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58f7e00640 1 -- 172.21.15.100:0/2244156698 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f58f815ba20 con 0x7f58e00043a0 2024-04-23T14:33:45.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58f7e00640 1 -- 172.21.15.100:0/2244156698 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f58e000ba00 con 0x7f58e00039f0 2024-04-23T14:33:45.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58f7e00640 1 -- 172.21.15.100:0/2244156698 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f58e000bc00 con 0x7f58f8073610 2024-04-23T14:33:45.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58fce00640 1 --2- 172.21.15.100:0/2244156698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e00043a0 0x7f58e0004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58fe200640 1 --2- 172.21.15.100:0/2244156698 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f8073610 0x7f58e00050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58fd800640 1 --2- 172.21.15.100:0/2244156698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58e00039f0 0x7f58e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58fce00640 1 -- 172.21.15.100:0/2244156698 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f8073610 msgr2=0x7f58e00050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:45.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58fce00640 1 --2- 172.21.15.100:0/2244156698 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f8073610 0x7f58e00050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58fce00640 1 -- 172.21.15.100:0/2244156698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58e00039f0 msgr2=0x7f58e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:45.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58fce00640 1 --2- 172.21.15.100:0/2244156698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58e00039f0 0x7f58e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58fce00640 1 -- 172.21.15.100:0/2244156698 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e0003530 con 0x7f58e00043a0 2024-04-23T14:33:45.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.564+0000 7f58fd800640 1 --2- 172.21.15.100:0/2244156698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58e00039f0 0x7f58e0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:45.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.568+0000 7f58fce00640 1 --2- 172.21.15.100:0/2244156698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e00043a0 0x7f58e0004870 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f58f806c0d0 tx=0x7f58e401cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc2f69b8a8fa93c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:45.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.568+0000 7f58f7e00640 1 -- 172.21.15.100:0/2244156698 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4007cf0 con 0x7f58e00043a0 2024-04-23T14:33:45.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.568+0000 7f58f7e00640 1 -- 172.21.15.100:0/2244156698 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58e40a9070 con 0x7f58e00043a0 2024-04-23T14:33:45.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.568+0000 7f58f7e00640 1 -- 172.21.15.100:0/2244156698 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4005d40 con 0x7f58e00043a0 2024-04-23T14:33:45.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.568+0000 7f5900200640 1 -- 172.21.15.100:0/2244156698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e00043a0 msgr2=0x7f58e0004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:45.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.568+0000 7f5900200640 1 --2- 172.21.15.100:0/2244156698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e00043a0 0x7f58e0004870 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f58f806c0d0 tx=0x7f58e401cbc0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.568+0000 7f58fe200640 1 -- 172.21.15.100:0/2244156698 reap_dead start 2024-04-23T14:33:45.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.568+0000 7f5900200640 1 -- 172.21.15.100:0/2244156698 shutdown_connections 2024-04-23T14:33:45.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.568+0000 7f5900200640 1 -- 172.21.15.100:0/2244156698 >> 172.21.15.100:0/2244156698 conn(0x7f58f80b1e80 msgr2=0x7f58f80b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:45.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.568+0000 7f5900200640 1 -- 172.21.15.100:0/2244156698 shutdown_connections 2024-04-23T14:33:45.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.568+0000 7f5900200640 1 -- 172.21.15.100:0/2244156698 wait complete. 2024-04-23T14:33:45.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.568+0000 7f5900200640 1 Processor -- start 2024-04-23T14:33:45.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.568+0000 7f5900200640 1 -- start start 2024-04-23T14:33:45.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f5900200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e00043a0 0x7f58f81f6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f5900200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58f814a160 0x7f58f80728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f5900200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f8072de0 0x7f58f815d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f5900200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f58f80cc750 con 0x7f58e00043a0 2024-04-23T14:33:45.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f5900200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f58f80af8e0 con 0x7f58f814a160 2024-04-23T14:33:45.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f5900200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f58f80cc8d0 con 0x7f58f8072de0 2024-04-23T14:33:45.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f58fd800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e00043a0 0x7f58f81f6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f58fd800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e00043a0 0x7f58f81f6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58350/0 (socket says 172.21.15.100:58350) 2024-04-23T14:33:45.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f58fd800640 1 -- 172.21.15.100:0/724177093 learned_addr learned my addr 172.21.15.100:0/724177093 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:45.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f58fce00640 1 --2- 172.21.15.100:0/724177093 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58f814a160 0x7f58f80728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f58fe200640 1 --2- 172.21.15.100:0/724177093 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f8072de0 0x7f58f815d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f58fd800640 1 -- 172.21.15.100:0/724177093 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f8072de0 msgr2=0x7f58f815d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:45.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f58fd800640 1 --2- 172.21.15.100:0/724177093 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f8072de0 0x7f58f815d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f58fd800640 1 -- 172.21.15.100:0/724177093 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58f814a160 msgr2=0x7f58f80728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:45.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f58fd800640 1 --2- 172.21.15.100:0/724177093 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58f814a160 0x7f58f80728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f58fd800640 1 -- 172.21.15.100:0/724177093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f80ccbe0 con 0x7f58e00043a0 2024-04-23T14:33:45.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f58fce00640 1 --2- 172.21.15.100:0/724177093 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58f814a160 0x7f58f80728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:45.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f58fd800640 1 --2- 172.21.15.100:0/724177093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e00043a0 0x7f58f81f6e00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f58ec010f40 tx=0x7f58ec00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:45.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f58f5600640 1 -- 172.21.15.100:0/724177093 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ec018020 con 0x7f58e00043a0 2024-04-23T14:33:45.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f58f5600640 1 -- 172.21.15.100:0/724177093 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58ec005270 con 0x7f58e00043a0 2024-04-23T14:33:45.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f58f5600640 1 -- 172.21.15.100:0/724177093 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ec00ee50 con 0x7f58e00043a0 2024-04-23T14:33:45.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f5900200640 1 -- 172.21.15.100:0/724177093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f80ccec0 con 0x7f58e00043a0 2024-04-23T14:33:45.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f5900200640 1 -- 172.21.15.100:0/724177093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f80cd480 con 0x7f58e00043a0 2024-04-23T14:33:45.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.572+0000 7f5900200640 1 -- 172.21.15.100:0/724177093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f8068ad0 con 0x7f58e00043a0 2024-04-23T14:33:45.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.576+0000 7f58f5600640 1 -- 172.21.15.100:0/724177093 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f58ec01b020 con 0x7f58e00043a0 2024-04-23T14:33:45.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.576+0000 7f58f5600640 1 --2- 172.21.15.100:0/724177093 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f58d8068380 0x7f58d806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.576+0000 7f58f5600640 1 -- 172.21.15.100:0/724177093 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f58ec08a260 con 0x7f58e00043a0 2024-04-23T14:33:45.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.576+0000 7f58fce00640 1 --2- 172.21.15.100:0/724177093 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f58d8068380 0x7f58d806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.576+0000 7f58fce00640 1 --2- 172.21.15.100:0/724177093 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f58d8068380 0x7f58d806a840 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f58e40ab000 tx=0x7f58e40b1040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:45.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.580+0000 7f58f5600640 1 -- 172.21.15.100:0/724177093 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f58ec022020 con 0x7f58e00043a0 2024-04-23T14:33:45.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.772+0000 7f5900200640 1 -- 172.21.15.100:0/724177093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f58f8159f30 con 0x7f58e00043a0 2024-04-23T14:33:45.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.772+0000 7f58f5600640 1 -- 172.21.15.100:0/724177093 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7f58ec05a680 con 0x7f58e00043a0 2024-04-23T14:33:45.776 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:33:45.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.772+0000 7f5900200640 1 -- 172.21.15.100:0/724177093 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f58d8068380 msgr2=0x7f58d806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:45.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.772+0000 7f5900200640 1 --2- 172.21.15.100:0/724177093 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f58d8068380 0x7f58d806a840 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f58e40ab000 tx=0x7f58e40b1040 comp rx=0 tx=0).stop 2024-04-23T14:33:45.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.772+0000 7f5900200640 1 -- 172.21.15.100:0/724177093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e00043a0 msgr2=0x7f58f81f6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:45.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.772+0000 7f5900200640 1 --2- 172.21.15.100:0/724177093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e00043a0 0x7f58f81f6e00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f58ec010f40 tx=0x7f58ec00e530 comp rx=0 tx=0).stop 2024-04-23T14:33:45.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.772+0000 7f5900200640 1 -- 172.21.15.100:0/724177093 shutdown_connections 2024-04-23T14:33:45.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.772+0000 7f5900200640 1 --2- 172.21.15.100:0/724177093 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f58d8068380 0x7f58d806a840 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.772+0000 7f5900200640 1 --2- 172.21.15.100:0/724177093 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f8072de0 0x7f58f815d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.772+0000 7f5900200640 1 --2- 172.21.15.100:0/724177093 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58f814a160 0x7f58f80728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.772+0000 7f5900200640 1 --2- 172.21.15.100:0/724177093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e00043a0 0x7f58f81f6e00 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.772+0000 7f5900200640 1 -- 172.21.15.100:0/724177093 >> 172.21.15.100:0/724177093 conn(0x7f58f80b1e80 msgr2=0x7f58f81528e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:45.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.772+0000 7f5900200640 1 -- 172.21.15.100:0/724177093 shutdown_connections 2024-04-23T14:33:45.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:45.772+0000 7f5900200640 1 -- 172.21.15.100:0/724177093 wait complete. 2024-04-23T14:33:45.797 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:33:45.797 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:33:45.797 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-04-23T14:33:45.797 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:33:45.797 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:45.797 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:33:45.797 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:33:45.797 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:33:45.797 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:33:45.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:45.798 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-04-23T14:33:45.806 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:33:45.806 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:33:45.807 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:33:45.807 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:45.807 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-23T14:33:45.807 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:45.807 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:33:45.807 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:33:45.807 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-04-23T14:33:45.807 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:33:45.807 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:33:45.807 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-04-23T14:33:45.807 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:33:45.807 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:33:45.807 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:33:45.807 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:33:45.808 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-23T14:33:45.808 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42358sec preferred_lft 42358sec 2024-04-23T14:33:45.808 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-04-23T14:33:45.808 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:45.808 INFO:teuthology.orchestra.run.smithi100.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-23T14:33:45.808 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 02:42:ff:fc:d1:cc brd ff:ff:ff:ff:ff:ff 2024-04-23T14:33:45.808 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-23T14:33:45.808 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:45.808 INFO:teuthology.orchestra.run.smithi100.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:33:45.808 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-23T14:33:45.808 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:33:45.808 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:45.808 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-23T14:33:45.808 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:45.808 INFO:teuthology.orchestra.run.smithi100.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:33:45.809 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:33:45.809 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-23T14:33:45.809 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:45.809 INFO:teuthology.orchestra.run.smithi100.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:33:45.809 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:33:45.809 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-23T14:33:45.809 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:45.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:45.810 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:33:45.868 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-04-23T14:33:45.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:45.870 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-04-23T14:33:45.921 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-04-23T14:33:45.921 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:33:45.921 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-23T14:33:45.921 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-23T14:33:45.921 INFO:teuthology.orchestra.run.smithi100.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-23T14:33:45.921 INFO:teuthology.orchestra.run.smithi100.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-23T14:33:45.921 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-23T14:33:45.921 INFO:teuthology.orchestra.run.smithi100.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:33:45.921 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:33:45.921 INFO:teuthology.orchestra.run.smithi100.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:33:45.921 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:33:45.922 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:33:45.922 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-23T14:33:45.922 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-23T14:33:45.922 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-23T14:33:45.922 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-23T14:33:45.922 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-23T14:33:46.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:33:46.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:33:46.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:46.059 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-23T14:33:46.068 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:33:46.068 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:33:46.068 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:33:46.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:46.068 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-04-23T14:33:46.129 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:33:46.130 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:33:46.130 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:46.182 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:33:46.182 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:33:46.182 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:46.233 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-23T14:33:46.294 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:33:46.344 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-23T14:33:46.344 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-23T14:33:46.344 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-23T14:33:46.344 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-23T14:33:46.344 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-23T14:33:46.344 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-23T14:33:46.344 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-23T14:33:46.344 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-23T14:33:46.344 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-23T14:33:46.344 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-23T14:33:46.344 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-23T14:33:46.344 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-23T14:33:46.344 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:33:46.344 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-23T14:33:46.345 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-23T14:33:46.345 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-23T14:33:46.345 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-23T14:33:46.345 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-23T14:33:46.345 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-23T14:33:46.345 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-23T14:33:46.345 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:33:46.345 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-23T14:33:46.345 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-23T14:33:46.345 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-23T14:33:46.345 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:33:46.345 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:33:46.345 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:33:46.345 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:33:46.345 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-23T14:33:46.345 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:33:46.346 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:33:46.346 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:33:46.346 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:33:46.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:46.347 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-23T14:33:46.398 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:33:46.399 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:33:46.445 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-23T14:33:46.465 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:33:46.519 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-23T14:33:46.520 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-23T14:33:46.520 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-23T14:33:46.520 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-23T14:33:46.520 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-23T14:33:46.520 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-23T14:33:46.520 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-23T14:33:46.520 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-23T14:33:46.520 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-23T14:33:46.520 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-23T14:33:46.520 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-23T14:33:46.520 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-23T14:33:46.520 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:33:46.521 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-23T14:33:46.521 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-23T14:33:46.521 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-23T14:33:46.521 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-23T14:33:46.521 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-23T14:33:46.521 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-23T14:33:46.521 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-23T14:33:46.521 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:33:46.521 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-23T14:33:46.521 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-23T14:33:46.521 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-23T14:33:46.521 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:33:46.521 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:33:46.521 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:33:46.521 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:33:46.521 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-23T14:33:46.522 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:33:46.522 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:33:46.522 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:33:46.522 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:33:46.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:46.523 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-23T14:33:46.538 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:2024-04-23T14:33:46.532+0000 7ff0d6b93580 -1 init, newargv = 0x55625df10e00 newargc=15 2024-04-23T14:33:46.538 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[39311]: starting ceph client 2024-04-23T14:33:46.546 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[39311]: starting fuse 2024-04-23T14:33:46.547 INFO:teuthology.orchestra.run.smithi100.stdout:44 2024-04-23T14:33:46.547 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-23T14:33:46.548 DEBUG:teuthology.orchestra.run.smithi100:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:33:46.548 DEBUG:teuthology.orchestra.run.smithi100:> import glob 2024-04-23T14:33:46.548 DEBUG:teuthology.orchestra.run.smithi100:> import re 2024-04-23T14:33:46.548 DEBUG:teuthology.orchestra.run.smithi100:> import os 2024-04-23T14:33:46.548 DEBUG:teuthology.orchestra.run.smithi100:> import subprocess 2024-04-23T14:33:46.548 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:33:46.548 DEBUG:teuthology.orchestra.run.smithi100:> def _find_admin_socket(client_name): 2024-04-23T14:33:46.548 DEBUG:teuthology.orchestra.run.smithi100:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:33:46.548 DEBUG:teuthology.orchestra.run.smithi100:> files = glob.glob(asok_path) 2024-04-23T14:33:46.548 DEBUG:teuthology.orchestra.run.smithi100:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:33:46.548 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:33:46.548 DEBUG:teuthology.orchestra.run.smithi100:> # Given a non-glob path, it better be there 2024-04-23T14:33:46.548 DEBUG:teuthology.orchestra.run.smithi100:> if "*" not in asok_path: 2024-04-23T14:33:46.548 DEBUG:teuthology.orchestra.run.smithi100:> assert(len(files) == 1) 2024-04-23T14:33:46.548 DEBUG:teuthology.orchestra.run.smithi100:> return files[0] 2024-04-23T14:33:46.548 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:33:46.549 DEBUG:teuthology.orchestra.run.smithi100:> for f in files: 2024-04-23T14:33:46.549 DEBUG:teuthology.orchestra.run.smithi100:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:33:46.549 DEBUG:teuthology.orchestra.run.smithi100:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:33:46.549 DEBUG:teuthology.orchestra.run.smithi100:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:33:46.549 DEBUG:teuthology.orchestra.run.smithi100:> contents = proc_f.read() 2024-04-23T14:33:46.549 DEBUG:teuthology.orchestra.run.smithi100:> if mountpoint in contents: 2024-04-23T14:33:46.549 DEBUG:teuthology.orchestra.run.smithi100:> return f 2024-04-23T14:33:46.549 DEBUG:teuthology.orchestra.run.smithi100:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:33:46.549 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:33:46.549 DEBUG:teuthology.orchestra.run.smithi100:> print(_find_admin_socket("client.0")) 2024-04-23T14:33:46.549 DEBUG:teuthology.orchestra.run.smithi100:> ' 2024-04-23T14:33:46.670 INFO:teuthology.orchestra.run.smithi100.stdout:/var/run/ceph/ceph-client.0.39311.asok 2024-04-23T14:33:46.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:33:46.681 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39311.asok 2024-04-23T14:33:46.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:46.681 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39311.asok status 2024-04-23T14:33:46.850 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:33:46.851 INFO:teuthology.orchestra.run.smithi100.stdout: "metadata": { 2024-04-23T14:33:46.851 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:33:46.851 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:33:46.851 INFO:teuthology.orchestra.run.smithi100.stdout: "entity_id": "0", 2024-04-23T14:33:46.851 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": "smithi100", 2024-04-23T14:33:46.851 INFO:teuthology.orchestra.run.smithi100.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:33:46.851 INFO:teuthology.orchestra.run.smithi100.stdout: "pid": "39311", 2024-04-23T14:33:46.851 INFO:teuthology.orchestra.run.smithi100.stdout: "root": "/" 2024-04-23T14:33:46.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:46.851 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_count": 0, 2024-04-23T14:33:46.851 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_pinned_count": 0, 2024-04-23T14:33:46.851 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 5679, 2024-04-23T14:33:46.851 INFO:teuthology.orchestra.run.smithi100.stdout: "inst": { 2024-04-23T14:33:46.852 INFO:teuthology.orchestra.run.smithi100.stdout: "name": { 2024-04-23T14:33:46.852 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "client", 2024-04-23T14:33:46.852 INFO:teuthology.orchestra.run.smithi100.stdout: "num": 5679 2024-04-23T14:33:46.852 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:46.852 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-23T14:33:46.852 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-23T14:33:46.852 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:33:46.852 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2838061550 2024-04-23T14:33:46.852 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:46.852 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:46.852 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-23T14:33:46.852 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-23T14:33:46.852 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:33:46.852 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2838061550 2024-04-23T14:33:46.852 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:46.852 INFO:teuthology.orchestra.run.smithi100.stdout: "inst_str": "client.5679 192.168.0.1:0/2838061550", 2024-04-23T14:33:46.853 INFO:teuthology.orchestra.run.smithi100.stdout: "addr_str": "192.168.0.1:0/2838061550", 2024-04-23T14:33:46.853 INFO:teuthology.orchestra.run.smithi100.stdout: "inode_count": 1, 2024-04-23T14:33:46.853 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_epoch": 18, 2024-04-23T14:33:46.853 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch": 38, 2024-04-23T14:33:46.853 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:33:46.853 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklisted": false, 2024-04-23T14:33:46.853 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs" 2024-04-23T14:33:46.853 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:33:46.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:46.865 DEBUG:teuthology.orchestra.run.smithi100:> stat --file-system '--printf=%T 2024-04-23T14:33:46.866 DEBUG:teuthology.orchestra.run.smithi100:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:46.874 INFO:teuthology.orchestra.run.smithi100.stdout:fuseblk 2024-04-23T14:33:46.874 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:46.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:46.875 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:46.937 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create data_new --pg_num_min 64 2024-04-23T14:33:47.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed15400640 1 Processor -- start 2024-04-23T14:33:47.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed15400640 1 -- start start 2024-04-23T14:33:47.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed15400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fed1015d600 0x7fed100d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:47.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed15400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed100d04a0 con 0x7fed1015c7e0 2024-04-23T14:33:47.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed15400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed100d0780 con 0x7fed100cca50 2024-04-23T14:33:47.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed15400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fed1015db00 con 0x7fed1015d600 2024-04-23T14:33:47.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed0de00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fed1015c7e0 0x7fed100d1260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57056/0 (socket says 172.21.15.100:57056) 2024-04-23T14:33:47.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed0de00640 1 -- 172.21.15.100:0/12643346 learned_addr learned my addr 172.21.15.100:0/12643346 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:47.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed0f200640 1 --2- 172.21.15.100:0/12643346 >> v2:172.21.15.151:3300/0 conn(0x7fed1015d600 0x7fed100d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:47.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed0d400640 1 -- 172.21.15.100:0/12643346 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2715312044 0 0) 0x7fed100d04a0 con 0x7fed1015c7e0 2024-04-23T14:33:47.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed0d400640 1 -- 172.21.15.100:0/12643346 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fecf4003530 con 0x7fed1015c7e0 2024-04-23T14:33:47.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed0d400640 1 -- 172.21.15.100:0/12643346 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3812926896 0 0) 0x7fecf4003530 con 0x7fed1015c7e0 2024-04-23T14:33:47.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed0d400640 1 -- 172.21.15.100:0/12643346 >> v2:172.21.15.151:3300/0 conn(0x7fed1015d600 msgr2=0x7fed100d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:47.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed0d400640 1 --2- 172.21.15.100:0/12643346 >> v2:172.21.15.151:3300/0 conn(0x7fed1015d600 0x7fed100d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed0d400640 1 -- 172.21.15.100:0/12643346 >> v1:172.21.15.151:6789/0 conn(0x7fed100cca50 legacy=0x7fed100cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:47.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed0d400640 1 -- 172.21.15.100:0/12643346 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed100d0a60 con 0x7fed1015c7e0 2024-04-23T14:33:47.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed0d400640 1 -- 172.21.15.100:0/12643346 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7fecfc0016a0 con 0x7fed1015c7e0 2024-04-23T14:33:47.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed0d400640 1 -- 172.21.15.100:0/12643346 >> v1:172.21.15.100:6789/0 conn(0x7fed1015c7e0 legacy=0x7fed100d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:47.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed0d400640 1 --2- 172.21.15.100:0/12643346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecf40039c0 0x7fecf4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:47.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed0d400640 1 --2- 172.21.15.100:0/12643346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecf4004810 0x7fecf4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:47.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed0d400640 1 --2- 172.21.15.100:0/12643346 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fecf40051d0 0x7fecf400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:47.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed0d400640 1 -- 172.21.15.100:0/12643346 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fed1015db00 con 0x7fecf40039c0 2024-04-23T14:33:47.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed0d400640 1 -- 172.21.15.100:0/12643346 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fecf400c170 con 0x7fecf4004810 2024-04-23T14:33:47.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.112+0000 7fed0d400640 1 -- 172.21.15.100:0/12643346 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fecf400c370 con 0x7fecf40051d0 2024-04-23T14:33:47.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.116+0000 7fed0e800640 1 --2- 172.21.15.100:0/12643346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecf40039c0 0x7fecf4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:47.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.116+0000 7fed0f200640 1 --2- 172.21.15.100:0/12643346 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fecf40051d0 0x7fecf400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:47.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.116+0000 7fed0de00640 1 --2- 172.21.15.100:0/12643346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecf4004810 0x7fecf4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:47.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.116+0000 7fed0e800640 1 -- 172.21.15.100:0/12643346 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fecf40051d0 msgr2=0x7fecf400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:47.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.116+0000 7fed0e800640 1 --2- 172.21.15.100:0/12643346 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fecf40051d0 0x7fecf400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.116+0000 7fed0e800640 1 -- 172.21.15.100:0/12643346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecf4004810 msgr2=0x7fecf4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:47.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.116+0000 7fed0e800640 1 --2- 172.21.15.100:0/12643346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecf4004810 0x7fecf4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.116+0000 7fed0e800640 1 -- 172.21.15.100:0/12643346 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf4003530 con 0x7fecf40039c0 2024-04-23T14:33:47.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.116+0000 7fed0f200640 1 -- 172.21.15.100:0/12643346 reap_dead start 2024-04-23T14:33:47.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.116+0000 7fed0e800640 1 --2- 172.21.15.100:0/12643346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecf40039c0 0x7fecf4003de0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fed00001780 tx=0x7fed0001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136a4e3202be8605 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:47.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.116+0000 7fed0d400640 1 -- 172.21.15.100:0/12643346 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed000013a0 con 0x7fecf40039c0 2024-04-23T14:33:47.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.116+0000 7fed0d400640 1 -- 172.21.15.100:0/12643346 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed000052f0 con 0x7fecf40039c0 2024-04-23T14:33:47.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.116+0000 7fed0d400640 1 -- 172.21.15.100:0/12643346 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed00003840 con 0x7fecf40039c0 2024-04-23T14:33:47.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed15400640 1 -- 172.21.15.100:0/12643346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecf40039c0 msgr2=0x7fecf4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:47.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed15400640 1 --2- 172.21.15.100:0/12643346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecf40039c0 0x7fecf4003de0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fed00001780 tx=0x7fed0001c3d0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed15400640 1 -- 172.21.15.100:0/12643346 shutdown_connections 2024-04-23T14:33:47.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed15400640 1 --2- 172.21.15.100:0/12643346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecf40039c0 0x7fecf4003de0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed15400640 1 -- 172.21.15.100:0/12643346 >> 172.21.15.100:0/12643346 conn(0x7fed100cee80 msgr2=0x7fed100cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:47.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed15400640 1 -- 172.21.15.100:0/12643346 shutdown_connections 2024-04-23T14:33:47.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed15400640 1 -- 172.21.15.100:0/12643346 wait complete. 2024-04-23T14:33:47.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed15400640 1 Processor -- start 2024-04-23T14:33:47.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed15400640 1 -- start start 2024-04-23T14:33:47.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed15400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecf40039c0 0x7fed100c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:47.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed15400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed100cbde0 0x7fed100c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:47.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed15400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed1014f570 0x7fed1014f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:47.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed15400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fed100b3d70 con 0x7fed100cbde0 2024-04-23T14:33:47.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed15400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fed100b3bf0 con 0x7fecf40039c0 2024-04-23T14:33:47.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed15400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fed10160980 con 0x7fed1014f570 2024-04-23T14:33:47.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed0de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed100cbde0 0x7fed100c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:47.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed0de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed100cbde0 0x7fed100c7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58374/0 (socket says 172.21.15.100:58374) 2024-04-23T14:33:47.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed0de00640 1 -- 172.21.15.100:0/1610455277 learned_addr learned my addr 172.21.15.100:0/1610455277 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:47.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed0f200640 1 --2- 172.21.15.100:0/1610455277 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed1014f570 0x7fed1014f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:47.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed0e800640 1 --2- 172.21.15.100:0/1610455277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecf40039c0 0x7fed100c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:47.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed0de00640 1 -- 172.21.15.100:0/1610455277 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed1014f570 msgr2=0x7fed1014f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:47.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed0de00640 1 --2- 172.21.15.100:0/1610455277 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed1014f570 0x7fed1014f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed0de00640 1 -- 172.21.15.100:0/1610455277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecf40039c0 msgr2=0x7fed100c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:47.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed0de00640 1 --2- 172.21.15.100:0/1610455277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecf40039c0 0x7fed100c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed0de00640 1 -- 172.21.15.100:0/1610455277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1014ff30 con 0x7fed100cbde0 2024-04-23T14:33:47.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed0de00640 1 --2- 172.21.15.100:0/1610455277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed100cbde0 0x7fed100c7cd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fecfc004ed0 tx=0x7fecfc007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:47.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.120+0000 7fed06a00640 1 -- 172.21.15.100:0/1610455277 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc00f7b0 con 0x7fed100cbde0 2024-04-23T14:33:47.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.124+0000 7fed06a00640 1 -- 172.21.15.100:0/1610455277 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecfc018070 con 0x7fed100cbde0 2024-04-23T14:33:47.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.124+0000 7fed06a00640 1 -- 172.21.15.100:0/1610455277 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc00fbd0 con 0x7fed100cbde0 2024-04-23T14:33:47.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.124+0000 7fed15400640 1 -- 172.21.15.100:0/1610455277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed100c6630 con 0x7fed100cbde0 2024-04-23T14:33:47.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.124+0000 7fed15400640 1 -- 172.21.15.100:0/1610455277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed100c6bf0 con 0x7fed100cbde0 2024-04-23T14:33:47.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.124+0000 7fed15400640 1 -- 172.21.15.100:0/1610455277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed10068ad0 con 0x7fed100cbde0 2024-04-23T14:33:47.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.124+0000 7fed06a00640 1 -- 172.21.15.100:0/1610455277 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fecfc01d460 con 0x7fed100cbde0 2024-04-23T14:33:47.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.124+0000 7fed06a00640 1 --2- 172.21.15.100:0/1610455277 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fecf00683b0 0x7fecf006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:47.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.124+0000 7fed06a00640 1 -- 172.21.15.100:0/1610455277 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fecfc086ad0 con 0x7fed100cbde0 2024-04-23T14:33:47.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.124+0000 7fed0e800640 1 --2- 172.21.15.100:0/1610455277 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fecf00683b0 0x7fecf006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:47.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.124+0000 7fed0e800640 1 --2- 172.21.15.100:0/1610455277 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fecf00683b0 0x7fecf006a870 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fed00001750 tx=0x7fed000011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:47.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.128+0000 7fed06a00640 1 -- 172.21.15.100:0/1610455277 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fecfc056e60 con 0x7fed100cbde0 2024-04-23T14:33:47.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.284+0000 7fed15400640 1 -- 172.21.15.100:0/1610455277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64} v 0) -- 0x7fed100c88f0 con 0x7fed100cbde0 2024-04-23T14:33:47.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.600+0000 7fed06a00640 1 -- 172.21.15.100:0/1610455277 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64}]=0 pool 'data_new' created v39) ==== 124+0+0 (secure 0 0 0) 0x7fecfc056760 con 0x7fed100cbde0 2024-04-23T14:33:47.606 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'data_new' created 2024-04-23T14:33:47.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.604+0000 7fed15400640 1 -- 172.21.15.100:0/1610455277 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fecf00683b0 msgr2=0x7fecf006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:47.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.604+0000 7fed15400640 1 --2- 172.21.15.100:0/1610455277 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fecf00683b0 0x7fecf006a870 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fed00001750 tx=0x7fed000011a0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.604+0000 7fed15400640 1 -- 172.21.15.100:0/1610455277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed100cbde0 msgr2=0x7fed100c7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:47.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.604+0000 7fed15400640 1 --2- 172.21.15.100:0/1610455277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed100cbde0 0x7fed100c7cd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fecfc004ed0 tx=0x7fecfc007430 comp rx=0 tx=0).stop 2024-04-23T14:33:47.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.604+0000 7fed15400640 1 -- 172.21.15.100:0/1610455277 shutdown_connections 2024-04-23T14:33:47.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.604+0000 7fed15400640 1 --2- 172.21.15.100:0/1610455277 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fecf00683b0 0x7fecf006a870 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.604+0000 7fed15400640 1 --2- 172.21.15.100:0/1610455277 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed1014f570 0x7fed1014f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.604+0000 7fed15400640 1 --2- 172.21.15.100:0/1610455277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed100cbde0 0x7fed100c7cd0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.604+0000 7fed15400640 1 --2- 172.21.15.100:0/1610455277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecf40039c0 0x7fed100c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.604+0000 7fed15400640 1 -- 172.21.15.100:0/1610455277 >> 172.21.15.100:0/1610455277 conn(0x7fed100cee80 msgr2=0x7fed10067810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:47.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.604+0000 7fed15400640 1 -- 172.21.15.100:0/1610455277 shutdown_connections 2024-04-23T14:33:47.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.604+0000 7fed15400640 1 -- 172.21.15.100:0/1610455277 wait complete. 2024-04-23T14:33:47.635 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs data_new 2024-04-23T14:33:47.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.768+0000 7f11af000640 1 Processor -- start 2024-04-23T14:33:47.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.768+0000 7f11af000640 1 -- start start 2024-04-23T14:33:47.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.768+0000 7f11af000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f11a806f8c0 0x7f11a806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:47.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.768+0000 7f11af000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f11a80d0170 0x7f11a80ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:47.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.768+0000 7f11af000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11a806dd60 con 0x7f11a80ce620 2024-04-23T14:33:47.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.768+0000 7f11af000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f11a80ad050 con 0x7f11a806f8c0 2024-04-23T14:33:47.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.768+0000 7f11af000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f11a80ad8f0 con 0x7f11a80d0170 2024-04-23T14:33:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.768+0000 7f11a7e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f11a806f8c0 0x7f11a806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.768+0000 7f11a7e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f11a806f8c0 0x7f11a806c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58386/0 (socket says 172.21.15.100:58386) 2024-04-23T14:33:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.768+0000 7f11a7e00640 1 -- 172.21.15.100:0/3538871271 learned_addr learned my addr 172.21.15.100:0/3538871271 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.768+0000 7f11ad000640 1 --1- 172.21.15.100:0/3538871271 >> v1:172.21.15.100:6789/0 conn(0x7f11a80ce620 0x7f11a815f0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57060/0 (socket says 172.21.15.100:57060) 2024-04-23T14:33:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.772+0000 7f11a7400640 1 --2- 172.21.15.100:0/3538871271 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f11a80d0170 0x7f11a80ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.772+0000 7f11a7e00640 1 -- 172.21.15.100:0/3538871271 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f11a80d0170 msgr2=0x7f11a80ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.772+0000 7f11a7e00640 1 --2- 172.21.15.100:0/3538871271 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f11a80d0170 0x7f11a80ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.772+0000 7f11a7e00640 1 -- 172.21.15.100:0/3538871271 >> v1:172.21.15.100:6789/0 conn(0x7f11a80ce620 legacy=0x7f11a815f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.772+0000 7f11a6a00640 1 -- 172.21.15.100:0/3538871271 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3708684206 0 0) 0x7f11a0001120 con 0x7f11a80ce620 2024-04-23T14:33:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.772+0000 7f11a7e00640 1 -- 172.21.15.100:0/3538871271 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a80d09c0 con 0x7f11a806f8c0 2024-04-23T14:33:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.772+0000 7f11a7400640 1 --2- 172.21.15.100:0/3538871271 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f11a80d0170 0x7f11a80ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:47.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.772+0000 7f11a7e00640 1 --2- 172.21.15.100:0/3538871271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11a806f8c0 0x7f11a806c6c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1198009e50 tx=0x7f119801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9702e77c234dc449 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:47.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.772+0000 7f11a6a00640 1 -- 172.21.15.100:0/3538871271 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11980a5720 con 0x7f11a806f8c0 2024-04-23T14:33:47.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.772+0000 7f11a6a00640 1 -- 172.21.15.100:0/3538871271 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11980a58c0 con 0x7f11a806f8c0 2024-04-23T14:33:47.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.772+0000 7f11a6a00640 1 -- 172.21.15.100:0/3538871271 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11980a5c90 con 0x7f11a806f8c0 2024-04-23T14:33:47.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.772+0000 7f11af000640 1 -- 172.21.15.100:0/3538871271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11a806f8c0 msgr2=0x7f11a806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:47.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.772+0000 7f11af000640 1 --2- 172.21.15.100:0/3538871271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11a806f8c0 0x7f11a806c6c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1198009e50 tx=0x7f119801ca30 comp rx=0 tx=0).stop 2024-04-23T14:33:47.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11af000640 1 -- 172.21.15.100:0/3538871271 shutdown_connections 2024-04-23T14:33:47.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11af000640 1 --2- 172.21.15.100:0/3538871271 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f11a80d0170 0x7f11a80ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11af000640 1 --2- 172.21.15.100:0/3538871271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11a806f8c0 0x7f11a806c6c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11af000640 1 -- 172.21.15.100:0/3538871271 >> 172.21.15.100:0/3538871271 conn(0x7f11a80b5200 msgr2=0x7f11a80b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:47.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11af000640 1 -- 172.21.15.100:0/3538871271 shutdown_connections 2024-04-23T14:33:47.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11af000640 1 -- 172.21.15.100:0/3538871271 wait complete. 2024-04-23T14:33:47.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11af000640 1 Processor -- start 2024-04-23T14:33:47.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11af000640 1 -- start start 2024-04-23T14:33:47.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11af000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f11a806f8c0 0x7f11a80c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:47.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11af000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11a80ce620 0x7f11a80b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:47.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11af000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11a80d0170 0x7f11a8072c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:47.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11af000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f11a80d06d0 con 0x7f11a80ce620 2024-04-23T14:33:47.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11af000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f11a80d0550 con 0x7f11a80d0170 2024-04-23T14:33:47.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11af000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f11a8168400 con 0x7f11a806f8c0 2024-04-23T14:33:47.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11a7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11a80ce620 0x7f11a80b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:47.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11a7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11a80ce620 0x7f11a80b2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58390/0 (socket says 172.21.15.100:58390) 2024-04-23T14:33:47.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11a7400640 1 -- 172.21.15.100:0/4034244276 learned_addr learned my addr 172.21.15.100:0/4034244276 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:47.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11a7e00640 1 --2- 172.21.15.100:0/4034244276 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f11a806f8c0 0x7f11a80c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:47.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11ad000640 1 --2- 172.21.15.100:0/4034244276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11a80d0170 0x7f11a8072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:47.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11a7400640 1 -- 172.21.15.100:0/4034244276 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f11a806f8c0 msgr2=0x7f11a80c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:47.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11a7400640 1 --2- 172.21.15.100:0/4034244276 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f11a806f8c0 0x7f11a80c6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11a7400640 1 -- 172.21.15.100:0/4034244276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11a80d0170 msgr2=0x7f11a8072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:47.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11a7400640 1 --2- 172.21.15.100:0/4034244276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11a80d0170 0x7f11a8072c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11a7400640 1 -- 172.21.15.100:0/4034244276 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a80b12c0 con 0x7f11a80ce620 2024-04-23T14:33:47.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.776+0000 7f11a7400640 1 --2- 172.21.15.100:0/4034244276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11a80ce620 0x7f11a80b2450 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f119400e810 tx=0x7f119400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:47.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.780+0000 7f119fe00640 1 -- 172.21.15.100:0/4034244276 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194017070 con 0x7f11a80ce620 2024-04-23T14:33:47.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.780+0000 7f119fe00640 1 -- 172.21.15.100:0/4034244276 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f119400c8d0 con 0x7f11a80ce620 2024-04-23T14:33:47.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.780+0000 7f119fe00640 1 -- 172.21.15.100:0/4034244276 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119400f040 con 0x7f11a80ce620 2024-04-23T14:33:47.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.780+0000 7f11af000640 1 -- 172.21.15.100:0/4034244276 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a80b15a0 con 0x7f11a80ce620 2024-04-23T14:33:47.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.780+0000 7f11af000640 1 -- 172.21.15.100:0/4034244276 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a80b1f30 con 0x7f11a80ce620 2024-04-23T14:33:47.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.780+0000 7f11af000640 1 -- 172.21.15.100:0/4034244276 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a80c70e0 con 0x7f11a80ce620 2024-04-23T14:33:47.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.780+0000 7f119fe00640 1 -- 172.21.15.100:0/4034244276 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f119401a070 con 0x7f11a80ce620 2024-04-23T14:33:47.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.780+0000 7f119fe00640 1 --2- 172.21.15.100:0/4034244276 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1188068380 0x7f118806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:47.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.780+0000 7f119fe00640 1 -- 172.21.15.100:0/4034244276 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6464+0+0 (secure 0 0 0) 0x7f1194086c60 con 0x7f11a80ce620 2024-04-23T14:33:47.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.780+0000 7f11a7e00640 1 --2- 172.21.15.100:0/4034244276 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1188068380 0x7f118806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:47.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.784+0000 7f11a7e00640 1 --2- 172.21.15.100:0/4034244276 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1188068380 0x7f118806a840 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f11980ae000 tx=0x7f11980a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:47.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.784+0000 7f119fe00640 1 -- 172.21.15.100:0/4034244276 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f11940568e0 con 0x7f11a80ce620 2024-04-23T14:33:47.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:47.976+0000 7f11af000640 1 -- 172.21.15.100:0/4034244276 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"} v 0) -- 0x7f11a8068ad0 con 0x7f11a80ce620 2024-04-23T14:33:48.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.616+0000 7f119fe00640 1 -- 172.21.15.100:0/4034244276 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"}]=0 added data pool 7 to fsmap v19) ==== 131+0+0 (secure 0 0 0) 0x7f119405abe0 con 0x7f11a80ce620 2024-04-23T14:33:48.627 INFO:teuthology.orchestra.run.smithi100.stderr:added data pool 7 to fsmap 2024-04-23T14:33:48.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.624+0000 7f11af000640 1 -- 172.21.15.100:0/4034244276 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1188068380 msgr2=0x7f118806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:48.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.624+0000 7f11af000640 1 --2- 172.21.15.100:0/4034244276 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1188068380 0x7f118806a840 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f11980ae000 tx=0x7f11980a7070 comp rx=0 tx=0).stop 2024-04-23T14:33:48.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.624+0000 7f11af000640 1 -- 172.21.15.100:0/4034244276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11a80ce620 msgr2=0x7f11a80b2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:48.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.624+0000 7f11af000640 1 --2- 172.21.15.100:0/4034244276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11a80ce620 0x7f11a80b2450 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f119400e810 tx=0x7f119400ecd0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.624+0000 7f11af000640 1 -- 172.21.15.100:0/4034244276 shutdown_connections 2024-04-23T14:33:48.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.624+0000 7f11af000640 1 --2- 172.21.15.100:0/4034244276 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1188068380 0x7f118806a840 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.624+0000 7f11af000640 1 --2- 172.21.15.100:0/4034244276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11a80d0170 0x7f11a8072c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.624+0000 7f11af000640 1 --2- 172.21.15.100:0/4034244276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11a80ce620 0x7f11a80b2450 secure :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f119400e810 tx=0x7f119400ecd0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.624+0000 7f11af000640 1 --2- 172.21.15.100:0/4034244276 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f11a806f8c0 0x7f11a80c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.624+0000 7f11af000640 1 -- 172.21.15.100:0/4034244276 >> 172.21.15.100:0/4034244276 conn(0x7f11a80b5200 msgr2=0x7f11a80ca810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:48.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.628+0000 7f11af000640 1 -- 172.21.15.100:0/4034244276 shutdown_connections 2024-04-23T14:33:48.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.628+0000 7f11af000640 1 -- 172.21.15.100:0/4034244276 wait complete. 2024-04-23T14:33:48.657 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:33:48.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.788+0000 7f31ad400640 1 Processor -- start 2024-04-23T14:33:48.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.788+0000 7f31ad400640 1 -- start start 2024-04-23T14:33:48.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.788+0000 7f31ad400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f31a815fa90 0x7f31a806d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:48.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.792+0000 7f31ad400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31a806dc70 con 0x7f31a806ef80 2024-04-23T14:33:48.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.792+0000 7f31ad400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31a80b47f0 con 0x7f31a806d350 2024-04-23T14:33:48.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.792+0000 7f31ad400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f31a806eaf0 con 0x7f31a815fa90 2024-04-23T14:33:48.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.792+0000 7f31a7200640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f31a806ef80 0x7f31a806f400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35426/0 (socket says 172.21.15.100:35426) 2024-04-23T14:33:48.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.792+0000 7f31a7200640 1 -- 172.21.15.100:0/1041519379 learned_addr learned my addr 172.21.15.100:0/1041519379 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:48.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.792+0000 7f31a5e00640 1 --2- 172.21.15.100:0/1041519379 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f31a815fa90 0x7f31a806d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:48.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.792+0000 7f31a5400640 1 -- 172.21.15.100:0/1041519379 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2488949425 0 0) 0x7f31a806dc70 con 0x7f31a806ef80 2024-04-23T14:33:48.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.792+0000 7f31a5400640 1 -- 172.21.15.100:0/1041519379 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f318c003530 con 0x7f31a806ef80 2024-04-23T14:33:48.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.792+0000 7f31a5400640 1 -- 172.21.15.100:0/1041519379 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2628125442 0 0) 0x7f31a80b47f0 con 0x7f31a806d350 2024-04-23T14:33:48.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.792+0000 7f31a5400640 1 -- 172.21.15.100:0/1041519379 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31a806dc70 con 0x7f31a806d350 2024-04-23T14:33:48.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.792+0000 7f31a5e00640 1 -- 172.21.15.100:0/1041519379 >> v1:172.21.15.151:6789/0 conn(0x7f31a806d350 legacy=0x7f31a8073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:48.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.792+0000 7f31a5e00640 1 -- 172.21.15.100:0/1041519379 >> v1:172.21.15.100:6789/0 conn(0x7f31a806ef80 legacy=0x7f31a806f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:48.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.792+0000 7f31a5e00640 1 -- 172.21.15.100:0/1041519379 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a80b4ad0 con 0x7f31a815fa90 2024-04-23T14:33:48.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.796+0000 7f31a5e00640 1 --2- 172.21.15.100:0/1041519379 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f31a815fa90 0x7f31a806d730 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f31940097f0 tx=0x7f319401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60c75f83ff83d3ac server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:48.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.796+0000 7f31a5400640 1 -- 172.21.15.100:0/1041519379 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31940b2070 con 0x7f31a815fa90 2024-04-23T14:33:48.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.796+0000 7f31a5400640 1 -- 172.21.15.100:0/1041519379 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31940a9070 con 0x7f31a815fa90 2024-04-23T14:33:48.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.796+0000 7f31a5400640 1 -- 172.21.15.100:0/1041519379 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31940ad880 con 0x7f31a815fa90 2024-04-23T14:33:48.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.796+0000 7f31ad400640 1 -- 172.21.15.100:0/1041519379 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f31a815fa90 msgr2=0x7f31a806d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:48.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.796+0000 7f31ad400640 1 --2- 172.21.15.100:0/1041519379 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f31a815fa90 0x7f31a806d730 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f31940097f0 tx=0x7f319401c450 comp rx=0 tx=0).stop 2024-04-23T14:33:48.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.796+0000 7f31ad400640 1 -- 172.21.15.100:0/1041519379 shutdown_connections 2024-04-23T14:33:48.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.796+0000 7f31ad400640 1 --2- 172.21.15.100:0/1041519379 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f31a815fa90 0x7f31a806d730 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.796+0000 7f31ad400640 1 -- 172.21.15.100:0/1041519379 >> 172.21.15.100:0/1041519379 conn(0x7f31a8072600 msgr2=0x7f31a8072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:48.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.796+0000 7f31ad400640 1 -- 172.21.15.100:0/1041519379 shutdown_connections 2024-04-23T14:33:48.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.796+0000 7f31ad400640 1 -- 172.21.15.100:0/1041519379 wait complete. 2024-04-23T14:33:48.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.796+0000 7f31ad400640 1 Processor -- start 2024-04-23T14:33:48.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.796+0000 7f31ad400640 1 -- start start 2024-04-23T14:33:48.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.796+0000 7f31ad400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31a806d350 0x7f31a814d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:48.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.796+0000 7f31ad400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f31a806ef80 0x7f31a8152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:48.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.796+0000 7f31ad400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31a815fa90 0x7f31a814c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:48.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.796+0000 7f31ad400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f31a816d330 con 0x7f31a806d350 2024-04-23T14:33:48.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.796+0000 7f31ad400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f31a816d1b0 con 0x7f31a815fa90 2024-04-23T14:33:48.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.796+0000 7f31ad400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f31a816d4b0 con 0x7f31a806ef80 2024-04-23T14:33:48.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.796+0000 7f31a6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31a806d350 0x7f31a814d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:48.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.796+0000 7f31a6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31a806d350 0x7f31a814d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36838/0 (socket says 172.21.15.100:36838) 2024-04-23T14:33:48.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.796+0000 7f31a6800640 1 -- 172.21.15.100:0/399562113 learned_addr learned my addr 172.21.15.100:0/399562113 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:48.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.800+0000 7f31a5e00640 1 --2- 172.21.15.100:0/399562113 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f31a806ef80 0x7f31a8152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:48.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.800+0000 7f31a7200640 1 --2- 172.21.15.100:0/399562113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31a815fa90 0x7f31a814c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:48.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.800+0000 7f31a6800640 1 -- 172.21.15.100:0/399562113 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f31a806ef80 msgr2=0x7f31a8152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:48.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.800+0000 7f31a6800640 1 --2- 172.21.15.100:0/399562113 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f31a806ef80 0x7f31a8152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.800+0000 7f31a6800640 1 -- 172.21.15.100:0/399562113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31a815fa90 msgr2=0x7f31a814c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:48.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.800+0000 7f31a6800640 1 --2- 172.21.15.100:0/399562113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31a815fa90 0x7f31a814c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.800+0000 7f31a6800640 1 -- 172.21.15.100:0/399562113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a814d920 con 0x7f31a806d350 2024-04-23T14:33:48.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.800+0000 7f31a7200640 1 --2- 172.21.15.100:0/399562113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31a815fa90 0x7f31a814c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:48.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.800+0000 7f31a5e00640 1 --2- 172.21.15.100:0/399562113 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f31a806ef80 0x7f31a8152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:48.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.800+0000 7f31a6800640 1 --2- 172.21.15.100:0/399562113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31a806d350 0x7f31a814d3e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f319800c770 tx=0x7f319800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:48.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.800+0000 7f319ea00640 1 -- 172.21.15.100:0/399562113 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3198003b80 con 0x7f31a806d350 2024-04-23T14:33:48.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.800+0000 7f319ea00640 1 -- 172.21.15.100:0/399562113 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3198003d20 con 0x7f31a806d350 2024-04-23T14:33:48.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.800+0000 7f31ad400640 1 -- 172.21.15.100:0/399562113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a8157600 con 0x7f31a806d350 2024-04-23T14:33:48.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.800+0000 7f319ea00640 1 -- 172.21.15.100:0/399562113 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319800aa90 con 0x7f31a806d350 2024-04-23T14:33:48.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.800+0000 7f31ad400640 1 -- 172.21.15.100:0/399562113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a8157bc0 con 0x7f31a806d350 2024-04-23T14:33:48.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.800+0000 7f31ad400640 1 -- 172.21.15.100:0/399562113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a8068ad0 con 0x7f31a806d350 2024-04-23T14:33:48.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.800+0000 7f319ea00640 1 -- 172.21.15.100:0/399562113 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f319800ac30 con 0x7f31a806d350 2024-04-23T14:33:48.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.804+0000 7f319ea00640 1 --2- 172.21.15.100:0/399562113 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3188068380 0x7f318806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:48.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.804+0000 7f319ea00640 1 -- 172.21.15.100:0/399562113 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6496+0+0 (secure 0 0 0) 0x7f3198086cf0 con 0x7f31a806d350 2024-04-23T14:33:48.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.804+0000 7f31a5e00640 1 --2- 172.21.15.100:0/399562113 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3188068380 0x7f318806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:48.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.804+0000 7f31a5e00640 1 --2- 172.21.15.100:0/399562113 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3188068380 0x7f318806a840 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f31940af000 tx=0x7f31940a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:48.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.808+0000 7f319ea00640 1 -- 172.21.15.100:0/399562113 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f319808b070 con 0x7f31a806d350 2024-04-23T14:33:49.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.996+0000 7f31ad400640 1 -- 172.21.15.100:0/399562113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31a8073120 con 0x7f31a806d350 2024-04-23T14:33:49.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:48.996+0000 7f319ea00640 1 -- 172.21.15.100:0/399562113 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5195 (secure 0 0 0) 0x7f3198056f10 con 0x7f31a806d350 2024-04-23T14:33:49.003 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:49.003 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2601071793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2601071793},{"type":"v1","addr":"172.21.15.100:6837","nonce":2601071793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2930424296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2930424296},{"type":"v1","addr":"172.21.15.151:6833","nonce":2930424296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1406976216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1406976216},{"type":"v1","addr":"172.21.15.151:6835","nonce":1406976216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:33:38.928245+0000","modified":"2024-04-23T14:33:47.983305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5466":{"gid":5466,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/2062210445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2062210445},{"type":"v1","addr":"172.21.15.100:6836","nonce":2062210445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5466,"qdb_cluster":[5466]},"id":2}]} 2024-04-23T14:33:49.003 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 19 2024-04-23T14:33:49.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.000+0000 7f31ad400640 1 -- 172.21.15.100:0/399562113 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3188068380 msgr2=0x7f318806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:49.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.000+0000 7f31ad400640 1 --2- 172.21.15.100:0/399562113 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3188068380 0x7f318806a840 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f31940af000 tx=0x7f31940a7000 comp rx=0 tx=0).stop 2024-04-23T14:33:49.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.000+0000 7f31ad400640 1 -- 172.21.15.100:0/399562113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31a806d350 msgr2=0x7f31a814d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:49.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.000+0000 7f31ad400640 1 --2- 172.21.15.100:0/399562113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31a806d350 0x7f31a814d3e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f319800c770 tx=0x7f319800cc30 comp rx=0 tx=0).stop 2024-04-23T14:33:49.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.000+0000 7f31ad400640 1 -- 172.21.15.100:0/399562113 shutdown_connections 2024-04-23T14:33:49.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.000+0000 7f31ad400640 1 --2- 172.21.15.100:0/399562113 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3188068380 0x7f318806a840 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.000+0000 7f31ad400640 1 --2- 172.21.15.100:0/399562113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31a815fa90 0x7f31a814c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.000+0000 7f31ad400640 1 --2- 172.21.15.100:0/399562113 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f31a806ef80 0x7f31a8152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.000+0000 7f31ad400640 1 --2- 172.21.15.100:0/399562113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31a806d350 0x7f31a814d3e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.000+0000 7f31ad400640 1 -- 172.21.15.100:0/399562113 >> 172.21.15.100:0/399562113 conn(0x7f31a8072600 msgr2=0x7f31a80ad990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:49.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.000+0000 7f31ad400640 1 -- 172.21.15.100:0/399562113 shutdown_connections 2024-04-23T14:33:49.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.000+0000 7f31ad400640 1 -- 172.21.15.100:0/399562113 wait complete. 2024-04-23T14:33:49.024 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:49.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.156+0000 7fab16a00640 1 Processor -- start 2024-04-23T14:33:49.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.156+0000 7fab16a00640 1 -- start start 2024-04-23T14:33:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.156+0000 7fab16a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fab1015ece0 0x7fab100b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.156+0000 7fab16a00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab100b6830 con 0x7fab10073590 2024-04-23T14:33:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.156+0000 7fab16a00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab100b4a50 con 0x7fab1015f750 2024-04-23T14:33:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.156+0000 7fab16a00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fab100b4d30 con 0x7fab1015ece0 2024-04-23T14:33:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.156+0000 7fab14a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fab1015ece0 0x7fab100b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.156+0000 7fab07400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fab10073590 0x7fab101603c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35432/0 (socket says 172.21.15.100:35432) 2024-04-23T14:33:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.156+0000 7fab07400640 1 -- 172.21.15.100:0/1675704932 learned_addr learned my addr 172.21.15.100:0/1675704932 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.156+0000 7fab14a00640 1 -- 172.21.15.100:0/1675704932 >> v1:172.21.15.151:6789/0 conn(0x7fab1015f750 legacy=0x7fab10072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.156+0000 7fab14a00640 1 -- 172.21.15.100:0/1675704932 >> v1:172.21.15.100:6789/0 conn(0x7fab10073590 legacy=0x7fab101603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.156+0000 7fab14a00640 1 -- 172.21.15.100:0/1675704932 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab100b5010 con 0x7fab1015ece0 2024-04-23T14:33:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.160+0000 7fab14a00640 1 --2- 172.21.15.100:0/1675704932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1015ece0 0x7fab100b4350 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fab08009d90 tx=0x7fab0801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9ea3e75fbd4190 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.160+0000 7fab0f400640 1 -- 172.21.15.100:0/1675704932 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab080b2070 con 0x7fab1015ece0 2024-04-23T14:33:49.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.160+0000 7fab0f400640 1 -- 172.21.15.100:0/1675704932 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab080a5900 con 0x7fab1015ece0 2024-04-23T14:33:49.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.160+0000 7fab0f400640 1 -- 172.21.15.100:0/1675704932 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab080a5cd0 con 0x7fab1015ece0 2024-04-23T14:33:49.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.160+0000 7fab16a00640 1 -- 172.21.15.100:0/1675704932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1015ece0 msgr2=0x7fab100b4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:49.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.160+0000 7fab16a00640 1 --2- 172.21.15.100:0/1675704932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1015ece0 0x7fab100b4350 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fab08009d90 tx=0x7fab0801c930 comp rx=0 tx=0).stop 2024-04-23T14:33:49.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.160+0000 7fab16a00640 1 -- 172.21.15.100:0/1675704932 shutdown_connections 2024-04-23T14:33:49.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.160+0000 7fab16a00640 1 --2- 172.21.15.100:0/1675704932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1015ece0 0x7fab100b4350 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.160+0000 7fab16a00640 1 -- 172.21.15.100:0/1675704932 >> 172.21.15.100:0/1675704932 conn(0x7fab100b3050 msgr2=0x7fab100b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:49.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.160+0000 7fab16a00640 1 -- 172.21.15.100:0/1675704932 shutdown_connections 2024-04-23T14:33:49.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.160+0000 7fab16a00640 1 -- 172.21.15.100:0/1675704932 wait complete. 2024-04-23T14:33:49.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab16a00640 1 Processor -- start 2024-04-23T14:33:49.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab16a00640 1 -- start start 2024-04-23T14:33:49.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab16a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab10073590 0x7fab101556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:49.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab16a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1015ece0 0x7fab10159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:49.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab16a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fab1015f750 0x7fab100b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:49.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab16a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fab1016c680 con 0x7fab1015ece0 2024-04-23T14:33:49.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab16a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fab101615b0 con 0x7fab10073590 2024-04-23T14:33:49.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab16a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fab1016c800 con 0x7fab1015f750 2024-04-23T14:33:49.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab07400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1015ece0 0x7fab10159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:49.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab07400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1015ece0 0x7fab10159160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36854/0 (socket says 172.21.15.100:36854) 2024-04-23T14:33:49.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab07400640 1 -- 172.21.15.100:0/1676863214 learned_addr learned my addr 172.21.15.100:0/1676863214 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:49.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab14a00640 1 --2- 172.21.15.100:0/1676863214 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fab1015f750 0x7fab100b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:49.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab0fe00640 1 --2- 172.21.15.100:0/1676863214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab10073590 0x7fab101556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:49.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab07400640 1 -- 172.21.15.100:0/1676863214 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fab1015f750 msgr2=0x7fab100b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:49.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab07400640 1 --2- 172.21.15.100:0/1676863214 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fab1015f750 0x7fab100b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab07400640 1 -- 172.21.15.100:0/1676863214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab10073590 msgr2=0x7fab101556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:49.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab07400640 1 --2- 172.21.15.100:0/1676863214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab10073590 0x7fab101556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab07400640 1 -- 172.21.15.100:0/1676863214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab100d24d0 con 0x7fab1015ece0 2024-04-23T14:33:49.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab07400640 1 --2- 172.21.15.100:0/1676863214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1015ece0 0x7fab10159160 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7faaf800c450 tx=0x7faaf800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:49.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab0cc00640 1 -- 172.21.15.100:0/1676863214 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8017070 con 0x7fab1015ece0 2024-04-23T14:33:49.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab0cc00640 1 -- 172.21.15.100:0/1676863214 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf8005cf0 con 0x7fab1015ece0 2024-04-23T14:33:49.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab0cc00640 1 -- 172.21.15.100:0/1676863214 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8007290 con 0x7fab1015ece0 2024-04-23T14:33:49.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab16a00640 1 -- 172.21.15.100:0/1676863214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab10159be0 con 0x7fab1015ece0 2024-04-23T14:33:49.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab16a00640 1 -- 172.21.15.100:0/1676863214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab1015a1a0 con 0x7fab1015ece0 2024-04-23T14:33:49.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.164+0000 7fab16a00640 1 -- 172.21.15.100:0/1676863214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab10068ad0 con 0x7fab1015ece0 2024-04-23T14:33:49.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.168+0000 7fab0cc00640 1 -- 172.21.15.100:0/1676863214 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7faaf801a070 con 0x7fab1015ece0 2024-04-23T14:33:49.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.168+0000 7fab0cc00640 1 --2- 172.21.15.100:0/1676863214 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faaec0683b0 0x7faaec06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:49.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.168+0000 7fab0cc00640 1 -- 172.21.15.100:0/1676863214 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6496+0+0 (secure 0 0 0) 0x7faaf8085f70 con 0x7fab1015ece0 2024-04-23T14:33:49.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.168+0000 7fab0fe00640 1 --2- 172.21.15.100:0/1676863214 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faaec0683b0 0x7faaec06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:49.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.168+0000 7fab0fe00640 1 --2- 172.21.15.100:0/1676863214 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faaec0683b0 0x7faaec06a870 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fab1015c830 tx=0x7fab00009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:49.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.172+0000 7fab0cc00640 1 -- 172.21.15.100:0/1676863214 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faaf8055b50 con 0x7fab1015ece0 2024-04-23T14:33:49.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.328+0000 7fab16a00640 1 -- 172.21.15.100:0/1676863214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab10073290 con 0x7fab1015ece0 2024-04-23T14:33:49.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.332+0000 7fab0cc00640 1 -- 172.21.15.100:0/1676863214 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+18779 (secure 0 0 0) 0x7faaf8059e50 con 0x7fab1015ece0 2024-04-23T14:33:49.336 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:49.336 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":40,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:33:47.982982+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:33:36.592437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-23T14:33:37.882390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-04-23T14:33:47.299378+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:49.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.332+0000 7fab16a00640 1 -- 172.21.15.100:0/1676863214 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faaec0683b0 msgr2=0x7faaec06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:49.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.332+0000 7fab16a00640 1 --2- 172.21.15.100:0/1676863214 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faaec0683b0 0x7faaec06a870 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fab1015c830 tx=0x7fab00009060 comp rx=0 tx=0).stop 2024-04-23T14:33:49.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.332+0000 7fab16a00640 1 -- 172.21.15.100:0/1676863214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1015ece0 msgr2=0x7fab10159160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:49.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.332+0000 7fab16a00640 1 --2- 172.21.15.100:0/1676863214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1015ece0 0x7fab10159160 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7faaf800c450 tx=0x7faaf800c910 comp rx=0 tx=0).stop 2024-04-23T14:33:49.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.332+0000 7fab16a00640 1 -- 172.21.15.100:0/1676863214 shutdown_connections 2024-04-23T14:33:49.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.332+0000 7fab16a00640 1 --2- 172.21.15.100:0/1676863214 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faaec0683b0 0x7faaec06a870 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.332+0000 7fab16a00640 1 --2- 172.21.15.100:0/1676863214 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fab1015f750 0x7fab100b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.332+0000 7fab16a00640 1 --2- 172.21.15.100:0/1676863214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1015ece0 0x7fab10159160 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.332+0000 7fab16a00640 1 --2- 172.21.15.100:0/1676863214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab10073590 0x7fab101556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.332+0000 7fab16a00640 1 -- 172.21.15.100:0/1676863214 >> 172.21.15.100:0/1676863214 conn(0x7fab100b3050 msgr2=0x7fab100b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:49.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.332+0000 7fab16a00640 1 -- 172.21.15.100:0/1676863214 shutdown_connections 2024-04-23T14:33:49.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.332+0000 7fab16a00640 1 -- 172.21.15.100:0/1676863214 wait complete. 2024-04-23T14:33:49.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:49.358 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch layoutfile) 2024-04-23T14:33:49.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:49.374 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir layoutdir) 2024-04-23T14:33:49.430 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rw' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2024-04-23T14:33:49.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.612+0000 7f3a4de00640 1 Processor -- start 2024-04-23T14:33:49.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.612+0000 7f3a4de00640 1 -- start start 2024-04-23T14:33:49.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.616+0000 7f3a4de00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3a4815e950 0x7f3a480cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:49.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.616+0000 7f3a4de00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f3a480b5fc0 0x7f3a4806cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:49.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.616+0000 7f3a4de00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a480cc6b0 con 0x7f3a4806e060 2024-04-23T14:33:49.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.616+0000 7f3a4de00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f3a480cc460 con 0x7f3a4815e950 2024-04-23T14:33:49.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.616+0000 7f3a4de00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f3a480cc990 con 0x7f3a480b5fc0 2024-04-23T14:33:49.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.616+0000 7f3a46a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3a4815e950 0x7f3a480cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:49.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.616+0000 7f3a47400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f3a4806e060 0x7f3a4815f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35438/0 (socket says 172.21.15.100:35438) 2024-04-23T14:33:49.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.616+0000 7f3a47400640 1 -- 172.21.15.100:0/976104013 learned_addr learned my addr 172.21.15.100:0/976104013 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:49.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.616+0000 7f3a46000640 1 --2- 172.21.15.100:0/976104013 >> v2:172.21.15.151:3300/0 conn(0x7f3a480b5fc0 0x7f3a4806cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:49.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.616+0000 7f3a46a00640 1 -- 172.21.15.100:0/976104013 >> v2:172.21.15.151:3300/0 conn(0x7f3a480b5fc0 msgr2=0x7f3a4806cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:49.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.616+0000 7f3a46a00640 1 --2- 172.21.15.100:0/976104013 >> v2:172.21.15.151:3300/0 conn(0x7f3a480b5fc0 0x7f3a4806cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.616+0000 7f3a46a00640 1 -- 172.21.15.100:0/976104013 >> v1:172.21.15.100:6789/0 conn(0x7f3a4806e060 legacy=0x7f3a4815f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:49.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.616+0000 7f3a46a00640 1 -- 172.21.15.100:0/976104013 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a480ccc70 con 0x7f3a4815e950 2024-04-23T14:33:49.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.620+0000 7f3a46a00640 1 --2- 172.21.15.100:0/976104013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4815e950 0x7f3a480cac60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3a480d0520 tx=0x7f3a3801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa907dbe3e1f4f11 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:49.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.620+0000 7f3a45600640 1 -- 172.21.15.100:0/976104013 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a380b0070 con 0x7f3a4815e950 2024-04-23T14:33:49.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.620+0000 7f3a45600640 1 -- 172.21.15.100:0/976104013 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a380ab600 con 0x7f3a4815e950 2024-04-23T14:33:49.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.620+0000 7f3a45600640 1 -- 172.21.15.100:0/976104013 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a380ab9d0 con 0x7f3a4815e950 2024-04-23T14:33:49.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.620+0000 7f3a4de00640 1 -- 172.21.15.100:0/976104013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4815e950 msgr2=0x7f3a480cac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:49.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.620+0000 7f3a4de00640 1 --2- 172.21.15.100:0/976104013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4815e950 0x7f3a480cac60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3a480d0520 tx=0x7f3a3801c360 comp rx=0 tx=0).stop 2024-04-23T14:33:49.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.620+0000 7f3a4de00640 1 -- 172.21.15.100:0/976104013 shutdown_connections 2024-04-23T14:33:49.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.620+0000 7f3a4de00640 1 --2- 172.21.15.100:0/976104013 >> v2:172.21.15.151:3300/0 conn(0x7f3a480b5fc0 0x7f3a4806cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.620+0000 7f3a4de00640 1 --2- 172.21.15.100:0/976104013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4815e950 0x7f3a480cac60 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.620+0000 7f3a4de00640 1 -- 172.21.15.100:0/976104013 >> 172.21.15.100:0/976104013 conn(0x7f3a480720d0 msgr2=0x7f3a480724e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:49.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.620+0000 7f3a4de00640 1 -- 172.21.15.100:0/976104013 shutdown_connections 2024-04-23T14:33:49.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.620+0000 7f3a4de00640 1 -- 172.21.15.100:0/976104013 wait complete. 2024-04-23T14:33:49.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.620+0000 7f3a4de00640 1 Processor -- start 2024-04-23T14:33:49.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.620+0000 7f3a4de00640 1 -- start start 2024-04-23T14:33:49.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a4de00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3a4806e060 0x7f3a48150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:49.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a4de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a480b5fc0 0x7f3a48151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:49.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a4de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a4815e950 0x7f3a48151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:49.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a4de00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4806d390 con 0x7f3a480b5fc0 2024-04-23T14:33:49.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a4de00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4806d210 con 0x7f3a4815e950 2024-04-23T14:33:49.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a4de00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f3a48160010 con 0x7f3a4806e060 2024-04-23T14:33:49.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a46000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a480b5fc0 0x7f3a48151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:49.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a46000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a480b5fc0 0x7f3a48151650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36876/0 (socket says 172.21.15.100:36876) 2024-04-23T14:33:49.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a46000640 1 -- 172.21.15.100:0/3565371310 learned_addr learned my addr 172.21.15.100:0/3565371310 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:49.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a46a00640 1 --2- 172.21.15.100:0/3565371310 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3a4806e060 0x7f3a48150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:49.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a47400640 1 --2- 172.21.15.100:0/3565371310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a4815e950 0x7f3a48151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:49.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a46000640 1 -- 172.21.15.100:0/3565371310 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3a4806e060 msgr2=0x7f3a48150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:49.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a46000640 1 --2- 172.21.15.100:0/3565371310 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3a4806e060 0x7f3a48150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a46000640 1 -- 172.21.15.100:0/3565371310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a4815e950 msgr2=0x7f3a48151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:49.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a46000640 1 --2- 172.21.15.100:0/3565371310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a4815e950 0x7f3a48151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a46000640 1 -- 172.21.15.100:0/3565371310 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a481576f0 con 0x7f3a480b5fc0 2024-04-23T14:33:49.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a47400640 1 --2- 172.21.15.100:0/3565371310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a4815e950 0x7f3a48151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:49.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a46000640 1 --2- 172.21.15.100:0/3565371310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a480b5fc0 0x7f3a48151650 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400d800 tx=0x7f3a3400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:49.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a3f400640 1 -- 172.21.15.100:0/3565371310 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3400b6c0 con 0x7f3a480b5fc0 2024-04-23T14:33:49.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a3f400640 1 -- 172.21.15.100:0/3565371310 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a34019070 con 0x7f3a480b5fc0 2024-04-23T14:33:49.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a3f400640 1 -- 172.21.15.100:0/3565371310 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3400ba60 con 0x7f3a480b5fc0 2024-04-23T14:33:49.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a4de00640 1 -- 172.21.15.100:0/3565371310 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a481579d0 con 0x7f3a480b5fc0 2024-04-23T14:33:49.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a4de00640 1 -- 172.21.15.100:0/3565371310 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a481584b0 con 0x7f3a480b5fc0 2024-04-23T14:33:49.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.624+0000 7f3a4de00640 1 -- 172.21.15.100:0/3565371310 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a480691e0 con 0x7f3a480b5fc0 2024-04-23T14:33:49.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.628+0000 7f3a3f400640 1 -- 172.21.15.100:0/3565371310 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3a3400bc00 con 0x7f3a480b5fc0 2024-04-23T14:33:49.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.628+0000 7f3a3f400640 1 --2- 172.21.15.100:0/3565371310 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3a28068380 0x7f3a2806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:49.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.628+0000 7f3a3f400640 1 -- 172.21.15.100:0/3565371310 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f3a340870f0 con 0x7f3a480b5fc0 2024-04-23T14:33:49.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.628+0000 7f3a46a00640 1 --2- 172.21.15.100:0/3565371310 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3a28068380 0x7f3a2806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:49.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.628+0000 7f3a46a00640 1 --2- 172.21.15.100:0/3565371310 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3a28068380 0x7f3a2806a840 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3a380ad000 tx=0x7f3a38008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:49.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.632+0000 7f3a3f400640 1 -- 172.21.15.100:0/3565371310 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3a340573b0 con 0x7f3a480b5fc0 2024-04-23T14:33:49.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.844+0000 7f3a4de00640 1 -- 172.21.15.100:0/3565371310 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) -- 0x7f3a480b1fe0 con 0x7f3a480b5fc0 2024-04-23T14:33:49.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.844+0000 7f3a3f400640 1 -- 172.21.15.100:0/3565371310 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v12) ==== 210+0+167 (secure 0 0 0) 0x7f3a34056cb0 con 0x7f3a480b5fc0 2024-04-23T14:33:49.851 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-04-23T14:33:49.851 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAtxidmyhImMBAAa49ApQEF2Lng6rY94F1HAQ== 2024-04-23T14:33:49.851 INFO:teuthology.orchestra.run.smithi100.stdout: caps mds = "allow rw" 2024-04-23T14:33:49.851 INFO:teuthology.orchestra.run.smithi100.stdout: caps mon = "allow r" 2024-04-23T14:33:49.851 INFO:teuthology.orchestra.run.smithi100.stdout: caps osd = "allow rw pool=cephfs_data,allow rw pool=data_new" 2024-04-23T14:33:49.851 INFO:teuthology.orchestra.run.smithi100.stderr:updated caps for client.0 2024-04-23T14:33:49.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.848+0000 7f3a4de00640 1 -- 172.21.15.100:0/3565371310 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3a28068380 msgr2=0x7f3a2806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:49.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.848+0000 7f3a4de00640 1 --2- 172.21.15.100:0/3565371310 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3a28068380 0x7f3a2806a840 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3a380ad000 tx=0x7f3a38008040 comp rx=0 tx=0).stop 2024-04-23T14:33:49.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.848+0000 7f3a4de00640 1 -- 172.21.15.100:0/3565371310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a480b5fc0 msgr2=0x7f3a48151650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:49.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.848+0000 7f3a4de00640 1 --2- 172.21.15.100:0/3565371310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a480b5fc0 0x7f3a48151650 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400d800 tx=0x7f3a3400dcc0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.848+0000 7f3a4de00640 1 -- 172.21.15.100:0/3565371310 shutdown_connections 2024-04-23T14:33:49.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.848+0000 7f3a4de00640 1 --2- 172.21.15.100:0/3565371310 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3a28068380 0x7f3a2806a840 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.848+0000 7f3a4de00640 1 --2- 172.21.15.100:0/3565371310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a4815e950 0x7f3a48151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.848+0000 7f3a4de00640 1 --2- 172.21.15.100:0/3565371310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a480b5fc0 0x7f3a48151650 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.848+0000 7f3a4de00640 1 --2- 172.21.15.100:0/3565371310 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3a4806e060 0x7f3a48150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.848+0000 7f3a4de00640 1 -- 172.21.15.100:0/3565371310 >> 172.21.15.100:0/3565371310 conn(0x7f3a480720d0 msgr2=0x7f3a480b4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:49.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.848+0000 7f3a4de00640 1 -- 172.21.15.100:0/3565371310 shutdown_connections 2024-04-23T14:33:49.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:49.848+0000 7f3a4de00640 1 -- 172.21.15.100:0/3565371310 wait complete. 2024-04-23T14:33:49.872 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:33:49.872 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:49.880 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:50.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.052+0000 7f1a20c00640 1 Processor -- start 2024-04-23T14:33:50.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.052+0000 7f1a20c00640 1 -- start start 2024-04-23T14:33:50.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.052+0000 7f1a20c00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f1a1c15b640 0x7f1a1c0adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.052+0000 7f1a20c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a1c0b2a10 0x7f1a1c072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.052+0000 7f1a20c00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a1c0af080 con 0x7f1a1c0cc750 2024-04-23T14:33:50.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.052+0000 7f1a20c00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f1a1c0afbc0 con 0x7f1a1c15b640 2024-04-23T14:33:50.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.052+0000 7f1a20c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1a1c15ba20 con 0x7f1a1c0b2a10 2024-04-23T14:33:50.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.052+0000 7f1a1a800640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1a1c0cc750 0x7f1a1c071ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35444/0 (socket says 172.21.15.100:35444) 2024-04-23T14:33:50.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.052+0000 7f1a1a800640 1 -- 172.21.15.100:0/3026923156 learned_addr learned my addr 172.21.15.100:0/3026923156 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:50.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.052+0000 7f1a19e00640 1 --2- 172.21.15.100:0/3026923156 >> v2:172.21.15.151:3300/0 conn(0x7f1a1c15b640 0x7f1a1c0adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.052+0000 7f1a19400640 1 --2- 172.21.15.100:0/3026923156 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a1c0b2a10 0x7f1a1c072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.052+0000 7f1a18a00640 1 -- 172.21.15.100:0/3026923156 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1630184596 0 0) 0x7f1a1c0af080 con 0x7f1a1c0cc750 2024-04-23T14:33:50.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.052+0000 7f1a18a00640 1 -- 172.21.15.100:0/3026923156 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a00003530 con 0x7f1a1c0cc750 2024-04-23T14:33:50.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.056+0000 7f1a19400640 1 -- 172.21.15.100:0/3026923156 >> v2:172.21.15.151:3300/0 conn(0x7f1a1c15b640 msgr2=0x7f1a1c0adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:50.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.056+0000 7f1a19400640 1 --2- 172.21.15.100:0/3026923156 >> v2:172.21.15.151:3300/0 conn(0x7f1a1c15b640 0x7f1a1c0adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.056+0000 7f1a19400640 1 -- 172.21.15.100:0/3026923156 >> v1:172.21.15.100:6789/0 conn(0x7f1a1c0cc750 legacy=0x7f1a1c071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.056+0000 7f1a19400640 1 -- 172.21.15.100:0/3026923156 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1c0af320 con 0x7f1a1c0b2a10 2024-04-23T14:33:50.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.056+0000 7f1a19e00640 1 --2- 172.21.15.100:0/3026923156 >> v2:172.21.15.151:3300/0 conn(0x7f1a1c15b640 0x7f1a1c0adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:33:50.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.056+0000 7f1a19400640 1 --2- 172.21.15.100:0/3026923156 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a1c0b2a10 0x7f1a1c072f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1a080097c0 tx=0x7f1a0801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd3060b4b17c06b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:50.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.056+0000 7f1a18a00640 1 -- 172.21.15.100:0/3026923156 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a080a7070 con 0x7f1a1c0b2a10 2024-04-23T14:33:50.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.056+0000 7f1a18a00640 1 -- 172.21.15.100:0/3026923156 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a080b0070 con 0x7f1a1c0b2a10 2024-04-23T14:33:50.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.056+0000 7f1a18a00640 1 -- 172.21.15.100:0/3026923156 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a080ab870 con 0x7f1a1c0b2a10 2024-04-23T14:33:50.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.056+0000 7f1a20c00640 1 -- 172.21.15.100:0/3026923156 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a1c0b2a10 msgr2=0x7f1a1c072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.056+0000 7f1a20c00640 1 --2- 172.21.15.100:0/3026923156 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a1c0b2a10 0x7f1a1c072f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1a080097c0 tx=0x7f1a0801c470 comp rx=0 tx=0).stop 2024-04-23T14:33:50.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.056+0000 7f1a20c00640 1 -- 172.21.15.100:0/3026923156 shutdown_connections 2024-04-23T14:33:50.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a20c00640 1 --2- 172.21.15.100:0/3026923156 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a1c0b2a10 0x7f1a1c072f00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a20c00640 1 --2- 172.21.15.100:0/3026923156 >> v2:172.21.15.151:3300/0 conn(0x7f1a1c15b640 0x7f1a1c0adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a20c00640 1 -- 172.21.15.100:0/3026923156 >> 172.21.15.100:0/3026923156 conn(0x7f1a1c0b1e80 msgr2=0x7f1a1c0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:50.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a20c00640 1 -- 172.21.15.100:0/3026923156 shutdown_connections 2024-04-23T14:33:50.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a20c00640 1 -- 172.21.15.100:0/3026923156 wait complete. 2024-04-23T14:33:50.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a20c00640 1 Processor -- start 2024-04-23T14:33:50.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a20c00640 1 -- start start 2024-04-23T14:33:50.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a20c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a1c0b2a10 0x7f1a1c1590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a20c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a1c0cc750 0x7f1a1c15e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a20c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a1c15b640 0x7f1a1c14d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a20c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c1683c0 con 0x7f1a1c0b2a10 2024-04-23T14:33:50.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a20c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c073850 con 0x7f1a1c0cc750 2024-04-23T14:33:50.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a20c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1a1c168540 con 0x7f1a1c15b640 2024-04-23T14:33:50.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a19e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a1c0b2a10 0x7f1a1c1590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a19e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a1c0b2a10 0x7f1a1c1590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36878/0 (socket says 172.21.15.100:36878) 2024-04-23T14:33:50.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a19e00640 1 -- 172.21.15.100:0/3181975830 learned_addr learned my addr 172.21.15.100:0/3181975830 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:50.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a19400640 1 --2- 172.21.15.100:0/3181975830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a1c0cc750 0x7f1a1c15e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a1a800640 1 --2- 172.21.15.100:0/3181975830 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a1c15b640 0x7f1a1c14d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a19e00640 1 -- 172.21.15.100:0/3181975830 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a1c15b640 msgr2=0x7f1a1c14d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a19e00640 1 --2- 172.21.15.100:0/3181975830 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a1c15b640 0x7f1a1c14d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a19e00640 1 -- 172.21.15.100:0/3181975830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a1c0cc750 msgr2=0x7f1a1c15e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a19e00640 1 --2- 172.21.15.100:0/3181975830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a1c0cc750 0x7f1a1c15e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a19e00640 1 -- 172.21.15.100:0/3181975830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1c151220 con 0x7f1a1c0b2a10 2024-04-23T14:33:50.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a19e00640 1 --2- 172.21.15.100:0/3181975830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a1c0b2a10 0x7f1a1c1590a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00efc0 tx=0x7f1a0c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:50.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.060+0000 7f1a12000640 1 -- 172.21.15.100:0/3181975830 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c018020 con 0x7f1a1c0b2a10 2024-04-23T14:33:50.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.064+0000 7f1a12000640 1 -- 172.21.15.100:0/3181975830 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a0c014680 con 0x7f1a1c0b2a10 2024-04-23T14:33:50.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.064+0000 7f1a12000640 1 -- 172.21.15.100:0/3181975830 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c014a50 con 0x7f1a1c0b2a10 2024-04-23T14:33:50.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.064+0000 7f1a20c00640 1 -- 172.21.15.100:0/3181975830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a1c153e70 con 0x7f1a1c0b2a10 2024-04-23T14:33:50.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.064+0000 7f1a20c00640 1 -- 172.21.15.100:0/3181975830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a1c154930 con 0x7f1a1c0b2a10 2024-04-23T14:33:50.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.064+0000 7f1a20c00640 1 -- 172.21.15.100:0/3181975830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a1c153c70 con 0x7f1a1c0b2a10 2024-04-23T14:33:50.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.064+0000 7f1a12000640 1 -- 172.21.15.100:0/3181975830 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1a0c01b020 con 0x7f1a1c0b2a10 2024-04-23T14:33:50.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.064+0000 7f1a12000640 1 --2- 172.21.15.100:0/3181975830 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f19fc068380 0x7f19fc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.064+0000 7f1a12000640 1 -- 172.21.15.100:0/3181975830 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f1a0c0878d0 con 0x7f1a1c0b2a10 2024-04-23T14:33:50.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.064+0000 7f1a19400640 1 --2- 172.21.15.100:0/3181975830 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f19fc068380 0x7f19fc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.064+0000 7f1a19400640 1 --2- 172.21.15.100:0/3181975830 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f19fc068380 0x7f19fc06a840 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1a080af000 tx=0x7f1a080a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:50.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.068+0000 7f1a12000640 1 -- 172.21.15.100:0/3181975830 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1a0c022030 con 0x7f1a1c0b2a10 2024-04-23T14:33:50.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.228+0000 7f1a20c00640 1 -- 172.21.15.100:0/3181975830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a1c072e80 con 0x7f1a1c0b2a10 2024-04-23T14:33:50.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.228+0000 7f1a12000640 1 -- 172.21.15.100:0/3181975830 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+18766 (secure 0 0 0) 0x7f1a0c010040 con 0x7f1a1c0b2a10 2024-04-23T14:33:50.233 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:50.233 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":41,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:33:49.609841+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:33:36.592437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-23T14:33:37.882390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-04-23T14:33:47.299378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:50.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.228+0000 7f1a20c00640 1 -- 172.21.15.100:0/3181975830 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f19fc068380 msgr2=0x7f19fc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.228+0000 7f1a20c00640 1 --2- 172.21.15.100:0/3181975830 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f19fc068380 0x7f19fc06a840 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1a080af000 tx=0x7f1a080a7000 comp rx=0 tx=0).stop 2024-04-23T14:33:50.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.228+0000 7f1a20c00640 1 -- 172.21.15.100:0/3181975830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a1c0b2a10 msgr2=0x7f1a1c1590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.228+0000 7f1a20c00640 1 --2- 172.21.15.100:0/3181975830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a1c0b2a10 0x7f1a1c1590a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00efc0 tx=0x7f1a0c00c490 comp rx=0 tx=0).stop 2024-04-23T14:33:50.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.228+0000 7f1a20c00640 1 -- 172.21.15.100:0/3181975830 shutdown_connections 2024-04-23T14:33:50.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.228+0000 7f1a20c00640 1 --2- 172.21.15.100:0/3181975830 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f19fc068380 0x7f19fc06a840 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.228+0000 7f1a20c00640 1 --2- 172.21.15.100:0/3181975830 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a1c15b640 0x7f1a1c14d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.228+0000 7f1a20c00640 1 --2- 172.21.15.100:0/3181975830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a1c0cc750 0x7f1a1c15e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.228+0000 7f1a20c00640 1 --2- 172.21.15.100:0/3181975830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a1c0b2a10 0x7f1a1c1590a0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.228+0000 7f1a20c00640 1 -- 172.21.15.100:0/3181975830 >> 172.21.15.100:0/3181975830 conn(0x7f1a1c0b1e80 msgr2=0x7f1a1c067810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:50.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.232+0000 7f1a20c00640 1 -- 172.21.15.100:0/3181975830 shutdown_connections 2024-04-23T14:33:50.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.232+0000 7f1a20c00640 1 -- 172.21.15.100:0/3181975830 wait complete. 2024-04-23T14:33:50.254 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:33:36.592437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:33:37.882390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-04-23T14:33:47.299378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:33:50.254 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:33:50.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.388+0000 7f2b5fc00640 1 Processor -- start 2024-04-23T14:33:50.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.392+0000 7f2b5fc00640 1 -- start start 2024-04-23T14:33:50.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.392+0000 7f2b5fc00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2b580ce7f0 0x7f2b581663f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.392+0000 7f2b5fc00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b5806dd60 con 0x7f2b5806f8c0 2024-04-23T14:33:50.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.392+0000 7f2b5fc00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b580d09c0 con 0x7f2b5806c6c0 2024-04-23T14:33:50.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.392+0000 7f2b5fc00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f2b580ad050 con 0x7f2b580ce7f0 2024-04-23T14:33:50.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.392+0000 7f2b5d200640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f2b5806f8c0 0x7f2b580d0170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35452/0 (socket says 172.21.15.100:35452) 2024-04-23T14:33:50.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.392+0000 7f2b5d200640 1 -- 172.21.15.100:0/3377755384 learned_addr learned my addr 172.21.15.100:0/3377755384 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:50.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.392+0000 7f2b5dc00640 1 --2- 172.21.15.100:0/3377755384 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2b580ce7f0 0x7f2b581663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.392+0000 7f2b57400640 1 -- 172.21.15.100:0/3377755384 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 748271680 0 0) 0x7f2b40001120 con 0x7f2b5806f8c0 2024-04-23T14:33:50.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.392+0000 7f2b57400640 1 -- 172.21.15.100:0/3377755384 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b3c003530 con 0x7f2b5806f8c0 2024-04-23T14:33:50.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.392+0000 7f2b57400640 1 -- 172.21.15.100:0/3377755384 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3201677678 0 0) 0x7f2b580d09c0 con 0x7f2b5806c6c0 2024-04-23T14:33:50.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.392+0000 7f2b57400640 1 -- 172.21.15.100:0/3377755384 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b40001120 con 0x7f2b5806c6c0 2024-04-23T14:33:50.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.392+0000 7f2b5dc00640 1 -- 172.21.15.100:0/3377755384 >> v1:172.21.15.151:6789/0 conn(0x7f2b5806c6c0 legacy=0x7f2b580ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.392+0000 7f2b5dc00640 1 -- 172.21.15.100:0/3377755384 >> v1:172.21.15.100:6789/0 conn(0x7f2b5806f8c0 legacy=0x7f2b580d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.392+0000 7f2b5dc00640 1 -- 172.21.15.100:0/3377755384 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b580ad2a0 con 0x7f2b580ce7f0 2024-04-23T14:33:50.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.396+0000 7f2b5dc00640 1 --2- 172.21.15.100:0/3377755384 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2b580ce7f0 0x7f2b581663f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2b580ad1d0 tx=0x7f2b5001c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69c429c77416dee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:50.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.396+0000 7f2b57400640 1 -- 172.21.15.100:0/3377755384 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b500b2070 con 0x7f2b580ce7f0 2024-04-23T14:33:50.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.396+0000 7f2b57400640 1 -- 172.21.15.100:0/3377755384 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b500a5960 con 0x7f2b580ce7f0 2024-04-23T14:33:50.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.396+0000 7f2b57400640 1 -- 172.21.15.100:0/3377755384 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b500a5d30 con 0x7f2b580ce7f0 2024-04-23T14:33:50.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.396+0000 7f2b5fc00640 1 -- 172.21.15.100:0/3377755384 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2b580ce7f0 msgr2=0x7f2b581663f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.396+0000 7f2b5fc00640 1 --2- 172.21.15.100:0/3377755384 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2b580ce7f0 0x7f2b581663f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2b580ad1d0 tx=0x7f2b5001c990 comp rx=0 tx=0).stop 2024-04-23T14:33:50.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.396+0000 7f2b5fc00640 1 -- 172.21.15.100:0/3377755384 shutdown_connections 2024-04-23T14:33:50.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.396+0000 7f2b5fc00640 1 --2- 172.21.15.100:0/3377755384 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2b580ce7f0 0x7f2b581663f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.396+0000 7f2b5fc00640 1 -- 172.21.15.100:0/3377755384 >> 172.21.15.100:0/3377755384 conn(0x7f2b580b5200 msgr2=0x7f2b580b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:50.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.396+0000 7f2b5fc00640 1 -- 172.21.15.100:0/3377755384 shutdown_connections 2024-04-23T14:33:50.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.396+0000 7f2b5fc00640 1 -- 172.21.15.100:0/3377755384 wait complete. 2024-04-23T14:33:50.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.396+0000 7f2b5fc00640 1 Processor -- start 2024-04-23T14:33:50.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.396+0000 7f2b5fc00640 1 -- start start 2024-04-23T14:33:50.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b5fc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b5806c6c0 0x7f2b580c5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b5fc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b5806f8c0 0x7f2b580c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b5fc00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2b580ce7f0 0x7f2b58157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b5fc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2b58169770 con 0x7f2b5806c6c0 2024-04-23T14:33:50.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b5fc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2b580ad8f0 con 0x7f2b5806f8c0 2024-04-23T14:33:50.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b5fc00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f2b581698f0 con 0x7f2b580ce7f0 2024-04-23T14:33:50.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b5d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b5806c6c0 0x7f2b580c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b5d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b5806c6c0 0x7f2b580c5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36886/0 (socket says 172.21.15.100:36886) 2024-04-23T14:33:50.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b5d200640 1 -- 172.21.15.100:0/2124847250 learned_addr learned my addr 172.21.15.100:0/2124847250 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:50.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b5dc00640 1 --2- 172.21.15.100:0/2124847250 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2b580ce7f0 0x7f2b58157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b57e00640 1 --2- 172.21.15.100:0/2124847250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b5806f8c0 0x7f2b580c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b5d200640 1 -- 172.21.15.100:0/2124847250 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2b580ce7f0 msgr2=0x7f2b58157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b5d200640 1 --2- 172.21.15.100:0/2124847250 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2b580ce7f0 0x7f2b58157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b5d200640 1 -- 172.21.15.100:0/2124847250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b5806f8c0 msgr2=0x7f2b580c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b5d200640 1 --2- 172.21.15.100:0/2124847250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b5806f8c0 0x7f2b580c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b5d200640 1 -- 172.21.15.100:0/2124847250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b580c5db0 con 0x7f2b5806c6c0 2024-04-23T14:33:50.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b57e00640 1 --2- 172.21.15.100:0/2124847250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b5806f8c0 0x7f2b580c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:50.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b5d200640 1 --2- 172.21.15.100:0/2124847250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b5806c6c0 0x7f2b580c5870 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000c690 tx=0x7f2b4000cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:50.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b54c00640 1 -- 172.21.15.100:0/2124847250 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40017070 con 0x7f2b5806c6c0 2024-04-23T14:33:50.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b54c00640 1 -- 172.21.15.100:0/2124847250 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b40005ae0 con 0x7f2b5806c6c0 2024-04-23T14:33:50.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b54c00640 1 -- 172.21.15.100:0/2124847250 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40007070 con 0x7f2b5806c6c0 2024-04-23T14:33:50.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b5fc00640 1 -- 172.21.15.100:0/2124847250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b580c77f0 con 0x7f2b5806c6c0 2024-04-23T14:33:50.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b5fc00640 1 -- 172.21.15.100:0/2124847250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b5815dea0 con 0x7f2b5806c6c0 2024-04-23T14:33:50.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b5fc00640 1 -- 172.21.15.100:0/2124847250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b58068ad0 con 0x7f2b5806c6c0 2024-04-23T14:33:50.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.400+0000 7f2b54c00640 1 -- 172.21.15.100:0/2124847250 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2b4001a070 con 0x7f2b5806c6c0 2024-04-23T14:33:50.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.404+0000 7f2b54c00640 1 --2- 172.21.15.100:0/2124847250 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2b34068380 0x7f2b3406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.404+0000 7f2b54c00640 1 -- 172.21.15.100:0/2124847250 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f2b40086260 con 0x7f2b5806c6c0 2024-04-23T14:33:50.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.404+0000 7f2b57e00640 1 --2- 172.21.15.100:0/2124847250 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2b34068380 0x7f2b3406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.404+0000 7f2b57e00640 1 --2- 172.21.15.100:0/2124847250 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2b34068380 0x7f2b3406a840 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2b58156a70 tx=0x7f2b48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:50.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.408+0000 7f2b54c00640 1 -- 172.21.15.100:0/2124847250 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2b5806dd60 con 0x7f2b5806c6c0 2024-04-23T14:33:50.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.564+0000 7f2b5fc00640 1 -- 172.21.15.100:0/2124847250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2b580c64f0 con 0x7f2b5806c6c0 2024-04-23T14:33:50.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.568+0000 7f2b54c00640 1 -- 172.21.15.100:0/2124847250 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7f2b40055e40 con 0x7f2b5806c6c0 2024-04-23T14:33:50.571 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:33:50.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.568+0000 7f2b5fc00640 1 -- 172.21.15.100:0/2124847250 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2b34068380 msgr2=0x7f2b3406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.568+0000 7f2b5fc00640 1 --2- 172.21.15.100:0/2124847250 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2b34068380 0x7f2b3406a840 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2b58156a70 tx=0x7f2b48009040 comp rx=0 tx=0).stop 2024-04-23T14:33:50.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.568+0000 7f2b5fc00640 1 -- 172.21.15.100:0/2124847250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b5806c6c0 msgr2=0x7f2b580c5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.568+0000 7f2b5fc00640 1 --2- 172.21.15.100:0/2124847250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b5806c6c0 0x7f2b580c5870 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000c690 tx=0x7f2b4000cb50 comp rx=0 tx=0).stop 2024-04-23T14:33:50.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.568+0000 7f2b5fc00640 1 -- 172.21.15.100:0/2124847250 shutdown_connections 2024-04-23T14:33:50.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.568+0000 7f2b5fc00640 1 --2- 172.21.15.100:0/2124847250 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2b34068380 0x7f2b3406a840 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.568+0000 7f2b5fc00640 1 --2- 172.21.15.100:0/2124847250 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2b580ce7f0 0x7f2b58157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.568+0000 7f2b5fc00640 1 --2- 172.21.15.100:0/2124847250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b5806f8c0 0x7f2b580c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.568+0000 7f2b5fc00640 1 --2- 172.21.15.100:0/2124847250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b5806c6c0 0x7f2b580c5870 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.568+0000 7f2b5fc00640 1 -- 172.21.15.100:0/2124847250 >> 172.21.15.100:0/2124847250 conn(0x7f2b580b5200 msgr2=0x7f2b580ca930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:50.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.568+0000 7f2b5fc00640 1 -- 172.21.15.100:0/2124847250 shutdown_connections 2024-04-23T14:33:50.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.568+0000 7f2b5fc00640 1 -- 172.21.15.100:0/2124847250 wait complete. 2024-04-23T14:33:50.592 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:33:50.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.724+0000 7f6796c00640 1 Processor -- start 2024-04-23T14:33:50.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.724+0000 7f6796c00640 1 -- start start 2024-04-23T14:33:50.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.728+0000 7f6796c00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f679006c6c0 0x7f67900ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.728+0000 7f6796c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f67900ce620 0x7f67900add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.728+0000 7f6796c00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f679006dd60 con 0x7f679006f8c0 2024-04-23T14:33:50.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.728+0000 7f6796c00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f67900ad050 con 0x7f67900ce620 2024-04-23T14:33:50.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.728+0000 7f6796c00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f67900ad8f0 con 0x7f679006c6c0 2024-04-23T14:33:50.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.728+0000 7f6794c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f67900ce620 0x7f67900add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.728+0000 7f678fe00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f679006f8c0 0x7f67900d0170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35454/0 (socket says 172.21.15.100:35454) 2024-04-23T14:33:50.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.728+0000 7f678fe00640 1 -- 172.21.15.100:0/3049467947 learned_addr learned my addr 172.21.15.100:0/3049467947 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:50.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.728+0000 7f6787400640 1 --2- 172.21.15.100:0/3049467947 >> v2:172.21.15.151:3300/0 conn(0x7f679006c6c0 0x7f67900ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.728+0000 7f6794c00640 1 -- 172.21.15.100:0/3049467947 >> v2:172.21.15.151:3300/0 conn(0x7f679006c6c0 msgr2=0x7f67900ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:50.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.728+0000 7f6794c00640 1 --2- 172.21.15.100:0/3049467947 >> v2:172.21.15.151:3300/0 conn(0x7f679006c6c0 0x7f67900ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.728+0000 7f6794c00640 1 -- 172.21.15.100:0/3049467947 >> v1:172.21.15.100:6789/0 conn(0x7f679006f8c0 legacy=0x7f67900d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.728+0000 7f6794c00640 1 -- 172.21.15.100:0/3049467947 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67900d09c0 con 0x7f67900ce620 2024-04-23T14:33:50.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.732+0000 7f6794c00640 1 --2- 172.21.15.100:0/3049467947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67900ce620 0x7f67900add80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f67900d00c0 tx=0x7f678801c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c672524ff4bd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:50.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.732+0000 7f678f400640 1 -- 172.21.15.100:0/3049467947 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67880b0070 con 0x7f67900ce620 2024-04-23T14:33:50.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.732+0000 7f678f400640 1 -- 172.21.15.100:0/3049467947 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67880ab600 con 0x7f67900ce620 2024-04-23T14:33:50.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.732+0000 7f678f400640 1 -- 172.21.15.100:0/3049467947 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67880ab9d0 con 0x7f67900ce620 2024-04-23T14:33:50.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.732+0000 7f6796c00640 1 -- 172.21.15.100:0/3049467947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67900ce620 msgr2=0x7f67900add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.732+0000 7f6796c00640 1 --2- 172.21.15.100:0/3049467947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67900ce620 0x7f67900add80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f67900d00c0 tx=0x7f678801c310 comp rx=0 tx=0).stop 2024-04-23T14:33:50.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.732+0000 7f6796c00640 1 -- 172.21.15.100:0/3049467947 shutdown_connections 2024-04-23T14:33:50.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.732+0000 7f6796c00640 1 --2- 172.21.15.100:0/3049467947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67900ce620 0x7f67900add80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.732+0000 7f6796c00640 1 --2- 172.21.15.100:0/3049467947 >> v2:172.21.15.151:3300/0 conn(0x7f679006c6c0 0x7f67900ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.732+0000 7f6796c00640 1 -- 172.21.15.100:0/3049467947 >> 172.21.15.100:0/3049467947 conn(0x7f67900b5200 msgr2=0x7f67900b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:50.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.732+0000 7f6796c00640 1 -- 172.21.15.100:0/3049467947 shutdown_connections 2024-04-23T14:33:50.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.732+0000 7f6796c00640 1 -- 172.21.15.100:0/3049467947 wait complete. 2024-04-23T14:33:50.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.732+0000 7f6796c00640 1 Processor -- start 2024-04-23T14:33:50.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.732+0000 7f6796c00640 1 -- start start 2024-04-23T14:33:50.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f6796c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f679006c6c0 0x7f67901552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f6796c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f679006f8c0 0x7f67900c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f6796c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f67900ce620 0x7f6790158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f6796c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f67900aee60 con 0x7f679006c6c0 2024-04-23T14:33:50.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f6796c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f67900aece0 con 0x7f679006f8c0 2024-04-23T14:33:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f6796c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f6790168400 con 0x7f67900ce620 2024-04-23T14:33:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f678fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f679006c6c0 0x7f67901552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f678fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f679006c6c0 0x7f67901552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36894/0 (socket says 172.21.15.100:36894) 2024-04-23T14:33:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f678fe00640 1 -- 172.21.15.100:0/1131028863 learned_addr learned my addr 172.21.15.100:0/1131028863 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f6787400640 1 --2- 172.21.15.100:0/1131028863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f679006f8c0 0x7f67900c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f6794c00640 1 --2- 172.21.15.100:0/1131028863 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f67900ce620 0x7f6790158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f678fe00640 1 -- 172.21.15.100:0/1131028863 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f67900ce620 msgr2=0x7f6790158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f678fe00640 1 --2- 172.21.15.100:0/1131028863 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f67900ce620 0x7f6790158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f678fe00640 1 -- 172.21.15.100:0/1131028863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f679006f8c0 msgr2=0x7f67900c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f678fe00640 1 --2- 172.21.15.100:0/1131028863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f679006f8c0 0x7f67900c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f678fe00640 1 -- 172.21.15.100:0/1131028863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6790156fe0 con 0x7f679006c6c0 2024-04-23T14:33:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f6794c00640 1 --2- 172.21.15.100:0/1131028863 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f67900ce620 0x7f6790158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f678fe00640 1 --2- 172.21.15.100:0/1131028863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f679006c6c0 0x7f67901552e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f678000cbd0 tx=0x7f678000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:50.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f678cc00640 1 -- 172.21.15.100:0/1131028863 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780018070 con 0x7f679006c6c0 2024-04-23T14:33:50.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f678cc00640 1 -- 172.21.15.100:0/1131028863 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6780004d70 con 0x7f679006c6c0 2024-04-23T14:33:50.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f6796c00640 1 -- 172.21.15.100:0/1131028863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67901572c0 con 0x7f679006c6c0 2024-04-23T14:33:50.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f678cc00640 1 -- 172.21.15.100:0/1131028863 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678000ad60 con 0x7f679006c6c0 2024-04-23T14:33:50.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f6796c00640 1 -- 172.21.15.100:0/1131028863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6790157880 con 0x7f679006c6c0 2024-04-23T14:33:50.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f6796c00640 1 -- 172.21.15.100:0/1131028863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6790068ad0 con 0x7f679006c6c0 2024-04-23T14:33:50.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.736+0000 7f678cc00640 1 -- 172.21.15.100:0/1131028863 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f678001b070 con 0x7f679006c6c0 2024-04-23T14:33:50.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.740+0000 7f678cc00640 1 --2- 172.21.15.100:0/1131028863 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f676c068380 0x7f676c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.740+0000 7f678cc00640 1 -- 172.21.15.100:0/1131028863 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f67800867b0 con 0x7f679006c6c0 2024-04-23T14:33:50.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.740+0000 7f6787400640 1 --2- 172.21.15.100:0/1131028863 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f676c068380 0x7f676c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.740+0000 7f6787400640 1 --2- 172.21.15.100:0/1131028863 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f676c068380 0x7f676c06a840 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f67901faee0 tx=0x7f6778009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:50.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.744+0000 7f678cc00640 1 -- 172.21.15.100:0/1131028863 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f679006dd60 con 0x7f679006c6c0 2024-04-23T14:33:50.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.900+0000 7f6796c00640 1 -- 172.21.15.100:0/1131028863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f679006cc00 con 0x7f679006c6c0 2024-04-23T14:33:50.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.900+0000 7f678cc00640 1 -- 172.21.15.100:0/1131028863 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) ==== 104+0+11 (secure 0 0 0) 0x7f67800564a0 con 0x7f679006c6c0 2024-04-23T14:33:50.906 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:50.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.904+0000 7f6796c00640 1 -- 172.21.15.100:0/1131028863 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f676c068380 msgr2=0x7f676c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.904+0000 7f6796c00640 1 --2- 172.21.15.100:0/1131028863 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f676c068380 0x7f676c06a840 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f67901faee0 tx=0x7f6778009190 comp rx=0 tx=0).stop 2024-04-23T14:33:50.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.904+0000 7f6796c00640 1 -- 172.21.15.100:0/1131028863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f679006c6c0 msgr2=0x7f67901552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.904+0000 7f6796c00640 1 --2- 172.21.15.100:0/1131028863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f679006c6c0 0x7f67901552e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f678000cbd0 tx=0x7f678000a430 comp rx=0 tx=0).stop 2024-04-23T14:33:50.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.904+0000 7f6796c00640 1 -- 172.21.15.100:0/1131028863 shutdown_connections 2024-04-23T14:33:50.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.904+0000 7f6796c00640 1 --2- 172.21.15.100:0/1131028863 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f676c068380 0x7f676c06a840 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.904+0000 7f6796c00640 1 --2- 172.21.15.100:0/1131028863 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f67900ce620 0x7f6790158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.904+0000 7f6796c00640 1 --2- 172.21.15.100:0/1131028863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f679006f8c0 0x7f67900c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.904+0000 7f6796c00640 1 --2- 172.21.15.100:0/1131028863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f679006c6c0 0x7f67901552e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.904+0000 7f6796c00640 1 -- 172.21.15.100:0/1131028863 >> 172.21.15.100:0/1131028863 conn(0x7f67900b5200 msgr2=0x7f6790159830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:50.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.904+0000 7f6796c00640 1 -- 172.21.15.100:0/1131028863 shutdown_connections 2024-04-23T14:33:50.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:50.904+0000 7f6796c00640 1 -- 172.21.15.100:0/1131028863 wait complete. 2024-04-23T14:33:50.927 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:33:51.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.060+0000 7efd6a200640 1 Processor -- start 2024-04-23T14:33:51.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.060+0000 7efd6a200640 1 -- start start 2024-04-23T14:33:51.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.060+0000 7efd6a200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efd6406d450 0x7efd6406d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:51.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.060+0000 7efd6a200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd6406ddb0 con 0x7efd6415ed80 2024-04-23T14:33:51.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.060+0000 7efd6a200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd640cefc0 con 0x7efd6415f890 2024-04-23T14:33:51.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.060+0000 7efd6a200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7efd640cf2a0 con 0x7efd6406d450 2024-04-23T14:33:51.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.060+0000 7efd63e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efd6406d450 0x7efd6406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:51.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.060+0000 7efd62a00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7efd6415ed80 0x7efd640d0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35456/0 (socket says 172.21.15.100:35456) 2024-04-23T14:33:51.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.060+0000 7efd62a00640 1 -- 172.21.15.100:0/2677822699 learned_addr learned my addr 172.21.15.100:0/2677822699 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:51.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.060+0000 7efd63e00640 1 -- 172.21.15.100:0/2677822699 >> v1:172.21.15.151:6789/0 conn(0x7efd6415f890 legacy=0x7efd640ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.060+0000 7efd63e00640 1 -- 172.21.15.100:0/2677822699 >> v1:172.21.15.100:6789/0 conn(0x7efd6415ed80 legacy=0x7efd640d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.060+0000 7efd62000640 1 -- 172.21.15.100:0/2677822699 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3419809348 0 0) 0x7efd6406ddb0 con 0x7efd6415ed80 2024-04-23T14:33:51.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.060+0000 7efd63e00640 1 -- 172.21.15.100:0/2677822699 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd640cf580 con 0x7efd6406d450 2024-04-23T14:33:51.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.064+0000 7efd63e00640 1 --2- 172.21.15.100:0/2677822699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6406d450 0x7efd6406d870 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009d90 tx=0x7efd5c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74acb6ee1b8a6ab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:51.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.064+0000 7efd62000640 1 -- 172.21.15.100:0/2677822699 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c0b2070 con 0x7efd6406d450 2024-04-23T14:33:51.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.064+0000 7efd62000640 1 -- 172.21.15.100:0/2677822699 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd5c0a5830 con 0x7efd6406d450 2024-04-23T14:33:51.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.064+0000 7efd62000640 1 -- 172.21.15.100:0/2677822699 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c0a5c00 con 0x7efd6406d450 2024-04-23T14:33:51.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.064+0000 7efd6a200640 1 -- 172.21.15.100:0/2677822699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6406d450 msgr2=0x7efd6406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.064+0000 7efd6a200640 1 --2- 172.21.15.100:0/2677822699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6406d450 0x7efd6406d870 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009d90 tx=0x7efd5c01c970 comp rx=0 tx=0).stop 2024-04-23T14:33:51.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.064+0000 7efd6a200640 1 -- 172.21.15.100:0/2677822699 shutdown_connections 2024-04-23T14:33:51.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.064+0000 7efd6a200640 1 --2- 172.21.15.100:0/2677822699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6406d450 0x7efd6406d870 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.064+0000 7efd6a200640 1 -- 172.21.15.100:0/2677822699 >> 172.21.15.100:0/2677822699 conn(0x7efd640719d0 msgr2=0x7efd64071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:51.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.064+0000 7efd6a200640 1 -- 172.21.15.100:0/2677822699 shutdown_connections 2024-04-23T14:33:51.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.064+0000 7efd6a200640 1 -- 172.21.15.100:0/2677822699 wait complete. 2024-04-23T14:33:51.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd6a200640 1 Processor -- start 2024-04-23T14:33:51.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd6a200640 1 -- start start 2024-04-23T14:33:51.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd6a200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6406d450 0x7efd6414ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:51.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd6a200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd6415ed80 0x7efd640b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:51.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd6a200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efd6415f890 0x7efd640b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:51.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd6a200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efd6416c720 con 0x7efd6406d450 2024-04-23T14:33:51.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd6a200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efd640cdb30 con 0x7efd6415ed80 2024-04-23T14:33:51.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd6a200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7efd6416c8a0 con 0x7efd6415f890 2024-04-23T14:33:51.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd63400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6406d450 0x7efd6414ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:51.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd63400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6406d450 0x7efd6414ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36910/0 (socket says 172.21.15.100:36910) 2024-04-23T14:33:51.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd63400640 1 -- 172.21.15.100:0/4230180359 learned_addr learned my addr 172.21.15.100:0/4230180359 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:51.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd63e00640 1 --2- 172.21.15.100:0/4230180359 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efd6415f890 0x7efd640b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:51.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd62a00640 1 --2- 172.21.15.100:0/4230180359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd6415ed80 0x7efd640b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:51.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd63400640 1 -- 172.21.15.100:0/4230180359 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efd6415f890 msgr2=0x7efd640b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd63400640 1 --2- 172.21.15.100:0/4230180359 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efd6415f890 0x7efd640b55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd63400640 1 -- 172.21.15.100:0/4230180359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd6415ed80 msgr2=0x7efd640b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd63400640 1 --2- 172.21.15.100:0/4230180359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd6415ed80 0x7efd640b50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd63400640 1 -- 172.21.15.100:0/4230180359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd641548b0 con 0x7efd6406d450 2024-04-23T14:33:51.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd63400640 1 --2- 172.21.15.100:0/4230180359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6406d450 0x7efd6414ee30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7efd5400c3c0 tx=0x7efd5400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:51.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd5b400640 1 -- 172.21.15.100:0/4230180359 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54017070 con 0x7efd6406d450 2024-04-23T14:33:51.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd5b400640 1 -- 172.21.15.100:0/4230180359 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd540052d0 con 0x7efd6406d450 2024-04-23T14:33:51.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd5b400640 1 -- 172.21.15.100:0/4230180359 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54003840 con 0x7efd6406d450 2024-04-23T14:33:51.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd6a200640 1 -- 172.21.15.100:0/4230180359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd6414f5f0 con 0x7efd6406d450 2024-04-23T14:33:51.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.068+0000 7efd6a200640 1 -- 172.21.15.100:0/4230180359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd6414fbb0 con 0x7efd6406d450 2024-04-23T14:33:51.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.072+0000 7efd6a200640 1 -- 172.21.15.100:0/4230180359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd64068ad0 con 0x7efd6406d450 2024-04-23T14:33:51.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.072+0000 7efd5b400640 1 -- 172.21.15.100:0/4230180359 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7efd5401a070 con 0x7efd6406d450 2024-04-23T14:33:51.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.072+0000 7efd5b400640 1 --2- 172.21.15.100:0/4230180359 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7efd44068380 0x7efd4406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:51.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.072+0000 7efd5b400640 1 -- 172.21.15.100:0/4230180359 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7efd54085fc0 con 0x7efd6406d450 2024-04-23T14:33:51.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.072+0000 7efd62a00640 1 --2- 172.21.15.100:0/4230180359 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7efd44068380 0x7efd4406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:51.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.072+0000 7efd62a00640 1 --2- 172.21.15.100:0/4230180359 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7efd44068380 0x7efd4406a840 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efd50007fd0 tx=0x7efd50007ba0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:51.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.076+0000 7efd5b400640 1 -- 172.21.15.100:0/4230180359 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efd54005480 con 0x7efd6406d450 2024-04-23T14:33:51.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.232+0000 7efd6a200640 1 -- 172.21.15.100:0/4230180359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efd640b8380 con 0x7efd6406d450 2024-04-23T14:33:51.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.232+0000 7efd5b400640 1 -- 172.21.15.100:0/4230180359 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) ==== 100+0+11 (secure 0 0 0) 0x7efd54055ba0 con 0x7efd6406d450 2024-04-23T14:33:51.238 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:33:51.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.236+0000 7efd6a200640 1 -- 172.21.15.100:0/4230180359 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7efd44068380 msgr2=0x7efd4406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.236+0000 7efd6a200640 1 --2- 172.21.15.100:0/4230180359 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7efd44068380 0x7efd4406a840 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efd50007fd0 tx=0x7efd50007ba0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.236+0000 7efd6a200640 1 -- 172.21.15.100:0/4230180359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6406d450 msgr2=0x7efd6414ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.236+0000 7efd6a200640 1 --2- 172.21.15.100:0/4230180359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6406d450 0x7efd6414ee30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7efd5400c3c0 tx=0x7efd5400c880 comp rx=0 tx=0).stop 2024-04-23T14:33:51.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.236+0000 7efd6a200640 1 -- 172.21.15.100:0/4230180359 shutdown_connections 2024-04-23T14:33:51.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.236+0000 7efd6a200640 1 --2- 172.21.15.100:0/4230180359 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7efd44068380 0x7efd4406a840 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.236+0000 7efd6a200640 1 --2- 172.21.15.100:0/4230180359 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efd6415f890 0x7efd640b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.236+0000 7efd6a200640 1 --2- 172.21.15.100:0/4230180359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd6415ed80 0x7efd640b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.236+0000 7efd6a200640 1 --2- 172.21.15.100:0/4230180359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6406d450 0x7efd6414ee30 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.236+0000 7efd6a200640 1 -- 172.21.15.100:0/4230180359 >> 172.21.15.100:0/4230180359 conn(0x7efd640719d0 msgr2=0x7efd64070240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:51.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.236+0000 7efd6a200640 1 -- 172.21.15.100:0/4230180359 shutdown_connections 2024-04-23T14:33:51.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.236+0000 7efd6a200640 1 -- 172.21.15.100:0/4230180359 wait complete. 2024-04-23T14:33:51.259 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-04-23T14:33:51.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.392+0000 7f7e57c00640 1 Processor -- start 2024-04-23T14:33:51.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.392+0000 7f7e57c00640 1 -- start start 2024-04-23T14:33:51.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.392+0000 7f7e57c00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f7e5006f8c0 0x7f7e5006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:51.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.392+0000 7f7e57c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7e500d0170 0x7f7e500ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:51.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.392+0000 7f7e57c00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e5006dd60 con 0x7f7e500ce620 2024-04-23T14:33:51.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.392+0000 7f7e57c00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f7e500ad050 con 0x7f7e5006f8c0 2024-04-23T14:33:51.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.392+0000 7f7e57c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7e500ad8f0 con 0x7f7e500d0170 2024-04-23T14:33:51.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.392+0000 7f7e55c00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f7e500ce620 0x7f7e5015f0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35464/0 (socket says 172.21.15.100:35464) 2024-04-23T14:33:51.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.392+0000 7f7e55c00640 1 -- 172.21.15.100:0/3663345234 learned_addr learned my addr 172.21.15.100:0/3663345234 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:51.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.392+0000 7f7e55200640 1 --2- 172.21.15.100:0/3663345234 >> v2:172.21.15.151:3300/0 conn(0x7f7e5006f8c0 0x7f7e5006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:51.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.392+0000 7f7e4fe00640 1 --2- 172.21.15.100:0/3663345234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7e500d0170 0x7f7e500ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:51.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.392+0000 7f7e4f400640 1 -- 172.21.15.100:0/3663345234 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3176905569 0 0) 0x7f7e480014a0 con 0x7f7e500ce620 2024-04-23T14:33:51.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.392+0000 7f7e4f400640 1 -- 172.21.15.100:0/3663345234 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e34003530 con 0x7f7e500ce620 2024-04-23T14:33:51.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.392+0000 7f7e4fe00640 1 -- 172.21.15.100:0/3663345234 >> v2:172.21.15.151:3300/0 conn(0x7f7e5006f8c0 msgr2=0x7f7e5006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:51.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.392+0000 7f7e4fe00640 1 --2- 172.21.15.100:0/3663345234 >> v2:172.21.15.151:3300/0 conn(0x7f7e5006f8c0 0x7f7e5006c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.392+0000 7f7e4fe00640 1 -- 172.21.15.100:0/3663345234 >> v1:172.21.15.100:6789/0 conn(0x7f7e500ce620 legacy=0x7f7e5015f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.392+0000 7f7e4fe00640 1 -- 172.21.15.100:0/3663345234 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e500d09c0 con 0x7f7e500d0170 2024-04-23T14:33:51.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.396+0000 7f7e55200640 1 --2- 172.21.15.100:0/3663345234 >> v2:172.21.15.151:3300/0 conn(0x7f7e5006f8c0 0x7f7e5006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:33:51.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.396+0000 7f7e4fe00640 1 --2- 172.21.15.100:0/3663345234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7e500d0170 0x7f7e500ce0e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7e380097c0 tx=0x7f7e3801e6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d089d9b8d24600e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:51.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.396+0000 7f7e4f400640 1 -- 172.21.15.100:0/3663345234 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e380ac650 con 0x7f7e500d0170 2024-04-23T14:33:51.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.396+0000 7f7e4f400640 1 -- 172.21.15.100:0/3663345234 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e380b1070 con 0x7f7e500d0170 2024-04-23T14:33:51.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.396+0000 7f7e4f400640 1 -- 172.21.15.100:0/3663345234 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e380aca10 con 0x7f7e500d0170 2024-04-23T14:33:51.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.396+0000 7f7e57c00640 1 -- 172.21.15.100:0/3663345234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7e500d0170 msgr2=0x7f7e500ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.396+0000 7f7e57c00640 1 --2- 172.21.15.100:0/3663345234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7e500d0170 0x7f7e500ce0e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7e380097c0 tx=0x7f7e3801e6b0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.396+0000 7f7e57c00640 1 -- 172.21.15.100:0/3663345234 shutdown_connections 2024-04-23T14:33:51.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.396+0000 7f7e57c00640 1 --2- 172.21.15.100:0/3663345234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7e500d0170 0x7f7e500ce0e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.396+0000 7f7e57c00640 1 --2- 172.21.15.100:0/3663345234 >> v2:172.21.15.151:3300/0 conn(0x7f7e5006f8c0 0x7f7e5006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.396+0000 7f7e57c00640 1 -- 172.21.15.100:0/3663345234 >> 172.21.15.100:0/3663345234 conn(0x7f7e500b5200 msgr2=0x7f7e500b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:51.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.396+0000 7f7e57c00640 1 -- 172.21.15.100:0/3663345234 shutdown_connections 2024-04-23T14:33:51.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.396+0000 7f7e57c00640 1 -- 172.21.15.100:0/3663345234 wait complete. 2024-04-23T14:33:51.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e57c00640 1 Processor -- start 2024-04-23T14:33:51.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e57c00640 1 -- start start 2024-04-23T14:33:51.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e57c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e5006f8c0 0x7f7e501493b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:51.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e57c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7e500ce620 0x7f7e50072520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:51.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e57c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e500d0170 0x7f7e5015e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:51.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e57c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7e500d0730 con 0x7f7e5006f8c0 2024-04-23T14:33:51.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e57c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7e500d05b0 con 0x7f7e500d0170 2024-04-23T14:33:51.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e57c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7e500ccc60 con 0x7f7e500ce620 2024-04-23T14:33:51.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e55200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e5006f8c0 0x7f7e501493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:51.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e4fe00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7e500ce620 0x7f7e50072520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:51.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e55200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e5006f8c0 0x7f7e501493b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36922/0 (socket says 172.21.15.100:36922) 2024-04-23T14:33:51.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e4fe00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7e500ce620 0x7f7e50072520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:35944/0 (socket says 172.21.15.100:35944) 2024-04-23T14:33:51.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e55200640 1 -- 172.21.15.100:0/1725217097 learned_addr learned my addr 172.21.15.100:0/1725217097 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:51.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e55c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e500d0170 0x7f7e5015e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:51.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e4fe00640 1 -- 172.21.15.100:0/1725217097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e500d0170 msgr2=0x7f7e5015e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e4fe00640 1 --2- 172.21.15.100:0/1725217097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e500d0170 0x7f7e5015e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e4fe00640 1 -- 172.21.15.100:0/1725217097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e5006f8c0 msgr2=0x7f7e501493b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e4fe00640 1 --2- 172.21.15.100:0/1725217097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e5006f8c0 0x7f7e501493b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e4fe00640 1 -- 172.21.15.100:0/1725217097 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e50156c10 con 0x7f7e500ce620 2024-04-23T14:33:51.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e55200640 1 --2- 172.21.15.100:0/1725217097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e5006f8c0 0x7f7e501493b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:33:51.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e4fe00640 1 --2- 172.21.15.100:0/1725217097 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7e500ce620 0x7f7e50072520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7e500ad1d0 tx=0x7f7e3801cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:51.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e4cc00640 1 -- 172.21.15.100:0/1725217097 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e38006e20 con 0x7f7e500ce620 2024-04-23T14:33:51.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e4cc00640 1 -- 172.21.15.100:0/1725217097 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e380a8040 con 0x7f7e500ce620 2024-04-23T14:33:51.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e4cc00640 1 -- 172.21.15.100:0/1725217097 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e380b5820 con 0x7f7e500ce620 2024-04-23T14:33:51.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.400+0000 7f7e57c00640 1 -- 172.21.15.100:0/1725217097 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e50072c20 con 0x7f7e500ce620 2024-04-23T14:33:51.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.404+0000 7f7e57c00640 1 -- 172.21.15.100:0/1725217097 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e500731e0 con 0x7f7e500ce620 2024-04-23T14:33:51.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.404+0000 7f7e57c00640 1 -- 172.21.15.100:0/1725217097 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e50068ad0 con 0x7f7e500ce620 2024-04-23T14:33:51.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.404+0000 7f7e4cc00640 1 -- 172.21.15.100:0/1725217097 <== mon.2 v2:172.21.15.151:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7e380b59c0 con 0x7f7e500ce620 2024-04-23T14:33:51.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.404+0000 7f7e4cc00640 1 --2- 172.21.15.100:0/1725217097 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7e2c068380 0x7f7e2c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:51.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.404+0000 7f7e4cc00640 1 -- 172.21.15.100:0/1725217097 <== mon.2 v2:172.21.15.151:3301/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f7e380be070 con 0x7f7e500ce620 2024-04-23T14:33:51.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.404+0000 7f7e55200640 1 --2- 172.21.15.100:0/1725217097 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7e2c068380 0x7f7e2c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:51.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.404+0000 7f7e55200640 1 --2- 172.21.15.100:0/1725217097 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7e2c068380 0x7f7e2c06a840 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7e40009580 tx=0x7f7e40009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:51.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.408+0000 7f7e4cc00640 1 -- 172.21.15.100:0/1725217097 <== mon.2 v2:172.21.15.151:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7e380182a0 con 0x7f7e500ce620 2024-04-23T14:33:51.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.564+0000 7f7e57c00640 1 -- 172.21.15.100:0/1725217097 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) -- 0x7f7e5006cc00 con 0x7f7e500ce620 2024-04-23T14:33:51.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.564+0000 7f7e4cc00640 1 -- 172.21.15.100:0/1725217097 <== mon.2 v2:172.21.15.151:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v41) ==== 97+0+11 (secure 0 0 0) 0x7f7e380efbc0 con 0x7f7e500ce620 2024-04-23T14:33:51.570 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:51.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.568+0000 7f7e57c00640 1 -- 172.21.15.100:0/1725217097 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7e2c068380 msgr2=0x7f7e2c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.568+0000 7f7e57c00640 1 --2- 172.21.15.100:0/1725217097 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7e2c068380 0x7f7e2c06a840 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7e40009580 tx=0x7f7e40009190 comp rx=0 tx=0).stop 2024-04-23T14:33:51.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.568+0000 7f7e57c00640 1 -- 172.21.15.100:0/1725217097 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7e500ce620 msgr2=0x7f7e50072520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.568+0000 7f7e57c00640 1 --2- 172.21.15.100:0/1725217097 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7e500ce620 0x7f7e50072520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7e500ad1d0 tx=0x7f7e3801cef0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.568+0000 7f7e57c00640 1 -- 172.21.15.100:0/1725217097 shutdown_connections 2024-04-23T14:33:51.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.568+0000 7f7e57c00640 1 --2- 172.21.15.100:0/1725217097 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7e2c068380 0x7f7e2c06a840 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.568+0000 7f7e57c00640 1 --2- 172.21.15.100:0/1725217097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e500d0170 0x7f7e5015e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.568+0000 7f7e57c00640 1 --2- 172.21.15.100:0/1725217097 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7e500ce620 0x7f7e50072520 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.568+0000 7f7e57c00640 1 --2- 172.21.15.100:0/1725217097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e5006f8c0 0x7f7e501493b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.568+0000 7f7e57c00640 1 -- 172.21.15.100:0/1725217097 >> 172.21.15.100:0/1725217097 conn(0x7f7e500b5200 msgr2=0x7f7e501645c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:51.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.568+0000 7f7e57c00640 1 -- 172.21.15.100:0/1725217097 shutdown_connections 2024-04-23T14:33:51.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.568+0000 7f7e57c00640 1 -- 172.21.15.100:0/1725217097 wait complete. 2024-04-23T14:33:51.591 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:33:51.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.728+0000 7fed2be00640 1 Processor -- start 2024-04-23T14:33:51.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.728+0000 7fed2be00640 1 -- start start 2024-04-23T14:33:51.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.728+0000 7fed2be00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed2c0d1a90 0x7fed2c0b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:51.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.728+0000 7fed2be00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed2c0ccbf0 con 0x7fed2c0b32b0 2024-04-23T14:33:51.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.728+0000 7fed2be00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed2c0cced0 con 0x7fed2c1609b0 2024-04-23T14:33:51.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.728+0000 7fed2be00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fed2c0b3690 con 0x7fed2c0d1a90 2024-04-23T14:33:51.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.728+0000 7fed2aa00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fed2c0b32b0 0x7fed2c15d360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35470/0 (socket says 172.21.15.100:35470) 2024-04-23T14:33:51.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.732+0000 7fed2aa00640 1 -- 172.21.15.100:0/2235347506 learned_addr learned my addr 172.21.15.100:0/2235347506 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:51.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.732+0000 7fed2b400640 1 --2- 172.21.15.100:0/2235347506 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed2c0d1a90 0x7fed2c0b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:51.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.732+0000 7fed29600640 1 -- 172.21.15.100:0/2235347506 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1956796052 0 0) 0x7fed2c0ccbf0 con 0x7fed2c0b32b0 2024-04-23T14:33:51.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.732+0000 7fed29600640 1 -- 172.21.15.100:0/2235347506 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed10003530 con 0x7fed2c0b32b0 2024-04-23T14:33:51.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.732+0000 7fed29600640 1 -- 172.21.15.100:0/2235347506 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2442262710 0 0) 0x7fed2c0cced0 con 0x7fed2c1609b0 2024-04-23T14:33:51.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.732+0000 7fed2b400640 1 -- 172.21.15.100:0/2235347506 >> v1:172.21.15.151:6789/0 conn(0x7fed2c1609b0 legacy=0x7fed2c0d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.732+0000 7fed2b400640 1 -- 172.21.15.100:0/2235347506 >> v1:172.21.15.100:6789/0 conn(0x7fed2c0b32b0 legacy=0x7fed2c15d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.732+0000 7fed2b400640 1 -- 172.21.15.100:0/2235347506 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c15fed0 con 0x7fed2c0d1a90 2024-04-23T14:33:51.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.736+0000 7fed2b400640 1 --2- 172.21.15.100:0/2235347506 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed2c0d1a90 0x7fed2c0b1bb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fed240097f0 tx=0x7fed2401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=498825d66cb90587 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:51.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.736+0000 7fed29600640 1 -- 172.21.15.100:0/2235347506 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed240b1070 con 0x7fed2c0d1a90 2024-04-23T14:33:51.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.736+0000 7fed29600640 1 -- 172.21.15.100:0/2235347506 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed240ac630 con 0x7fed2c0d1a90 2024-04-23T14:33:51.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.736+0000 7fed29600640 1 -- 172.21.15.100:0/2235347506 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed240a8050 con 0x7fed2c0d1a90 2024-04-23T14:33:51.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.736+0000 7fed2be00640 1 -- 172.21.15.100:0/2235347506 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed2c0d1a90 msgr2=0x7fed2c0b1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.736+0000 7fed2be00640 1 --2- 172.21.15.100:0/2235347506 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed2c0d1a90 0x7fed2c0b1bb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fed240097f0 tx=0x7fed2401c4e0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.736+0000 7fed2be00640 1 -- 172.21.15.100:0/2235347506 shutdown_connections 2024-04-23T14:33:51.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.736+0000 7fed2be00640 1 --2- 172.21.15.100:0/2235347506 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed2c0d1a90 0x7fed2c0b1bb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.736+0000 7fed2be00640 1 -- 172.21.15.100:0/2235347506 >> 172.21.15.100:0/2235347506 conn(0x7fed2c0d2270 msgr2=0x7fed2c0d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:51.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.736+0000 7fed2be00640 1 -- 172.21.15.100:0/2235347506 shutdown_connections 2024-04-23T14:33:51.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.736+0000 7fed2be00640 1 -- 172.21.15.100:0/2235347506 wait complete. 2024-04-23T14:33:51.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.736+0000 7fed2be00640 1 Processor -- start 2024-04-23T14:33:51.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.736+0000 7fed2be00640 1 -- start start 2024-04-23T14:33:51.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed2be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed2c0b32b0 0x7fed2c150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:51.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed2be00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed2c0d1a90 0x7fed2c1589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:51.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed2be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed2c1609b0 0x7fed2c155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:51.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed2be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c0ce970 con 0x7fed2c0b32b0 2024-04-23T14:33:51.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed2be00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c0ce7f0 con 0x7fed2c1609b0 2024-04-23T14:33:51.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed2be00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fed2c0b70c0 con 0x7fed2c0d1a90 2024-04-23T14:33:51.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed2aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed2c0b32b0 0x7fed2c150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:51.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed2aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed2c0b32b0 0x7fed2c150100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36938/0 (socket says 172.21.15.100:36938) 2024-04-23T14:33:51.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed2aa00640 1 -- 172.21.15.100:0/4150958650 learned_addr learned my addr 172.21.15.100:0/4150958650 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:51.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed2a000640 1 --2- 172.21.15.100:0/4150958650 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed2c0d1a90 0x7fed2c1589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:51.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed2b400640 1 --2- 172.21.15.100:0/4150958650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed2c1609b0 0x7fed2c155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:51.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed2aa00640 1 -- 172.21.15.100:0/4150958650 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed2c0d1a90 msgr2=0x7fed2c1589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed2aa00640 1 --2- 172.21.15.100:0/4150958650 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed2c0d1a90 0x7fed2c1589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed2aa00640 1 -- 172.21.15.100:0/4150958650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed2c1609b0 msgr2=0x7fed2c155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed2aa00640 1 --2- 172.21.15.100:0/4150958650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed2c1609b0 0x7fed2c155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed2aa00640 1 -- 172.21.15.100:0/4150958650 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c157c40 con 0x7fed2c0b32b0 2024-04-23T14:33:51.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed2aa00640 1 --2- 172.21.15.100:0/4150958650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed2c0b32b0 0x7fed2c150100 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00cb70 tx=0x7fed1c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:51.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed22a00640 1 -- 172.21.15.100:0/4150958650 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c017070 con 0x7fed2c0b32b0 2024-04-23T14:33:51.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed22a00640 1 -- 172.21.15.100:0/4150958650 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed1c00ad40 con 0x7fed2c0b32b0 2024-04-23T14:33:51.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed22a00640 1 -- 172.21.15.100:0/4150958650 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c012620 con 0x7fed2c0b32b0 2024-04-23T14:33:51.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed2be00640 1 -- 172.21.15.100:0/4150958650 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2c157f00 con 0x7fed2c0b32b0 2024-04-23T14:33:51.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed2be00640 1 -- 172.21.15.100:0/4150958650 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2c150640 con 0x7fed2c0b32b0 2024-04-23T14:33:51.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed2be00640 1 -- 172.21.15.100:0/4150958650 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed2c075c20 con 0x7fed2c0b32b0 2024-04-23T14:33:51.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.740+0000 7fed22a00640 1 -- 172.21.15.100:0/4150958650 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fed1c01a070 con 0x7fed2c0b32b0 2024-04-23T14:33:51.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.744+0000 7fed22a00640 1 --2- 172.21.15.100:0/4150958650 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fed0c068380 0x7fed0c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:51.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.744+0000 7fed22a00640 1 -- 172.21.15.100:0/4150958650 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fed1c086a90 con 0x7fed2c0b32b0 2024-04-23T14:33:51.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.744+0000 7fed2a000640 1 --2- 172.21.15.100:0/4150958650 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fed0c068380 0x7fed0c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:51.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.744+0000 7fed2a000640 1 --2- 172.21.15.100:0/4150958650 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fed0c068380 0x7fed0c06a840 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fed2c1fa760 tx=0x7fed18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:51.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.748+0000 7fed22a00640 1 -- 172.21.15.100:0/4150958650 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fed1c056670 con 0x7fed2c0b32b0 2024-04-23T14:33:51.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.932+0000 7fed2be00640 1 -- 172.21.15.100:0/4150958650 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fed2c0b3ad0 con 0x7fed2c0b32b0 2024-04-23T14:33:51.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.932+0000 7fed22a00640 1 -- 172.21.15.100:0/4150958650 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) ==== 80+0+0 (secure 0 0 0) 0x7fed1c05a970 con 0x7fed2c0b32b0 2024-04-23T14:33:51.937 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-04-23T14:33:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.936+0000 7fed2be00640 1 -- 172.21.15.100:0/4150958650 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fed0c068380 msgr2=0x7fed0c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.936+0000 7fed2be00640 1 --2- 172.21.15.100:0/4150958650 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fed0c068380 0x7fed0c06a840 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fed2c1fa760 tx=0x7fed18009040 comp rx=0 tx=0).stop 2024-04-23T14:33:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.936+0000 7fed2be00640 1 -- 172.21.15.100:0/4150958650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed2c0b32b0 msgr2=0x7fed2c150100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.936+0000 7fed2be00640 1 --2- 172.21.15.100:0/4150958650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed2c0b32b0 0x7fed2c150100 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00cb70 tx=0x7fed1c00a430 comp rx=0 tx=0).stop 2024-04-23T14:33:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.936+0000 7fed2be00640 1 -- 172.21.15.100:0/4150958650 shutdown_connections 2024-04-23T14:33:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.936+0000 7fed2be00640 1 --2- 172.21.15.100:0/4150958650 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fed0c068380 0x7fed0c06a840 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.936+0000 7fed2be00640 1 --2- 172.21.15.100:0/4150958650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed2c1609b0 0x7fed2c155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.936+0000 7fed2be00640 1 --2- 172.21.15.100:0/4150958650 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fed2c0d1a90 0x7fed2c1589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.936+0000 7fed2be00640 1 --2- 172.21.15.100:0/4150958650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed2c0b32b0 0x7fed2c150100 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.936+0000 7fed2be00640 1 -- 172.21.15.100:0/4150958650 >> 172.21.15.100:0/4150958650 conn(0x7fed2c0d2270 msgr2=0x7fed2c0c8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.936+0000 7fed2be00640 1 -- 172.21.15.100:0/4150958650 shutdown_connections 2024-04-23T14:33:51.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:51.936+0000 7fed2be00640 1 -- 172.21.15.100:0/4150958650 wait complete. 2024-04-23T14:33:51.959 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:33:51.959 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:51.968 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:52.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.140+0000 7f470a600640 1 Processor -- start 2024-04-23T14:33:52.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.140+0000 7f470a600640 1 -- start start 2024-04-23T14:33:52.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.140+0000 7f470a600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4704072040 0x7f47040724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.140+0000 7f470a600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4704162950 con 0x7f470415e850 2024-04-23T14:33:52.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.140+0000 7f470a600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f470406f760 con 0x7f4704073010 2024-04-23T14:33:52.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.140+0000 7f470a600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f470406df70 con 0x7f4704072040 2024-04-23T14:33:52.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.140+0000 7f4709200640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f470415e850 0x7f470406cdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35474/0 (socket says 172.21.15.100:35474) 2024-04-23T14:33:52.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.140+0000 7f4709200640 1 -- 172.21.15.100:0/1105755712 learned_addr learned my addr 172.21.15.100:0/1105755712 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:52.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.140+0000 7f4709c00640 1 --2- 172.21.15.100:0/1105755712 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4704072040 0x7f47040724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.140+0000 7f4703e00640 1 -- 172.21.15.100:0/1105755712 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 654606034 0 0) 0x7f4704162950 con 0x7f470415e850 2024-04-23T14:33:52.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.140+0000 7f4703e00640 1 -- 172.21.15.100:0/1105755712 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46e8003530 con 0x7f470415e850 2024-04-23T14:33:52.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.140+0000 7f4703e00640 1 -- 172.21.15.100:0/1105755712 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 870306029 0 0) 0x7f470406f760 con 0x7f4704073010 2024-04-23T14:33:52.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.140+0000 7f4703e00640 1 -- 172.21.15.100:0/1105755712 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4704162950 con 0x7f4704073010 2024-04-23T14:33:52.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.140+0000 7f4709c00640 1 -- 172.21.15.100:0/1105755712 >> v1:172.21.15.151:6789/0 conn(0x7f4704073010 legacy=0x7f4704162240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.144+0000 7f4709c00640 1 -- 172.21.15.100:0/1105755712 >> v1:172.21.15.100:6789/0 conn(0x7f470415e850 legacy=0x7f470406cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.144+0000 7f4709c00640 1 -- 172.21.15.100:0/1105755712 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470406fa40 con 0x7f4704072040 2024-04-23T14:33:52.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.144+0000 7f4709c00640 1 --2- 172.21.15.100:0/1105755712 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4704072040 0x7f47040724d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009dc0 tx=0x7f46fc01c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6d39d6102b6d95a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:52.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.144+0000 7f4703e00640 1 -- 172.21.15.100:0/1105755712 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc0a66c0 con 0x7f4704072040 2024-04-23T14:33:52.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.144+0000 7f4703e00640 1 -- 172.21.15.100:0/1105755712 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46fc0a6860 con 0x7f4704072040 2024-04-23T14:33:52.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.144+0000 7f4703e00640 1 -- 172.21.15.100:0/1105755712 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc0a6c30 con 0x7f4704072040 2024-04-23T14:33:52.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.144+0000 7f470a600640 1 -- 172.21.15.100:0/1105755712 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4704072040 msgr2=0x7f47040724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.144+0000 7f470a600640 1 --2- 172.21.15.100:0/1105755712 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4704072040 0x7f47040724d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009dc0 tx=0x7f46fc01c900 comp rx=0 tx=0).stop 2024-04-23T14:33:52.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.144+0000 7f470a600640 1 -- 172.21.15.100:0/1105755712 shutdown_connections 2024-04-23T14:33:52.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.144+0000 7f470a600640 1 --2- 172.21.15.100:0/1105755712 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4704072040 0x7f47040724d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.144+0000 7f470a600640 1 -- 172.21.15.100:0/1105755712 >> 172.21.15.100:0/1105755712 conn(0x7f47040d0580 msgr2=0x7f47040d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:52.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f470a600640 1 -- 172.21.15.100:0/1105755712 shutdown_connections 2024-04-23T14:33:52.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f470a600640 1 -- 172.21.15.100:0/1105755712 wait complete. 2024-04-23T14:33:52.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f470a600640 1 Processor -- start 2024-04-23T14:33:52.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f470a600640 1 -- start start 2024-04-23T14:33:52.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f470a600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4704072040 0x7f47041565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f470a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4704073010 0x7f470415b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f470a600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f470415e850 0x7f470415a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f470a600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f470416c960 con 0x7f4704073010 2024-04-23T14:33:52.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f470a600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f470416c7e0 con 0x7f4704072040 2024-04-23T14:33:52.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f470a600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f470416cae0 con 0x7f470415e850 2024-04-23T14:33:52.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f46fbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4704073010 0x7f470415b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f46fbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4704073010 0x7f470415b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36940/0 (socket says 172.21.15.100:36940) 2024-04-23T14:33:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f46fbe00640 1 -- 172.21.15.100:0/3748478051 learned_addr learned my addr 172.21.15.100:0/3748478051 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f4709200640 1 --2- 172.21.15.100:0/3748478051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4704072040 0x7f47041565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f4709c00640 1 --2- 172.21.15.100:0/3748478051 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f470415e850 0x7f470415a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f46fbe00640 1 -- 172.21.15.100:0/3748478051 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f470415e850 msgr2=0x7f470415a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f46fbe00640 1 --2- 172.21.15.100:0/3748478051 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f470415e850 0x7f470415a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f46fbe00640 1 -- 172.21.15.100:0/3748478051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4704072040 msgr2=0x7f47041565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f46fbe00640 1 --2- 172.21.15.100:0/3748478051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4704072040 0x7f47041565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f46fbe00640 1 -- 172.21.15.100:0/3748478051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470415a850 con 0x7f4704073010 2024-04-23T14:33:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f4709c00640 1 --2- 172.21.15.100:0/3748478051 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f470415e850 0x7f470415a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f46fbe00640 1 --2- 172.21.15.100:0/3748478051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4704073010 0x7f470415b1e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00c500 tx=0x7f46ec00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.148+0000 7f4701600640 1 -- 172.21.15.100:0/3748478051 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec0052d0 con 0x7f4704073010 2024-04-23T14:33:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.152+0000 7f4701600640 1 -- 172.21.15.100:0/3748478051 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ec005490 con 0x7f4704073010 2024-04-23T14:33:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.152+0000 7f470a600640 1 -- 172.21.15.100:0/3748478051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4704155800 con 0x7f4704073010 2024-04-23T14:33:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.152+0000 7f4701600640 1 -- 172.21.15.100:0/3748478051 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec012850 con 0x7f4704073010 2024-04-23T14:33:52.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.152+0000 7f470a600640 1 -- 172.21.15.100:0/3748478051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4704155dc0 con 0x7f4704073010 2024-04-23T14:33:52.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.152+0000 7f470a600640 1 -- 172.21.15.100:0/3748478051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4704075c20 con 0x7f4704073010 2024-04-23T14:33:52.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.152+0000 7f4701600640 1 -- 172.21.15.100:0/3748478051 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f46ec0129f0 con 0x7f4704073010 2024-04-23T14:33:52.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.152+0000 7f4701600640 1 --2- 172.21.15.100:0/3748478051 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f46e0068380 0x7f46e006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.152+0000 7f4701600640 1 -- 172.21.15.100:0/3748478051 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f46ec086960 con 0x7f4704073010 2024-04-23T14:33:52.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.152+0000 7f4709200640 1 --2- 172.21.15.100:0/3748478051 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f46e0068380 0x7f46e006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.152+0000 7f4709200640 1 --2- 172.21.15.100:0/3748478051 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f46e0068380 0x7f46e006a840 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f46f40011f0 tx=0x7f46f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:52.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.156+0000 7f4701600640 1 -- 172.21.15.100:0/3748478051 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f46ec056c20 con 0x7f4704073010 2024-04-23T14:33:52.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.316+0000 7f470a600640 1 -- 172.21.15.100:0/3748478051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4704169fe0 con 0x7f4704073010 2024-04-23T14:33:52.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.316+0000 7f4701600640 1 -- 172.21.15.100:0/3748478051 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+18766 (secure 0 0 0) 0x7f46ec056520 con 0x7f4704073010 2024-04-23T14:33:52.321 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:52.321 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":41,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:33:49.609841+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:33:36.592437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-23T14:33:37.882390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-04-23T14:33:47.299378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:52.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.316+0000 7f470a600640 1 -- 172.21.15.100:0/3748478051 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f46e0068380 msgr2=0x7f46e006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.316+0000 7f470a600640 1 --2- 172.21.15.100:0/3748478051 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f46e0068380 0x7f46e006a840 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f46f40011f0 tx=0x7f46f4009040 comp rx=0 tx=0).stop 2024-04-23T14:33:52.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.316+0000 7f470a600640 1 -- 172.21.15.100:0/3748478051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4704073010 msgr2=0x7f470415b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.316+0000 7f470a600640 1 --2- 172.21.15.100:0/3748478051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4704073010 0x7f470415b1e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00c500 tx=0x7f46ec00c9c0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.320+0000 7f470a600640 1 -- 172.21.15.100:0/3748478051 shutdown_connections 2024-04-23T14:33:52.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.320+0000 7f470a600640 1 --2- 172.21.15.100:0/3748478051 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f46e0068380 0x7f46e006a840 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.320+0000 7f470a600640 1 --2- 172.21.15.100:0/3748478051 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f470415e850 0x7f470415a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.320+0000 7f470a600640 1 --2- 172.21.15.100:0/3748478051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4704073010 0x7f470415b1e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.320+0000 7f470a600640 1 --2- 172.21.15.100:0/3748478051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4704072040 0x7f47041565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.320+0000 7f470a600640 1 -- 172.21.15.100:0/3748478051 >> 172.21.15.100:0/3748478051 conn(0x7f47040d0580 msgr2=0x7f47040d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:52.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.320+0000 7f470a600640 1 -- 172.21.15.100:0/3748478051 shutdown_connections 2024-04-23T14:33:52.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.320+0000 7f470a600640 1 -- 172.21.15.100:0/3748478051 wait complete. 2024-04-23T14:33:52.343 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:33:36.592437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:33:37.882390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-04-23T14:33:47.299378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:33:52.344 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:33:52.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.476+0000 7f1e9c200640 1 Processor -- start 2024-04-23T14:33:52.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.476+0000 7f1e9c200640 1 -- start start 2024-04-23T14:33:52.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.476+0000 7f1e9c200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1e9406f8c0 0x7f1e9406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.476+0000 7f1e9c200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e940ce7f0 0x7f1e940add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.476+0000 7f1e9c200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e9406dd60 con 0x7f1e940d0170 2024-04-23T14:33:52.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.476+0000 7f1e9c200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f1e940ad050 con 0x7f1e9406f8c0 2024-04-23T14:33:52.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.476+0000 7f1e9c200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1e940ad8f0 con 0x7f1e940ce7f0 2024-04-23T14:33:52.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.476+0000 7f1e99800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1e9406f8c0 0x7f1e9406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.476+0000 7f1e99800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1e9406f8c0 0x7f1e9406c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36948/0 (socket says 172.21.15.100:36948) 2024-04-23T14:33:52.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.476+0000 7f1e99800640 1 -- 172.21.15.100:0/1470185605 learned_addr learned my addr 172.21.15.100:0/1470185605 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:52.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.476+0000 7f1e98e00640 1 --1- 172.21.15.100:0/1470185605 >> v1:172.21.15.100:6789/0 conn(0x7f1e940d0170 0x7f1e940ce0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35478/0 (socket says 172.21.15.100:35478) 2024-04-23T14:33:52.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.476+0000 7f1e9a200640 1 --2- 172.21.15.100:0/1470185605 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e940ce7f0 0x7f1e940add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.476+0000 7f1e99800640 1 -- 172.21.15.100:0/1470185605 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e940ce7f0 msgr2=0x7f1e940add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:52.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.476+0000 7f1e99800640 1 --2- 172.21.15.100:0/1470185605 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e940ce7f0 0x7f1e940add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.476+0000 7f1e99800640 1 -- 172.21.15.100:0/1470185605 >> v1:172.21.15.100:6789/0 conn(0x7f1e940d0170 legacy=0x7f1e940ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.476+0000 7f1e99800640 1 -- 172.21.15.100:0/1470185605 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e940d09c0 con 0x7f1e9406f8c0 2024-04-23T14:33:52.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.476+0000 7f1e9a200640 1 --2- 172.21.15.100:0/1470185605 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e940ce7f0 0x7f1e940add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:52.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.480+0000 7f1e99800640 1 --2- 172.21.15.100:0/1470185605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e9406f8c0 0x7f1e9406c6c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009870 tx=0x7f1e8801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be31f188890bbce3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:52.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.480+0000 7f1e93e00640 1 -- 172.21.15.100:0/1470185605 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e880a9620 con 0x7f1e9406f8c0 2024-04-23T14:33:52.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.480+0000 7f1e93e00640 1 -- 172.21.15.100:0/1470185605 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e880a97c0 con 0x7f1e9406f8c0 2024-04-23T14:33:52.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.480+0000 7f1e93e00640 1 -- 172.21.15.100:0/1470185605 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e880a9b90 con 0x7f1e9406f8c0 2024-04-23T14:33:52.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.480+0000 7f1e9c200640 1 -- 172.21.15.100:0/1470185605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e9406f8c0 msgr2=0x7f1e9406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.480+0000 7f1e9c200640 1 --2- 172.21.15.100:0/1470185605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e9406f8c0 0x7f1e9406c6c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009870 tx=0x7f1e8801c560 comp rx=0 tx=0).stop 2024-04-23T14:33:52.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.480+0000 7f1e9c200640 1 -- 172.21.15.100:0/1470185605 shutdown_connections 2024-04-23T14:33:52.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.480+0000 7f1e9c200640 1 --2- 172.21.15.100:0/1470185605 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e940ce7f0 0x7f1e940add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.480+0000 7f1e9c200640 1 --2- 172.21.15.100:0/1470185605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e9406f8c0 0x7f1e9406c6c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.480+0000 7f1e9c200640 1 -- 172.21.15.100:0/1470185605 >> 172.21.15.100:0/1470185605 conn(0x7f1e940b5200 msgr2=0x7f1e940b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:52.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.480+0000 7f1e9c200640 1 -- 172.21.15.100:0/1470185605 shutdown_connections 2024-04-23T14:33:52.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.480+0000 7f1e9c200640 1 -- 172.21.15.100:0/1470185605 wait complete. 2024-04-23T14:33:52.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.484+0000 7f1e9c200640 1 Processor -- start 2024-04-23T14:33:52.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.484+0000 7f1e9c200640 1 -- start start 2024-04-23T14:33:52.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.484+0000 7f1e9c200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e9406f8c0 0x7f1e94155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.484+0000 7f1e9c200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e940ce7f0 0x7f1e94165c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.484+0000 7f1e9c200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e940d0170 0x7f1e9414f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.484+0000 7f1e9c200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1e940aead0 con 0x7f1e9406f8c0 2024-04-23T14:33:52.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.484+0000 7f1e9c200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e940d0760 con 0x7f1e940ce7f0 2024-04-23T14:33:52.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.484+0000 7f1e9c200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1e940aec50 con 0x7f1e940d0170 2024-04-23T14:33:52.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.484+0000 7f1e99800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e9406f8c0 0x7f1e94155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.484+0000 7f1e99800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e9406f8c0 0x7f1e94155960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36960/0 (socket says 172.21.15.100:36960) 2024-04-23T14:33:52.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.484+0000 7f1e99800640 1 -- 172.21.15.100:0/421919617 learned_addr learned my addr 172.21.15.100:0/421919617 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:52.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.484+0000 7f1e99800640 1 -- 172.21.15.100:0/421919617 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e940d0170 msgr2=0x7f1e9414f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.484+0000 7f1e99800640 1 --2- 172.21.15.100:0/421919617 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e940d0170 0x7f1e9414f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.484+0000 7f1e99800640 1 -- 172.21.15.100:0/421919617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e940ce7f0 msgr2=0x7f1e94165c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.484+0000 7f1e99800640 1 --2- 172.21.15.100:0/421919617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e940ce7f0 0x7f1e94165c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.484+0000 7f1e99800640 1 -- 172.21.15.100:0/421919617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e94166170 con 0x7f1e9406f8c0 2024-04-23T14:33:52.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.484+0000 7f1e99800640 1 --2- 172.21.15.100:0/421919617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e9406f8c0 0x7f1e94155960 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1e880ab040 tx=0x7f1e8801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:52.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.484+0000 7f1e91600640 1 -- 172.21.15.100:0/421919617 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e880bc650 con 0x7f1e9406f8c0 2024-04-23T14:33:52.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.484+0000 7f1e91600640 1 -- 172.21.15.100:0/421919617 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e880bc830 con 0x7f1e9406f8c0 2024-04-23T14:33:52.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.484+0000 7f1e9c200640 1 -- 172.21.15.100:0/421919617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e9414f9b0 con 0x7f1e9406f8c0 2024-04-23T14:33:52.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.484+0000 7f1e91600640 1 -- 172.21.15.100:0/421919617 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e880bcc20 con 0x7f1e9406f8c0 2024-04-23T14:33:52.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.484+0000 7f1e9c200640 1 -- 172.21.15.100:0/421919617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e9414e990 con 0x7f1e9406f8c0 2024-04-23T14:33:52.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.488+0000 7f1e9c200640 1 -- 172.21.15.100:0/421919617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e94068ad0 con 0x7f1e9406f8c0 2024-04-23T14:33:52.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.488+0000 7f1e91600640 1 -- 172.21.15.100:0/421919617 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1e880c53f0 con 0x7f1e9406f8c0 2024-04-23T14:33:52.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.488+0000 7f1e91600640 1 --2- 172.21.15.100:0/421919617 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e74068380 0x7f1e7406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.488+0000 7f1e91600640 1 -- 172.21.15.100:0/421919617 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f1e881214d0 con 0x7f1e9406f8c0 2024-04-23T14:33:52.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.488+0000 7f1e98e00640 1 --2- 172.21.15.100:0/421919617 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e74068380 0x7f1e7406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.488+0000 7f1e98e00640 1 --2- 172.21.15.100:0/421919617 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e74068380 0x7f1e7406a840 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1e80000f70 tx=0x7f1e80007a10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:52.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.492+0000 7f1e91600640 1 -- 172.21.15.100:0/421919617 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1e94166170 con 0x7f1e9406f8c0 2024-04-23T14:33:52.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.648+0000 7f1e9c200640 1 -- 172.21.15.100:0/421919617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1e9406e960 con 0x7f1e9406f8c0 2024-04-23T14:33:52.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.652+0000 7f1e91600640 1 -- 172.21.15.100:0/421919617 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7f1e880ad020 con 0x7f1e9406f8c0 2024-04-23T14:33:52.655 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:33:52.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.652+0000 7f1e9c200640 1 -- 172.21.15.100:0/421919617 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e74068380 msgr2=0x7f1e7406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.652+0000 7f1e9c200640 1 --2- 172.21.15.100:0/421919617 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e74068380 0x7f1e7406a840 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1e80000f70 tx=0x7f1e80007a10 comp rx=0 tx=0).stop 2024-04-23T14:33:52.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.652+0000 7f1e9c200640 1 -- 172.21.15.100:0/421919617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e9406f8c0 msgr2=0x7f1e94155960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.652+0000 7f1e9c200640 1 --2- 172.21.15.100:0/421919617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e9406f8c0 0x7f1e94155960 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1e880ab040 tx=0x7f1e8801ada0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.652+0000 7f1e9c200640 1 -- 172.21.15.100:0/421919617 shutdown_connections 2024-04-23T14:33:52.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.652+0000 7f1e9c200640 1 --2- 172.21.15.100:0/421919617 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e74068380 0x7f1e7406a840 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.652+0000 7f1e9c200640 1 --2- 172.21.15.100:0/421919617 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e940d0170 0x7f1e9414f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.652+0000 7f1e9c200640 1 --2- 172.21.15.100:0/421919617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e940ce7f0 0x7f1e94165c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.652+0000 7f1e9c200640 1 --2- 172.21.15.100:0/421919617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e9406f8c0 0x7f1e94155960 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.652+0000 7f1e9c200640 1 -- 172.21.15.100:0/421919617 >> 172.21.15.100:0/421919617 conn(0x7f1e940b5200 msgr2=0x7f1e9406bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:52.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.652+0000 7f1e9c200640 1 -- 172.21.15.100:0/421919617 shutdown_connections 2024-04-23T14:33:52.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.652+0000 7f1e9c200640 1 -- 172.21.15.100:0/421919617 wait complete. 2024-04-23T14:33:52.677 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:33:52.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.812+0000 7f121e000640 1 Processor -- start 2024-04-23T14:33:52.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.812+0000 7f121e000640 1 -- start start 2024-04-23T14:33:52.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.812+0000 7f121e000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f121815ae60 0x7f12180cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.812+0000 7f121e000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12180d0a70 0x7f12180ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.812+0000 7f121e000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12180cfc70 con 0x7f121806c9a0 2024-04-23T14:33:52.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.812+0000 7f121e000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f121815b240 con 0x7f121815ae60 2024-04-23T14:33:52.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.812+0000 7f121e000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f12180cff10 con 0x7f12180d0a70 2024-04-23T14:33:52.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.812+0000 7f1217400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f121815ae60 0x7f12180cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.812+0000 7f1217400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f121815ae60 0x7f12180cf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36974/0 (socket says 172.21.15.100:36974) 2024-04-23T14:33:52.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.812+0000 7f1217400640 1 -- 172.21.15.100:0/1280272577 learned_addr learned my addr 172.21.15.100:0/1280272577 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:52.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.812+0000 7f1216a00640 1 --2- 172.21.15.100:0/1280272577 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12180d0a70 0x7f12180ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.812+0000 7f1217400640 1 -- 172.21.15.100:0/1280272577 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12180d0a70 msgr2=0x7f12180ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:52.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.812+0000 7f1217400640 1 --2- 172.21.15.100:0/1280272577 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12180d0a70 0x7f12180ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.812+0000 7f1217400640 1 -- 172.21.15.100:0/1280272577 >> v1:172.21.15.151:6789/0 conn(0x7f121806c9a0 legacy=0x7f121806ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:52.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.812+0000 7f1217400640 1 -- 172.21.15.100:0/1280272577 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121806bbe0 con 0x7f121815ae60 2024-04-23T14:33:52.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.816+0000 7f1217400640 1 --2- 172.21.15.100:0/1280272577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f121815ae60 0x7f12180cf730 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1208009870 tx=0x7f120801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b59fb56b3bbd97 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:52.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.816+0000 7f1216000640 1 -- 172.21.15.100:0/1280272577 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12080aa600 con 0x7f121815ae60 2024-04-23T14:33:52.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.816+0000 7f1216000640 1 -- 172.21.15.100:0/1280272577 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12080af070 con 0x7f121815ae60 2024-04-23T14:33:52.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.816+0000 7f1216000640 1 -- 172.21.15.100:0/1280272577 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12080aa9c0 con 0x7f121815ae60 2024-04-23T14:33:52.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.816+0000 7f121e000640 1 -- 172.21.15.100:0/1280272577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f121815ae60 msgr2=0x7f12180cf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.816+0000 7f121e000640 1 --2- 172.21.15.100:0/1280272577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f121815ae60 0x7f12180cf730 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1208009870 tx=0x7f120801c4a0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.816+0000 7f121e000640 1 -- 172.21.15.100:0/1280272577 shutdown_connections 2024-04-23T14:33:52.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.816+0000 7f121e000640 1 --2- 172.21.15.100:0/1280272577 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12180d0a70 0x7f12180ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.816+0000 7f121e000640 1 --2- 172.21.15.100:0/1280272577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f121815ae60 0x7f12180cf730 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.816+0000 7f121e000640 1 -- 172.21.15.100:0/1280272577 >> 172.21.15.100:0/1280272577 conn(0x7f12180b5200 msgr2=0x7f12180b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:52.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.816+0000 7f121e000640 1 -- 172.21.15.100:0/1280272577 shutdown_connections 2024-04-23T14:33:52.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.816+0000 7f121e000640 1 -- 172.21.15.100:0/1280272577 wait complete. 2024-04-23T14:33:52.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.816+0000 7f121e000640 1 Processor -- start 2024-04-23T14:33:52.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f121e000640 1 -- start start 2024-04-23T14:33:52.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f121e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f121806c9a0 0x7f121815a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f121e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12180d0a70 0x7f12180c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f121e000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f121815ae60 0x7f12180c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f121e000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f12180caef0 con 0x7f12180d0a70 2024-04-23T14:33:52.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f121e000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f12180cad70 con 0x7f121806c9a0 2024-04-23T14:33:52.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f121e000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f12180cd230 con 0x7f121815ae60 2024-04-23T14:33:52.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f1216a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12180d0a70 0x7f12180c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f1216a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12180d0a70 0x7f12180c7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36984/0 (socket says 172.21.15.100:36984) 2024-04-23T14:33:52.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f1216a00640 1 -- 172.21.15.100:0/994448087 learned_addr learned my addr 172.21.15.100:0/994448087 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:52.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f1217400640 1 --2- 172.21.15.100:0/994448087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f121806c9a0 0x7f121815a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f1217e00640 1 --2- 172.21.15.100:0/994448087 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f121815ae60 0x7f12180c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f1216a00640 1 -- 172.21.15.100:0/994448087 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f121815ae60 msgr2=0x7f12180c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f1216a00640 1 --2- 172.21.15.100:0/994448087 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f121815ae60 0x7f12180c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f1216a00640 1 -- 172.21.15.100:0/994448087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f121806c9a0 msgr2=0x7f121815a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f1216a00640 1 --2- 172.21.15.100:0/994448087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f121806c9a0 0x7f121815a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f1216a00640 1 -- 172.21.15.100:0/994448087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1218164a50 con 0x7f12180d0a70 2024-04-23T14:33:52.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f1217e00640 1 --2- 172.21.15.100:0/994448087 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f121815ae60 0x7f12180c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:52.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f1216a00640 1 --2- 172.21.15.100:0/994448087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12180d0a70 0x7f12180c7ca0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f120400d800 tx=0x7f120400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:52.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f120f400640 1 -- 172.21.15.100:0/994448087 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120400b6c0 con 0x7f12180d0a70 2024-04-23T14:33:52.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f120f400640 1 -- 172.21.15.100:0/994448087 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1204019070 con 0x7f12180d0a70 2024-04-23T14:33:52.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f121e000640 1 -- 172.21.15.100:0/994448087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f121815c750 con 0x7f12180d0a70 2024-04-23T14:33:52.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f120f400640 1 -- 172.21.15.100:0/994448087 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120400ba60 con 0x7f12180d0a70 2024-04-23T14:33:52.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f121e000640 1 -- 172.21.15.100:0/994448087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12181f7b20 con 0x7f12180d0a70 2024-04-23T14:33:52.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f121e000640 1 -- 172.21.15.100:0/994448087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1218068ad0 con 0x7f12180d0a70 2024-04-23T14:33:52.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.820+0000 7f120f400640 1 -- 172.21.15.100:0/994448087 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f120400bc00 con 0x7f12180d0a70 2024-04-23T14:33:52.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.824+0000 7f120f400640 1 --2- 172.21.15.100:0/994448087 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f11f8068380 0x7f11f806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.824+0000 7f120f400640 1 -- 172.21.15.100:0/994448087 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f1204087130 con 0x7f12180d0a70 2024-04-23T14:33:52.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.824+0000 7f1217400640 1 --2- 172.21.15.100:0/994448087 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f11f8068380 0x7f11f806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.824+0000 7f1217400640 1 --2- 172.21.15.100:0/994448087 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f11f8068380 0x7f11f806a840 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f12080ae000 tx=0x7f12080a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:52.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.828+0000 7f120f400640 1 -- 172.21.15.100:0/994448087 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1204057360 con 0x7f12180d0a70 2024-04-23T14:33:52.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.984+0000 7f121e000640 1 -- 172.21.15.100:0/994448087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f12180b1770 con 0x7f12180d0a70 2024-04-23T14:33:52.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.984+0000 7f120f400640 1 -- 172.21.15.100:0/994448087 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) ==== 104+0+11 (secure 0 0 0) 0x7f1204056c60 con 0x7f12180d0a70 2024-04-23T14:33:52.988 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:52.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.984+0000 7f121e000640 1 -- 172.21.15.100:0/994448087 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f11f8068380 msgr2=0x7f11f806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.984+0000 7f121e000640 1 --2- 172.21.15.100:0/994448087 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f11f8068380 0x7f11f806a840 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f12080ae000 tx=0x7f12080a7000 comp rx=0 tx=0).stop 2024-04-23T14:33:52.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.984+0000 7f121e000640 1 -- 172.21.15.100:0/994448087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12180d0a70 msgr2=0x7f12180c7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.984+0000 7f121e000640 1 --2- 172.21.15.100:0/994448087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12180d0a70 0x7f12180c7ca0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f120400d800 tx=0x7f120400dcc0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.984+0000 7f121e000640 1 -- 172.21.15.100:0/994448087 shutdown_connections 2024-04-23T14:33:52.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.984+0000 7f121e000640 1 --2- 172.21.15.100:0/994448087 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f11f8068380 0x7f11f806a840 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.984+0000 7f121e000640 1 --2- 172.21.15.100:0/994448087 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f121815ae60 0x7f12180c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.984+0000 7f121e000640 1 --2- 172.21.15.100:0/994448087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12180d0a70 0x7f12180c7ca0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.984+0000 7f121e000640 1 --2- 172.21.15.100:0/994448087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f121806c9a0 0x7f121815a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.984+0000 7f121e000640 1 -- 172.21.15.100:0/994448087 >> 172.21.15.100:0/994448087 conn(0x7f12180b5200 msgr2=0x7f1218071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:52.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.984+0000 7f121e000640 1 -- 172.21.15.100:0/994448087 shutdown_connections 2024-04-23T14:33:52.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:52.984+0000 7f121e000640 1 -- 172.21.15.100:0/994448087 wait complete. 2024-04-23T14:33:53.010 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:33:53.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.144+0000 7faa4cc00640 1 Processor -- start 2024-04-23T14:33:53.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.144+0000 7faa4cc00640 1 -- start start 2024-04-23T14:33:53.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.144+0000 7faa4cc00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7faa4815c7a0 0x7faa480af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.144+0000 7faa4cc00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa4806bb20 0x7faa480ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.144+0000 7faa4cc00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa4806cbb0 con 0x7faa480cd130 2024-04-23T14:33:53.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.144+0000 7faa4cc00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7faa480cd5a0 con 0x7faa4815c7a0 2024-04-23T14:33:53.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.144+0000 7faa4cc00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7faa4806d450 con 0x7faa4806bb20 2024-04-23T14:33:53.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.144+0000 7faa45400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa4806bb20 0x7faa480ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.144+0000 7faa45e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7faa4815c7a0 0x7faa480af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.144+0000 7faa46800640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7faa480cd130 0x7faa4806e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:42062/0 (socket says 172.21.15.100:42062) 2024-04-23T14:33:53.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.144+0000 7faa45e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7faa4815c7a0 0x7faa480af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:48506/0 (socket says 172.21.15.100:48506) 2024-04-23T14:33:53.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.144+0000 7faa46800640 1 -- 172.21.15.100:0/1551890478 learned_addr learned my addr 172.21.15.100:0/1551890478 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:53.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.144+0000 7faa45400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa4806bb20 0x7faa480ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:36014/0 (socket says 172.21.15.100:36014) 2024-04-23T14:33:53.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.144+0000 7faa45e00640 1 -- 172.21.15.100:0/1551890478 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa4806bb20 msgr2=0x7faa480ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:53.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.144+0000 7faa45e00640 1 --2- 172.21.15.100:0/1551890478 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa4806bb20 0x7faa480ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.144+0000 7faa45e00640 1 -- 172.21.15.100:0/1551890478 >> v1:172.21.15.151:6789/0 conn(0x7faa480cd130 legacy=0x7faa4806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.144+0000 7faa45e00640 1 -- 172.21.15.100:0/1551890478 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4806ce90 con 0x7faa4815c7a0 2024-04-23T14:33:53.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.144+0000 7faa45400640 1 --2- 172.21.15.100:0/1551890478 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa4806bb20 0x7faa480ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:33:53.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.148+0000 7faa45e00640 1 --2- 172.21.15.100:0/1551890478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa4815c7a0 0x7faa480af190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faa380097c0 tx=0x7faa3801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ed6dbab5c85a9f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:53.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.148+0000 7faa44a00640 1 -- 172.21.15.100:0/1551890478 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa380b0070 con 0x7faa4815c7a0 2024-04-23T14:33:53.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.148+0000 7faa44a00640 1 -- 172.21.15.100:0/1551890478 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa380ab600 con 0x7faa4815c7a0 2024-04-23T14:33:53.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.148+0000 7faa44a00640 1 -- 172.21.15.100:0/1551890478 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa380ab9d0 con 0x7faa4815c7a0 2024-04-23T14:33:53.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.148+0000 7faa4cc00640 1 -- 172.21.15.100:0/1551890478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa4815c7a0 msgr2=0x7faa480af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.148+0000 7faa4cc00640 1 --2- 172.21.15.100:0/1551890478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa4815c7a0 0x7faa480af190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faa380097c0 tx=0x7faa3801c410 comp rx=0 tx=0).stop 2024-04-23T14:33:53.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.148+0000 7faa4cc00640 1 -- 172.21.15.100:0/1551890478 shutdown_connections 2024-04-23T14:33:53.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.148+0000 7faa4cc00640 1 --2- 172.21.15.100:0/1551890478 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa4806bb20 0x7faa480ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.148+0000 7faa4cc00640 1 --2- 172.21.15.100:0/1551890478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa4815c7a0 0x7faa480af190 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.148+0000 7faa4cc00640 1 -- 172.21.15.100:0/1551890478 >> 172.21.15.100:0/1551890478 conn(0x7faa480b5200 msgr2=0x7faa480b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:53.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.148+0000 7faa4cc00640 1 -- 172.21.15.100:0/1551890478 shutdown_connections 2024-04-23T14:33:53.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.148+0000 7faa4cc00640 1 -- 172.21.15.100:0/1551890478 wait complete. 2024-04-23T14:33:53.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa4cc00640 1 Processor -- start 2024-04-23T14:33:53.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa4cc00640 1 -- start start 2024-04-23T14:33:53.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa4cc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa4806bb20 0x7faa480c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa4cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa480cd130 0x7faa48099670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa4cc00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa4815c7a0 0x7faa4815f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa4cc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faa48169590 con 0x7faa480cd130 2024-04-23T14:33:53.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa4cc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faa48169410 con 0x7faa4806bb20 2024-04-23T14:33:53.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa4cc00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7faa48169710 con 0x7faa4815c7a0 2024-04-23T14:33:53.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa45400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa480cd130 0x7faa48099670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa45400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa480cd130 0x7faa48099670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36998/0 (socket says 172.21.15.100:36998) 2024-04-23T14:33:53.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa45400640 1 -- 172.21.15.100:0/133400868 learned_addr learned my addr 172.21.15.100:0/133400868 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:53.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa45e00640 1 --2- 172.21.15.100:0/133400868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa4806bb20 0x7faa480c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa46800640 1 --2- 172.21.15.100:0/133400868 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa4815c7a0 0x7faa4815f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa45400640 1 -- 172.21.15.100:0/133400868 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa4815c7a0 msgr2=0x7faa4815f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa45400640 1 --2- 172.21.15.100:0/133400868 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa4815c7a0 0x7faa4815f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa45400640 1 -- 172.21.15.100:0/133400868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa4806bb20 msgr2=0x7faa480c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa45400640 1 --2- 172.21.15.100:0/133400868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa4806bb20 0x7faa480c6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa45400640 1 -- 172.21.15.100:0/133400868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa480c6b40 con 0x7faa480cd130 2024-04-23T14:33:53.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa45400640 1 --2- 172.21.15.100:0/133400868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa480cd130 0x7faa48099670 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7faa3400e9b0 tx=0x7faa3400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:53.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa3e000640 1 -- 172.21.15.100:0/133400868 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa34017070 con 0x7faa480cd130 2024-04-23T14:33:53.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa3e000640 1 -- 172.21.15.100:0/133400868 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa3400c9a0 con 0x7faa480cd130 2024-04-23T14:33:53.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa3e000640 1 -- 172.21.15.100:0/133400868 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3400cd90 con 0x7faa480cd130 2024-04-23T14:33:53.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa4cc00640 1 -- 172.21.15.100:0/133400868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa4815e7b0 con 0x7faa480cd130 2024-04-23T14:33:53.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa4cc00640 1 -- 172.21.15.100:0/133400868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa4815ed70 con 0x7faa480cd130 2024-04-23T14:33:53.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.152+0000 7faa4cc00640 1 -- 172.21.15.100:0/133400868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa48068ad0 con 0x7faa480cd130 2024-04-23T14:33:53.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.156+0000 7faa3e000640 1 -- 172.21.15.100:0/133400868 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7faa3401a070 con 0x7faa480cd130 2024-04-23T14:33:53.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.156+0000 7faa3e000640 1 --2- 172.21.15.100:0/133400868 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faa28068380 0x7faa2806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.156+0000 7faa3e000640 1 -- 172.21.15.100:0/133400868 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7faa34086bc0 con 0x7faa480cd130 2024-04-23T14:33:53.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.156+0000 7faa45e00640 1 --2- 172.21.15.100:0/133400868 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faa28068380 0x7faa2806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.156+0000 7faa45e00640 1 --2- 172.21.15.100:0/133400868 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faa28068380 0x7faa2806a840 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faa380ad000 tx=0x7faa38008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:53.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.160+0000 7faa3e000640 1 -- 172.21.15.100:0/133400868 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faa340567a0 con 0x7faa480cd130 2024-04-23T14:33:53.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.316+0000 7faa4cc00640 1 -- 172.21.15.100:0/133400868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7faa480cf210 con 0x7faa480cd130 2024-04-23T14:33:53.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.316+0000 7faa3e000640 1 -- 172.21.15.100:0/133400868 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) ==== 100+0+11 (secure 0 0 0) 0x7faa3405aaa0 con 0x7faa480cd130 2024-04-23T14:33:53.320 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:33:53.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.316+0000 7faa4cc00640 1 -- 172.21.15.100:0/133400868 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faa28068380 msgr2=0x7faa2806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.316+0000 7faa4cc00640 1 --2- 172.21.15.100:0/133400868 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faa28068380 0x7faa2806a840 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faa380ad000 tx=0x7faa38008040 comp rx=0 tx=0).stop 2024-04-23T14:33:53.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.316+0000 7faa4cc00640 1 -- 172.21.15.100:0/133400868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa480cd130 msgr2=0x7faa48099670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.316+0000 7faa4cc00640 1 --2- 172.21.15.100:0/133400868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa480cd130 0x7faa48099670 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7faa3400e9b0 tx=0x7faa3400ee70 comp rx=0 tx=0).stop 2024-04-23T14:33:53.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.316+0000 7faa4cc00640 1 -- 172.21.15.100:0/133400868 shutdown_connections 2024-04-23T14:33:53.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.316+0000 7faa4cc00640 1 --2- 172.21.15.100:0/133400868 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faa28068380 0x7faa2806a840 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.316+0000 7faa4cc00640 1 --2- 172.21.15.100:0/133400868 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa4815c7a0 0x7faa4815f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.316+0000 7faa4cc00640 1 --2- 172.21.15.100:0/133400868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa480cd130 0x7faa48099670 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.316+0000 7faa4cc00640 1 --2- 172.21.15.100:0/133400868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa4806bb20 0x7faa480c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.316+0000 7faa4cc00640 1 -- 172.21.15.100:0/133400868 >> 172.21.15.100:0/133400868 conn(0x7faa480b5200 msgr2=0x7faa480cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:53.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.316+0000 7faa4cc00640 1 -- 172.21.15.100:0/133400868 shutdown_connections 2024-04-23T14:33:53.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.316+0000 7faa4cc00640 1 -- 172.21.15.100:0/133400868 wait complete. 2024-04-23T14:33:53.341 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-04-23T14:33:53.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.476+0000 7ffb77a00640 1 Processor -- start 2024-04-23T14:33:53.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.476+0000 7ffb77a00640 1 -- start start 2024-04-23T14:33:53.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.476+0000 7ffb77a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ffb7006d450 0x7ffb7006d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.476+0000 7ffb77a00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ffb700cea60 0x7ffb700d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.476+0000 7ffb77a00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb7006dd70 con 0x7ffb700cd460 2024-04-23T14:33:53.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.476+0000 7ffb77a00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7ffb700b1f90 con 0x7ffb700cea60 2024-04-23T14:33:53.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.476+0000 7ffb77a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ffb700b2270 con 0x7ffb7006d450 2024-04-23T14:33:53.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.476+0000 7ffb75a00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ffb700cd460 0x7ffb700b2d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35484/0 (socket says 172.21.15.100:35484) 2024-04-23T14:33:53.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.476+0000 7ffb75a00640 1 -- 172.21.15.100:0/1197741166 learned_addr learned my addr 172.21.15.100:0/1197741166 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:53.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.476+0000 7ffb6fe00640 1 --2- 172.21.15.100:0/1197741166 >> v2:172.21.15.151:3300/0 conn(0x7ffb700cea60 0x7ffb700d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.476+0000 7ffb75000640 1 --2- 172.21.15.100:0/1197741166 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ffb7006d450 0x7ffb7006d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.476+0000 7ffb6f400640 1 -- 172.21.15.100:0/1197741166 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1252111044 0 0) 0x7ffb7006dd70 con 0x7ffb700cd460 2024-04-23T14:33:53.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.476+0000 7ffb6f400640 1 -- 172.21.15.100:0/1197741166 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb54003530 con 0x7ffb700cd460 2024-04-23T14:33:53.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.476+0000 7ffb6fe00640 1 -- 172.21.15.100:0/1197741166 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ffb7006d450 msgr2=0x7ffb7006d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:53.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.476+0000 7ffb6fe00640 1 --2- 172.21.15.100:0/1197741166 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ffb7006d450 0x7ffb7006d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.476+0000 7ffb6fe00640 1 -- 172.21.15.100:0/1197741166 >> v1:172.21.15.100:6789/0 conn(0x7ffb700cd460 legacy=0x7ffb700b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.476+0000 7ffb6fe00640 1 -- 172.21.15.100:0/1197741166 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb700b2550 con 0x7ffb700cea60 2024-04-23T14:33:53.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.480+0000 7ffb6fe00640 1 --2- 172.21.15.100:0/1197741166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb700cea60 0x7ffb700d0300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c0097c0 tx=0x7ffb5c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8ed9e43acafa89a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:53.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.480+0000 7ffb6f400640 1 -- 172.21.15.100:0/1197741166 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0b0070 con 0x7ffb700cea60 2024-04-23T14:33:53.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.480+0000 7ffb6f400640 1 -- 172.21.15.100:0/1197741166 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb5c0ab600 con 0x7ffb700cea60 2024-04-23T14:33:53.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.480+0000 7ffb6f400640 1 -- 172.21.15.100:0/1197741166 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0ab9d0 con 0x7ffb700cea60 2024-04-23T14:33:53.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.480+0000 7ffb77a00640 1 -- 172.21.15.100:0/1197741166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb700cea60 msgr2=0x7ffb700d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.480+0000 7ffb77a00640 1 --2- 172.21.15.100:0/1197741166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb700cea60 0x7ffb700d0300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c0097c0 tx=0x7ffb5c01c410 comp rx=0 tx=0).stop 2024-04-23T14:33:53.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.480+0000 7ffb77a00640 1 -- 172.21.15.100:0/1197741166 shutdown_connections 2024-04-23T14:33:53.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.480+0000 7ffb77a00640 1 --2- 172.21.15.100:0/1197741166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb700cea60 0x7ffb700d0300 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.480+0000 7ffb77a00640 1 --2- 172.21.15.100:0/1197741166 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ffb7006d450 0x7ffb7006d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.480+0000 7ffb77a00640 1 -- 172.21.15.100:0/1197741166 >> 172.21.15.100:0/1197741166 conn(0x7ffb700cee80 msgr2=0x7ffb700cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:53.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.480+0000 7ffb77a00640 1 -- 172.21.15.100:0/1197741166 shutdown_connections 2024-04-23T14:33:53.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.480+0000 7ffb77a00640 1 -- 172.21.15.100:0/1197741166 wait complete. 2024-04-23T14:33:53.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb77a00640 1 Processor -- start 2024-04-23T14:33:53.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb77a00640 1 -- start start 2024-04-23T14:33:53.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb77a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ffb7006d450 0x7ffb701559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb77a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb700cd460 0x7ffb700c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb77a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb700cea60 0x7ffb7015ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb77a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7015fc70 con 0x7ffb700cea60 2024-04-23T14:33:53.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb77a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7015faf0 con 0x7ffb700cd460 2024-04-23T14:33:53.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb77a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ffb7016bbc0 con 0x7ffb7006d450 2024-04-23T14:33:53.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb75a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb700cea60 0x7ffb7015ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb75a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb700cea60 0x7ffb7015ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37008/0 (socket says 172.21.15.100:37008) 2024-04-23T14:33:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb75a00640 1 -- 172.21.15.100:0/2331376026 learned_addr learned my addr 172.21.15.100:0/2331376026 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb75000640 1 --2- 172.21.15.100:0/2331376026 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ffb7006d450 0x7ffb701559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb6fe00640 1 --2- 172.21.15.100:0/2331376026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb700cd460 0x7ffb700c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb75a00640 1 -- 172.21.15.100:0/2331376026 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ffb7006d450 msgr2=0x7ffb701559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb75a00640 1 --2- 172.21.15.100:0/2331376026 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ffb7006d450 0x7ffb701559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb75a00640 1 -- 172.21.15.100:0/2331376026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb700cd460 msgr2=0x7ffb700c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb75a00640 1 --2- 172.21.15.100:0/2331376026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb700cd460 0x7ffb700c83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb75a00640 1 -- 172.21.15.100:0/2331376026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb700c64a0 con 0x7ffb700cea60 2024-04-23T14:33:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb75a00640 1 --2- 172.21.15.100:0/2331376026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb700cea60 0x7ffb7015ff10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ffb6800c3c0 tx=0x7ffb6800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb6cc00640 1 -- 172.21.15.100:0/2331376026 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb68017070 con 0x7ffb700cea60 2024-04-23T14:33:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb6cc00640 1 -- 172.21.15.100:0/2331376026 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb68005cf0 con 0x7ffb700cea60 2024-04-23T14:33:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb6cc00640 1 -- 172.21.15.100:0/2331376026 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb68007310 con 0x7ffb700cea60 2024-04-23T14:33:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb77a00640 1 -- 172.21.15.100:0/2331376026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb700c6780 con 0x7ffb700cea60 2024-04-23T14:33:53.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.484+0000 7ffb77a00640 1 -- 172.21.15.100:0/2331376026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb700c6d40 con 0x7ffb700cea60 2024-04-23T14:33:53.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.488+0000 7ffb77a00640 1 -- 172.21.15.100:0/2331376026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb70068ad0 con 0x7ffb700cea60 2024-04-23T14:33:53.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.488+0000 7ffb6cc00640 1 -- 172.21.15.100:0/2331376026 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ffb6801a070 con 0x7ffb700cea60 2024-04-23T14:33:53.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.488+0000 7ffb6cc00640 1 --2- 172.21.15.100:0/2331376026 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ffb50068380 0x7ffb5006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.488+0000 7ffb6cc00640 1 -- 172.21.15.100:0/2331376026 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7ffb68085f70 con 0x7ffb700cea60 2024-04-23T14:33:53.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.488+0000 7ffb75000640 1 --2- 172.21.15.100:0/2331376026 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ffb50068380 0x7ffb5006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.488+0000 7ffb75000640 1 --2- 172.21.15.100:0/2331376026 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ffb50068380 0x7ffb5006a840 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ffb700d1bf0 tx=0x7ffb6000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:53.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.492+0000 7ffb6cc00640 1 -- 172.21.15.100:0/2331376026 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffb68055b50 con 0x7ffb700cea60 2024-04-23T14:33:53.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.652+0000 7ffb77a00640 1 -- 172.21.15.100:0/2331376026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) -- 0x7ffb70161610 con 0x7ffb700cea60 2024-04-23T14:33:53.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.652+0000 7ffb6cc00640 1 -- 172.21.15.100:0/2331376026 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v41) ==== 97+0+11 (secure 0 0 0) 0x7ffb68059e50 con 0x7ffb700cea60 2024-04-23T14:33:53.656 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.652+0000 7ffb77a00640 1 -- 172.21.15.100:0/2331376026 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ffb50068380 msgr2=0x7ffb5006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.652+0000 7ffb77a00640 1 --2- 172.21.15.100:0/2331376026 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ffb50068380 0x7ffb5006a840 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ffb700d1bf0 tx=0x7ffb6000b040 comp rx=0 tx=0).stop 2024-04-23T14:33:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.652+0000 7ffb77a00640 1 -- 172.21.15.100:0/2331376026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb700cea60 msgr2=0x7ffb7015ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.652+0000 7ffb77a00640 1 --2- 172.21.15.100:0/2331376026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb700cea60 0x7ffb7015ff10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ffb6800c3c0 tx=0x7ffb6800c880 comp rx=0 tx=0).stop 2024-04-23T14:33:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.652+0000 7ffb77a00640 1 -- 172.21.15.100:0/2331376026 shutdown_connections 2024-04-23T14:33:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.652+0000 7ffb77a00640 1 --2- 172.21.15.100:0/2331376026 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ffb50068380 0x7ffb5006a840 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.652+0000 7ffb77a00640 1 --2- 172.21.15.100:0/2331376026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb700cea60 0x7ffb7015ff10 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.652+0000 7ffb77a00640 1 --2- 172.21.15.100:0/2331376026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb700cd460 0x7ffb700c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.652+0000 7ffb77a00640 1 --2- 172.21.15.100:0/2331376026 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ffb7006d450 0x7ffb701559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.652+0000 7ffb77a00640 1 -- 172.21.15.100:0/2331376026 >> 172.21.15.100:0/2331376026 conn(0x7ffb700cee80 msgr2=0x7ffb700ce1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.652+0000 7ffb77a00640 1 -- 172.21.15.100:0/2331376026 shutdown_connections 2024-04-23T14:33:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.652+0000 7ffb77a00640 1 -- 172.21.15.100:0/2331376026 wait complete. 2024-04-23T14:33:53.676 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:33:53.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.808+0000 7f983dc00640 1 Processor -- start 2024-04-23T14:33:53.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.808+0000 7f983dc00640 1 -- start start 2024-04-23T14:33:53.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.808+0000 7f983dc00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f983815b660 0x7f98380c6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.808+0000 7f983dc00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98380cee00 con 0x7f98380d08a0 2024-04-23T14:33:53.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.808+0000 7f983dc00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f983806bc50 con 0x7f98380c6d60 2024-04-23T14:33:53.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.808+0000 7f983dc00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f983815ba40 con 0x7f983815b660 2024-04-23T14:33:53.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.808+0000 7f9837400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f98380d08a0 0x7f98380c5160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35488/0 (socket says 172.21.15.100:35488) 2024-04-23T14:33:53.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.808+0000 7f9837400640 1 -- 172.21.15.100:0/1492565554 learned_addr learned my addr 172.21.15.100:0/1492565554 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:53.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.812+0000 7f9836a00640 1 --2- 172.21.15.100:0/1492565554 >> v2:172.21.15.151:3300/0 conn(0x7f983815b660 0x7f98380c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.812+0000 7f9836000640 1 -- 172.21.15.100:0/1492565554 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 967845760 0 0) 0x7f98300014a0 con 0x7f98380d08a0 2024-04-23T14:33:53.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.812+0000 7f9836000640 1 -- 172.21.15.100:0/1492565554 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f981c003530 con 0x7f98380d08a0 2024-04-23T14:33:53.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.812+0000 7f9836a00640 1 -- 172.21.15.100:0/1492565554 >> v1:172.21.15.151:6789/0 conn(0x7f98380c6d60 legacy=0x7f98380d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.812+0000 7f9836a00640 1 -- 172.21.15.100:0/1492565554 >> v1:172.21.15.100:6789/0 conn(0x7f98380d08a0 legacy=0x7f98380c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.812+0000 7f9836000640 1 -- 172.21.15.100:0/1492565554 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1665158024 0 0) 0x7f981c003530 con 0x7f98380d08a0 2024-04-23T14:33:53.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.812+0000 7f9836a00640 1 -- 172.21.15.100:0/1492565554 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f983806bf30 con 0x7f983815b660 2024-04-23T14:33:53.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.812+0000 7f9836a00640 1 --2- 172.21.15.100:0/1492565554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983815b660 0x7f98380c6820 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f98280097c0 tx=0x7f982801c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=116b1ee459c62f12 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:53.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f9836000640 1 -- 172.21.15.100:0/1492565554 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98280b1070 con 0x7f983815b660 2024-04-23T14:33:53.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f9836000640 1 -- 172.21.15.100:0/1492565554 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98280ac440 con 0x7f983815b660 2024-04-23T14:33:53.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f9836000640 1 -- 172.21.15.100:0/1492565554 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830001bd0 con 0x7f983815b660 2024-04-23T14:33:53.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f983dc00640 1 -- 172.21.15.100:0/1492565554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983815b660 msgr2=0x7f98380c6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f983dc00640 1 --2- 172.21.15.100:0/1492565554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983815b660 0x7f98380c6820 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f98280097c0 tx=0x7f982801c390 comp rx=0 tx=0).stop 2024-04-23T14:33:53.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f983dc00640 1 -- 172.21.15.100:0/1492565554 shutdown_connections 2024-04-23T14:33:53.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f983dc00640 1 --2- 172.21.15.100:0/1492565554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983815b660 0x7f98380c6820 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f983dc00640 1 -- 172.21.15.100:0/1492565554 >> 172.21.15.100:0/1492565554 conn(0x7f98380b5310 msgr2=0x7f98380b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:53.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f983dc00640 1 -- 172.21.15.100:0/1492565554 shutdown_connections 2024-04-23T14:33:53.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f983dc00640 1 -- 172.21.15.100:0/1492565554 wait complete. 2024-04-23T14:33:53.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f983dc00640 1 Processor -- start 2024-04-23T14:33:53.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f983dc00640 1 -- start start 2024-04-23T14:33:53.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f983dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98380c6d60 0x7f983806fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f983dc00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f98380d08a0 0x7f983814c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f983dc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983815b660 0x7f98381567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f983dc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f98380c6200 con 0x7f98380c6d60 2024-04-23T14:33:53.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f983dc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f98380c6080 con 0x7f983815b660 2024-04-23T14:33:53.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f983dc00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f98380c6380 con 0x7f98380d08a0 2024-04-23T14:33:53.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f9836a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98380c6d60 0x7f983806fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f9836a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98380c6d60 0x7f983806fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37018/0 (socket says 172.21.15.100:37018) 2024-04-23T14:33:53.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f9836a00640 1 -- 172.21.15.100:0/2277540207 learned_addr learned my addr 172.21.15.100:0/2277540207 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:53.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f9837400640 1 --2- 172.21.15.100:0/2277540207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983815b660 0x7f98381567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.816+0000 7f982fe00640 1 --2- 172.21.15.100:0/2277540207 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f98380d08a0 0x7f983814c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.820+0000 7f9836a00640 1 -- 172.21.15.100:0/2277540207 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f98380d08a0 msgr2=0x7f983814c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.820+0000 7f9836a00640 1 --2- 172.21.15.100:0/2277540207 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f98380d08a0 0x7f983814c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.820+0000 7f9836a00640 1 -- 172.21.15.100:0/2277540207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983815b660 msgr2=0x7f98381567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.820+0000 7f9836a00640 1 --2- 172.21.15.100:0/2277540207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983815b660 0x7f98381567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.820+0000 7f9836a00640 1 -- 172.21.15.100:0/2277540207 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98380cbdc0 con 0x7f98380c6d60 2024-04-23T14:33:53.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.820+0000 7f9836a00640 1 --2- 172.21.15.100:0/2277540207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98380c6d60 0x7f983806fb70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9828018fa0 tx=0x7f982801abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:53.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.820+0000 7f982ea00640 1 -- 172.21.15.100:0/2277540207 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98280b1050 con 0x7f98380c6d60 2024-04-23T14:33:53.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.820+0000 7f982ea00640 1 -- 172.21.15.100:0/2277540207 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98280bd070 con 0x7f98380c6d60 2024-04-23T14:33:53.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.820+0000 7f982ea00640 1 -- 172.21.15.100:0/2277540207 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98280b8850 con 0x7f98380c6d60 2024-04-23T14:33:53.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.820+0000 7f983dc00640 1 -- 172.21.15.100:0/2277540207 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98380cc0a0 con 0x7f98380c6d60 2024-04-23T14:33:53.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.820+0000 7f983dc00640 1 -- 172.21.15.100:0/2277540207 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98380cc660 con 0x7f98380c6d60 2024-04-23T14:33:53.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.820+0000 7f983dc00640 1 -- 172.21.15.100:0/2277540207 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9838068ad0 con 0x7f98380c6d60 2024-04-23T14:33:53.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.820+0000 7f982ea00640 1 -- 172.21.15.100:0/2277540207 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f98280b89f0 con 0x7f98380c6d60 2024-04-23T14:33:53.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.820+0000 7f982ea00640 1 --2- 172.21.15.100:0/2277540207 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9814068380 0x7f981406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.820+0000 7f982ea00640 1 -- 172.21.15.100:0/2277540207 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f9828121630 con 0x7f98380c6d60 2024-04-23T14:33:53.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.824+0000 7f982fe00640 1 --2- 172.21.15.100:0/2277540207 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9814068380 0x7f981406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.824+0000 7f982fe00640 1 --2- 172.21.15.100:0/2277540207 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9814068380 0x7f981406a840 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f98381507f0 tx=0x7f9820009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:53.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:53.824+0000 7f982ea00640 1 -- 172.21.15.100:0/2277540207 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9828015fb0 con 0x7f98380c6d60 2024-04-23T14:33:54.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.004+0000 7f983dc00640 1 -- 172.21.15.100:0/2277540207 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f98380ceaa0 con 0x7f98380c6d60 2024-04-23T14:33:54.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.004+0000 7f982ea00640 1 -- 172.21.15.100:0/2277540207 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) ==== 80+0+0 (secure 0 0 0) 0x7f98280f1980 con 0x7f98380c6d60 2024-04-23T14:33:54.009 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-04-23T14:33:54.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.004+0000 7f983dc00640 1 -- 172.21.15.100:0/2277540207 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9814068380 msgr2=0x7f981406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:54.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.004+0000 7f983dc00640 1 --2- 172.21.15.100:0/2277540207 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9814068380 0x7f981406a840 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f98381507f0 tx=0x7f9820009040 comp rx=0 tx=0).stop 2024-04-23T14:33:54.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.004+0000 7f983dc00640 1 -- 172.21.15.100:0/2277540207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98380c6d60 msgr2=0x7f983806fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:54.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.004+0000 7f983dc00640 1 --2- 172.21.15.100:0/2277540207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98380c6d60 0x7f983806fb70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9828018fa0 tx=0x7f982801abd0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.004+0000 7f983dc00640 1 -- 172.21.15.100:0/2277540207 shutdown_connections 2024-04-23T14:33:54.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.004+0000 7f983dc00640 1 --2- 172.21.15.100:0/2277540207 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9814068380 0x7f981406a840 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.004+0000 7f983dc00640 1 --2- 172.21.15.100:0/2277540207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983815b660 0x7f98381567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.004+0000 7f983dc00640 1 --2- 172.21.15.100:0/2277540207 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f98380d08a0 0x7f983814c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.004+0000 7f983dc00640 1 --2- 172.21.15.100:0/2277540207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98380c6d60 0x7f983806fb70 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.004+0000 7f983dc00640 1 -- 172.21.15.100:0/2277540207 >> 172.21.15.100:0/2277540207 conn(0x7f98380b5310 msgr2=0x7f98381595b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:54.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.004+0000 7f983dc00640 1 -- 172.21.15.100:0/2277540207 shutdown_connections 2024-04-23T14:33:54.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.004+0000 7f983dc00640 1 -- 172.21.15.100:0/2277540207 wait complete. 2024-04-23T14:33:54.030 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi100.front.sepia.ceph.com... 2024-04-23T14:33:54.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:54.030 DEBUG:teuthology.orchestra.run.smithi100:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:54.048 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[39311]: fuse finished with error 0 and tester_r 0 2024-04-23T14:33:54.051 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:33:54.127 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:33:54.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:54.128 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:33:54.137 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:33:54.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.312+0000 7f1b87400640 1 Processor -- start 2024-04-23T14:33:54.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.312+0000 7f1b87400640 1 -- start start 2024-04-23T14:33:54.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.312+0000 7f1b87400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1b8015c470 0x7f1b800b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:54.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.312+0000 7f1b87400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f1b80160de0 0x7f1b800ad160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:54.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.312+0000 7f1b87400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b800ad6a0 0x7f1b8006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:54.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.312+0000 7f1b87400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f1b800adab0 con 0x7f1b8015c470 2024-04-23T14:33:54.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.312+0000 7f1b87400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f1b80160320 con 0x7f1b80160de0 2024-04-23T14:33:54.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.312+0000 7f1b87400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1b800cc7e0 con 0x7f1b800ad6a0 2024-04-23T14:33:54.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.312+0000 7f1b84a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1b8015c470 0x7f1b800b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:54.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.316+0000 7f1b84a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1b8015c470 0x7f1b800b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37034/0 (socket says 172.21.15.100:37034) 2024-04-23T14:33:54.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.316+0000 7f1b84a00640 1 -- 172.21.15.100:0/2482406367 learned_addr learned my addr 172.21.15.100:0/2482406367 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:54.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.316+0000 7f1b85400640 1 --2- 172.21.15.100:0/2482406367 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b800ad6a0 0x7f1b8006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:54.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.316+0000 7f1b7fe00640 1 --2- 172.21.15.100:0/2482406367 >> v2:172.21.15.151:3300/0 conn(0x7f1b80160de0 0x7f1b800ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:54.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.316+0000 7f1b84a00640 1 -- 172.21.15.100:0/2482406367 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b800ad6a0 msgr2=0x7f1b8006bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:54.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.316+0000 7f1b84a00640 1 --2- 172.21.15.100:0/2482406367 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b800ad6a0 0x7f1b8006bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.316+0000 7f1b84a00640 1 -- 172.21.15.100:0/2482406367 >> v2:172.21.15.151:3300/0 conn(0x7f1b80160de0 msgr2=0x7f1b800ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:54.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.316+0000 7f1b84a00640 1 --2- 172.21.15.100:0/2482406367 >> v2:172.21.15.151:3300/0 conn(0x7f1b80160de0 0x7f1b800ad160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.316+0000 7f1b84a00640 1 -- 172.21.15.100:0/2482406367 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b801605e0 con 0x7f1b8015c470 2024-04-23T14:33:54.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.316+0000 7f1b84a00640 1 --2- 172.21.15.100:0/2482406367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b8015c470 0x7f1b800b29a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1b800742b0 tx=0x7f1b7401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecda765241b5eeca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:54.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.316+0000 7f1b7f400640 1 -- 172.21.15.100:0/2482406367 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b740b2070 con 0x7f1b8015c470 2024-04-23T14:33:54.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.316+0000 7f1b7f400640 1 -- 172.21.15.100:0/2482406367 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b740a9070 con 0x7f1b8015c470 2024-04-23T14:33:54.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.316+0000 7f1b7f400640 1 -- 172.21.15.100:0/2482406367 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b740ad880 con 0x7f1b8015c470 2024-04-23T14:33:54.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b87400640 1 -- 172.21.15.100:0/2482406367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b8015c470 msgr2=0x7f1b800b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:54.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b87400640 1 --2- 172.21.15.100:0/2482406367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b8015c470 0x7f1b800b29a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1b800742b0 tx=0x7f1b7401c490 comp rx=0 tx=0).stop 2024-04-23T14:33:54.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b87400640 1 -- 172.21.15.100:0/2482406367 shutdown_connections 2024-04-23T14:33:54.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b87400640 1 --2- 172.21.15.100:0/2482406367 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b800ad6a0 0x7f1b8006bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b87400640 1 --2- 172.21.15.100:0/2482406367 >> v2:172.21.15.151:3300/0 conn(0x7f1b80160de0 0x7f1b800ad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b87400640 1 --2- 172.21.15.100:0/2482406367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b8015c470 0x7f1b800b29a0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b87400640 1 -- 172.21.15.100:0/2482406367 >> 172.21.15.100:0/2482406367 conn(0x7f1b800b1e80 msgr2=0x7f1b800b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:54.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b87400640 1 -- 172.21.15.100:0/2482406367 shutdown_connections 2024-04-23T14:33:54.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b87400640 1 -- 172.21.15.100:0/2482406367 wait complete. 2024-04-23T14:33:54.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b87400640 1 Processor -- start 2024-04-23T14:33:54.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b87400640 1 -- start start 2024-04-23T14:33:54.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b87400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b800ad6a0 0x7f1b800ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:54.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b87400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b8015c470 0x7f1b8009d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:54.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b87400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b80160de0 0x7f1b801508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:54.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b87400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1b800cc370 con 0x7f1b800ad6a0 2024-04-23T14:33:54.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b87400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8015ca60 con 0x7f1b8015c470 2024-04-23T14:33:54.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b87400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1b800cc4f0 con 0x7f1b80160de0 2024-04-23T14:33:54.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b84a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b800ad6a0 0x7f1b800ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:54.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b84a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b800ad6a0 0x7f1b800ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37042/0 (socket says 172.21.15.100:37042) 2024-04-23T14:33:54.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b84a00640 1 -- 172.21.15.100:0/906145900 learned_addr learned my addr 172.21.15.100:0/906145900 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:54.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b7fe00640 1 --2- 172.21.15.100:0/906145900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b8015c470 0x7f1b8009d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:54.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b85400640 1 --2- 172.21.15.100:0/906145900 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b80160de0 0x7f1b801508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:54.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b84a00640 1 -- 172.21.15.100:0/906145900 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b80160de0 msgr2=0x7f1b801508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:54.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b84a00640 1 --2- 172.21.15.100:0/906145900 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b80160de0 0x7f1b801508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b84a00640 1 -- 172.21.15.100:0/906145900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b8015c470 msgr2=0x7f1b8009d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:54.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b84a00640 1 --2- 172.21.15.100:0/906145900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b8015c470 0x7f1b8009d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b84a00640 1 -- 172.21.15.100:0/906145900 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b800cd1a0 con 0x7f1b800ad6a0 2024-04-23T14:33:54.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b7fe00640 1 --2- 172.21.15.100:0/906145900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b8015c470 0x7f1b8009d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:54.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b84a00640 1 --2- 172.21.15.100:0/906145900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b800ad6a0 0x7f1b800ccc60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1b7401ab40 tx=0x7f1b74019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:54.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.320+0000 7f1b7cc00640 1 -- 172.21.15.100:0/906145900 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b740b2040 con 0x7f1b800ad6a0 2024-04-23T14:33:54.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.324+0000 7f1b7cc00640 1 -- 172.21.15.100:0/906145900 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b740bb070 con 0x7f1b800ad6a0 2024-04-23T14:33:54.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.324+0000 7f1b7cc00640 1 -- 172.21.15.100:0/906145900 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b740b6440 con 0x7f1b800ad6a0 2024-04-23T14:33:54.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.324+0000 7f1b87400640 1 -- 172.21.15.100:0/906145900 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b8009df30 con 0x7f1b800ad6a0 2024-04-23T14:33:54.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.324+0000 7f1b87400640 1 -- 172.21.15.100:0/906145900 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b8015d060 con 0x7f1b800ad6a0 2024-04-23T14:33:54.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.324+0000 7f1b87400640 1 -- 172.21.15.100:0/906145900 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b80068ad0 con 0x7f1b800ad6a0 2024-04-23T14:33:54.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.324+0000 7f1b7cc00640 1 -- 172.21.15.100:0/906145900 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1b740ab070 con 0x7f1b800ad6a0 2024-04-23T14:33:54.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.324+0000 7f1b7cc00640 1 --2- 172.21.15.100:0/906145900 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1b60068380 0x7f1b6006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:54.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.324+0000 7f1b7cc00640 1 -- 172.21.15.100:0/906145900 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f1b741204a0 con 0x7f1b800ad6a0 2024-04-23T14:33:54.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.324+0000 7f1b7fe00640 1 --2- 172.21.15.100:0/906145900 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1b60068380 0x7f1b6006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:54.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.324+0000 7f1b7fe00640 1 --2- 172.21.15.100:0/906145900 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1b60068380 0x7f1b6006a840 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1b80144c90 tx=0x7f1b6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:54.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.328+0000 7f1b7cc00640 1 -- 172.21.15.100:0/906145900 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1b74016110 con 0x7f1b800ad6a0 2024-04-23T14:33:54.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.520+0000 7f1b87400640 1 -- 172.21.15.100:0/906145900 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1b800b5460 con 0x7f1b800ad6a0 2024-04-23T14:33:54.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.520+0000 7f1b7cc00640 1 -- 172.21.15.100:0/906145900 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) ==== 53+0+82 (secure 0 0 0) 0x7f1b740f01a0 con 0x7f1b800ad6a0 2024-04-23T14:33:54.525 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2024-04-23T14:33:54.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.520+0000 7f1b87400640 1 -- 172.21.15.100:0/906145900 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1b60068380 msgr2=0x7f1b6006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:54.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.520+0000 7f1b87400640 1 --2- 172.21.15.100:0/906145900 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1b60068380 0x7f1b6006a840 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1b80144c90 tx=0x7f1b6c008040 comp rx=0 tx=0).stop 2024-04-23T14:33:54.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.520+0000 7f1b87400640 1 -- 172.21.15.100:0/906145900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b800ad6a0 msgr2=0x7f1b800ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:54.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.520+0000 7f1b87400640 1 --2- 172.21.15.100:0/906145900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b800ad6a0 0x7f1b800ccc60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1b7401ab40 tx=0x7f1b74019230 comp rx=0 tx=0).stop 2024-04-23T14:33:54.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.520+0000 7f1b87400640 1 -- 172.21.15.100:0/906145900 shutdown_connections 2024-04-23T14:33:54.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.520+0000 7f1b87400640 1 --2- 172.21.15.100:0/906145900 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1b60068380 0x7f1b6006a840 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.520+0000 7f1b87400640 1 --2- 172.21.15.100:0/906145900 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1b80160de0 0x7f1b801508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.520+0000 7f1b87400640 1 --2- 172.21.15.100:0/906145900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b8015c470 0x7f1b8009d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.520+0000 7f1b87400640 1 --2- 172.21.15.100:0/906145900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b800ad6a0 0x7f1b800ccc60 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.520+0000 7f1b87400640 1 -- 172.21.15.100:0/906145900 >> 172.21.15.100:0/906145900 conn(0x7f1b800b1e80 msgr2=0x7f1b80143220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:54.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.520+0000 7f1b87400640 1 -- 172.21.15.100:0/906145900 shutdown_connections 2024-04-23T14:33:54.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54.524+0000 7f1b87400640 1 -- 172.21.15.100:0/906145900 wait complete. 2024-04-23T14:33:54.546 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:33:54.546 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:33:54.546 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-04-23T14:33:54.546 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:33:54.546 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:54.546 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:33:54.546 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:33:54.546 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:33:54.546 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:33:54.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:54.547 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-04-23T14:33:54.557 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:33:54.557 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:33:54.557 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:33:54.557 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:54.557 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-23T14:33:54.557 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:54.557 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:33:54.557 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:33:54.557 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-04-23T14:33:54.557 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:33:54.557 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:33:54.558 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-04-23T14:33:54.558 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:33:54.558 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:33:54.558 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:33:54.558 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:33:54.558 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-23T14:33:54.558 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42349sec preferred_lft 42349sec 2024-04-23T14:33:54.558 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-04-23T14:33:54.558 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:54.558 INFO:teuthology.orchestra.run.smithi100.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-23T14:33:54.558 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 02:42:ff:fc:d1:cc brd ff:ff:ff:ff:ff:ff 2024-04-23T14:33:54.558 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-23T14:33:54.558 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:54.559 INFO:teuthology.orchestra.run.smithi100.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:33:54.559 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-23T14:33:54.559 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:33:54.559 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:54.559 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-23T14:33:54.559 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:54.559 INFO:teuthology.orchestra.run.smithi100.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:33:54.559 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:33:54.559 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-23T14:33:54.559 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:54.559 INFO:teuthology.orchestra.run.smithi100.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:33:54.559 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:33:54.559 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-23T14:33:54.559 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:54.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:54.560 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:33:54.616 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-04-23T14:33:54.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:54.618 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-04-23T14:33:54.674 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-04-23T14:33:54.674 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:33:54.674 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-23T14:33:54.674 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-23T14:33:54.674 INFO:teuthology.orchestra.run.smithi100.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-23T14:33:54.674 INFO:teuthology.orchestra.run.smithi100.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-23T14:33:54.674 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-23T14:33:54.674 INFO:teuthology.orchestra.run.smithi100.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:33:54.674 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:33:54.674 INFO:teuthology.orchestra.run.smithi100.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:33:54.674 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:33:54.675 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:33:54.675 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-23T14:33:54.675 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-23T14:33:54.675 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-23T14:33:54.675 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-23T14:33:54.675 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-23T14:33:54.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:33:54.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:33:54.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:54.812 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-23T14:33:54.821 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:33:54.821 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:33:54.821 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:33:54.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:54.821 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-04-23T14:33:54.882 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:33:54.882 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:33:54.882 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:54.935 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:33:54.935 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:33:54.935 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:54.990 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-23T14:33:55.050 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:33:55.100 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-23T14:33:55.100 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-23T14:33:55.100 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-23T14:33:55.100 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-23T14:33:55.100 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-23T14:33:55.100 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-23T14:33:55.100 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-23T14:33:55.100 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-23T14:33:55.100 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-23T14:33:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-23T14:33:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-23T14:33:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-23T14:33:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:33:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-23T14:33:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-23T14:33:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-23T14:33:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-23T14:33:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-23T14:33:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-23T14:33:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-23T14:33:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:33:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-23T14:33:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-23T14:33:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-23T14:33:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:33:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:33:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:33:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:33:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-23T14:33:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:33:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:33:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:33:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:33:55.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:55.104 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-23T14:33:55.154 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:33:55.154 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:33:55.197 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-23T14:33:55.217 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:33:55.269 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-23T14:33:55.269 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-23T14:33:55.269 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-23T14:33:55.269 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-23T14:33:55.269 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-23T14:33:55.269 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-23T14:33:55.269 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-23T14:33:55.269 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-23T14:33:55.269 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-23T14:33:55.270 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-23T14:33:55.270 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-23T14:33:55.270 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-23T14:33:55.270 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:33:55.270 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-23T14:33:55.270 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-23T14:33:55.270 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-23T14:33:55.270 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-23T14:33:55.270 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-23T14:33:55.270 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-23T14:33:55.270 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-23T14:33:55.271 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:33:55.271 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-23T14:33:55.271 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-23T14:33:55.271 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-23T14:33:55.271 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:33:55.271 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:33:55.271 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:33:55.271 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:33:55.271 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-23T14:33:55.272 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:33:55.272 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:33:55.272 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:33:55.272 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:33:55.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:55.273 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-23T14:33:55.291 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:2024-04-23T14:33:55.284+0000 7f0a802e2580 -1 init, newargv = 0x558167a3abf0 newargc=15 2024-04-23T14:33:55.291 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[39870]: starting ceph client 2024-04-23T14:33:55.298 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[39870]: starting fuse 2024-04-23T14:33:55.299 INFO:teuthology.orchestra.run.smithi100.stdout:44 2024-04-23T14:33:55.300 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-23T14:33:55.300 DEBUG:teuthology.orchestra.run.smithi100:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:33:55.300 DEBUG:teuthology.orchestra.run.smithi100:> import glob 2024-04-23T14:33:55.300 DEBUG:teuthology.orchestra.run.smithi100:> import re 2024-04-23T14:33:55.300 DEBUG:teuthology.orchestra.run.smithi100:> import os 2024-04-23T14:33:55.300 DEBUG:teuthology.orchestra.run.smithi100:> import subprocess 2024-04-23T14:33:55.300 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:33:55.300 DEBUG:teuthology.orchestra.run.smithi100:> def _find_admin_socket(client_name): 2024-04-23T14:33:55.300 DEBUG:teuthology.orchestra.run.smithi100:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:33:55.300 DEBUG:teuthology.orchestra.run.smithi100:> files = glob.glob(asok_path) 2024-04-23T14:33:55.301 DEBUG:teuthology.orchestra.run.smithi100:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:33:55.301 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:33:55.301 DEBUG:teuthology.orchestra.run.smithi100:> # Given a non-glob path, it better be there 2024-04-23T14:33:55.301 DEBUG:teuthology.orchestra.run.smithi100:> if "*" not in asok_path: 2024-04-23T14:33:55.301 DEBUG:teuthology.orchestra.run.smithi100:> assert(len(files) == 1) 2024-04-23T14:33:55.301 DEBUG:teuthology.orchestra.run.smithi100:> return files[0] 2024-04-23T14:33:55.301 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:33:55.301 DEBUG:teuthology.orchestra.run.smithi100:> for f in files: 2024-04-23T14:33:55.301 DEBUG:teuthology.orchestra.run.smithi100:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:33:55.301 DEBUG:teuthology.orchestra.run.smithi100:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:33:55.301 DEBUG:teuthology.orchestra.run.smithi100:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:33:55.301 DEBUG:teuthology.orchestra.run.smithi100:> contents = proc_f.read() 2024-04-23T14:33:55.301 DEBUG:teuthology.orchestra.run.smithi100:> if mountpoint in contents: 2024-04-23T14:33:55.302 DEBUG:teuthology.orchestra.run.smithi100:> return f 2024-04-23T14:33:55.302 DEBUG:teuthology.orchestra.run.smithi100:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:33:55.302 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:33:55.302 DEBUG:teuthology.orchestra.run.smithi100:> print(_find_admin_socket("client.0")) 2024-04-23T14:33:55.302 DEBUG:teuthology.orchestra.run.smithi100:> ' 2024-04-23T14:33:55.422 INFO:teuthology.orchestra.run.smithi100.stdout:/var/run/ceph/ceph-client.0.39870.asok 2024-04-23T14:33:55.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:33:55.433 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39870.asok 2024-04-23T14:33:55.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:55.434 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39870.asok status 2024-04-23T14:33:55.605 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:33:55.606 INFO:teuthology.orchestra.run.smithi100.stdout: "metadata": { 2024-04-23T14:33:55.606 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:33:55.606 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:33:55.606 INFO:teuthology.orchestra.run.smithi100.stdout: "entity_id": "0", 2024-04-23T14:33:55.606 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": "smithi100", 2024-04-23T14:33:55.606 INFO:teuthology.orchestra.run.smithi100.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:33:55.606 INFO:teuthology.orchestra.run.smithi100.stdout: "pid": "39870", 2024-04-23T14:33:55.606 INFO:teuthology.orchestra.run.smithi100.stdout: "root": "/" 2024-04-23T14:33:55.606 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:55.606 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_count": 0, 2024-04-23T14:33:55.606 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_pinned_count": 0, 2024-04-23T14:33:55.606 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 5811, 2024-04-23T14:33:55.607 INFO:teuthology.orchestra.run.smithi100.stdout: "inst": { 2024-04-23T14:33:55.607 INFO:teuthology.orchestra.run.smithi100.stdout: "name": { 2024-04-23T14:33:55.607 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "client", 2024-04-23T14:33:55.607 INFO:teuthology.orchestra.run.smithi100.stdout: "num": 5811 2024-04-23T14:33:55.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:55.607 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-23T14:33:55.607 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-23T14:33:55.607 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:33:55.607 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2019750758 2024-04-23T14:33:55.607 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:33:55.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:55.607 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-23T14:33:55.607 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-23T14:33:55.607 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:33:55.608 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2019750758 2024-04-23T14:33:55.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:33:55.608 INFO:teuthology.orchestra.run.smithi100.stdout: "inst_str": "client.5811 192.168.0.1:0/2019750758", 2024-04-23T14:33:55.608 INFO:teuthology.orchestra.run.smithi100.stdout: "addr_str": "192.168.0.1:0/2019750758", 2024-04-23T14:33:55.608 INFO:teuthology.orchestra.run.smithi100.stdout: "inode_count": 1, 2024-04-23T14:33:55.608 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_epoch": 19, 2024-04-23T14:33:55.608 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch": 41, 2024-04-23T14:33:55.608 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:33:55.608 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklisted": false, 2024-04-23T14:33:55.608 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs" 2024-04-23T14:33:55.608 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:33:55.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:55.621 DEBUG:teuthology.orchestra.run.smithi100:> stat --file-system '--printf=%T 2024-04-23T14:33:55.621 DEBUG:teuthology.orchestra.run.smithi100:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:55.630 INFO:teuthology.orchestra.run.smithi100.stdout:fuseblk 2024-04-23T14:33:55.630 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:55.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:55.630 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:55.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:55.694 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2024-04-23T14:33:55.748 INFO:teuthology.orchestra.run.smithi100.stderr:setfattr: layoutfile: Permission denied 2024-04-23T14:33:55.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:33:55.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:55.749 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2024-04-23T14:33:55.800 INFO:teuthology.orchestra.run.smithi100.stderr:setfattr: layoutdir: Permission denied 2024-04-23T14:33:55.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:33:55.801 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:33:55.801 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:55.850 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:56.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.024+0000 7f12d7600640 1 Processor -- start 2024-04-23T14:33:56.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.024+0000 7f12d7600640 1 -- start start 2024-04-23T14:33:56.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.024+0000 7f12d7600640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f12d015c7f0 0x7f12d00d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:56.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.024+0000 7f12d7600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12d00b2220 con 0x7f12d00cf350 2024-04-23T14:33:56.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.024+0000 7f12d7600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12d00b2500 con 0x7f12d00d2090 2024-04-23T14:33:56.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.024+0000 7f12d7600640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f12d015cbd0 con 0x7f12d015c7f0 2024-04-23T14:33:56.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.024+0000 7f12d5600640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f12d00cf350 0x7f12d00b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35494/0 (socket says 172.21.15.100:35494) 2024-04-23T14:33:56.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.024+0000 7f12d5600640 1 -- 172.21.15.100:0/3760085795 learned_addr learned my addr 172.21.15.100:0/3760085795 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:56.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.024+0000 7f12d4c00640 1 --2- 172.21.15.100:0/3760085795 >> v2:172.21.15.151:3300/0 conn(0x7f12d015c7f0 0x7f12d00d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:56.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.024+0000 7f12cf400640 1 -- 172.21.15.100:0/3760085795 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1539185895 0 0) 0x7f12d00b2220 con 0x7f12d00cf350 2024-04-23T14:33:56.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.024+0000 7f12cf400640 1 -- 172.21.15.100:0/3760085795 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12b4003530 con 0x7f12d00cf350 2024-04-23T14:33:56.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.024+0000 7f12d4c00640 1 -- 172.21.15.100:0/3760085795 >> v1:172.21.15.151:6789/0 conn(0x7f12d00d2090 legacy=0x7f12d00cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:56.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.024+0000 7f12d4c00640 1 -- 172.21.15.100:0/3760085795 >> v1:172.21.15.100:6789/0 conn(0x7f12d00cf350 legacy=0x7f12d00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:56.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.024+0000 7f12d4c00640 1 -- 172.21.15.100:0/3760085795 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d00b27e0 con 0x7f12d015c7f0 2024-04-23T14:33:56.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.028+0000 7f12d4c00640 1 --2- 172.21.15.100:0/3760085795 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d015c7f0 0x7f12d00d1b50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f12c00097c0 tx=0x7f12c001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23e07bd3be41adbc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:56.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.028+0000 7f12cf400640 1 -- 172.21.15.100:0/3760085795 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c00b0070 con 0x7f12d015c7f0 2024-04-23T14:33:56.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.028+0000 7f12cf400640 1 -- 172.21.15.100:0/3760085795 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12c00ab600 con 0x7f12d015c7f0 2024-04-23T14:33:56.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.028+0000 7f12cf400640 1 -- 172.21.15.100:0/3760085795 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c00ab9d0 con 0x7f12d015c7f0 2024-04-23T14:33:56.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.028+0000 7f12d7600640 1 -- 172.21.15.100:0/3760085795 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d015c7f0 msgr2=0x7f12d00d1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:56.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.028+0000 7f12d7600640 1 --2- 172.21.15.100:0/3760085795 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d015c7f0 0x7f12d00d1b50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f12c00097c0 tx=0x7f12c001c410 comp rx=0 tx=0).stop 2024-04-23T14:33:56.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.028+0000 7f12d7600640 1 -- 172.21.15.100:0/3760085795 shutdown_connections 2024-04-23T14:33:56.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.028+0000 7f12d7600640 1 --2- 172.21.15.100:0/3760085795 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d015c7f0 0x7f12d00d1b50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.028+0000 7f12d7600640 1 -- 172.21.15.100:0/3760085795 >> 172.21.15.100:0/3760085795 conn(0x7f12d00b4050 msgr2=0x7f12d00b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:56.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.028+0000 7f12d7600640 1 -- 172.21.15.100:0/3760085795 shutdown_connections 2024-04-23T14:33:56.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.028+0000 7f12d7600640 1 -- 172.21.15.100:0/3760085795 wait complete. 2024-04-23T14:33:56.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12d7600640 1 Processor -- start 2024-04-23T14:33:56.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12d7600640 1 -- start start 2024-04-23T14:33:56.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12d7600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12d00cf350 0x7f12d0153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:56.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12d7600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d015c7f0 0x7f12d0158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:56.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12d7600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12d0158fb0 0x7f12d00c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:56.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12d7600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f12d00b6700 con 0x7f12d0158fb0 2024-04-23T14:33:56.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12d7600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f12d00b6580 con 0x7f12d015c7f0 2024-04-23T14:33:56.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12d7600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f12d016a2c0 con 0x7f12d00cf350 2024-04-23T14:33:56.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12d5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12d0158fb0 0x7f12d00c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:56.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12d5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12d0158fb0 0x7f12d00c9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37046/0 (socket says 172.21.15.100:37046) 2024-04-23T14:33:56.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12d5600640 1 -- 172.21.15.100:0/1169688798 learned_addr learned my addr 172.21.15.100:0/1169688798 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:56.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12d4c00640 1 --2- 172.21.15.100:0/1169688798 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12d00cf350 0x7f12d0153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:56.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12cfe00640 1 --2- 172.21.15.100:0/1169688798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d015c7f0 0x7f12d0158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:56.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12d5600640 1 -- 172.21.15.100:0/1169688798 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12d00cf350 msgr2=0x7f12d0153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:56.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12d5600640 1 --2- 172.21.15.100:0/1169688798 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12d00cf350 0x7f12d0153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12d5600640 1 -- 172.21.15.100:0/1169688798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d015c7f0 msgr2=0x7f12d0158a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:56.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12d5600640 1 --2- 172.21.15.100:0/1169688798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d015c7f0 0x7f12d0158a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12d5600640 1 -- 172.21.15.100:0/1169688798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d00ca260 con 0x7f12d0158fb0 2024-04-23T14:33:56.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12cfe00640 1 --2- 172.21.15.100:0/1169688798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d015c7f0 0x7f12d0158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:56.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12d5600640 1 --2- 172.21.15.100:0/1169688798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12d0158fb0 0x7f12d00c9550 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f12c800caa0 tx=0x7f12c800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:56.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12ccc00640 1 -- 172.21.15.100:0/1169688798 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c8017070 con 0x7f12d0158fb0 2024-04-23T14:33:56.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12ccc00640 1 -- 172.21.15.100:0/1169688798 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12c800ac00 con 0x7f12d0158fb0 2024-04-23T14:33:56.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12ccc00640 1 -- 172.21.15.100:0/1169688798 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c8012400 con 0x7f12d0158fb0 2024-04-23T14:33:56.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12d7600640 1 -- 172.21.15.100:0/1169688798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12d00ca540 con 0x7f12d0158fb0 2024-04-23T14:33:56.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.032+0000 7f12d7600640 1 -- 172.21.15.100:0/1169688798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12d00cab00 con 0x7f12d0158fb0 2024-04-23T14:33:56.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.036+0000 7f12d7600640 1 -- 172.21.15.100:0/1169688798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12d0068ad0 con 0x7f12d0158fb0 2024-04-23T14:33:56.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.036+0000 7f12ccc00640 1 -- 172.21.15.100:0/1169688798 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f12c801a070 con 0x7f12d0158fb0 2024-04-23T14:33:56.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.036+0000 7f12ccc00640 1 --2- 172.21.15.100:0/1169688798 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f12b0068380 0x7f12b006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:56.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.036+0000 7f12ccc00640 1 -- 172.21.15.100:0/1169688798 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f12c8086a80 con 0x7f12d0158fb0 2024-04-23T14:33:56.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.036+0000 7f12d4c00640 1 --2- 172.21.15.100:0/1169688798 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f12b0068380 0x7f12b006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:56.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.036+0000 7f12d4c00640 1 --2- 172.21.15.100:0/1169688798 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f12b0068380 0x7f12b006a840 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f12c00ad000 tx=0x7f12c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:56.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.040+0000 7f12ccc00640 1 -- 172.21.15.100:0/1169688798 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f12c8088e20 con 0x7f12d0158fb0 2024-04-23T14:33:56.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.196+0000 7f12d7600640 1 -- 172.21.15.100:0/1169688798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12d00d0cc0 con 0x7f12d0158fb0 2024-04-23T14:33:56.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.200+0000 7f12ccc00640 1 -- 172.21.15.100:0/1169688798 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+18766 (secure 0 0 0) 0x7f12c8056d40 con 0x7f12d0158fb0 2024-04-23T14:33:56.204 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:56.204 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":41,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:33:49.609841+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:33:36.592437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-23T14:33:37.882390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-04-23T14:33:47.299378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:56.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.200+0000 7f12d7600640 1 -- 172.21.15.100:0/1169688798 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f12b0068380 msgr2=0x7f12b006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:56.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.200+0000 7f12d7600640 1 --2- 172.21.15.100:0/1169688798 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f12b0068380 0x7f12b006a840 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f12c00ad000 tx=0x7f12c0008040 comp rx=0 tx=0).stop 2024-04-23T14:33:56.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.200+0000 7f12d7600640 1 -- 172.21.15.100:0/1169688798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12d0158fb0 msgr2=0x7f12d00c9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:56.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.200+0000 7f12d7600640 1 --2- 172.21.15.100:0/1169688798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12d0158fb0 0x7f12d00c9550 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f12c800caa0 tx=0x7f12c800cf60 comp rx=0 tx=0).stop 2024-04-23T14:33:56.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.200+0000 7f12d7600640 1 -- 172.21.15.100:0/1169688798 shutdown_connections 2024-04-23T14:33:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.200+0000 7f12d7600640 1 --2- 172.21.15.100:0/1169688798 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f12b0068380 0x7f12b006a840 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.200+0000 7f12d7600640 1 --2- 172.21.15.100:0/1169688798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12d0158fb0 0x7f12d00c9550 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.200+0000 7f12d7600640 1 --2- 172.21.15.100:0/1169688798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d015c7f0 0x7f12d0158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.200+0000 7f12d7600640 1 --2- 172.21.15.100:0/1169688798 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12d00cf350 0x7f12d0153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.200+0000 7f12d7600640 1 -- 172.21.15.100:0/1169688798 >> 172.21.15.100:0/1169688798 conn(0x7f12d00b4050 msgr2=0x7f12d015d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.200+0000 7f12d7600640 1 -- 172.21.15.100:0/1169688798 shutdown_connections 2024-04-23T14:33:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.200+0000 7f12d7600640 1 -- 172.21.15.100:0/1169688798 wait complete. 2024-04-23T14:33:56.226 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:33:36.592437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:33:37.882390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-04-23T14:33:47.299378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:33:56.226 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:33:56.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.360+0000 7fcf01e00640 1 Processor -- start 2024-04-23T14:33:56.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.360+0000 7fcf01e00640 1 -- start start 2024-04-23T14:33:56.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.360+0000 7fcf01e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcefc15e9c0 0x7fcefc0cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:56.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.360+0000 7fcf01e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fcefc06d450 0x7fcefc06d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:56.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.360+0000 7fcf01e00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcefc0ccaf0 con 0x7fcefc0718f0 2024-04-23T14:33:56.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.360+0000 7fcf01e00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fcefc15eda0 con 0x7fcefc06d450 2024-04-23T14:33:56.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.360+0000 7fcf01e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fcefc0ccdd0 con 0x7fcefc15e9c0 2024-04-23T14:33:56.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.360+0000 7fcef2000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fcefc06d450 0x7fcefc06d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:56.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.360+0000 7fcefaa00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcefc15e9c0 0x7fcefc0cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:56.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.360+0000 7fcef2000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fcefc06d450 0x7fcefc06d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:48610/0 (socket says 172.21.15.100:48610) 2024-04-23T14:33:56.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.360+0000 7fcef2000640 1 -- 172.21.15.100:0/3906466219 learned_addr learned my addr 172.21.15.100:0/3906466219 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:56.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.360+0000 7fcef2000640 1 -- 172.21.15.100:0/3906466219 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcefc15e9c0 msgr2=0x7fcefc0cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:56.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.360+0000 7fcef2000640 1 --2- 172.21.15.100:0/3906466219 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcefc15e9c0 0x7fcefc0cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.360+0000 7fcef2000640 1 -- 172.21.15.100:0/3906466219 >> v1:172.21.15.151:6789/0 conn(0x7fcefc0718f0 legacy=0x7fcefc0d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:56.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.360+0000 7fcef2000640 1 -- 172.21.15.100:0/3906466219 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcefc0cd0b0 con 0x7fcefc06d450 2024-04-23T14:33:56.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.364+0000 7fcef2000640 1 --2- 172.21.15.100:0/3906466219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcefc06d450 0x7fcefc06d8b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009870 tx=0x7fcee401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71ba65b4c1eec239 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:56.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.364+0000 7fcefa000640 1 -- 172.21.15.100:0/3906466219 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee40ab600 con 0x7fcefc06d450 2024-04-23T14:33:56.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.364+0000 7fcefa000640 1 -- 172.21.15.100:0/3906466219 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcee40b0070 con 0x7fcefc06d450 2024-04-23T14:33:56.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.364+0000 7fcefa000640 1 -- 172.21.15.100:0/3906466219 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee40ab9c0 con 0x7fcefc06d450 2024-04-23T14:33:56.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.364+0000 7fcf01e00640 1 -- 172.21.15.100:0/3906466219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcefc06d450 msgr2=0x7fcefc06d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:56.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.364+0000 7fcf01e00640 1 --2- 172.21.15.100:0/3906466219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcefc06d450 0x7fcefc06d8b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009870 tx=0x7fcee401c4a0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.364+0000 7fcf01e00640 1 -- 172.21.15.100:0/3906466219 shutdown_connections 2024-04-23T14:33:56.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.364+0000 7fcf01e00640 1 --2- 172.21.15.100:0/3906466219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcefc06d450 0x7fcefc06d8b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.364+0000 7fcf01e00640 1 --2- 172.21.15.100:0/3906466219 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcefc15e9c0 0x7fcefc0cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.364+0000 7fcf01e00640 1 -- 172.21.15.100:0/3906466219 >> 172.21.15.100:0/3906466219 conn(0x7fcefc0d2400 msgr2=0x7fcefc0d2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:56.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.364+0000 7fcf01e00640 1 -- 172.21.15.100:0/3906466219 shutdown_connections 2024-04-23T14:33:56.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.364+0000 7fcf01e00640 1 -- 172.21.15.100:0/3906466219 wait complete. 2024-04-23T14:33:56.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcf01e00640 1 Processor -- start 2024-04-23T14:33:56.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcf01e00640 1 -- start start 2024-04-23T14:33:56.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcf01e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcefc06d450 0x7fcefc152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:56.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcf01e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcefc0718f0 0x7fcefc0c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:56.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcf01e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcefc15e9c0 0x7fcefc0c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:56.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcf01e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc072110 con 0x7fcefc0718f0 2024-04-23T14:33:56.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcf01e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc0d0e20 con 0x7fcefc15e9c0 2024-04-23T14:33:56.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcf01e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fcefc0b6890 con 0x7fcefc06d450 2024-04-23T14:33:56.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcef2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcefc0718f0 0x7fcefc0c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:56.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcef2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcefc0718f0 0x7fcefc0c5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37058/0 (socket says 172.21.15.100:37058) 2024-04-23T14:33:56.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcef2000640 1 -- 172.21.15.100:0/316507288 learned_addr learned my addr 172.21.15.100:0/316507288 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:56.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcefaa00640 1 --2- 172.21.15.100:0/316507288 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcefc06d450 0x7fcefc152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:56.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcef2000640 1 -- 172.21.15.100:0/316507288 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcefc06d450 msgr2=0x7fcefc152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:56.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcefb400640 1 --2- 172.21.15.100:0/316507288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcefc15e9c0 0x7fcefc0c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:56.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcef2000640 1 --2- 172.21.15.100:0/316507288 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcefc06d450 0x7fcefc152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcef2000640 1 -- 172.21.15.100:0/316507288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcefc15e9c0 msgr2=0x7fcefc0c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:56.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcef2000640 1 --2- 172.21.15.100:0/316507288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcefc15e9c0 0x7fcefc0c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcef2000640 1 -- 172.21.15.100:0/316507288 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcefc0c88e0 con 0x7fcefc0718f0 2024-04-23T14:33:56.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcefaa00640 1 --2- 172.21.15.100:0/316507288 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcefc06d450 0x7fcefc152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:56.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcefb400640 1 --2- 172.21.15.100:0/316507288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcefc15e9c0 0x7fcefc0c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:56.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcef2000640 1 --2- 172.21.15.100:0/316507288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcefc0718f0 0x7fcefc0c5f30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fcee401ab50 tx=0x7fcee401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:56.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcef3e00640 1 -- 172.21.15.100:0/316507288 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee40be5f0 con 0x7fcefc0718f0 2024-04-23T14:33:56.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcef3e00640 1 -- 172.21.15.100:0/316507288 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcee40be7b0 con 0x7fcefc0718f0 2024-04-23T14:33:56.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcef3e00640 1 -- 172.21.15.100:0/316507288 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee40beba0 con 0x7fcefc0718f0 2024-04-23T14:33:56.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcf01e00640 1 -- 172.21.15.100:0/316507288 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcefc0b5c80 con 0x7fcefc0718f0 2024-04-23T14:33:56.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.368+0000 7fcf01e00640 1 -- 172.21.15.100:0/316507288 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcefc0b6240 con 0x7fcefc0718f0 2024-04-23T14:33:56.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.372+0000 7fcf01e00640 1 -- 172.21.15.100:0/316507288 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcefc0b6a10 con 0x7fcefc0718f0 2024-04-23T14:33:56.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.372+0000 7fcef3e00640 1 -- 172.21.15.100:0/316507288 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcee40aa070 con 0x7fcefc0718f0 2024-04-23T14:33:56.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.372+0000 7fcef3e00640 1 --2- 172.21.15.100:0/316507288 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fced80683b0 0x7fced806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:56.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.372+0000 7fcef3e00640 1 -- 172.21.15.100:0/316507288 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fcee4120990 con 0x7fcefc0718f0 2024-04-23T14:33:56.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.372+0000 7fcefaa00640 1 --2- 172.21.15.100:0/316507288 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fced80683b0 0x7fced806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:56.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.372+0000 7fcefaa00640 1 --2- 172.21.15.100:0/316507288 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fced80683b0 0x7fced806a870 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fceec009ae0 tx=0x7fceec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:56.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.376+0000 7fcef3e00640 1 -- 172.21.15.100:0/316507288 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcefc0c88e0 con 0x7fcefc0718f0 2024-04-23T14:33:56.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.532+0000 7fcf01e00640 1 -- 172.21.15.100:0/316507288 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcefc068ad0 con 0x7fcefc0718f0 2024-04-23T14:33:56.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.532+0000 7fcef3e00640 1 -- 172.21.15.100:0/316507288 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7fcee4016150 con 0x7fcefc0718f0 2024-04-23T14:33:56.538 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:33:56.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.536+0000 7fcf01e00640 1 -- 172.21.15.100:0/316507288 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fced80683b0 msgr2=0x7fced806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:56.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.536+0000 7fcf01e00640 1 --2- 172.21.15.100:0/316507288 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fced80683b0 0x7fced806a870 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fceec009ae0 tx=0x7fceec008040 comp rx=0 tx=0).stop 2024-04-23T14:33:56.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.536+0000 7fcf01e00640 1 -- 172.21.15.100:0/316507288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcefc0718f0 msgr2=0x7fcefc0c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:56.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.536+0000 7fcf01e00640 1 --2- 172.21.15.100:0/316507288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcefc0718f0 0x7fcefc0c5f30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fcee401ab50 tx=0x7fcee401ace0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.536+0000 7fcf01e00640 1 -- 172.21.15.100:0/316507288 shutdown_connections 2024-04-23T14:33:56.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.536+0000 7fcf01e00640 1 --2- 172.21.15.100:0/316507288 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fced80683b0 0x7fced806a870 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.536+0000 7fcf01e00640 1 --2- 172.21.15.100:0/316507288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcefc15e9c0 0x7fcefc0c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.536+0000 7fcf01e00640 1 --2- 172.21.15.100:0/316507288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcefc0718f0 0x7fcefc0c5f30 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.536+0000 7fcf01e00640 1 --2- 172.21.15.100:0/316507288 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcefc06d450 0x7fcefc152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.536+0000 7fcf01e00640 1 -- 172.21.15.100:0/316507288 >> 172.21.15.100:0/316507288 conn(0x7fcefc0d2400 msgr2=0x7fcefc0d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:56.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.536+0000 7fcf01e00640 1 -- 172.21.15.100:0/316507288 shutdown_connections 2024-04-23T14:33:56.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.536+0000 7fcf01e00640 1 -- 172.21.15.100:0/316507288 wait complete. 2024-04-23T14:33:56.558 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:33:56.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.696+0000 7f4539c00640 1 Processor -- start 2024-04-23T14:33:56.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.696+0000 7f4539c00640 1 -- start start 2024-04-23T14:33:56.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.696+0000 7f4539c00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f453415c7a0 0x7f45340af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:56.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.696+0000 7f4539c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f45340cd300 0x7f45340ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:56.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.696+0000 7f4539c00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f453406cbb0 con 0x7f453406bb20 2024-04-23T14:33:56.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.696+0000 7f4539c00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f453406d450 con 0x7f45340cd300 2024-04-23T14:33:56.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.696+0000 7f4539c00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f453406ce90 con 0x7f453415c7a0 2024-04-23T14:33:56.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.696+0000 7f4533400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f45340cd300 0x7f45340ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:56.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.696+0000 7f4533400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f45340cd300 0x7f45340ad430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37072/0 (socket says 172.21.15.100:37072) 2024-04-23T14:33:56.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.696+0000 7f4533400640 1 -- 172.21.15.100:0/3838727248 learned_addr learned my addr 172.21.15.100:0/3838727248 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:56.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.696+0000 7f4532a00640 1 --2- 172.21.15.100:0/3838727248 >> v2:172.21.15.151:3300/0 conn(0x7f453415c7a0 0x7f45340af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:56.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.696+0000 7f4533400640 1 -- 172.21.15.100:0/3838727248 >> v2:172.21.15.151:3300/0 conn(0x7f453415c7a0 msgr2=0x7f45340af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:56.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.696+0000 7f4533400640 1 --2- 172.21.15.100:0/3838727248 >> v2:172.21.15.151:3300/0 conn(0x7f453415c7a0 0x7f45340af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.696+0000 7f4533400640 1 -- 172.21.15.100:0/3838727248 >> v1:172.21.15.151:6789/0 conn(0x7f453406bb20 legacy=0x7f45340ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:56.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.696+0000 7f4533400640 1 -- 172.21.15.100:0/3838727248 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453406d170 con 0x7f45340cd300 2024-04-23T14:33:56.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.700+0000 7f4533400640 1 --2- 172.21.15.100:0/3838727248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45340cd300 0x7f45340ad430 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f45340716b0 tx=0x7f452c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3665d9b1da7d42 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:56.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.700+0000 7f4532000640 1 -- 172.21.15.100:0/3838727248 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c0a5620 con 0x7f45340cd300 2024-04-23T14:33:56.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.700+0000 7f4532000640 1 -- 172.21.15.100:0/3838727248 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f452c0a57c0 con 0x7f45340cd300 2024-04-23T14:33:56.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.700+0000 7f4532000640 1 -- 172.21.15.100:0/3838727248 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c0a5b90 con 0x7f45340cd300 2024-04-23T14:33:56.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.700+0000 7f4539c00640 1 -- 172.21.15.100:0/3838727248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45340cd300 msgr2=0x7f45340ad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:56.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.700+0000 7f4539c00640 1 --2- 172.21.15.100:0/3838727248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45340cd300 0x7f45340ad430 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f45340716b0 tx=0x7f452c01c960 comp rx=0 tx=0).stop 2024-04-23T14:33:56.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.700+0000 7f4539c00640 1 -- 172.21.15.100:0/3838727248 shutdown_connections 2024-04-23T14:33:56.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.700+0000 7f4539c00640 1 --2- 172.21.15.100:0/3838727248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45340cd300 0x7f45340ad430 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.700+0000 7f4539c00640 1 --2- 172.21.15.100:0/3838727248 >> v2:172.21.15.151:3300/0 conn(0x7f453415c7a0 0x7f45340af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.700+0000 7f4539c00640 1 -- 172.21.15.100:0/3838727248 >> 172.21.15.100:0/3838727248 conn(0x7f45340b5200 msgr2=0x7f45340b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:56.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.700+0000 7f4539c00640 1 -- 172.21.15.100:0/3838727248 shutdown_connections 2024-04-23T14:33:56.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.700+0000 7f4539c00640 1 -- 172.21.15.100:0/3838727248 wait complete. 2024-04-23T14:33:56.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f4539c00640 1 Processor -- start 2024-04-23T14:33:56.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f4539c00640 1 -- start start 2024-04-23T14:33:56.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f4539c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f453406bb20 0x7f4534157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:56.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f4539c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f45340cd300 0x7f453415da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:56.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f4539c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f453415c7a0 0x7f4534157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:56.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f4539c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f45340ae590 con 0x7f453415c7a0 2024-04-23T14:33:56.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f4539c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f45340ae410 con 0x7f453406bb20 2024-04-23T14:33:56.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f4539c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f45340ae710 con 0x7f45340cd300 2024-04-23T14:33:56.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f4533400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f453415c7a0 0x7f4534157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:56.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f4533400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f453415c7a0 0x7f4534157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37082/0 (socket says 172.21.15.100:37082) 2024-04-23T14:33:56.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f4533400640 1 -- 172.21.15.100:0/4292928574 learned_addr learned my addr 172.21.15.100:0/4292928574 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:56.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f4532a00640 1 --2- 172.21.15.100:0/4292928574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f453406bb20 0x7f4534157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:56.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f452a000640 1 --2- 172.21.15.100:0/4292928574 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f45340cd300 0x7f453415da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:56.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f4533400640 1 -- 172.21.15.100:0/4292928574 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f45340cd300 msgr2=0x7f453415da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:56.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f4533400640 1 --2- 172.21.15.100:0/4292928574 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f45340cd300 0x7f453415da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f4533400640 1 -- 172.21.15.100:0/4292928574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f453406bb20 msgr2=0x7f4534157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:56.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f4533400640 1 --2- 172.21.15.100:0/4292928574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f453406bb20 0x7f4534157730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f4533400640 1 -- 172.21.15.100:0/4292928574 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453415dfb0 con 0x7f453415c7a0 2024-04-23T14:33:56.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f4532a00640 1 --2- 172.21.15.100:0/4292928574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f453406bb20 0x7f4534157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:56.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f452a000640 1 --2- 172.21.15.100:0/4292928574 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f45340cd300 0x7f453415da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:56.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f4533400640 1 --2- 172.21.15.100:0/4292928574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f453415c7a0 0x7f4534157d30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f452c0af040 tx=0x7f452c019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:56.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f452b400640 1 -- 172.21.15.100:0/4292928574 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c0bc020 con 0x7f453415c7a0 2024-04-23T14:33:56.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f452b400640 1 -- 172.21.15.100:0/4292928574 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f452c0b88a0 con 0x7f453415c7a0 2024-04-23T14:33:56.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f452b400640 1 -- 172.21.15.100:0/4292928574 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c0a5620 con 0x7f453415c7a0 2024-04-23T14:33:56.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f4539c00640 1 -- 172.21.15.100:0/4292928574 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f453415f5d0 con 0x7f453415c7a0 2024-04-23T14:33:56.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f4539c00640 1 -- 172.21.15.100:0/4292928574 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f453415ccb0 con 0x7f453415c7a0 2024-04-23T14:33:56.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.704+0000 7f4539c00640 1 -- 172.21.15.100:0/4292928574 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4534068ad0 con 0x7f453415c7a0 2024-04-23T14:33:56.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.708+0000 7f452b400640 1 -- 172.21.15.100:0/4292928574 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f452c0bf020 con 0x7f453415c7a0 2024-04-23T14:33:56.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.708+0000 7f452b400640 1 --2- 172.21.15.100:0/4292928574 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f45100683b0 0x7f451006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:56.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.708+0000 7f452b400640 1 -- 172.21.15.100:0/4292928574 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f452c121dd0 con 0x7f453415c7a0 2024-04-23T14:33:56.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.708+0000 7f4532a00640 1 --2- 172.21.15.100:0/4292928574 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f45100683b0 0x7f451006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:56.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.708+0000 7f4532a00640 1 --2- 172.21.15.100:0/4292928574 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f45100683b0 0x7f451006a870 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f45340b2d50 tx=0x7f4524008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:56.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.712+0000 7f452b400640 1 -- 172.21.15.100:0/4292928574 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f452c0c6030 con 0x7f453415c7a0 2024-04-23T14:33:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.872+0000 7f4539c00640 1 -- 172.21.15.100:0/4292928574 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4534073c10 con 0x7f453415c7a0 2024-04-23T14:33:56.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.872+0000 7f452b400640 1 -- 172.21.15.100:0/4292928574 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) ==== 104+0+11 (secure 0 0 0) 0x7f452c0165e0 con 0x7f453415c7a0 2024-04-23T14:33:56.876 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.872+0000 7f4539c00640 1 -- 172.21.15.100:0/4292928574 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f45100683b0 msgr2=0x7f451006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.872+0000 7f4539c00640 1 --2- 172.21.15.100:0/4292928574 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f45100683b0 0x7f451006a870 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f45340b2d50 tx=0x7f4524008040 comp rx=0 tx=0).stop 2024-04-23T14:33:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.872+0000 7f4539c00640 1 -- 172.21.15.100:0/4292928574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f453415c7a0 msgr2=0x7f4534157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.872+0000 7f4539c00640 1 --2- 172.21.15.100:0/4292928574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f453415c7a0 0x7f4534157d30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f452c0af040 tx=0x7f452c019700 comp rx=0 tx=0).stop 2024-04-23T14:33:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.872+0000 7f4539c00640 1 -- 172.21.15.100:0/4292928574 shutdown_connections 2024-04-23T14:33:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.872+0000 7f4539c00640 1 --2- 172.21.15.100:0/4292928574 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f45100683b0 0x7f451006a870 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.872+0000 7f4539c00640 1 --2- 172.21.15.100:0/4292928574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f453415c7a0 0x7f4534157d30 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.872+0000 7f4539c00640 1 --2- 172.21.15.100:0/4292928574 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f45340cd300 0x7f453415da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.872+0000 7f4539c00640 1 --2- 172.21.15.100:0/4292928574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f453406bb20 0x7f4534157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:56.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.872+0000 7f4539c00640 1 -- 172.21.15.100:0/4292928574 >> 172.21.15.100:0/4292928574 conn(0x7f45340b5200 msgr2=0x7f453406e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:56.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.872+0000 7f4539c00640 1 -- 172.21.15.100:0/4292928574 shutdown_connections 2024-04-23T14:33:56.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:56.872+0000 7f4539c00640 1 -- 172.21.15.100:0/4292928574 wait complete. 2024-04-23T14:33:56.897 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:33:57.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.028+0000 7fe5df200640 1 Processor -- start 2024-04-23T14:33:57.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.028+0000 7fe5df200640 1 -- start start 2024-04-23T14:33:57.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.032+0000 7fe5df200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe5d80cca50 0x7fe5d80cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.032+0000 7fe5df200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fe5d815d600 0x7fe5d8160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.032+0000 7fe5df200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5d80cd370 con 0x7fe5d815c7e0 2024-04-23T14:33:57.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.032+0000 7fe5df200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fe5d80d2270 con 0x7fe5d815d600 2024-04-23T14:33:57.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.032+0000 7fe5df200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe5d815db00 con 0x7fe5d80cca50 2024-04-23T14:33:57.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.032+0000 7fe5cf400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe5d815c7e0 0x7fe5d8160170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35504/0 (socket says 172.21.15.100:35504) 2024-04-23T14:33:57.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.032+0000 7fe5cf400640 1 -- 172.21.15.100:0/2962825200 learned_addr learned my addr 172.21.15.100:0/2962825200 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:57.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.032+0000 7fe5d7e00640 1 --2- 172.21.15.100:0/2962825200 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe5d80cca50 0x7fe5d80cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.032+0000 7fe5dd200640 1 --2- 172.21.15.100:0/2962825200 >> v2:172.21.15.151:3300/0 conn(0x7fe5d815d600 0x7fe5d8160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.032+0000 7fe5d7400640 1 -- 172.21.15.100:0/2962825200 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 967122306 0 0) 0x7fe5d80cd370 con 0x7fe5d815c7e0 2024-04-23T14:33:57.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.032+0000 7fe5d7400640 1 -- 172.21.15.100:0/2962825200 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5bc003530 con 0x7fe5d815c7e0 2024-04-23T14:33:57.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.032+0000 7fe5d7e00640 1 -- 172.21.15.100:0/2962825200 >> v2:172.21.15.151:3300/0 conn(0x7fe5d815d600 msgr2=0x7fe5d8160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:57.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.032+0000 7fe5d7e00640 1 --2- 172.21.15.100:0/2962825200 >> v2:172.21.15.151:3300/0 conn(0x7fe5d815d600 0x7fe5d8160880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.032+0000 7fe5d7e00640 1 -- 172.21.15.100:0/2962825200 >> v1:172.21.15.100:6789/0 conn(0x7fe5d815c7e0 legacy=0x7fe5d8160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:57.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.032+0000 7fe5d7e00640 1 -- 172.21.15.100:0/2962825200 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d80d04a0 con 0x7fe5d80cca50 2024-04-23T14:33:57.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.036+0000 7fe5d7e00640 1 --2- 172.21.15.100:0/2962825200 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe5d80cca50 0x7fe5d80cce30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009d90 tx=0x7fe5c801c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6015ba336491968b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:57.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.036+0000 7fe5d7400640 1 -- 172.21.15.100:0/2962825200 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c80b2070 con 0x7fe5d80cca50 2024-04-23T14:33:57.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.036+0000 7fe5d7400640 1 -- 172.21.15.100:0/2962825200 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c80a5970 con 0x7fe5d80cca50 2024-04-23T14:33:57.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.036+0000 7fe5d7400640 1 -- 172.21.15.100:0/2962825200 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c80a5d40 con 0x7fe5d80cca50 2024-04-23T14:33:57.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.036+0000 7fe5df200640 1 -- 172.21.15.100:0/2962825200 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe5d80cca50 msgr2=0x7fe5d80cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:57.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.036+0000 7fe5df200640 1 --2- 172.21.15.100:0/2962825200 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe5d80cca50 0x7fe5d80cce30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009d90 tx=0x7fe5c801c8b0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.036+0000 7fe5df200640 1 -- 172.21.15.100:0/2962825200 shutdown_connections 2024-04-23T14:33:57.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.036+0000 7fe5df200640 1 --2- 172.21.15.100:0/2962825200 >> v2:172.21.15.151:3300/0 conn(0x7fe5d815d600 0x7fe5d8160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.036+0000 7fe5df200640 1 --2- 172.21.15.100:0/2962825200 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe5d80cca50 0x7fe5d80cce30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.036+0000 7fe5df200640 1 -- 172.21.15.100:0/2962825200 >> 172.21.15.100:0/2962825200 conn(0x7fe5d80cee80 msgr2=0x7fe5d80cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:57.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.036+0000 7fe5df200640 1 -- 172.21.15.100:0/2962825200 shutdown_connections 2024-04-23T14:33:57.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.036+0000 7fe5df200640 1 -- 172.21.15.100:0/2962825200 wait complete. 2024-04-23T14:33:57.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.036+0000 7fe5df200640 1 Processor -- start 2024-04-23T14:33:57.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.036+0000 7fe5df200640 1 -- start start 2024-04-23T14:33:57.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5df200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5d80cca50 0x7fe5d814fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5df200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe5d815c7e0 0x7fe5d80c7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5df200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5d815d600 0x7fe5d8165690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5df200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d80b5430 con 0x7fe5d815d600 2024-04-23T14:33:57.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5df200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d80b52b0 con 0x7fe5d80cca50 2024-04-23T14:33:57.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5df200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe5d80b3bf0 con 0x7fe5d815c7e0 2024-04-23T14:33:57.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5dd200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5d815d600 0x7fe5d8165690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5dd200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5d815d600 0x7fe5d8165690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37088/0 (socket says 172.21.15.100:37088) 2024-04-23T14:33:57.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5dd200640 1 -- 172.21.15.100:0/2608939134 learned_addr learned my addr 172.21.15.100:0/2608939134 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:57.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5d7e00640 1 --2- 172.21.15.100:0/2608939134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5d80cca50 0x7fe5d814fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5cf400640 1 --2- 172.21.15.100:0/2608939134 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe5d815c7e0 0x7fe5d80c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5dd200640 1 -- 172.21.15.100:0/2608939134 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe5d815c7e0 msgr2=0x7fe5d80c7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:57.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5dd200640 1 --2- 172.21.15.100:0/2608939134 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe5d815c7e0 0x7fe5d80c7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5dd200640 1 -- 172.21.15.100:0/2608939134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5d80cca50 msgr2=0x7fe5d814fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:57.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5dd200640 1 --2- 172.21.15.100:0/2608939134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5d80cca50 0x7fe5d814fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5dd200640 1 -- 172.21.15.100:0/2608939134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d80b50e0 con 0x7fe5d815d600 2024-04-23T14:33:57.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5cf400640 1 --2- 172.21.15.100:0/2608939134 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe5d815c7e0 0x7fe5d80c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:57.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5dd200640 1 --2- 172.21.15.100:0/2608939134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5d815d600 0x7fe5d8165690 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000e880 tx=0x7fe5d000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:57.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5d4c00640 1 -- 172.21.15.100:0/2608939134 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0017070 con 0x7fe5d815d600 2024-04-23T14:33:57.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5d4c00640 1 -- 172.21.15.100:0/2608939134 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5d000c8e0 con 0x7fe5d815d600 2024-04-23T14:33:57.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5d4c00640 1 -- 172.21.15.100:0/2608939134 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d000ccb0 con 0x7fe5d815d600 2024-04-23T14:33:57.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5df200640 1 -- 172.21.15.100:0/2608939134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d80c63c0 con 0x7fe5d815d600 2024-04-23T14:33:57.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5df200640 1 -- 172.21.15.100:0/2608939134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d80c6980 con 0x7fe5d815d600 2024-04-23T14:33:57.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5df200640 1 -- 172.21.15.100:0/2608939134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5d8068ad0 con 0x7fe5d815d600 2024-04-23T14:33:57.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.040+0000 7fe5d4c00640 1 -- 172.21.15.100:0/2608939134 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe5d001a070 con 0x7fe5d815d600 2024-04-23T14:33:57.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.044+0000 7fe5d4c00640 1 --2- 172.21.15.100:0/2608939134 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe5b4068380 0x7fe5b406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.044+0000 7fe5d4c00640 1 -- 172.21.15.100:0/2608939134 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fe5d0086fc0 con 0x7fe5d815d600 2024-04-23T14:33:57.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.044+0000 7fe5d7e00640 1 --2- 172.21.15.100:0/2608939134 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe5b4068380 0x7fe5b406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.044+0000 7fe5d7e00640 1 --2- 172.21.15.100:0/2608939134 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe5b4068380 0x7fe5b406a840 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe5c80af000 tx=0x7fe5c80a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:57.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.048+0000 7fe5d4c00640 1 -- 172.21.15.100:0/2608939134 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe5d0056c20 con 0x7fe5d815d600 2024-04-23T14:33:57.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.204+0000 7fe5df200640 1 -- 172.21.15.100:0/2608939134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe5d80ce0d0 con 0x7fe5d815d600 2024-04-23T14:33:57.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.204+0000 7fe5d4c00640 1 -- 172.21.15.100:0/2608939134 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) ==== 100+0+11 (secure 0 0 0) 0x7fe5d005af20 con 0x7fe5d815d600 2024-04-23T14:33:57.209 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:33:57.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.204+0000 7fe5df200640 1 -- 172.21.15.100:0/2608939134 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe5b4068380 msgr2=0x7fe5b406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:57.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.204+0000 7fe5df200640 1 --2- 172.21.15.100:0/2608939134 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe5b4068380 0x7fe5b406a840 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe5c80af000 tx=0x7fe5c80a7070 comp rx=0 tx=0).stop 2024-04-23T14:33:57.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.204+0000 7fe5df200640 1 -- 172.21.15.100:0/2608939134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5d815d600 msgr2=0x7fe5d8165690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:57.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.204+0000 7fe5df200640 1 --2- 172.21.15.100:0/2608939134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5d815d600 0x7fe5d8165690 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000e880 tx=0x7fe5d000ed40 comp rx=0 tx=0).stop 2024-04-23T14:33:57.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.204+0000 7fe5df200640 1 -- 172.21.15.100:0/2608939134 shutdown_connections 2024-04-23T14:33:57.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.204+0000 7fe5df200640 1 --2- 172.21.15.100:0/2608939134 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe5b4068380 0x7fe5b406a840 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.204+0000 7fe5df200640 1 --2- 172.21.15.100:0/2608939134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5d815d600 0x7fe5d8165690 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.204+0000 7fe5df200640 1 --2- 172.21.15.100:0/2608939134 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe5d815c7e0 0x7fe5d80c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.204+0000 7fe5df200640 1 --2- 172.21.15.100:0/2608939134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5d80cca50 0x7fe5d814fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.204+0000 7fe5df200640 1 -- 172.21.15.100:0/2608939134 >> 172.21.15.100:0/2608939134 conn(0x7fe5d80cee80 msgr2=0x7fe5d80cce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:57.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.204+0000 7fe5df200640 1 -- 172.21.15.100:0/2608939134 shutdown_connections 2024-04-23T14:33:57.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.204+0000 7fe5df200640 1 -- 172.21.15.100:0/2608939134 wait complete. 2024-04-23T14:33:57.230 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-04-23T14:33:57.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.364+0000 7ff2ce800640 1 Processor -- start 2024-04-23T14:33:57.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.364+0000 7ff2ce800640 1 -- start start 2024-04-23T14:33:57.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.364+0000 7ff2ce800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff2c80c31b0 0x7ff2c806e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.364+0000 7ff2ce800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff2c80cfc10 0x7ff2c80c3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.364+0000 7ff2ce800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c80cf740 con 0x7ff2c8166730 2024-04-23T14:33:57.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.364+0000 7ff2ce800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff2c80c3ad0 con 0x7ff2c80c31b0 2024-04-23T14:33:57.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.364+0000 7ff2ce800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7ff2c80cda50 con 0x7ff2c80cfc10 2024-04-23T14:33:57.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.364+0000 7ff2c7400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff2c80c31b0 0x7ff2c806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.364+0000 7ff2c7400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff2c80c31b0 0x7ff2c806e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37092/0 (socket says 172.21.15.100:37092) 2024-04-23T14:33:57.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.364+0000 7ff2c7400640 1 -- 172.21.15.100:0/2129742215 learned_addr learned my addr 172.21.15.100:0/2129742215 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:57.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.364+0000 7ff2c6a00640 1 --2- 172.21.15.100:0/2129742215 >> v2:172.21.15.151:3300/0 conn(0x7ff2c80cfc10 0x7ff2c80c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.364+0000 7ff2c7400640 1 -- 172.21.15.100:0/2129742215 >> v2:172.21.15.151:3300/0 conn(0x7ff2c80cfc10 msgr2=0x7ff2c80c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:57.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.364+0000 7ff2c7400640 1 --2- 172.21.15.100:0/2129742215 >> v2:172.21.15.151:3300/0 conn(0x7ff2c80cfc10 0x7ff2c80c3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.364+0000 7ff2c7400640 1 -- 172.21.15.100:0/2129742215 >> v1:172.21.15.151:6789/0 conn(0x7ff2c8166730 legacy=0x7ff2c8166bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:57.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.364+0000 7ff2c7400640 1 -- 172.21.15.100:0/2129742215 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c80cced0 con 0x7ff2c80c31b0 2024-04-23T14:33:57.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.364+0000 7ff2c6a00640 1 --2- 172.21.15.100:0/2129742215 >> v2:172.21.15.151:3300/0 conn(0x7ff2c80cfc10 0x7ff2c80c3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:57.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.368+0000 7ff2c7400640 1 --2- 172.21.15.100:0/2129742215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2c80c31b0 0x7ff2c806e770 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff2c806dae0 tx=0x7ff2b801c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16014e1f075aca2f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:57.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.368+0000 7ff2c6000640 1 -- 172.21.15.100:0/2129742215 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80b0070 con 0x7ff2c80c31b0 2024-04-23T14:33:57.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.368+0000 7ff2c6000640 1 -- 172.21.15.100:0/2129742215 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b80ab600 con 0x7ff2c80c31b0 2024-04-23T14:33:57.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.368+0000 7ff2c6000640 1 -- 172.21.15.100:0/2129742215 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80ab9d0 con 0x7ff2c80c31b0 2024-04-23T14:33:57.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.368+0000 7ff2ce800640 1 -- 172.21.15.100:0/2129742215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2c80c31b0 msgr2=0x7ff2c806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:57.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.368+0000 7ff2ce800640 1 --2- 172.21.15.100:0/2129742215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2c80c31b0 0x7ff2c806e770 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff2c806dae0 tx=0x7ff2b801c2d0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.368+0000 7ff2ce800640 1 -- 172.21.15.100:0/2129742215 shutdown_connections 2024-04-23T14:33:57.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.368+0000 7ff2ce800640 1 --2- 172.21.15.100:0/2129742215 >> v2:172.21.15.151:3300/0 conn(0x7ff2c80cfc10 0x7ff2c80c3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.368+0000 7ff2ce800640 1 --2- 172.21.15.100:0/2129742215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2c80c31b0 0x7ff2c806e770 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.368+0000 7ff2ce800640 1 -- 172.21.15.100:0/2129742215 >> 172.21.15.100:0/2129742215 conn(0x7ff2c80b5200 msgr2=0x7ff2c80b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:57.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.368+0000 7ff2ce800640 1 -- 172.21.15.100:0/2129742215 shutdown_connections 2024-04-23T14:33:57.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.368+0000 7ff2ce800640 1 -- 172.21.15.100:0/2129742215 wait complete. 2024-04-23T14:33:57.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.368+0000 7ff2ce800640 1 Processor -- start 2024-04-23T14:33:57.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.372+0000 7ff2ce800640 1 -- start start 2024-04-23T14:33:57.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.372+0000 7ff2ce800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2c80c31b0 0x7ff2c80c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.372+0000 7ff2ce800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c80cfc10 0x7ff2c81624b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.372+0000 7ff2ce800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2c8166730 0x7ff2c815c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.372+0000 7ff2ce800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c806bab0 con 0x7ff2c80c31b0 2024-04-23T14:33:57.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.372+0000 7ff2ce800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c8162f70 con 0x7ff2c80cfc10 2024-04-23T14:33:57.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.372+0000 7ff2ce800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c806bc30 con 0x7ff2c8166730 2024-04-23T14:33:57.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.372+0000 7ff2c7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2c80c31b0 0x7ff2c80c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.372+0000 7ff2c7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2c80c31b0 0x7ff2c80c5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37094/0 (socket says 172.21.15.100:37094) 2024-04-23T14:33:57.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.372+0000 7ff2c7400640 1 -- 172.21.15.100:0/99274053 learned_addr learned my addr 172.21.15.100:0/99274053 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:57.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.372+0000 7ff2c7400640 1 -- 172.21.15.100:0/99274053 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2c8166730 msgr2=0x7ff2c815c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:57.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.372+0000 7ff2c7400640 1 --2- 172.21.15.100:0/99274053 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2c8166730 0x7ff2c815c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.372+0000 7ff2c7400640 1 -- 172.21.15.100:0/99274053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c80cfc10 msgr2=0x7ff2c81624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:57.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.372+0000 7ff2c7400640 1 --2- 172.21.15.100:0/99274053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c80cfc10 0x7ff2c81624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.372+0000 7ff2c7400640 1 -- 172.21.15.100:0/99274053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c80c5a00 con 0x7ff2c80c31b0 2024-04-23T14:33:57.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.372+0000 7ff2c7400640 1 --2- 172.21.15.100:0/99274053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2c80c31b0 0x7ff2c80c5390 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8019f00 tx=0x7ff2b8019070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:57.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.372+0000 7ff2bf400640 1 -- 172.21.15.100:0/99274053 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80b0040 con 0x7ff2c80c31b0 2024-04-23T14:33:57.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.372+0000 7ff2bf400640 1 -- 172.21.15.100:0/99274053 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b80b9070 con 0x7ff2c80c31b0 2024-04-23T14:33:57.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.372+0000 7ff2ce800640 1 -- 172.21.15.100:0/99274053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c80c5ce0 con 0x7ff2c80c31b0 2024-04-23T14:33:57.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.372+0000 7ff2bf400640 1 -- 172.21.15.100:0/99274053 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80b4880 con 0x7ff2c80c31b0 2024-04-23T14:33:57.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.372+0000 7ff2ce800640 1 -- 172.21.15.100:0/99274053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c80c4240 con 0x7ff2c80c31b0 2024-04-23T14:33:57.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.372+0000 7ff2ce800640 1 -- 172.21.15.100:0/99274053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c8068ad0 con 0x7ff2c80c31b0 2024-04-23T14:33:57.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.376+0000 7ff2bf400640 1 -- 172.21.15.100:0/99274053 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff2b80ab4c0 con 0x7ff2c80c31b0 2024-04-23T14:33:57.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.376+0000 7ff2bf400640 1 --2- 172.21.15.100:0/99274053 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff2a8068380 0x7ff2a806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.376+0000 7ff2bf400640 1 -- 172.21.15.100:0/99274053 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7ff2b8120b30 con 0x7ff2c80c31b0 2024-04-23T14:33:57.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.376+0000 7ff2c6a00640 1 --2- 172.21.15.100:0/99274053 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff2a8068380 0x7ff2a806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.376+0000 7ff2c6a00640 1 --2- 172.21.15.100:0/99274053 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff2a8068380 0x7ff2a806a840 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff2c81ff370 tx=0x7ff2b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:57.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.380+0000 7ff2bf400640 1 -- 172.21.15.100:0/99274053 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff2c80cf740 con 0x7ff2c80c31b0 2024-04-23T14:33:57.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.536+0000 7ff2ce800640 1 -- 172.21.15.100:0/99274053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) -- 0x7ff2c80af430 con 0x7ff2c80c31b0 2024-04-23T14:33:57.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.536+0000 7ff2bf400640 1 -- 172.21.15.100:0/99274053 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v41) ==== 97+0+11 (secure 0 0 0) 0x7ff2b80f0df0 con 0x7ff2c80c31b0 2024-04-23T14:33:57.543 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:57.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.540+0000 7ff2ce800640 1 -- 172.21.15.100:0/99274053 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff2a8068380 msgr2=0x7ff2a806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:57.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.540+0000 7ff2ce800640 1 --2- 172.21.15.100:0/99274053 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff2a8068380 0x7ff2a806a840 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff2c81ff370 tx=0x7ff2b4008040 comp rx=0 tx=0).stop 2024-04-23T14:33:57.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.540+0000 7ff2ce800640 1 -- 172.21.15.100:0/99274053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2c80c31b0 msgr2=0x7ff2c80c5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:57.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.540+0000 7ff2ce800640 1 --2- 172.21.15.100:0/99274053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2c80c31b0 0x7ff2c80c5390 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8019f00 tx=0x7ff2b8019070 comp rx=0 tx=0).stop 2024-04-23T14:33:57.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.540+0000 7ff2ce800640 1 -- 172.21.15.100:0/99274053 shutdown_connections 2024-04-23T14:33:57.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.540+0000 7ff2ce800640 1 --2- 172.21.15.100:0/99274053 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff2a8068380 0x7ff2a806a840 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.540+0000 7ff2ce800640 1 --2- 172.21.15.100:0/99274053 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2c8166730 0x7ff2c815c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.540+0000 7ff2ce800640 1 --2- 172.21.15.100:0/99274053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c80cfc10 0x7ff2c81624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.540+0000 7ff2ce800640 1 --2- 172.21.15.100:0/99274053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2c80c31b0 0x7ff2c80c5390 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.540+0000 7ff2ce800640 1 -- 172.21.15.100:0/99274053 >> 172.21.15.100:0/99274053 conn(0x7ff2c80b5200 msgr2=0x7ff2c806bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:57.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.540+0000 7ff2ce800640 1 -- 172.21.15.100:0/99274053 shutdown_connections 2024-04-23T14:33:57.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.540+0000 7ff2ce800640 1 -- 172.21.15.100:0/99274053 wait complete. 2024-04-23T14:33:57.563 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:33:57.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5083200640 1 Processor -- start 2024-04-23T14:33:57.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5083200640 1 -- start start 2024-04-23T14:33:57.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5083200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f508415e4b0 0x7f50840ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5083200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f50840afa70 0x7f50840b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5083200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50840ae7c0 con 0x7f508415e890 2024-04-23T14:33:57.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5083200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f50840b4a00 con 0x7f50840afa70 2024-04-23T14:33:57.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5083200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f50840b40d0 con 0x7f508415e4b0 2024-04-23T14:33:57.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5081400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f508415e890 0x7f50840af360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35512/0 (socket says 172.21.15.100:35512) 2024-04-23T14:33:57.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5081400640 1 -- 172.21.15.100:0/2701859144 learned_addr learned my addr 172.21.15.100:0/2701859144 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:57.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5081e00640 1 --2- 172.21.15.100:0/2701859144 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f508415e4b0 0x7f50840ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5082800640 1 --2- 172.21.15.100:0/2701859144 >> v2:172.21.15.151:3300/0 conn(0x7f50840afa70 0x7f50840b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5080a00640 1 -- 172.21.15.100:0/2701859144 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 81399002 0 0) 0x7f50840ae7c0 con 0x7f508415e890 2024-04-23T14:33:57.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5080a00640 1 -- 172.21.15.100:0/2701859144 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5068003530 con 0x7f508415e890 2024-04-23T14:33:57.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5080a00640 1 -- 172.21.15.100:0/2701859144 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 419542385 0 0) 0x7f5068003530 con 0x7f508415e890 2024-04-23T14:33:57.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5080a00640 1 -- 172.21.15.100:0/2701859144 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f508415e4b0 msgr2=0x7f50840ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:57.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5080a00640 1 --2- 172.21.15.100:0/2701859144 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f508415e4b0 0x7f50840ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5080a00640 1 -- 172.21.15.100:0/2701859144 >> v2:172.21.15.151:3300/0 conn(0x7f50840afa70 msgr2=0x7f50840b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:57.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5080a00640 1 --2- 172.21.15.100:0/2701859144 >> v2:172.21.15.151:3300/0 conn(0x7f50840afa70 0x7f50840b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5080a00640 1 -- 172.21.15.100:0/2701859144 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50840b43b0 con 0x7f508415e890 2024-04-23T14:33:57.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5080a00640 1 -- 172.21.15.100:0/2701859144 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f5070001680 con 0x7f508415e890 2024-04-23T14:33:57.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5080a00640 1 -- 172.21.15.100:0/2701859144 >> v1:172.21.15.100:6789/0 conn(0x7f508415e890 legacy=0x7f50840af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:57.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5080a00640 1 --2- 172.21.15.100:0/2701859144 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f508415e4b0 0x7f50680039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.696+0000 7f5080a00640 1 --2- 172.21.15.100:0/2701859144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5068003f00 0x7f5068004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.700+0000 7f5080a00640 1 --2- 172.21.15.100:0/2701859144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5068004d50 0x7f50680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.700+0000 7f5080a00640 1 -- 172.21.15.100:0/2701859144 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f50840b4a00 con 0x7f5068003f00 2024-04-23T14:33:57.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.700+0000 7f5080a00640 1 -- 172.21.15.100:0/2701859144 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f50840b40d0 con 0x7f5068004d50 2024-04-23T14:33:57.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.700+0000 7f5080a00640 1 -- 172.21.15.100:0/2701859144 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f506800b9d0 con 0x7f508415e4b0 2024-04-23T14:33:57.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.700+0000 7f5081400640 1 --2- 172.21.15.100:0/2701859144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5068003f00 0x7f5068004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.700+0000 7f5082800640 1 --2- 172.21.15.100:0/2701859144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5068004d50 0x7f50680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.700+0000 7f5081e00640 1 --2- 172.21.15.100:0/2701859144 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f508415e4b0 0x7f50680039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.700+0000 7f5081400640 1 -- 172.21.15.100:0/2701859144 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f508415e4b0 msgr2=0x7f50680039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:57.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.700+0000 7f5081400640 1 --2- 172.21.15.100:0/2701859144 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f508415e4b0 0x7f50680039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.700+0000 7f5081400640 1 -- 172.21.15.100:0/2701859144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5068004d50 msgr2=0x7f50680051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:57.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.700+0000 7f5081400640 1 --2- 172.21.15.100:0/2701859144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5068004d50 0x7f50680051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.700+0000 7f5081400640 1 -- 172.21.15.100:0/2701859144 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5068003530 con 0x7f5068003f00 2024-04-23T14:33:57.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.700+0000 7f5081400640 1 --2- 172.21.15.100:0/2701859144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5068003f00 0x7f5068004810 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f50700050b0 tx=0x7f507001fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37cb786e3f7217f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:57.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.700+0000 7f5080a00640 1 -- 172.21.15.100:0/2701859144 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070004cb0 con 0x7f5068003f00 2024-04-23T14:33:57.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.700+0000 7f5080a00640 1 -- 172.21.15.100:0/2701859144 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5070005ae0 con 0x7f5068003f00 2024-04-23T14:33:57.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.700+0000 7f5080a00640 1 -- 172.21.15.100:0/2701859144 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070008110 con 0x7f5068003f00 2024-04-23T14:33:57.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5083200640 1 -- 172.21.15.100:0/2701859144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5068003f00 msgr2=0x7f5068004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:57.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5083200640 1 --2- 172.21.15.100:0/2701859144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5068003f00 0x7f5068004810 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f50700050b0 tx=0x7f507001fa90 comp rx=0 tx=0).stop 2024-04-23T14:33:57.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5082800640 1 -- 172.21.15.100:0/2701859144 reap_dead start 2024-04-23T14:33:57.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5083200640 1 -- 172.21.15.100:0/2701859144 shutdown_connections 2024-04-23T14:33:57.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5083200640 1 -- 172.21.15.100:0/2701859144 >> 172.21.15.100:0/2701859144 conn(0x7f5084072bd0 msgr2=0x7f5084072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:57.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5083200640 1 -- 172.21.15.100:0/2701859144 shutdown_connections 2024-04-23T14:33:57.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5083200640 1 -- 172.21.15.100:0/2701859144 wait complete. 2024-04-23T14:33:57.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5083200640 1 Processor -- start 2024-04-23T14:33:57.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5083200640 1 -- start start 2024-04-23T14:33:57.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5083200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5068003f00 0x7f50841f4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5083200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5084161270 0x7f50841559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5083200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50841616f0 0x7f5084155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5083200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f508406f580 con 0x7f5068003f00 2024-04-23T14:33:57.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5083200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f508406fc40 con 0x7f50841616f0 2024-04-23T14:33:57.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5083200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f508406f700 con 0x7f5084161270 2024-04-23T14:33:57.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5081e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5068003f00 0x7f50841f4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5081e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5068003f00 0x7f50841f4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37102/0 (socket says 172.21.15.100:37102) 2024-04-23T14:33:57.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5081e00640 1 -- 172.21.15.100:0/1585872483 learned_addr learned my addr 172.21.15.100:0/1585872483 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:57.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5081400640 1 --2- 172.21.15.100:0/1585872483 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5084161270 0x7f50841559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5082800640 1 --2- 172.21.15.100:0/1585872483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50841616f0 0x7f5084155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5081e00640 1 -- 172.21.15.100:0/1585872483 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5084161270 msgr2=0x7f50841559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:57.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5081e00640 1 --2- 172.21.15.100:0/1585872483 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5084161270 0x7f50841559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5081e00640 1 -- 172.21.15.100:0/1585872483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50841616f0 msgr2=0x7f5084155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:57.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5081e00640 1 --2- 172.21.15.100:0/1585872483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50841616f0 0x7f5084155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5081e00640 1 -- 172.21.15.100:0/1585872483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508415eec0 con 0x7f5068003f00 2024-04-23T14:33:57.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5081400640 1 --2- 172.21.15.100:0/1585872483 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5084161270 0x7f50841559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:57.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.704+0000 7f5082800640 1 --2- 172.21.15.100:0/1585872483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50841616f0 0x7f5084155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:57.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.708+0000 7f5081e00640 1 --2- 172.21.15.100:0/1585872483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5068003f00 0x7f50841f4f00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5074011e30 tx=0x7f507400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:57.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.708+0000 7f507a000640 1 -- 172.21.15.100:0/1585872483 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5074015630 con 0x7f5068003f00 2024-04-23T14:33:57.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.708+0000 7f507a000640 1 -- 172.21.15.100:0/1585872483 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50740157d0 con 0x7f5068003f00 2024-04-23T14:33:57.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.708+0000 7f507a000640 1 -- 172.21.15.100:0/1585872483 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5074015ba0 con 0x7f5068003f00 2024-04-23T14:33:57.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.708+0000 7f5083200640 1 -- 172.21.15.100:0/1585872483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f508406eb50 con 0x7f5068003f00 2024-04-23T14:33:57.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.708+0000 7f5083200640 1 -- 172.21.15.100:0/1585872483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f508415f160 con 0x7f5068003f00 2024-04-23T14:33:57.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.708+0000 7f5083200640 1 -- 172.21.15.100:0/1585872483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5084075c20 con 0x7f5068003f00 2024-04-23T14:33:57.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.708+0000 7f507a000640 1 -- 172.21.15.100:0/1585872483 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f507401d400 con 0x7f5068003f00 2024-04-23T14:33:57.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.708+0000 7f507a000640 1 --2- 172.21.15.100:0/1585872483 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5064068380 0x7f506406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.708+0000 7f507a000640 1 -- 172.21.15.100:0/1585872483 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f5074087d80 con 0x7f5068003f00 2024-04-23T14:33:57.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.708+0000 7f5081400640 1 --2- 172.21.15.100:0/1585872483 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5064068380 0x7f506406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.712+0000 7f5081400640 1 --2- 172.21.15.100:0/1585872483 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5064068380 0x7f506406a840 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f508415ecf0 tx=0x7f5070004a50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:57.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.712+0000 7f507a000640 1 -- 172.21.15.100:0/1585872483 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5074019030 con 0x7f5068003f00 2024-04-23T14:33:57.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.884+0000 7f5083200640 1 -- 172.21.15.100:0/1585872483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f50840b6220 con 0x7f5068003f00 2024-04-23T14:33:57.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.884+0000 7f507a000640 1 -- 172.21.15.100:0/1585872483 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) ==== 80+0+0 (secure 0 0 0) 0x7f5074010040 con 0x7f5068003f00 2024-04-23T14:33:57.888 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-04-23T14:33:57.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.884+0000 7f5083200640 1 -- 172.21.15.100:0/1585872483 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5064068380 msgr2=0x7f506406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:57.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.884+0000 7f5083200640 1 --2- 172.21.15.100:0/1585872483 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5064068380 0x7f506406a840 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f508415ecf0 tx=0x7f5070004a50 comp rx=0 tx=0).stop 2024-04-23T14:33:57.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.884+0000 7f5083200640 1 -- 172.21.15.100:0/1585872483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5068003f00 msgr2=0x7f50841f4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:57.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.884+0000 7f5083200640 1 --2- 172.21.15.100:0/1585872483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5068003f00 0x7f50841f4f00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5074011e30 tx=0x7f507400f580 comp rx=0 tx=0).stop 2024-04-23T14:33:57.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.884+0000 7f5083200640 1 -- 172.21.15.100:0/1585872483 shutdown_connections 2024-04-23T14:33:57.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.884+0000 7f5083200640 1 --2- 172.21.15.100:0/1585872483 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5064068380 0x7f506406a840 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.884+0000 7f5083200640 1 --2- 172.21.15.100:0/1585872483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50841616f0 0x7f5084155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.884+0000 7f5083200640 1 --2- 172.21.15.100:0/1585872483 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5084161270 0x7f50841559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.884+0000 7f5083200640 1 --2- 172.21.15.100:0/1585872483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5068003f00 0x7f50841f4f00 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.884+0000 7f5083200640 1 -- 172.21.15.100:0/1585872483 >> 172.21.15.100:0/1585872483 conn(0x7f5084072bd0 msgr2=0x7f50840b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:57.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.884+0000 7f5083200640 1 -- 172.21.15.100:0/1585872483 shutdown_connections 2024-04-23T14:33:57.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:57.884+0000 7f5083200640 1 -- 172.21.15.100:0/1585872483 wait complete. 2024-04-23T14:33:57.909 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:33:57.909 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:57.918 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:58.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.092+0000 7fc599400640 1 Processor -- start 2024-04-23T14:33:58.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.092+0000 7fc599400640 1 -- start start 2024-04-23T14:33:58.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.092+0000 7fc599400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc59406d450 0x7fc59406d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.092+0000 7fc599400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc5940cd460 0x7fc5940b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.092+0000 7fc599400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc59406dd70 con 0x7fc5940cea60 2024-04-23T14:33:58.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.092+0000 7fc599400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc5940b1f90 con 0x7fc5940cd460 2024-04-23T14:33:58.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.092+0000 7fc599400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc5940b2270 con 0x7fc59406d450 2024-04-23T14:33:58.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.092+0000 7fc593200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc5940cd460 0x7fc5940b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.092+0000 7fc593200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc5940cd460 0x7fc5940b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37112/0 (socket says 172.21.15.100:37112) 2024-04-23T14:33:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.092+0000 7fc593200640 1 -- 172.21.15.100:0/2342576672 learned_addr learned my addr 172.21.15.100:0/2342576672 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.092+0000 7fc589e00640 1 --1- 172.21.15.100:0/2342576672 >> v1:172.21.15.100:6789/0 conn(0x7fc5940cea60 0x7fc5940d0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35522/0 (socket says 172.21.15.100:35522) 2024-04-23T14:33:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.092+0000 7fc592800640 1 --2- 172.21.15.100:0/2342576672 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc59406d450 0x7fc59406d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.092+0000 7fc593200640 1 -- 172.21.15.100:0/2342576672 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc59406d450 msgr2=0x7fc59406d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.092+0000 7fc593200640 1 --2- 172.21.15.100:0/2342576672 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc59406d450 0x7fc59406d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.092+0000 7fc593200640 1 -- 172.21.15.100:0/2342576672 >> v1:172.21.15.100:6789/0 conn(0x7fc5940cea60 legacy=0x7fc5940d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.092+0000 7fc593200640 1 -- 172.21.15.100:0/2342576672 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5940b2550 con 0x7fc5940cd460 2024-04-23T14:33:58.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.096+0000 7fc593200640 1 --2- 172.21.15.100:0/2342576672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5940cd460 0x7fc5940b2d50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009e50 tx=0x7fc58c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4a3268913e6853 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:58.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.096+0000 7fc591e00640 1 -- 172.21.15.100:0/2342576672 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c0a5660 con 0x7fc5940cd460 2024-04-23T14:33:58.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.096+0000 7fc591e00640 1 -- 172.21.15.100:0/2342576672 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc58c0a5800 con 0x7fc5940cd460 2024-04-23T14:33:58.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.096+0000 7fc591e00640 1 -- 172.21.15.100:0/2342576672 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c0a5bd0 con 0x7fc5940cd460 2024-04-23T14:33:58.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.096+0000 7fc599400640 1 -- 172.21.15.100:0/2342576672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5940cd460 msgr2=0x7fc5940b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.096+0000 7fc599400640 1 --2- 172.21.15.100:0/2342576672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5940cd460 0x7fc5940b2d50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009e50 tx=0x7fc58c01c9a0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.096+0000 7fc599400640 1 -- 172.21.15.100:0/2342576672 shutdown_connections 2024-04-23T14:33:58.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.096+0000 7fc599400640 1 --2- 172.21.15.100:0/2342576672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5940cd460 0x7fc5940b2d50 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.096+0000 7fc599400640 1 --2- 172.21.15.100:0/2342576672 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc59406d450 0x7fc59406d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.096+0000 7fc599400640 1 -- 172.21.15.100:0/2342576672 >> 172.21.15.100:0/2342576672 conn(0x7fc5940cee80 msgr2=0x7fc5940cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:58.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.096+0000 7fc599400640 1 -- 172.21.15.100:0/2342576672 shutdown_connections 2024-04-23T14:33:58.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.096+0000 7fc599400640 1 -- 172.21.15.100:0/2342576672 wait complete. 2024-04-23T14:33:58.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc599400640 1 Processor -- start 2024-04-23T14:33:58.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc599400640 1 -- start start 2024-04-23T14:33:58.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc599400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc59406d450 0x7fc5940c4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc599400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5940cd460 0x7fc59415a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc599400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5940cea60 0x7fc59415a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc599400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc59416b740 con 0x7fc5940cd460 2024-04-23T14:33:58.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc599400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc59416b5c0 con 0x7fc59406d450 2024-04-23T14:33:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc599400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc59416b8c0 con 0x7fc5940cea60 2024-04-23T14:33:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc589e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5940cd460 0x7fc59415a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc589e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5940cd460 0x7fc59415a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37120/0 (socket says 172.21.15.100:37120) 2024-04-23T14:33:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc589e00640 1 -- 172.21.15.100:0/1029711413 learned_addr learned my addr 172.21.15.100:0/1029711413 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc592800640 1 --2- 172.21.15.100:0/1029711413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc59406d450 0x7fc5940c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc593200640 1 --2- 172.21.15.100:0/1029711413 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5940cea60 0x7fc59415a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc589e00640 1 -- 172.21.15.100:0/1029711413 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5940cea60 msgr2=0x7fc59415a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc589e00640 1 --2- 172.21.15.100:0/1029711413 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5940cea60 0x7fc59415a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc589e00640 1 -- 172.21.15.100:0/1029711413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc59406d450 msgr2=0x7fc5940c4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc589e00640 1 --2- 172.21.15.100:0/1029711413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc59406d450 0x7fc5940c4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc589e00640 1 -- 172.21.15.100:0/1029711413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc594156d60 con 0x7fc5940cd460 2024-04-23T14:33:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc589e00640 1 --2- 172.21.15.100:0/1029711413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5940cd460 0x7fc59415a210 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00c450 tx=0x7fc57c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:58.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc58b400640 1 -- 172.21.15.100:0/1029711413 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c018070 con 0x7fc5940cd460 2024-04-23T14:33:58.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc58b400640 1 -- 172.21.15.100:0/1029711413 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc57c007480 con 0x7fc5940cd460 2024-04-23T14:33:58.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc58b400640 1 -- 172.21.15.100:0/1029711413 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c0047f0 con 0x7fc5940cd460 2024-04-23T14:33:58.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc599400640 1 -- 172.21.15.100:0/1029711413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc594157040 con 0x7fc5940cd460 2024-04-23T14:33:58.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.100+0000 7fc599400640 1 -- 172.21.15.100:0/1029711413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5940cad80 con 0x7fc5940cd460 2024-04-23T14:33:58.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.104+0000 7fc599400640 1 -- 172.21.15.100:0/1029711413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5940caf80 con 0x7fc5940cd460 2024-04-23T14:33:58.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.104+0000 7fc58b400640 1 -- 172.21.15.100:0/1029711413 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc57c01b070 con 0x7fc5940cd460 2024-04-23T14:33:58.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.104+0000 7fc58b400640 1 --2- 172.21.15.100:0/1029711413 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc5700683b0 0x7fc57006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.104+0000 7fc58b400640 1 -- 172.21.15.100:0/1029711413 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fc57c085f90 con 0x7fc5940cd460 2024-04-23T14:33:58.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.104+0000 7fc592800640 1 --2- 172.21.15.100:0/1029711413 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc5700683b0 0x7fc57006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.104+0000 7fc592800640 1 --2- 172.21.15.100:0/1029711413 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc5700683b0 0x7fc57006a870 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc58400b930 tx=0x7fc584008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:58.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.108+0000 7fc58b400640 1 -- 172.21.15.100:0/1029711413 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc5940caf80 con 0x7fc5940cd460 2024-04-23T14:33:58.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.264+0000 7fc599400640 1 -- 172.21.15.100:0/1029711413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5940c4cb0 con 0x7fc5940cd460 2024-04-23T14:33:58.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.268+0000 7fc58b400640 1 -- 172.21.15.100:0/1029711413 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+18766 (secure 0 0 0) 0x7fc57c0561c0 con 0x7fc5940cd460 2024-04-23T14:33:58.272 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:33:58.272 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":41,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:33:49.609841+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:33:36.592437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-23T14:33:37.882390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-04-23T14:33:47.299378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.268+0000 7fc599400640 1 -- 172.21.15.100:0/1029711413 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc5700683b0 msgr2=0x7fc57006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.268+0000 7fc599400640 1 --2- 172.21.15.100:0/1029711413 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc5700683b0 0x7fc57006a870 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc58400b930 tx=0x7fc584008040 comp rx=0 tx=0).stop 2024-04-23T14:33:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.268+0000 7fc599400640 1 -- 172.21.15.100:0/1029711413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5940cd460 msgr2=0x7fc59415a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.268+0000 7fc599400640 1 --2- 172.21.15.100:0/1029711413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5940cd460 0x7fc59415a210 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00c450 tx=0x7fc57c00c910 comp rx=0 tx=0).stop 2024-04-23T14:33:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.268+0000 7fc599400640 1 -- 172.21.15.100:0/1029711413 shutdown_connections 2024-04-23T14:33:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.268+0000 7fc599400640 1 --2- 172.21.15.100:0/1029711413 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc5700683b0 0x7fc57006a870 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.268+0000 7fc599400640 1 --2- 172.21.15.100:0/1029711413 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5940cea60 0x7fc59415a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.268+0000 7fc599400640 1 --2- 172.21.15.100:0/1029711413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5940cd460 0x7fc59415a210 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.268+0000 7fc599400640 1 --2- 172.21.15.100:0/1029711413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc59406d450 0x7fc5940c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.268+0000 7fc599400640 1 -- 172.21.15.100:0/1029711413 >> 172.21.15.100:0/1029711413 conn(0x7fc5940cee80 msgr2=0x7fc5940d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:58.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.268+0000 7fc599400640 1 -- 172.21.15.100:0/1029711413 shutdown_connections 2024-04-23T14:33:58.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.268+0000 7fc599400640 1 -- 172.21.15.100:0/1029711413 wait complete. 2024-04-23T14:33:58.293 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:33:36.592437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:33:37.882390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-04-23T14:33:47.299378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:33:58.293 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:33:58.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f564ce00640 1 Processor -- start 2024-04-23T14:33:58.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f564ce00640 1 -- start start 2024-04-23T14:33:58.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f564ce00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f5648145ca0 0x7f56480ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f564ce00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56480701b0 0x7f5648070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f564ce00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56480cd8e0 con 0x7f56480cf590 2024-04-23T14:33:58.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f564ce00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f5648070b50 con 0x7f5648145ca0 2024-04-23T14:33:58.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f564ce00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f56480ce210 con 0x7f56480701b0 2024-04-23T14:33:58.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5647e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f56480cf590 0x7f56480cfa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35536/0 (socket says 172.21.15.100:35536) 2024-04-23T14:33:58.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5647e00640 1 -- 172.21.15.100:0/3571707243 learned_addr learned my addr 172.21.15.100:0/3571707243 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:58.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5647400640 1 --2- 172.21.15.100:0/3571707243 >> v2:172.21.15.151:3300/0 conn(0x7f5648145ca0 0x7f56480ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646a00640 1 --2- 172.21.15.100:0/3571707243 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56480701b0 0x7f5648070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646000640 1 -- 172.21.15.100:0/3571707243 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 717111709 0 0) 0x7f56480cd8e0 con 0x7f56480cf590 2024-04-23T14:33:58.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646000640 1 -- 172.21.15.100:0/3571707243 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f562c003530 con 0x7f56480cf590 2024-04-23T14:33:58.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646000640 1 -- 172.21.15.100:0/3571707243 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1344657381 0 0) 0x7f562c003530 con 0x7f56480cf590 2024-04-23T14:33:58.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646000640 1 -- 172.21.15.100:0/3571707243 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56480701b0 msgr2=0x7f5648070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:58.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646000640 1 --2- 172.21.15.100:0/3571707243 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56480701b0 0x7f5648070610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646000640 1 -- 172.21.15.100:0/3571707243 >> v2:172.21.15.151:3300/0 conn(0x7f5648145ca0 msgr2=0x7f56480ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:58.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646000640 1 --2- 172.21.15.100:0/3571707243 >> v2:172.21.15.151:3300/0 conn(0x7f5648145ca0 0x7f56480ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646000640 1 -- 172.21.15.100:0/3571707243 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56480cdbc0 con 0x7f56480cf590 2024-04-23T14:33:58.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646000640 1 -- 172.21.15.100:0/3571707243 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f5640001680 con 0x7f56480cf590 2024-04-23T14:33:58.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646000640 1 -- 172.21.15.100:0/3571707243 >> v1:172.21.15.100:6789/0 conn(0x7f56480cf590 legacy=0x7f56480cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646000640 1 --2- 172.21.15.100:0/3571707243 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56480701b0 0x7f562c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646000640 1 --2- 172.21.15.100:0/3571707243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f562c003f40 0x7f562c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646000640 1 --2- 172.21.15.100:0/3571707243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f562c004d50 0x7f562c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646000640 1 -- 172.21.15.100:0/3571707243 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f56480ce210 con 0x7f562c003f40 2024-04-23T14:33:58.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646000640 1 -- 172.21.15.100:0/3571707243 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f562c00b9d0 con 0x7f562c004d50 2024-04-23T14:33:58.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646000640 1 -- 172.21.15.100:0/3571707243 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f562c00bba0 con 0x7f56480701b0 2024-04-23T14:33:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646a00640 1 --2- 172.21.15.100:0/3571707243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f562c003f40 0x7f562c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5647e00640 1 --2- 172.21.15.100:0/3571707243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f562c004d50 0x7f562c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5647400640 1 --2- 172.21.15.100:0/3571707243 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56480701b0 0x7f562c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646a00640 1 -- 172.21.15.100:0/3571707243 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56480701b0 msgr2=0x7f562c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646a00640 1 --2- 172.21.15.100:0/3571707243 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56480701b0 0x7f562c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646a00640 1 -- 172.21.15.100:0/3571707243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f562c004d50 msgr2=0x7f562c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646a00640 1 --2- 172.21.15.100:0/3571707243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f562c004d50 0x7f562c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5646a00640 1 -- 172.21.15.100:0/3571707243 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562c003530 con 0x7f562c003f40 2024-04-23T14:33:58.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.428+0000 7f5647400640 1 --2- 172.21.15.100:0/3571707243 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56480701b0 0x7f562c0039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.432+0000 7f5646a00640 1 --2- 172.21.15.100:0/3571707243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f562c003f40 0x7f562c004810 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5634009b10 tx=0x7f563401ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84fc52198f1ea6e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.432+0000 7f5646000640 1 -- 172.21.15.100:0/3571707243 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56340b2070 con 0x7f562c003f40 2024-04-23T14:33:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.432+0000 7f5646000640 1 -- 172.21.15.100:0/3571707243 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56340a5520 con 0x7f562c003f40 2024-04-23T14:33:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.432+0000 7f5646000640 1 -- 172.21.15.100:0/3571707243 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56340096b0 con 0x7f562c003f40 2024-04-23T14:33:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.432+0000 7f564ce00640 1 -- 172.21.15.100:0/3571707243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f562c003f40 msgr2=0x7f562c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.432+0000 7f564ce00640 1 --2- 172.21.15.100:0/3571707243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f562c003f40 0x7f562c004810 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5634009b10 tx=0x7f563401ca40 comp rx=0 tx=0).stop 2024-04-23T14:33:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.432+0000 7f5647e00640 1 -- 172.21.15.100:0/3571707243 reap_dead start 2024-04-23T14:33:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.432+0000 7f564ce00640 1 -- 172.21.15.100:0/3571707243 shutdown_connections 2024-04-23T14:33:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.432+0000 7f564ce00640 1 -- 172.21.15.100:0/3571707243 >> 172.21.15.100:0/3571707243 conn(0x7f56480d15f0 msgr2=0x7f56480d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.432+0000 7f564ce00640 1 -- 172.21.15.100:0/3571707243 shutdown_connections 2024-04-23T14:33:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.432+0000 7f564ce00640 1 -- 172.21.15.100:0/3571707243 wait complete. 2024-04-23T14:33:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f564ce00640 1 Processor -- start 2024-04-23T14:33:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f564ce00640 1 -- start start 2024-04-23T14:33:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f564ce00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f562c003f40 0x7f56480c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f564ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f564815beb0 0x7f56480c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f564ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f564815c310 0x7f56480c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f564ce00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5648146200 con 0x7f564815c310 2024-04-23T14:33:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f564ce00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5648146080 con 0x7f564815beb0 2024-04-23T14:33:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f564ce00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f564806f780 con 0x7f562c003f40 2024-04-23T14:33:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f5647e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f564815c310 0x7f56480c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f5647400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f562c003f40 0x7f56480c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f5647e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f564815c310 0x7f56480c6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37140/0 (socket says 172.21.15.100:37140) 2024-04-23T14:33:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f5647e00640 1 -- 172.21.15.100:0/1764566475 learned_addr learned my addr 172.21.15.100:0/1764566475 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f5646a00640 1 --2- 172.21.15.100:0/1764566475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f564815beb0 0x7f56480c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f5647e00640 1 -- 172.21.15.100:0/1764566475 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f562c003f40 msgr2=0x7f56480c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f5647e00640 1 --2- 172.21.15.100:0/1764566475 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f562c003f40 0x7f56480c7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f5647e00640 1 -- 172.21.15.100:0/1764566475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f564815beb0 msgr2=0x7f56480c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f5647e00640 1 --2- 172.21.15.100:0/1764566475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f564815beb0 0x7f56480c6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f5647e00640 1 -- 172.21.15.100:0/1764566475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564813ffd0 con 0x7f564815c310 2024-04-23T14:33:58.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f5647e00640 1 --2- 172.21.15.100:0/1764566475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f564815c310 0x7f56480c6690 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f56480d22f0 tx=0x7f5640007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:58.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f563f400640 1 -- 172.21.15.100:0/1764566475 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564001a070 con 0x7f564815c310 2024-04-23T14:33:58.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f563f400640 1 -- 172.21.15.100:0/1764566475 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f564000f6a0 con 0x7f564815c310 2024-04-23T14:33:58.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f563f400640 1 -- 172.21.15.100:0/1764566475 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564000fa90 con 0x7f564815c310 2024-04-23T14:33:58.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f564ce00640 1 -- 172.21.15.100:0/1764566475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56480c6d30 con 0x7f564815c310 2024-04-23T14:33:58.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.436+0000 7f564ce00640 1 -- 172.21.15.100:0/1764566475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56480c72f0 con 0x7f564815c310 2024-04-23T14:33:58.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.440+0000 7f564ce00640 1 -- 172.21.15.100:0/1764566475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5648068ad0 con 0x7f564815c310 2024-04-23T14:33:58.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.440+0000 7f563f400640 1 -- 172.21.15.100:0/1764566475 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f564000fc30 con 0x7f564815c310 2024-04-23T14:33:58.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.440+0000 7f563f400640 1 --2- 172.21.15.100:0/1764566475 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5628068380 0x7f562806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.440+0000 7f563f400640 1 -- 172.21.15.100:0/1764566475 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f56400861a0 con 0x7f564815c310 2024-04-23T14:33:58.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.440+0000 7f5647400640 1 --2- 172.21.15.100:0/1764566475 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5628068380 0x7f562806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.440+0000 7f5647400640 1 --2- 172.21.15.100:0/1764566475 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5628068380 0x7f562806a840 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5638007540 tx=0x7f563800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:58.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.444+0000 7f563f400640 1 -- 172.21.15.100:0/1764566475 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f564813ffd0 con 0x7f564815c310 2024-04-23T14:33:58.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.600+0000 7f564ce00640 1 -- 172.21.15.100:0/1764566475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f56481605d0 con 0x7f564815c310 2024-04-23T14:33:58.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.604+0000 7f563f400640 1 -- 172.21.15.100:0/1764566475 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7f5640055e90 con 0x7f564815c310 2024-04-23T14:33:58.608 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:33:58.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.604+0000 7f564ce00640 1 -- 172.21.15.100:0/1764566475 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5628068380 msgr2=0x7f562806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.604+0000 7f564ce00640 1 --2- 172.21.15.100:0/1764566475 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5628068380 0x7f562806a840 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5638007540 tx=0x7f563800b040 comp rx=0 tx=0).stop 2024-04-23T14:33:58.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.604+0000 7f564ce00640 1 -- 172.21.15.100:0/1764566475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f564815c310 msgr2=0x7f56480c6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.604+0000 7f564ce00640 1 --2- 172.21.15.100:0/1764566475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f564815c310 0x7f56480c6690 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f56480d22f0 tx=0x7f5640007430 comp rx=0 tx=0).stop 2024-04-23T14:33:58.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.604+0000 7f564ce00640 1 -- 172.21.15.100:0/1764566475 shutdown_connections 2024-04-23T14:33:58.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.604+0000 7f564ce00640 1 --2- 172.21.15.100:0/1764566475 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5628068380 0x7f562806a840 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.604+0000 7f564ce00640 1 --2- 172.21.15.100:0/1764566475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f564815c310 0x7f56480c6690 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.604+0000 7f564ce00640 1 --2- 172.21.15.100:0/1764566475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f564815beb0 0x7f56480c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.604+0000 7f564ce00640 1 --2- 172.21.15.100:0/1764566475 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f562c003f40 0x7f56480c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.604+0000 7f564ce00640 1 -- 172.21.15.100:0/1764566475 >> 172.21.15.100:0/1764566475 conn(0x7f56480d15f0 msgr2=0x7f56480b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:58.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.604+0000 7f564ce00640 1 -- 172.21.15.100:0/1764566475 shutdown_connections 2024-04-23T14:33:58.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.604+0000 7f564ce00640 1 -- 172.21.15.100:0/1764566475 wait complete. 2024-04-23T14:33:58.629 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:33:58.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.764+0000 7f135be00640 1 Processor -- start 2024-04-23T14:33:58.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.764+0000 7f135be00640 1 -- start start 2024-04-23T14:33:58.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.764+0000 7f135be00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f135415c7a0 0x7f13540af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.764+0000 7f135be00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f135406bb20 0x7f13540ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.764+0000 7f135be00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13540cd540 con 0x7f13540cd130 2024-04-23T14:33:58.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.764+0000 7f135be00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f135406d450 con 0x7f135415c7a0 2024-04-23T14:33:58.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.764+0000 7f135be00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f135406cbb0 con 0x7f135406bb20 2024-04-23T14:33:58.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.764+0000 7f1359400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f135415c7a0 0x7f13540af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.764+0000 7f1359400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f135415c7a0 0x7f13540af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42810/0 (socket says 172.21.15.100:42810) 2024-04-23T14:33:58.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.768+0000 7f1359400640 1 -- 172.21.15.100:0/3187303095 learned_addr learned my addr 172.21.15.100:0/3187303095 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:58.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.764+0000 7f1359e00640 1 --1- 172.21.15.100:0/3187303095 >> v1:172.21.15.100:6789/0 conn(0x7f13540cd130 0x7f135406e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:41906/0 (socket says 172.21.15.100:41906) 2024-04-23T14:33:58.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.768+0000 7f1359400640 1 -- 172.21.15.100:0/3187303095 >> v2:172.21.15.151:3300/0 conn(0x7f135406bb20 msgr2=0x7f13540ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:58.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.768+0000 7f1359400640 1 --2- 172.21.15.100:0/3187303095 >> v2:172.21.15.151:3300/0 conn(0x7f135406bb20 0x7f13540ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.768+0000 7f1359400640 1 -- 172.21.15.100:0/3187303095 >> v1:172.21.15.100:6789/0 conn(0x7f13540cd130 legacy=0x7f135406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.768+0000 7f1359400640 1 -- 172.21.15.100:0/3187303095 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135406ce90 con 0x7f135415c7a0 2024-04-23T14:33:58.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.768+0000 7f1359400640 1 --2- 172.21.15.100:0/3187303095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f135415c7a0 0x7f13540af190 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f13540d1280 tx=0x7f134801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e042e7cf8ff9e8a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:58.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.768+0000 7f1353e00640 1 -- 172.21.15.100:0/3187303095 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13480b0070 con 0x7f135415c7a0 2024-04-23T14:33:58.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.768+0000 7f1353e00640 1 -- 172.21.15.100:0/3187303095 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13480ab600 con 0x7f135415c7a0 2024-04-23T14:33:58.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.768+0000 7f1353e00640 1 -- 172.21.15.100:0/3187303095 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13480ab9d0 con 0x7f135415c7a0 2024-04-23T14:33:58.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.768+0000 7f135be00640 1 -- 172.21.15.100:0/3187303095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f135415c7a0 msgr2=0x7f13540af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.768+0000 7f135be00640 1 --2- 172.21.15.100:0/3187303095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f135415c7a0 0x7f13540af190 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f13540d1280 tx=0x7f134801c490 comp rx=0 tx=0).stop 2024-04-23T14:33:58.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f135be00640 1 -- 172.21.15.100:0/3187303095 shutdown_connections 2024-04-23T14:33:58.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f135be00640 1 --2- 172.21.15.100:0/3187303095 >> v2:172.21.15.151:3300/0 conn(0x7f135406bb20 0x7f13540ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f135be00640 1 --2- 172.21.15.100:0/3187303095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f135415c7a0 0x7f13540af190 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f135be00640 1 -- 172.21.15.100:0/3187303095 >> 172.21.15.100:0/3187303095 conn(0x7f13540b5200 msgr2=0x7f13540b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:58.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f135be00640 1 -- 172.21.15.100:0/3187303095 shutdown_connections 2024-04-23T14:33:58.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f135be00640 1 -- 172.21.15.100:0/3187303095 wait complete. 2024-04-23T14:33:58.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f135be00640 1 Processor -- start 2024-04-23T14:33:58.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f135be00640 1 -- start start 2024-04-23T14:33:58.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f135be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f135406bb20 0x7f13540cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f135be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13540cd130 0x7f13540cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f135be00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f135415c7a0 0x7f13541f7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f135be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1354169590 con 0x7f13540cd130 2024-04-23T14:33:58.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f135be00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1354169410 con 0x7f135406bb20 2024-04-23T14:33:58.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f135be00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1354169710 con 0x7f135415c7a0 2024-04-23T14:33:58.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f1358a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13540cd130 0x7f13540cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f1358a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13540cd130 0x7f13540cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42826/0 (socket says 172.21.15.100:42826) 2024-04-23T14:33:58.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f1358a00640 1 -- 172.21.15.100:0/3739766214 learned_addr learned my addr 172.21.15.100:0/3739766214 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:58.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f1359400640 1 --2- 172.21.15.100:0/3739766214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f135406bb20 0x7f13540cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f1359e00640 1 --2- 172.21.15.100:0/3739766214 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f135415c7a0 0x7f13541f7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f1358a00640 1 -- 172.21.15.100:0/3739766214 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f135415c7a0 msgr2=0x7f13541f7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f1358a00640 1 --2- 172.21.15.100:0/3739766214 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f135415c7a0 0x7f13541f7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f1358a00640 1 -- 172.21.15.100:0/3739766214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f135406bb20 msgr2=0x7f13540cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f1358a00640 1 --2- 172.21.15.100:0/3739766214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f135406bb20 0x7f13540cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f1358a00640 1 -- 172.21.15.100:0/3739766214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1354153a80 con 0x7f13540cd130 2024-04-23T14:33:58.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.772+0000 7f1358a00640 1 --2- 172.21.15.100:0/3739766214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13540cd130 0x7f13540cbaf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f134000b480 tx=0x7f134000b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:58.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.776+0000 7f1351600640 1 -- 172.21.15.100:0/3739766214 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1340014070 con 0x7f13540cd130 2024-04-23T14:33:58.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.776+0000 7f1351600640 1 -- 172.21.15.100:0/3739766214 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1340010870 con 0x7f13540cd130 2024-04-23T14:33:58.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.776+0000 7f1351600640 1 -- 172.21.15.100:0/3739766214 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1340010c60 con 0x7f13540cd130 2024-04-23T14:33:58.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.776+0000 7f135be00640 1 -- 172.21.15.100:0/3739766214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13541f8da0 con 0x7f13540cd130 2024-04-23T14:33:58.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.776+0000 7f135be00640 1 -- 172.21.15.100:0/3739766214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13540cc480 con 0x7f13540cd130 2024-04-23T14:33:58.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.776+0000 7f135be00640 1 -- 172.21.15.100:0/3739766214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1354068ad0 con 0x7f13540cd130 2024-04-23T14:33:58.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.776+0000 7f1351600640 1 -- 172.21.15.100:0/3739766214 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1340017070 con 0x7f13540cd130 2024-04-23T14:33:58.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.776+0000 7f1351600640 1 --2- 172.21.15.100:0/3739766214 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1334068380 0x7f133406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.776+0000 7f1351600640 1 -- 172.21.15.100:0/3739766214 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f1340086ac0 con 0x7f13540cd130 2024-04-23T14:33:58.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.776+0000 7f1359400640 1 --2- 172.21.15.100:0/3739766214 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1334068380 0x7f133406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.776+0000 7f1359400640 1 --2- 172.21.15.100:0/3739766214 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1334068380 0x7f133406a840 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f13480ad000 tx=0x7f1348008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:58.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.780+0000 7f1351600640 1 -- 172.21.15.100:0/3739766214 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1340056d80 con 0x7f13540cd130 2024-04-23T14:33:58.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.936+0000 7f135be00640 1 -- 172.21.15.100:0/3739766214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f13540af1a0 con 0x7f13540cd130 2024-04-23T14:33:58.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.936+0000 7f1351600640 1 -- 172.21.15.100:0/3739766214 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) ==== 104+0+11 (secure 0 0 0) 0x7f1340056680 con 0x7f13540cd130 2024-04-23T14:33:58.942 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.940+0000 7f135be00640 1 -- 172.21.15.100:0/3739766214 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1334068380 msgr2=0x7f133406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.940+0000 7f135be00640 1 --2- 172.21.15.100:0/3739766214 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1334068380 0x7f133406a840 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f13480ad000 tx=0x7f1348008040 comp rx=0 tx=0).stop 2024-04-23T14:33:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.940+0000 7f135be00640 1 -- 172.21.15.100:0/3739766214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13540cd130 msgr2=0x7f13540cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.940+0000 7f135be00640 1 --2- 172.21.15.100:0/3739766214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13540cd130 0x7f13540cbaf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f134000b480 tx=0x7f134000b940 comp rx=0 tx=0).stop 2024-04-23T14:33:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.940+0000 7f135be00640 1 -- 172.21.15.100:0/3739766214 shutdown_connections 2024-04-23T14:33:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.940+0000 7f135be00640 1 --2- 172.21.15.100:0/3739766214 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1334068380 0x7f133406a840 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.940+0000 7f135be00640 1 --2- 172.21.15.100:0/3739766214 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f135415c7a0 0x7f13541f7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.940+0000 7f135be00640 1 --2- 172.21.15.100:0/3739766214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13540cd130 0x7f13540cbaf0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.940+0000 7f135be00640 1 --2- 172.21.15.100:0/3739766214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f135406bb20 0x7f13540cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.940+0000 7f135be00640 1 -- 172.21.15.100:0/3739766214 >> 172.21.15.100:0/3739766214 conn(0x7f13540b5200 msgr2=0x7f1354152b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.940+0000 7f135be00640 1 -- 172.21.15.100:0/3739766214 shutdown_connections 2024-04-23T14:33:58.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:58.940+0000 7f135be00640 1 -- 172.21.15.100:0/3739766214 wait complete. 2024-04-23T14:33:58.963 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:33:59.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.096+0000 7fb4c3400640 1 Processor -- start 2024-04-23T14:33:59.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.096+0000 7fb4c3400640 1 -- start start 2024-04-23T14:33:59.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.096+0000 7fb4c3400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb4bc0cea60 0x7fb4bc0d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.096+0000 7fb4c3400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb4bc0cd460 0x7fb4bc0b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.096+0000 7fb4c3400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4bc0b1f90 con 0x7fb4bc06d450 2024-04-23T14:33:59.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.096+0000 7fb4c3400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fb4bc0b2270 con 0x7fb4bc0cd460 2024-04-23T14:33:59.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.096+0000 7fb4c3400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb4bc0b2550 con 0x7fb4bc0cea60 2024-04-23T14:33:59.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.096+0000 7fb4c1400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb4bc0cd460 0x7fb4bc0b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.096+0000 7fb4c1400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb4bc0cd460 0x7fb4bc0b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42834/0 (socket says 172.21.15.100:42834) 2024-04-23T14:33:59.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.096+0000 7fb4c1400640 1 -- 172.21.15.100:0/1881825254 learned_addr learned my addr 172.21.15.100:0/1881825254 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:59.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.096+0000 7fb4c0a00640 1 --1- 172.21.15.100:0/1881825254 >> v1:172.21.15.100:6789/0 conn(0x7fb4bc06d450 0x7fb4bc06d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:41908/0 (socket says 172.21.15.100:41908) 2024-04-23T14:33:59.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.096+0000 7fb4b3e00640 1 --2- 172.21.15.100:0/1881825254 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb4bc0cea60 0x7fb4bc0d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.096+0000 7fb4c1400640 1 -- 172.21.15.100:0/1881825254 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb4bc0cea60 msgr2=0x7fb4bc0d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.096+0000 7fb4c1400640 1 --2- 172.21.15.100:0/1881825254 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb4bc0cea60 0x7fb4bc0d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.096+0000 7fb4c1400640 1 -- 172.21.15.100:0/1881825254 >> v1:172.21.15.100:6789/0 conn(0x7fb4bc06d450 legacy=0x7fb4bc06d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.096+0000 7fb4c1400640 1 -- 172.21.15.100:0/1881825254 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4bc0b2830 con 0x7fb4bc0cd460 2024-04-23T14:33:59.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.100+0000 7fb4c1400640 1 --2- 172.21.15.100:0/1881825254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4bc0cd460 0x7fb4bc0b2d50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4009e50 tx=0x7fb4b401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23c9587260aa5c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.100+0000 7fb4bbe00640 1 -- 172.21.15.100:0/1881825254 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b40a5660 con 0x7fb4bc0cd460 2024-04-23T14:33:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.100+0000 7fb4bbe00640 1 -- 172.21.15.100:0/1881825254 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4b40a5800 con 0x7fb4bc0cd460 2024-04-23T14:33:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.100+0000 7fb4bbe00640 1 -- 172.21.15.100:0/1881825254 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b40a5bd0 con 0x7fb4bc0cd460 2024-04-23T14:33:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.100+0000 7fb4c3400640 1 -- 172.21.15.100:0/1881825254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4bc0cd460 msgr2=0x7fb4bc0b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.100+0000 7fb4c3400640 1 --2- 172.21.15.100:0/1881825254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4bc0cd460 0x7fb4bc0b2d50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4009e50 tx=0x7fb4b401c9a0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.100+0000 7fb4c3400640 1 -- 172.21.15.100:0/1881825254 shutdown_connections 2024-04-23T14:33:59.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.100+0000 7fb4c3400640 1 --2- 172.21.15.100:0/1881825254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4bc0cd460 0x7fb4bc0b2d50 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.100+0000 7fb4c3400640 1 --2- 172.21.15.100:0/1881825254 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb4bc0cea60 0x7fb4bc0d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.100+0000 7fb4c3400640 1 -- 172.21.15.100:0/1881825254 >> 172.21.15.100:0/1881825254 conn(0x7fb4bc0cee80 msgr2=0x7fb4bc0cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:59.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.100+0000 7fb4c3400640 1 -- 172.21.15.100:0/1881825254 shutdown_connections 2024-04-23T14:33:59.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.100+0000 7fb4c3400640 1 -- 172.21.15.100:0/1881825254 wait complete. 2024-04-23T14:33:59.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4c3400640 1 Processor -- start 2024-04-23T14:33:59.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4c3400640 1 -- start start 2024-04-23T14:33:59.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4c3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4bc06d450 0x7fb4bc1faea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4c3400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb4bc0cd460 0x7fb4bc0d1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4c3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4bc0cea60 0x7fb4bc152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4c3400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc16b740 con 0x7fb4bc06d450 2024-04-23T14:33:59.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4c3400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc16b5c0 con 0x7fb4bc0cea60 2024-04-23T14:33:59.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4c3400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb4bc16b8c0 con 0x7fb4bc0cd460 2024-04-23T14:33:59.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4c0a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4bc06d450 0x7fb4bc1faea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4c0a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4bc06d450 0x7fb4bc1faea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42850/0 (socket says 172.21.15.100:42850) 2024-04-23T14:33:59.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4c0a00640 1 -- 172.21.15.100:0/2306790397 learned_addr learned my addr 172.21.15.100:0/2306790397 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:59.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4b3e00640 1 --2- 172.21.15.100:0/2306790397 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb4bc0cd460 0x7fb4bc0d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4c1400640 1 --2- 172.21.15.100:0/2306790397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4bc0cea60 0x7fb4bc152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4c0a00640 1 -- 172.21.15.100:0/2306790397 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb4bc0cd460 msgr2=0x7fb4bc0d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4c0a00640 1 --2- 172.21.15.100:0/2306790397 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb4bc0cd460 0x7fb4bc0d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4c0a00640 1 -- 172.21.15.100:0/2306790397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4bc0cea60 msgr2=0x7fb4bc152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4c0a00640 1 --2- 172.21.15.100:0/2306790397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4bc0cea60 0x7fb4bc152f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4c0a00640 1 -- 172.21.15.100:0/2306790397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4bc1fb5a0 con 0x7fb4bc06d450 2024-04-23T14:33:59.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4c1400640 1 --2- 172.21.15.100:0/2306790397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4bc0cea60 0x7fb4bc152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:59.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4c0a00640 1 --2- 172.21.15.100:0/2306790397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4bc06d450 0x7fb4bc1faea0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00c370 tx=0x7fb4ac00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4b9600640 1 -- 172.21.15.100:0/2306790397 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac0074e0 con 0x7fb4bc06d450 2024-04-23T14:33:59.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4b9600640 1 -- 172.21.15.100:0/2306790397 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4ac007680 con 0x7fb4bc06d450 2024-04-23T14:33:59.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4c3400640 1 -- 172.21.15.100:0/2306790397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4bc1fb880 con 0x7fb4bc06d450 2024-04-23T14:33:59.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4b9600640 1 -- 172.21.15.100:0/2306790397 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac004a30 con 0x7fb4bc06d450 2024-04-23T14:33:59.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.104+0000 7fb4c3400640 1 -- 172.21.15.100:0/2306790397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4bc1553c0 con 0x7fb4bc06d450 2024-04-23T14:33:59.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.108+0000 7fb4c3400640 1 -- 172.21.15.100:0/2306790397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4bc068ad0 con 0x7fb4bc06d450 2024-04-23T14:33:59.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.108+0000 7fb4b9600640 1 -- 172.21.15.100:0/2306790397 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb4ac01a020 con 0x7fb4bc06d450 2024-04-23T14:33:59.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.108+0000 7fb4b9600640 1 --2- 172.21.15.100:0/2306790397 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb498068380 0x7fb49806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.108+0000 7fb4b9600640 1 -- 172.21.15.100:0/2306790397 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fb4ac086470 con 0x7fb4bc06d450 2024-04-23T14:33:59.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.108+0000 7fb4b3e00640 1 --2- 172.21.15.100:0/2306790397 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb498068380 0x7fb49806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.108+0000 7fb4b3e00640 1 --2- 172.21.15.100:0/2306790397 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb498068380 0x7fb49806a840 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0b1c40 tx=0x7fb4a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.112+0000 7fb4b9600640 1 -- 172.21.15.100:0/2306790397 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb4ac056730 con 0x7fb4bc06d450 2024-04-23T14:33:59.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.268+0000 7fb4c3400640 1 -- 172.21.15.100:0/2306790397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb4bc0cdef0 con 0x7fb4bc06d450 2024-04-23T14:33:59.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.268+0000 7fb4b9600640 1 -- 172.21.15.100:0/2306790397 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) ==== 100+0+11 (secure 0 0 0) 0x7fb4ac056030 con 0x7fb4bc06d450 2024-04-23T14:33:59.272 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:33:59.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.268+0000 7fb4c3400640 1 -- 172.21.15.100:0/2306790397 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb498068380 msgr2=0x7fb49806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.268+0000 7fb4c3400640 1 --2- 172.21.15.100:0/2306790397 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb498068380 0x7fb49806a840 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0b1c40 tx=0x7fb4a4008040 comp rx=0 tx=0).stop 2024-04-23T14:33:59.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.268+0000 7fb4c3400640 1 -- 172.21.15.100:0/2306790397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4bc06d450 msgr2=0x7fb4bc1faea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.268+0000 7fb4c3400640 1 --2- 172.21.15.100:0/2306790397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4bc06d450 0x7fb4bc1faea0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00c370 tx=0x7fb4ac00c830 comp rx=0 tx=0).stop 2024-04-23T14:33:59.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.268+0000 7fb4c3400640 1 -- 172.21.15.100:0/2306790397 shutdown_connections 2024-04-23T14:33:59.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.268+0000 7fb4c3400640 1 --2- 172.21.15.100:0/2306790397 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb498068380 0x7fb49806a840 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.268+0000 7fb4c3400640 1 --2- 172.21.15.100:0/2306790397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4bc0cea60 0x7fb4bc152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.268+0000 7fb4c3400640 1 --2- 172.21.15.100:0/2306790397 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb4bc0cd460 0x7fb4bc0d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.268+0000 7fb4c3400640 1 --2- 172.21.15.100:0/2306790397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4bc06d450 0x7fb4bc1faea0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.268+0000 7fb4c3400640 1 -- 172.21.15.100:0/2306790397 >> 172.21.15.100:0/2306790397 conn(0x7fb4bc0cee80 msgr2=0x7fb4bc06f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:59.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.268+0000 7fb4c3400640 1 -- 172.21.15.100:0/2306790397 shutdown_connections 2024-04-23T14:33:59.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.268+0000 7fb4c3400640 1 -- 172.21.15.100:0/2306790397 wait complete. 2024-04-23T14:33:59.292 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-04-23T14:33:59.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.424+0000 7fc4f7e00640 1 Processor -- start 2024-04-23T14:33:59.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.424+0000 7fc4f7e00640 1 -- start start 2024-04-23T14:33:59.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.428+0000 7fc4f7e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc4f006c6c0 0x7fc4f00ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.428+0000 7fc4f7e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc4f00ce620 0x7fc4f00add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.428+0000 7fc4f7e00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4f006dd60 con 0x7fc4f006f8c0 2024-04-23T14:33:59.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.428+0000 7fc4f7e00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f00ad050 con 0x7fc4f006c6c0 2024-04-23T14:33:59.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.428+0000 7fc4f7e00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f00ad8f0 con 0x7fc4f00ce620 2024-04-23T14:33:59.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.428+0000 7fc4f5400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fc4f006f8c0 0x7fc4f00d0170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:41924/0 (socket says 172.21.15.100:41924) 2024-04-23T14:33:59.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.428+0000 7fc4f5400640 1 -- 172.21.15.100:0/2731500846 learned_addr learned my addr 172.21.15.100:0/2731500846 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:33:59.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.428+0000 7fc4f4a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc4f006c6c0 0x7fc4f00ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.428+0000 7fc4f5e00640 1 --2- 172.21.15.100:0/2731500846 >> v2:172.21.15.151:3300/0 conn(0x7fc4f00ce620 0x7fc4f00add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.428+0000 7fc4f4a00640 1 -- 172.21.15.100:0/2731500846 >> v2:172.21.15.151:3300/0 conn(0x7fc4f00ce620 msgr2=0x7fc4f00add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.428+0000 7fc4f4a00640 1 --2- 172.21.15.100:0/2731500846 >> v2:172.21.15.151:3300/0 conn(0x7fc4f00ce620 0x7fc4f00add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.428+0000 7fc4f4a00640 1 -- 172.21.15.100:0/2731500846 >> v1:172.21.15.100:6789/0 conn(0x7fc4f006f8c0 legacy=0x7fc4f00d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.428+0000 7fc4f4a00640 1 -- 172.21.15.100:0/2731500846 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f00d09c0 con 0x7fc4f006c6c0 2024-04-23T14:33:59.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.428+0000 7fc4efe00640 1 -- 172.21.15.100:0/2731500846 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1370133185 0 0) 0x7fc4e4001120 con 0x7fc4f006f8c0 2024-04-23T14:33:59.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.428+0000 7fc4f4a00640 1 --2- 172.21.15.100:0/2731500846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4f006c6c0 0x7fc4f00ce0e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00b3120 tx=0x7fc4dc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46feb936dea6001 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4efe00640 1 -- 172.21.15.100:0/2731500846 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc0b1070 con 0x7fc4f006c6c0 2024-04-23T14:33:59.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4efe00640 1 -- 172.21.15.100:0/2731500846 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4dc0ac5f0 con 0x7fc4f006c6c0 2024-04-23T14:33:59.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4efe00640 1 -- 172.21.15.100:0/2731500846 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc0ac9c0 con 0x7fc4f006c6c0 2024-04-23T14:33:59.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4f7e00640 1 -- 172.21.15.100:0/2731500846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4f006c6c0 msgr2=0x7fc4f00ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4f7e00640 1 --2- 172.21.15.100:0/2731500846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4f006c6c0 0x7fc4f00ce0e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00b3120 tx=0x7fc4dc01c480 comp rx=0 tx=0).stop 2024-04-23T14:33:59.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4f7e00640 1 -- 172.21.15.100:0/2731500846 shutdown_connections 2024-04-23T14:33:59.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4f7e00640 1 --2- 172.21.15.100:0/2731500846 >> v2:172.21.15.151:3300/0 conn(0x7fc4f00ce620 0x7fc4f00add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4f7e00640 1 --2- 172.21.15.100:0/2731500846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4f006c6c0 0x7fc4f00ce0e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4f7e00640 1 -- 172.21.15.100:0/2731500846 >> 172.21.15.100:0/2731500846 conn(0x7fc4f00b5200 msgr2=0x7fc4f00b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:59.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4f7e00640 1 -- 172.21.15.100:0/2731500846 shutdown_connections 2024-04-23T14:33:59.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4f7e00640 1 -- 172.21.15.100:0/2731500846 wait complete. 2024-04-23T14:33:59.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4f7e00640 1 Processor -- start 2024-04-23T14:33:59.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4f7e00640 1 -- start start 2024-04-23T14:33:59.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4f7e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc4f006c6c0 0x7fc4f01555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4f7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4f006f8c0 0x7fc4f00731f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4f7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4f00ce620 0x7fc4f0073730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4f7e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f00aee60 con 0x7fc4f00ce620 2024-04-23T14:33:59.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4f7e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f00aece0 con 0x7fc4f006f8c0 2024-04-23T14:33:59.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4f7e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f0168400 con 0x7fc4f006c6c0 2024-04-23T14:33:59.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4f5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4f00ce620 0x7fc4f0073730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4f5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4f00ce620 0x7fc4f0073730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42858/0 (socket says 172.21.15.100:42858) 2024-04-23T14:33:59.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4f5e00640 1 -- 172.21.15.100:0/245073770 learned_addr learned my addr 172.21.15.100:0/245073770 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:59.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4f4a00640 1 --2- 172.21.15.100:0/245073770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4f006f8c0 0x7fc4f00731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.432+0000 7fc4f5400640 1 --2- 172.21.15.100:0/245073770 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc4f006c6c0 0x7fc4f01555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.436+0000 7fc4f5e00640 1 -- 172.21.15.100:0/245073770 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc4f006c6c0 msgr2=0x7fc4f01555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.436+0000 7fc4f5e00640 1 --2- 172.21.15.100:0/245073770 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc4f006c6c0 0x7fc4f01555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.436+0000 7fc4f5e00640 1 -- 172.21.15.100:0/245073770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4f006f8c0 msgr2=0x7fc4f00731f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.436+0000 7fc4f5e00640 1 --2- 172.21.15.100:0/245073770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4f006f8c0 0x7fc4f00731f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.436+0000 7fc4f5e00640 1 -- 172.21.15.100:0/245073770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f00c7f60 con 0x7fc4f00ce620 2024-04-23T14:33:59.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.436+0000 7fc4f5e00640 1 --2- 172.21.15.100:0/245073770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4f00ce620 0x7fc4f0073730 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8007e70 tx=0x7fc4e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.436+0000 7fc4ed600640 1 -- 172.21.15.100:0/245073770 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8014620 con 0x7fc4f00ce620 2024-04-23T14:33:59.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.436+0000 7fc4ed600640 1 -- 172.21.15.100:0/245073770 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e8019070 con 0x7fc4f00ce620 2024-04-23T14:33:59.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.436+0000 7fc4ed600640 1 -- 172.21.15.100:0/245073770 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8014a20 con 0x7fc4f00ce620 2024-04-23T14:33:59.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.436+0000 7fc4f7e00640 1 -- 172.21.15.100:0/245073770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f00c8240 con 0x7fc4f00ce620 2024-04-23T14:33:59.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.436+0000 7fc4f7e00640 1 -- 172.21.15.100:0/245073770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f00c8800 con 0x7fc4f00ce620 2024-04-23T14:33:59.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.436+0000 7fc4f7e00640 1 -- 172.21.15.100:0/245073770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f00c8a00 con 0x7fc4f00ce620 2024-04-23T14:33:59.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.436+0000 7fc4ed600640 1 -- 172.21.15.100:0/245073770 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc4e8014bc0 con 0x7fc4f00ce620 2024-04-23T14:33:59.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.436+0000 7fc4ed600640 1 --2- 172.21.15.100:0/245073770 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc4d00683b0 0x7fc4d006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.436+0000 7fc4ed600640 1 -- 172.21.15.100:0/245073770 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fc4e8087470 con 0x7fc4f00ce620 2024-04-23T14:33:59.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.436+0000 7fc4f5400640 1 --2- 172.21.15.100:0/245073770 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc4d00683b0 0x7fc4d006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.440+0000 7fc4f5400640 1 --2- 172.21.15.100:0/245073770 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc4d00683b0 0x7fc4d006a870 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4000f70 tx=0x7fc4e4007b60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.440+0000 7fc4ed600640 1 -- 172.21.15.100:0/245073770 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc4f00c8a00 con 0x7fc4f00ce620 2024-04-23T14:33:59.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.600+0000 7fc4f7e00640 1 -- 172.21.15.100:0/245073770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) -- 0x7fc4f0068ad0 con 0x7fc4f00ce620 2024-04-23T14:33:59.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.600+0000 7fc4ed600640 1 -- 172.21.15.100:0/245073770 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v41) ==== 97+0+11 (secure 0 0 0) 0x7fc4e80570e0 con 0x7fc4f00ce620 2024-04-23T14:33:59.606 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:33:59.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.604+0000 7fc4f7e00640 1 -- 172.21.15.100:0/245073770 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc4d00683b0 msgr2=0x7fc4d006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.604+0000 7fc4f7e00640 1 --2- 172.21.15.100:0/245073770 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc4d00683b0 0x7fc4d006a870 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4000f70 tx=0x7fc4e4007b60 comp rx=0 tx=0).stop 2024-04-23T14:33:59.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.604+0000 7fc4f7e00640 1 -- 172.21.15.100:0/245073770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4f00ce620 msgr2=0x7fc4f0073730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.604+0000 7fc4f7e00640 1 --2- 172.21.15.100:0/245073770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4f00ce620 0x7fc4f0073730 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8007e70 tx=0x7fc4e800e4b0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.604+0000 7fc4f7e00640 1 -- 172.21.15.100:0/245073770 shutdown_connections 2024-04-23T14:33:59.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.604+0000 7fc4f7e00640 1 --2- 172.21.15.100:0/245073770 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc4d00683b0 0x7fc4d006a870 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.604+0000 7fc4f7e00640 1 --2- 172.21.15.100:0/245073770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4f00ce620 0x7fc4f0073730 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.604+0000 7fc4f7e00640 1 --2- 172.21.15.100:0/245073770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4f006f8c0 0x7fc4f00731f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.604+0000 7fc4f7e00640 1 --2- 172.21.15.100:0/245073770 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc4f006c6c0 0x7fc4f01555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.604+0000 7fc4f7e00640 1 -- 172.21.15.100:0/245073770 >> 172.21.15.100:0/245073770 conn(0x7fc4f00b5200 msgr2=0x7fc4f00b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:59.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.604+0000 7fc4f7e00640 1 -- 172.21.15.100:0/245073770 shutdown_connections 2024-04-23T14:33:59.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.604+0000 7fc4f7e00640 1 -- 172.21.15.100:0/245073770 wait complete. 2024-04-23T14:33:59.627 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:33:59.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.760+0000 7fc122200640 1 Processor -- start 2024-04-23T14:33:59.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.760+0000 7fc122200640 1 -- start start 2024-04-23T14:33:59.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.760+0000 7fc122200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc11c15c600 0x7fc11c0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.760+0000 7fc122200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc11c0d1eb0 0x7fc11c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.760+0000 7fc122200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc11c0b57f0 0x7fc11c0cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.760+0000 7fc122200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc11c0cf570 con 0x7fc11c0d1eb0 2024-04-23T14:33:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.760+0000 7fc122200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fc11c0d0e90 con 0x7fc11c15c600 2024-04-23T14:33:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.760+0000 7fc122200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc11c0d1170 con 0x7fc11c0b57f0 2024-04-23T14:33:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.760+0000 7fc11aa00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc11c0d1eb0 0x7fc11c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.760+0000 7fc11aa00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc11c0d1eb0 0x7fc11c0b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42874/0 (socket says 172.21.15.100:42874) 2024-04-23T14:33:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.760+0000 7fc11aa00640 1 -- 172.21.15.100:0/2308456861 learned_addr learned my addr 172.21.15.100:0/2308456861 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.760+0000 7fc11be00640 1 --2- 172.21.15.100:0/2308456861 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc11c0b57f0 0x7fc11c0cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.760+0000 7fc11b400640 1 --2- 172.21.15.100:0/2308456861 >> v2:172.21.15.151:3300/0 conn(0x7fc11c15c600 0x7fc11c0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.760+0000 7fc11aa00640 1 -- 172.21.15.100:0/2308456861 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc11c0b57f0 msgr2=0x7fc11c0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.760+0000 7fc11aa00640 1 --2- 172.21.15.100:0/2308456861 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc11c0b57f0 0x7fc11c0cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.760+0000 7fc11aa00640 1 -- 172.21.15.100:0/2308456861 >> v2:172.21.15.151:3300/0 conn(0x7fc11c15c600 msgr2=0x7fc11c0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.760+0000 7fc11aa00640 1 --2- 172.21.15.100:0/2308456861 >> v2:172.21.15.151:3300/0 conn(0x7fc11c15c600 0x7fc11c0d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.760+0000 7fc11aa00640 1 -- 172.21.15.100:0/2308456861 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11c0d1450 con 0x7fc11c0d1eb0 2024-04-23T14:33:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.760+0000 7fc11be00640 1 --2- 172.21.15.100:0/2308456861 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc11c0b57f0 0x7fc11c0cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:59.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.760+0000 7fc11b400640 1 --2- 172.21.15.100:0/2308456861 >> v2:172.21.15.151:3300/0 conn(0x7fc11c15c600 0x7fc11c0d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:59.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.764+0000 7fc11aa00640 1 --2- 172.21.15.100:0/2308456861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc11c0d1eb0 0x7fc11c0b52b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0686b0 tx=0x7fc10801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d029191b33bd78d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.764+0000 7fc11a000640 1 -- 172.21.15.100:0/2308456861 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1080b0070 con 0x7fc11c0d1eb0 2024-04-23T14:33:59.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.764+0000 7fc11a000640 1 -- 172.21.15.100:0/2308456861 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1080ab600 con 0x7fc11c0d1eb0 2024-04-23T14:33:59.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.764+0000 7fc11a000640 1 -- 172.21.15.100:0/2308456861 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1080ab9d0 con 0x7fc11c0d1eb0 2024-04-23T14:33:59.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.764+0000 7fc122200640 1 -- 172.21.15.100:0/2308456861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc11c0d1eb0 msgr2=0x7fc11c0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.764+0000 7fc122200640 1 --2- 172.21.15.100:0/2308456861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc11c0d1eb0 0x7fc11c0b52b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0686b0 tx=0x7fc10801c520 comp rx=0 tx=0).stop 2024-04-23T14:33:59.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.764+0000 7fc122200640 1 -- 172.21.15.100:0/2308456861 shutdown_connections 2024-04-23T14:33:59.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.764+0000 7fc122200640 1 --2- 172.21.15.100:0/2308456861 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc11c0b57f0 0x7fc11c0cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.764+0000 7fc122200640 1 --2- 172.21.15.100:0/2308456861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc11c0d1eb0 0x7fc11c0b52b0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.764+0000 7fc122200640 1 --2- 172.21.15.100:0/2308456861 >> v2:172.21.15.151:3300/0 conn(0x7fc11c15c600 0x7fc11c0d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.764+0000 7fc122200640 1 -- 172.21.15.100:0/2308456861 >> 172.21.15.100:0/2308456861 conn(0x7fc11c0719d0 msgr2=0x7fc11c071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:59.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.764+0000 7fc122200640 1 -- 172.21.15.100:0/2308456861 shutdown_connections 2024-04-23T14:33:59.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.764+0000 7fc122200640 1 -- 172.21.15.100:0/2308456861 wait complete. 2024-04-23T14:33:59.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.768+0000 7fc122200640 1 Processor -- start 2024-04-23T14:33:59.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.768+0000 7fc122200640 1 -- start start 2024-04-23T14:33:59.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.768+0000 7fc122200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc11c0b57f0 0x7fc11c0cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.768+0000 7fc122200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc11c0d1eb0 0x7fc11c15e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.768+0000 7fc122200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc11c15c600 0x7fc11c0c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.768+0000 7fc122200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c160550 con 0x7fc11c0d1eb0 2024-04-23T14:33:59.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.768+0000 7fc122200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c1603d0 con 0x7fc11c0b57f0 2024-04-23T14:33:59.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.768+0000 7fc122200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc11c1606d0 con 0x7fc11c15c600 2024-04-23T14:33:59.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.768+0000 7fc11aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc11c0d1eb0 0x7fc11c15e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.768+0000 7fc11aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc11c0d1eb0 0x7fc11c15e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42884/0 (socket says 172.21.15.100:42884) 2024-04-23T14:33:59.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.768+0000 7fc11aa00640 1 -- 172.21.15.100:0/4117974905 learned_addr learned my addr 172.21.15.100:0/4117974905 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:33:59.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.768+0000 7fc11aa00640 1 -- 172.21.15.100:0/4117974905 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc11c15c600 msgr2=0x7fc11c0c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.768+0000 7fc11aa00640 1 --2- 172.21.15.100:0/4117974905 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc11c15c600 0x7fc11c0c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.768+0000 7fc11aa00640 1 -- 172.21.15.100:0/4117974905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc11c0b57f0 msgr2=0x7fc11c0cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.768+0000 7fc11aa00640 1 --2- 172.21.15.100:0/4117974905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc11c0b57f0 0x7fc11c0cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.768+0000 7fc11aa00640 1 -- 172.21.15.100:0/4117974905 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11c1473e0 con 0x7fc11c0d1eb0 2024-04-23T14:33:59.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.768+0000 7fc11aa00640 1 --2- 172.21.15.100:0/4117974905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc11c0d1eb0 0x7fc11c15e320 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc10801abd0 tx=0x7fc1080192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.768+0000 7fc113400640 1 -- 172.21.15.100:0/4117974905 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1080ba070 con 0x7fc11c0d1eb0 2024-04-23T14:33:59.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.768+0000 7fc113400640 1 -- 172.21.15.100:0/4117974905 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1080b4420 con 0x7fc11c0d1eb0 2024-04-23T14:33:59.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.768+0000 7fc113400640 1 -- 172.21.15.100:0/4117974905 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1080b0040 con 0x7fc11c0d1eb0 2024-04-23T14:33:59.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.768+0000 7fc122200640 1 -- 172.21.15.100:0/4117974905 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11c1476c0 con 0x7fc11c0d1eb0 2024-04-23T14:33:59.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.768+0000 7fc122200640 1 -- 172.21.15.100:0/4117974905 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc11c147c80 con 0x7fc11c0d1eb0 2024-04-23T14:33:59.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.772+0000 7fc122200640 1 -- 172.21.15.100:0/4117974905 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc11c068ad0 con 0x7fc11c0d1eb0 2024-04-23T14:33:59.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.772+0000 7fc113400640 1 -- 172.21.15.100:0/4117974905 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc1080b47f0 con 0x7fc11c0d1eb0 2024-04-23T14:33:59.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.772+0000 7fc113400640 1 --2- 172.21.15.100:0/4117974905 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc0fc068380 0x7fc0fc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.772+0000 7fc113400640 1 -- 172.21.15.100:0/4117974905 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fc108120040 con 0x7fc11c0d1eb0 2024-04-23T14:33:59.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.772+0000 7fc11b400640 1 --2- 172.21.15.100:0/4117974905 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc0fc068380 0x7fc0fc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.772+0000 7fc11b400640 1 --2- 172.21.15.100:0/4117974905 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc0fc068380 0x7fc0fc06a840 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0094c0 tx=0x7fc10c009570 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.776+0000 7fc113400640 1 -- 172.21.15.100:0/4117974905 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc108016110 con 0x7fc11c0d1eb0 2024-04-23T14:33:59.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.944+0000 7fc122200640 1 -- 172.21.15.100:0/4117974905 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc11c15e860 con 0x7fc11c0d1eb0 2024-04-23T14:33:59.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.944+0000 7fc113400640 1 -- 172.21.15.100:0/4117974905 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) ==== 80+0+0 (secure 0 0 0) 0x7fc1080f0270 con 0x7fc11c0d1eb0 2024-04-23T14:33:59.948 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-04-23T14:33:59.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.944+0000 7fc122200640 1 -- 172.21.15.100:0/4117974905 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc0fc068380 msgr2=0x7fc0fc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.944+0000 7fc122200640 1 --2- 172.21.15.100:0/4117974905 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc0fc068380 0x7fc0fc06a840 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0094c0 tx=0x7fc10c009570 comp rx=0 tx=0).stop 2024-04-23T14:33:59.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.944+0000 7fc122200640 1 -- 172.21.15.100:0/4117974905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc11c0d1eb0 msgr2=0x7fc11c15e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.944+0000 7fc122200640 1 --2- 172.21.15.100:0/4117974905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc11c0d1eb0 0x7fc11c15e320 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc10801abd0 tx=0x7fc1080192c0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.944+0000 7fc122200640 1 -- 172.21.15.100:0/4117974905 shutdown_connections 2024-04-23T14:33:59.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.944+0000 7fc122200640 1 --2- 172.21.15.100:0/4117974905 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc0fc068380 0x7fc0fc06a840 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.944+0000 7fc122200640 1 --2- 172.21.15.100:0/4117974905 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc11c15c600 0x7fc11c0c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.944+0000 7fc122200640 1 --2- 172.21.15.100:0/4117974905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc11c0d1eb0 0x7fc11c15e320 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.944+0000 7fc122200640 1 --2- 172.21.15.100:0/4117974905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc11c0b57f0 0x7fc11c0cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.944+0000 7fc122200640 1 -- 172.21.15.100:0/4117974905 >> 172.21.15.100:0/4117974905 conn(0x7fc11c0719d0 msgr2=0x7fc11c160b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:59.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.944+0000 7fc122200640 1 -- 172.21.15.100:0/4117974905 shutdown_connections 2024-04-23T14:33:59.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:33:59.944+0000 7fc122200640 1 -- 172.21.15.100:0/4117974905 wait complete. 2024-04-23T14:33:59.970 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi100.front.sepia.ceph.com... 2024-04-23T14:33:59.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:59.970 DEBUG:teuthology.orchestra.run.smithi100:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:59.988 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[39870]: fuse finished with error 0 and tester_r 0 2024-04-23T14:33:59.990 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:34:00.077 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:34:00.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:00.078 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:34:00.087 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rwp' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2024-04-23T14:34:00.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.268+0000 7fb8b5000640 1 Processor -- start 2024-04-23T14:34:00.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.268+0000 7fb8b5000640 1 -- start start 2024-04-23T14:34:00.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.272+0000 7fb8b5000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fb8b00cca50 0x7fb8b00cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:00.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.272+0000 7fb8b5000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb8b015d600 0x7fb8b0160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:00.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.272+0000 7fb8b5000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8b00cd370 con 0x7fb8b015c7e0 2024-04-23T14:34:00.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.272+0000 7fb8b5000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fb8b00d2270 con 0x7fb8b015d600 2024-04-23T14:34:00.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.272+0000 7fb8b5000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fb8b015db00 con 0x7fb8b00cca50 2024-04-23T14:34:00.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.272+0000 7fb8ae800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb8b015d600 0x7fb8b0160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:00.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.272+0000 7fb8ae800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb8b015d600 0x7fb8b0160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42894/0 (socket says 172.21.15.100:42894) 2024-04-23T14:34:00.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.272+0000 7fb8a7e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fb8b015c7e0 0x7fb8b0160170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:41940/0 (socket says 172.21.15.100:41940) 2024-04-23T14:34:00.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.272+0000 7fb8ae800640 1 -- 172.21.15.100:0/1316505814 learned_addr learned my addr 172.21.15.100:0/1316505814 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:00.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.272+0000 7fb8ade00640 1 --2- 172.21.15.100:0/1316505814 >> v2:172.21.15.151:3300/0 conn(0x7fb8b00cca50 0x7fb8b00cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:00.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.272+0000 7fb8ae800640 1 -- 172.21.15.100:0/1316505814 >> v2:172.21.15.151:3300/0 conn(0x7fb8b00cca50 msgr2=0x7fb8b00cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:00.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.272+0000 7fb8ae800640 1 --2- 172.21.15.100:0/1316505814 >> v2:172.21.15.151:3300/0 conn(0x7fb8b00cca50 0x7fb8b00cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:00.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.272+0000 7fb8ae800640 1 -- 172.21.15.100:0/1316505814 >> v1:172.21.15.100:6789/0 conn(0x7fb8b015c7e0 legacy=0x7fb8b0160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:00.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.272+0000 7fb8ae800640 1 -- 172.21.15.100:0/1316505814 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b00d04a0 con 0x7fb8b015d600 2024-04-23T14:34:00.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.276+0000 7fb8ae800640 1 --2- 172.21.15.100:0/1316505814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b015d600 0x7fb8b0160880 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00741f0 tx=0x7fb8a801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326e80731facb145 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:00.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.276+0000 7fb8ad400640 1 -- 172.21.15.100:0/1316505814 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a80a57f0 con 0x7fb8b015d600 2024-04-23T14:34:00.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.276+0000 7fb8ad400640 1 -- 172.21.15.100:0/1316505814 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8a80a5990 con 0x7fb8b015d600 2024-04-23T14:34:00.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.276+0000 7fb8ad400640 1 -- 172.21.15.100:0/1316505814 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a80a5d60 con 0x7fb8b015d600 2024-04-23T14:34:00.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.276+0000 7fb8b5000640 1 -- 172.21.15.100:0/1316505814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b015d600 msgr2=0x7fb8b0160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:00.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.276+0000 7fb8b5000640 1 --2- 172.21.15.100:0/1316505814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b015d600 0x7fb8b0160880 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00741f0 tx=0x7fb8a801c930 comp rx=0 tx=0).stop 2024-04-23T14:34:00.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.276+0000 7fb8b5000640 1 -- 172.21.15.100:0/1316505814 shutdown_connections 2024-04-23T14:34:00.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.276+0000 7fb8b5000640 1 --2- 172.21.15.100:0/1316505814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b015d600 0x7fb8b0160880 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:00.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.276+0000 7fb8b5000640 1 --2- 172.21.15.100:0/1316505814 >> v2:172.21.15.151:3300/0 conn(0x7fb8b00cca50 0x7fb8b00cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:00.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.276+0000 7fb8b5000640 1 -- 172.21.15.100:0/1316505814 >> 172.21.15.100:0/1316505814 conn(0x7fb8b00cee80 msgr2=0x7fb8b00cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:00.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.276+0000 7fb8b5000640 1 -- 172.21.15.100:0/1316505814 shutdown_connections 2024-04-23T14:34:00.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.276+0000 7fb8b5000640 1 -- 172.21.15.100:0/1316505814 wait complete. 2024-04-23T14:34:00.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.276+0000 7fb8b5000640 1 Processor -- start 2024-04-23T14:34:00.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.276+0000 7fb8b5000640 1 -- start start 2024-04-23T14:34:00.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.276+0000 7fb8b5000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8b00cca50 0x7fb8b0155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:00.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.276+0000 7fb8b5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b015c7e0 0x7fb8b014f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:00.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.276+0000 7fb8b5000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb8b015d600 0x7fb8b014f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:00.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.276+0000 7fb8b5000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b00b5430 con 0x7fb8b015c7e0 2024-04-23T14:34:00.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.276+0000 7fb8b5000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b00b52b0 con 0x7fb8b00cca50 2024-04-23T14:34:00.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.276+0000 7fb8b5000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb8b00b3bf0 con 0x7fb8b015d600 2024-04-23T14:34:00.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.280+0000 7fb8a7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b015c7e0 0x7fb8b014f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:00.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.280+0000 7fb8a7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b015c7e0 0x7fb8b014f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42900/0 (socket says 172.21.15.100:42900) 2024-04-23T14:34:00.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.280+0000 7fb8a7e00640 1 -- 172.21.15.100:0/436049281 learned_addr learned my addr 172.21.15.100:0/436049281 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:00.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.280+0000 7fb8ade00640 1 --2- 172.21.15.100:0/436049281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8b00cca50 0x7fb8b0155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:00.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.280+0000 7fb8ae800640 1 --2- 172.21.15.100:0/436049281 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb8b015d600 0x7fb8b014f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:00.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.280+0000 7fb8a7e00640 1 -- 172.21.15.100:0/436049281 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb8b015d600 msgr2=0x7fb8b014f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:00.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.280+0000 7fb8a7e00640 1 --2- 172.21.15.100:0/436049281 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb8b015d600 0x7fb8b014f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:00.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.280+0000 7fb8a7e00640 1 -- 172.21.15.100:0/436049281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8b00cca50 msgr2=0x7fb8b0155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:00.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.280+0000 7fb8a7e00640 1 --2- 172.21.15.100:0/436049281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8b00cca50 0x7fb8b0155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:00.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.280+0000 7fb8a7e00640 1 -- 172.21.15.100:0/436049281 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b00ca700 con 0x7fb8b015c7e0 2024-04-23T14:34:00.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.280+0000 7fb8ae800640 1 --2- 172.21.15.100:0/436049281 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb8b015d600 0x7fb8b014f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:00.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.280+0000 7fb8a7e00640 1 --2- 172.21.15.100:0/436049281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b015c7e0 0x7fb8b014f390 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb89800c420 tx=0x7fb89800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:00.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.280+0000 7fb8a6000640 1 -- 172.21.15.100:0/436049281 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8980074e0 con 0x7fb8b015c7e0 2024-04-23T14:34:00.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.280+0000 7fb8a6000640 1 -- 172.21.15.100:0/436049281 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb898007680 con 0x7fb8b015c7e0 2024-04-23T14:34:00.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.280+0000 7fb8b5000640 1 -- 172.21.15.100:0/436049281 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b00ca9e0 con 0x7fb8b015c7e0 2024-04-23T14:34:00.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.280+0000 7fb8a6000640 1 -- 172.21.15.100:0/436049281 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb898004a30 con 0x7fb8b015c7e0 2024-04-23T14:34:00.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.280+0000 7fb8b5000640 1 -- 172.21.15.100:0/436049281 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b00cafa0 con 0x7fb8b015c7e0 2024-04-23T14:34:00.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.280+0000 7fb8b5000640 1 -- 172.21.15.100:0/436049281 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8b0068ad0 con 0x7fb8b015c7e0 2024-04-23T14:34:00.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.280+0000 7fb8a6000640 1 -- 172.21.15.100:0/436049281 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb89801a020 con 0x7fb8b015c7e0 2024-04-23T14:34:00.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.284+0000 7fb8a6000640 1 --2- 172.21.15.100:0/436049281 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb88c068380 0x7fb88c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:00.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.284+0000 7fb8a6000640 1 -- 172.21.15.100:0/436049281 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fb898086500 con 0x7fb8b015c7e0 2024-04-23T14:34:00.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.284+0000 7fb8ade00640 1 --2- 172.21.15.100:0/436049281 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb88c068380 0x7fb88c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:00.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.284+0000 7fb8ade00640 1 --2- 172.21.15.100:0/436049281 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb88c068380 0x7fb88c06a840 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009af0 tx=0x7fb8a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:00.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.288+0000 7fb8a6000640 1 -- 172.21.15.100:0/436049281 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb8980567c0 con 0x7fb8b015c7e0 2024-04-23T14:34:00.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.500+0000 7fb8b5000640 1 -- 172.21.15.100:0/436049281 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) -- 0x7fb8b01534c0 con 0x7fb8b015c7e0 2024-04-23T14:34:00.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.504+0000 7fb8a6000640 1 -- 172.21.15.100:0/436049281 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v13) ==== 211+0+168 (secure 0 0 0) 0x7fb8980560c0 con 0x7fb8b015c7e0 2024-04-23T14:34:00.508 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-04-23T14:34:00.508 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAtxidmyhImMBAAa49ApQEF2Lng6rY94F1HAQ== 2024-04-23T14:34:00.508 INFO:teuthology.orchestra.run.smithi100.stdout: caps mds = "allow rwp" 2024-04-23T14:34:00.508 INFO:teuthology.orchestra.run.smithi100.stdout: caps mon = "allow r" 2024-04-23T14:34:00.508 INFO:teuthology.orchestra.run.smithi100.stdout: caps osd = "allow rw pool=cephfs_data,allow rw pool=data_new" 2024-04-23T14:34:00.508 INFO:teuthology.orchestra.run.smithi100.stderr:updated caps for client.0 2024-04-23T14:34:00.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.504+0000 7fb8b5000640 1 -- 172.21.15.100:0/436049281 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb88c068380 msgr2=0x7fb88c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:00.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.504+0000 7fb8b5000640 1 --2- 172.21.15.100:0/436049281 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb88c068380 0x7fb88c06a840 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009af0 tx=0x7fb8a0008040 comp rx=0 tx=0).stop 2024-04-23T14:34:00.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.504+0000 7fb8b5000640 1 -- 172.21.15.100:0/436049281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b015c7e0 msgr2=0x7fb8b014f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:00.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.504+0000 7fb8b5000640 1 --2- 172.21.15.100:0/436049281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b015c7e0 0x7fb8b014f390 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb89800c420 tx=0x7fb89800c8e0 comp rx=0 tx=0).stop 2024-04-23T14:34:00.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.504+0000 7fb8b5000640 1 -- 172.21.15.100:0/436049281 shutdown_connections 2024-04-23T14:34:00.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.504+0000 7fb8b5000640 1 --2- 172.21.15.100:0/436049281 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb88c068380 0x7fb88c06a840 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:00.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.504+0000 7fb8b5000640 1 --2- 172.21.15.100:0/436049281 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb8b015d600 0x7fb8b014f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:00.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.504+0000 7fb8b5000640 1 --2- 172.21.15.100:0/436049281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b015c7e0 0x7fb8b014f390 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:00.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.504+0000 7fb8b5000640 1 --2- 172.21.15.100:0/436049281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8b00cca50 0x7fb8b0155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:00.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.504+0000 7fb8b5000640 1 -- 172.21.15.100:0/436049281 >> 172.21.15.100:0/436049281 conn(0x7fb8b00cee80 msgr2=0x7fb8b0152b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:00.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.504+0000 7fb8b5000640 1 -- 172.21.15.100:0/436049281 shutdown_connections 2024-04-23T14:34:00.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.504+0000 7fb8b5000640 1 -- 172.21.15.100:0/436049281 wait complete. 2024-04-23T14:34:00.529 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:34:00.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.660+0000 7f5f76c00640 1 Processor -- start 2024-04-23T14:34:00.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.660+0000 7f5f76c00640 1 -- start start 2024-04-23T14:34:00.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.664+0000 7f5f76c00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f5f7015c7e0 0x7f5f700d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:00.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.664+0000 7f5f76c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f7015d600 0x7f5f700d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:00.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.664+0000 7f5f76c00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f700d04a0 con 0x7f5f700cca50 2024-04-23T14:34:00.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.664+0000 7f5f76c00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f5f7015db00 con 0x7f5f7015c7e0 2024-04-23T14:34:00.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.664+0000 7f5f76c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5f700d0780 con 0x7f5f7015d600 2024-04-23T14:34:00.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.664+0000 7f5f6fe00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f5f700cca50 0x7f5f700cce30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:41952/0 (socket says 172.21.15.100:41952) 2024-04-23T14:34:00.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.664+0000 7f5f6fe00640 1 -- 172.21.15.100:0/425854259 learned_addr learned my addr 172.21.15.100:0/425854259 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:00.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.664+0000 7f5f6f400640 1 --2- 172.21.15.100:0/425854259 >> v2:172.21.15.151:3300/0 conn(0x7f5f7015c7e0 0x7f5f700d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:00.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.664+0000 7f5f74c00640 1 --2- 172.21.15.100:0/425854259 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f7015d600 0x7f5f700d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:00.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.664+0000 7f5f6ea00640 1 -- 172.21.15.100:0/425854259 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2840302356 0 0) 0x7f5f700d04a0 con 0x7f5f700cca50 2024-04-23T14:34:00.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.664+0000 7f5f6ea00640 1 -- 172.21.15.100:0/425854259 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f54003530 con 0x7f5f700cca50 2024-04-23T14:34:00.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.664+0000 7f5f74c00640 1 -- 172.21.15.100:0/425854259 >> v2:172.21.15.151:3300/0 conn(0x7f5f7015c7e0 msgr2=0x7f5f700d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:00.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.664+0000 7f5f74c00640 1 --2- 172.21.15.100:0/425854259 >> v2:172.21.15.151:3300/0 conn(0x7f5f7015c7e0 0x7f5f700d1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:00.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.664+0000 7f5f74c00640 1 -- 172.21.15.100:0/425854259 >> v1:172.21.15.100:6789/0 conn(0x7f5f700cca50 legacy=0x7f5f700cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:00.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.664+0000 7f5f74c00640 1 -- 172.21.15.100:0/425854259 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f700d0a60 con 0x7f5f7015d600 2024-04-23T14:34:00.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.668+0000 7f5f74c00640 1 --2- 172.21.15.100:0/425854259 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f7015d600 0x7f5f700d17a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5f680097c0 tx=0x7f5f6801c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f814e84de8eee127 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:00.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.668+0000 7f5f6ea00640 1 -- 172.21.15.100:0/425854259 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f680ab600 con 0x7f5f7015d600 2024-04-23T14:34:00.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.668+0000 7f5f6ea00640 1 -- 172.21.15.100:0/425854259 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f680b0070 con 0x7f5f7015d600 2024-04-23T14:34:00.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.668+0000 7f5f6ea00640 1 -- 172.21.15.100:0/425854259 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f680ab9c0 con 0x7f5f7015d600 2024-04-23T14:34:00.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.668+0000 7f5f76c00640 1 -- 172.21.15.100:0/425854259 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f7015d600 msgr2=0x7f5f700d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:00.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.668+0000 7f5f76c00640 1 --2- 172.21.15.100:0/425854259 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f7015d600 0x7f5f700d17a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5f680097c0 tx=0x7f5f6801c3e0 comp rx=0 tx=0).stop 2024-04-23T14:34:00.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.668+0000 7f5f76c00640 1 -- 172.21.15.100:0/425854259 shutdown_connections 2024-04-23T14:34:00.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.668+0000 7f5f76c00640 1 --2- 172.21.15.100:0/425854259 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f7015d600 0x7f5f700d17a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:00.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.668+0000 7f5f76c00640 1 --2- 172.21.15.100:0/425854259 >> v2:172.21.15.151:3300/0 conn(0x7f5f7015c7e0 0x7f5f700d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:00.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.668+0000 7f5f76c00640 1 -- 172.21.15.100:0/425854259 >> 172.21.15.100:0/425854259 conn(0x7f5f700cee80 msgr2=0x7f5f700cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:00.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.668+0000 7f5f76c00640 1 -- 172.21.15.100:0/425854259 shutdown_connections 2024-04-23T14:34:00.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.668+0000 7f5f76c00640 1 -- 172.21.15.100:0/425854259 wait complete. 2024-04-23T14:34:00.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.668+0000 7f5f76c00640 1 Processor -- start 2024-04-23T14:34:00.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.668+0000 7f5f76c00640 1 -- start start 2024-04-23T14:34:00.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.668+0000 7f5f76c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f700cca50 0x7f5f7014c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:00.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.668+0000 7f5f76c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f7015c7e0 0x7f5f7014ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:00.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.668+0000 7f5f76c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f7015d600 0x7f5f700b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:00.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.668+0000 7f5f76c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5f700b3d70 con 0x7f5f700cca50 2024-04-23T14:34:00.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.668+0000 7f5f76c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5f700b3bf0 con 0x7f5f7015d600 2024-04-23T14:34:00.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.668+0000 7f5f76c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5f70169ad0 con 0x7f5f7015c7e0 2024-04-23T14:34:00.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.672+0000 7f5f6fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f700cca50 0x7f5f7014c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:00.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.672+0000 7f5f6fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f700cca50 0x7f5f7014c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42906/0 (socket says 172.21.15.100:42906) 2024-04-23T14:34:00.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.672+0000 7f5f6fe00640 1 -- 172.21.15.100:0/1923483240 learned_addr learned my addr 172.21.15.100:0/1923483240 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:00.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.672+0000 7f5f6f400640 1 --2- 172.21.15.100:0/1923483240 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f7015c7e0 0x7f5f7014ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:00.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.672+0000 7f5f74c00640 1 --2- 172.21.15.100:0/1923483240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f7015d600 0x7f5f700b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:00.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.672+0000 7f5f6fe00640 1 -- 172.21.15.100:0/1923483240 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f7015c7e0 msgr2=0x7f5f7014ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:00.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.672+0000 7f5f6fe00640 1 --2- 172.21.15.100:0/1923483240 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f7015c7e0 0x7f5f7014ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:00.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.672+0000 7f5f6fe00640 1 -- 172.21.15.100:0/1923483240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f7015d600 msgr2=0x7f5f700b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:00.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.672+0000 7f5f6fe00640 1 --2- 172.21.15.100:0/1923483240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f7015d600 0x7f5f700b55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:00.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.672+0000 7f5f6fe00640 1 -- 172.21.15.100:0/1923483240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f701507a0 con 0x7f5f700cca50 2024-04-23T14:34:00.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.672+0000 7f5f6f400640 1 --2- 172.21.15.100:0/1923483240 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f7015c7e0 0x7f5f7014ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:00.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.672+0000 7f5f6fe00640 1 --2- 172.21.15.100:0/1923483240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f700cca50 0x7f5f7014c330 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000cb40 tx=0x7f5f6000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:00.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.672+0000 7f5f67e00640 1 -- 172.21.15.100:0/1923483240 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60017070 con 0x7f5f700cca50 2024-04-23T14:34:00.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.672+0000 7f5f67e00640 1 -- 172.21.15.100:0/1923483240 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f6000aca0 con 0x7f5f700cca50 2024-04-23T14:34:00.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.672+0000 7f5f76c00640 1 -- 172.21.15.100:0/1923483240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f7014c870 con 0x7f5f700cca50 2024-04-23T14:34:00.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.672+0000 7f5f67e00640 1 -- 172.21.15.100:0/1923483240 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60012620 con 0x7f5f700cca50 2024-04-23T14:34:00.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.672+0000 7f5f76c00640 1 -- 172.21.15.100:0/1923483240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f70150a10 con 0x7f5f700cca50 2024-04-23T14:34:00.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.672+0000 7f5f76c00640 1 -- 172.21.15.100:0/1923483240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f70068ad0 con 0x7f5f700cca50 2024-04-23T14:34:00.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.672+0000 7f5f67e00640 1 -- 172.21.15.100:0/1923483240 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5f6001a070 con 0x7f5f700cca50 2024-04-23T14:34:00.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.672+0000 7f5f67e00640 1 --2- 172.21.15.100:0/1923483240 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5f50068380 0x7f5f5006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:00.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.672+0000 7f5f67e00640 1 -- 172.21.15.100:0/1923483240 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f5f60086ac0 con 0x7f5f700cca50 2024-04-23T14:34:00.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.676+0000 7f5f6f400640 1 --2- 172.21.15.100:0/1923483240 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5f50068380 0x7f5f5006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:00.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.676+0000 7f5f6f400640 1 --2- 172.21.15.100:0/1923483240 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5f50068380 0x7f5f5006a840 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009bb0 tx=0x7f5f5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:00.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.676+0000 7f5f67e00640 1 -- 172.21.15.100:0/1923483240 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5f60056d80 con 0x7f5f700cca50 2024-04-23T14:34:00.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.868+0000 7f5f76c00640 1 -- 172.21.15.100:0/1923483240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5f70167090 con 0x7f5f700cca50 2024-04-23T14:34:00.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.868+0000 7f5f67e00640 1 -- 172.21.15.100:0/1923483240 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) ==== 53+0+82 (secure 0 0 0) 0x7f5f60056680 con 0x7f5f700cca50 2024-04-23T14:34:00.874 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2024-04-23T14:34:00.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.872+0000 7f5f76c00640 1 -- 172.21.15.100:0/1923483240 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5f50068380 msgr2=0x7f5f5006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:00.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.872+0000 7f5f76c00640 1 --2- 172.21.15.100:0/1923483240 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5f50068380 0x7f5f5006a840 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009bb0 tx=0x7f5f5c008040 comp rx=0 tx=0).stop 2024-04-23T14:34:00.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.872+0000 7f5f76c00640 1 -- 172.21.15.100:0/1923483240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f700cca50 msgr2=0x7f5f7014c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:00.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.872+0000 7f5f76c00640 1 --2- 172.21.15.100:0/1923483240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f700cca50 0x7f5f7014c330 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000cb40 tx=0x7f5f6000a060 comp rx=0 tx=0).stop 2024-04-23T14:34:00.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.872+0000 7f5f76c00640 1 -- 172.21.15.100:0/1923483240 shutdown_connections 2024-04-23T14:34:00.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.872+0000 7f5f76c00640 1 --2- 172.21.15.100:0/1923483240 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5f50068380 0x7f5f5006a840 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:00.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.872+0000 7f5f76c00640 1 --2- 172.21.15.100:0/1923483240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f7015d600 0x7f5f700b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:00.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.872+0000 7f5f76c00640 1 --2- 172.21.15.100:0/1923483240 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5f7015c7e0 0x7f5f7014ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:00.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.872+0000 7f5f76c00640 1 --2- 172.21.15.100:0/1923483240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f700cca50 0x7f5f7014c330 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:00.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.872+0000 7f5f76c00640 1 -- 172.21.15.100:0/1923483240 >> 172.21.15.100:0/1923483240 conn(0x7f5f700cee80 msgr2=0x7f5f700b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:00.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.872+0000 7f5f76c00640 1 -- 172.21.15.100:0/1923483240 shutdown_connections 2024-04-23T14:34:00.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:00.872+0000 7f5f76c00640 1 -- 172.21.15.100:0/1923483240 wait complete. 2024-04-23T14:34:00.895 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:34:00.896 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:34:00.896 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-04-23T14:34:00.896 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:34:00.896 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:00.896 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:34:00.896 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:34:00.896 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:34:00.896 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:34:00.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:00.896 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-04-23T14:34:00.905 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:34:00.905 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:34:00.905 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:34:00.905 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:00.906 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-23T14:34:00.906 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:00.906 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:34:00.906 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:00.906 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-04-23T14:34:00.906 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:34:00.906 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:00.906 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-04-23T14:34:00.906 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:34:00.906 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:00.906 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:34:00.906 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:00.906 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-23T14:34:00.907 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42343sec preferred_lft 42343sec 2024-04-23T14:34:00.907 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-04-23T14:34:00.907 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:00.907 INFO:teuthology.orchestra.run.smithi100.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-23T14:34:00.907 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 02:42:ff:fc:d1:cc brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:00.907 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-23T14:34:00.907 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:00.907 INFO:teuthology.orchestra.run.smithi100.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:34:00.907 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:00.907 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:34:00.907 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:00.907 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-23T14:34:00.907 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:00.907 INFO:teuthology.orchestra.run.smithi100.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:34:00.908 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:34:00.908 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-23T14:34:00.908 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:00.908 INFO:teuthology.orchestra.run.smithi100.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:34:00.908 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:34:00.908 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-23T14:34:00.908 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:00.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:00.908 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:34:00.963 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-04-23T14:34:00.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:00.965 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-04-23T14:34:01.017 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-04-23T14:34:01.018 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:34:01.018 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-23T14:34:01.018 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-23T14:34:01.018 INFO:teuthology.orchestra.run.smithi100.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-23T14:34:01.018 INFO:teuthology.orchestra.run.smithi100.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-23T14:34:01.018 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-23T14:34:01.018 INFO:teuthology.orchestra.run.smithi100.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:34:01.018 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:34:01.018 INFO:teuthology.orchestra.run.smithi100.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:34:01.018 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:34:01.019 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:34:01.019 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-23T14:34:01.019 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-23T14:34:01.019 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-23T14:34:01.019 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-23T14:34:01.019 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-23T14:34:01.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:34:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:34:01.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:01.151 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-23T14:34:01.159 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:34:01.160 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:34:01.160 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:34:01.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:01.160 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-04-23T14:34:01.222 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:34:01.222 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:34:01.222 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:01.270 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:34:01.271 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:34:01.271 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:01.326 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-23T14:34:01.390 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:34:01.439 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-23T14:34:01.439 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-23T14:34:01.440 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-23T14:34:01.440 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-23T14:34:01.440 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-23T14:34:01.440 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-23T14:34:01.440 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-23T14:34:01.440 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-23T14:34:01.440 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-23T14:34:01.440 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-23T14:34:01.440 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-23T14:34:01.440 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-23T14:34:01.440 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:34:01.440 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-23T14:34:01.440 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-23T14:34:01.441 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-23T14:34:01.441 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-23T14:34:01.441 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-23T14:34:01.441 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-23T14:34:01.441 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-23T14:34:01.441 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:34:01.441 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-23T14:34:01.441 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-23T14:34:01.441 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-23T14:34:01.441 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:34:01.441 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:34:01.441 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:34:01.441 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:34:01.441 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-23T14:34:01.441 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:34:01.441 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:34:01.442 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:34:01.442 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:34:01.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:01.443 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-23T14:34:01.498 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:34:01.498 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:34:01.541 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-23T14:34:01.561 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:34:01.612 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-23T14:34:01.612 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-23T14:34:01.612 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-23T14:34:01.612 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-23T14:34:01.612 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-23T14:34:01.612 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-23T14:34:01.612 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-23T14:34:01.612 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-23T14:34:01.612 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-23T14:34:01.612 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-23T14:34:01.613 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-23T14:34:01.613 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-23T14:34:01.613 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:34:01.613 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-23T14:34:01.613 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-23T14:34:01.613 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-23T14:34:01.613 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-23T14:34:01.613 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-23T14:34:01.613 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-23T14:34:01.613 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-23T14:34:01.613 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:34:01.613 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-23T14:34:01.613 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-23T14:34:01.613 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-23T14:34:01.613 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:34:01.614 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:34:01.614 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:34:01.614 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:34:01.614 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-23T14:34:01.614 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:34:01.614 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:34:01.614 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:34:01.614 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:34:01.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:01.615 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-23T14:34:01.634 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:2024-04-23T14:34:01.628+0000 7ff2fc67e580 -1 init, newargv = 0x559201540e40 newargc=15 2024-04-23T14:34:01.635 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[40322]: starting ceph client 2024-04-23T14:34:01.642 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[40322]: starting fuse 2024-04-23T14:34:01.644 INFO:teuthology.orchestra.run.smithi100.stdout:44 2024-04-23T14:34:01.644 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-23T14:34:01.644 DEBUG:teuthology.orchestra.run.smithi100:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:34:01.644 DEBUG:teuthology.orchestra.run.smithi100:> import glob 2024-04-23T14:34:01.644 DEBUG:teuthology.orchestra.run.smithi100:> import re 2024-04-23T14:34:01.644 DEBUG:teuthology.orchestra.run.smithi100:> import os 2024-04-23T14:34:01.644 DEBUG:teuthology.orchestra.run.smithi100:> import subprocess 2024-04-23T14:34:01.645 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:34:01.645 DEBUG:teuthology.orchestra.run.smithi100:> def _find_admin_socket(client_name): 2024-04-23T14:34:01.645 DEBUG:teuthology.orchestra.run.smithi100:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:34:01.645 DEBUG:teuthology.orchestra.run.smithi100:> files = glob.glob(asok_path) 2024-04-23T14:34:01.645 DEBUG:teuthology.orchestra.run.smithi100:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:34:01.645 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:34:01.645 DEBUG:teuthology.orchestra.run.smithi100:> # Given a non-glob path, it better be there 2024-04-23T14:34:01.645 DEBUG:teuthology.orchestra.run.smithi100:> if "*" not in asok_path: 2024-04-23T14:34:01.645 DEBUG:teuthology.orchestra.run.smithi100:> assert(len(files) == 1) 2024-04-23T14:34:01.645 DEBUG:teuthology.orchestra.run.smithi100:> return files[0] 2024-04-23T14:34:01.645 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:34:01.645 DEBUG:teuthology.orchestra.run.smithi100:> for f in files: 2024-04-23T14:34:01.645 DEBUG:teuthology.orchestra.run.smithi100:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:34:01.645 DEBUG:teuthology.orchestra.run.smithi100:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:34:01.645 DEBUG:teuthology.orchestra.run.smithi100:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:34:01.645 DEBUG:teuthology.orchestra.run.smithi100:> contents = proc_f.read() 2024-04-23T14:34:01.645 DEBUG:teuthology.orchestra.run.smithi100:> if mountpoint in contents: 2024-04-23T14:34:01.646 DEBUG:teuthology.orchestra.run.smithi100:> return f 2024-04-23T14:34:01.646 DEBUG:teuthology.orchestra.run.smithi100:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:34:01.646 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:34:01.646 DEBUG:teuthology.orchestra.run.smithi100:> print(_find_admin_socket("client.0")) 2024-04-23T14:34:01.646 DEBUG:teuthology.orchestra.run.smithi100:> ' 2024-04-23T14:34:01.767 INFO:teuthology.orchestra.run.smithi100.stdout:/var/run/ceph/ceph-client.0.40322.asok 2024-04-23T14:34:01.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:34:01.779 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.40322.asok 2024-04-23T14:34:01.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:01.779 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.40322.asok status 2024-04-23T14:34:01.948 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:34:01.948 INFO:teuthology.orchestra.run.smithi100.stdout: "metadata": { 2024-04-23T14:34:01.948 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:34:01.948 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:34:01.948 INFO:teuthology.orchestra.run.smithi100.stdout: "entity_id": "0", 2024-04-23T14:34:01.948 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": "smithi100", 2024-04-23T14:34:01.948 INFO:teuthology.orchestra.run.smithi100.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:34:01.949 INFO:teuthology.orchestra.run.smithi100.stdout: "pid": "40322", 2024-04-23T14:34:01.949 INFO:teuthology.orchestra.run.smithi100.stdout: "root": "/" 2024-04-23T14:34:01.949 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:01.949 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_count": 0, 2024-04-23T14:34:01.949 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_pinned_count": 0, 2024-04-23T14:34:01.949 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 5913, 2024-04-23T14:34:01.949 INFO:teuthology.orchestra.run.smithi100.stdout: "inst": { 2024-04-23T14:34:01.949 INFO:teuthology.orchestra.run.smithi100.stdout: "name": { 2024-04-23T14:34:01.949 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "client", 2024-04-23T14:34:01.949 INFO:teuthology.orchestra.run.smithi100.stdout: "num": 5913 2024-04-23T14:34:01.949 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:01.949 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-23T14:34:01.949 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-23T14:34:01.950 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:34:01.950 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 646614947 2024-04-23T14:34:01.950 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:01.950 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:01.950 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-23T14:34:01.950 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-23T14:34:01.950 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:34:01.950 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 646614947 2024-04-23T14:34:01.950 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:01.950 INFO:teuthology.orchestra.run.smithi100.stdout: "inst_str": "client.5913 192.168.0.1:0/646614947", 2024-04-23T14:34:01.950 INFO:teuthology.orchestra.run.smithi100.stdout: "addr_str": "192.168.0.1:0/646614947", 2024-04-23T14:34:01.950 INFO:teuthology.orchestra.run.smithi100.stdout: "inode_count": 1, 2024-04-23T14:34:01.950 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_epoch": 19, 2024-04-23T14:34:01.950 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch": 41, 2024-04-23T14:34:01.951 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:34:01.951 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklisted": false, 2024-04-23T14:34:01.951 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs" 2024-04-23T14:34:01.951 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:34:01.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:01.963 DEBUG:teuthology.orchestra.run.smithi100:> stat --file-system '--printf=%T 2024-04-23T14:34:01.963 DEBUG:teuthology.orchestra.run.smithi100:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:01.972 INFO:teuthology.orchestra.run.smithi100.stdout:fuseblk 2024-04-23T14:34:01.972 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:01.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:01.972 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:02.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:02.037 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2024-04-23T14:34:02.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:02.090 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2024-04-23T14:34:02.144 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:34:02.145 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:02.195 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:02.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.368+0000 7f5925a00640 1 Processor -- start 2024-04-23T14:34:02.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.368+0000 7f5925a00640 1 -- start start 2024-04-23T14:34:02.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.368+0000 7f5925a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f592006d350 0x7f5920073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:02.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.368+0000 7f5925a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f592015fa90 0x7f592006d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:02.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.368+0000 7f5925a00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f5920162390 0x7f59201627f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:02.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.368+0000 7f5925a00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f592006eaf0 con 0x7f592015fa90 2024-04-23T14:34:02.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.368+0000 7f5925a00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f592006e530 con 0x7f5920162390 2024-04-23T14:34:02.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.368+0000 7f5925a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5920073880 con 0x7f592006d350 2024-04-23T14:34:02.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.368+0000 7f591e000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f592015fa90 0x7f592006d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:02.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.368+0000 7f591e000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f592015fa90 0x7f592006d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42912/0 (socket says 172.21.15.100:42912) 2024-04-23T14:34:02.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.368+0000 7f591e000640 1 -- 172.21.15.100:0/934346911 learned_addr learned my addr 172.21.15.100:0/934346911 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:02.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.368+0000 7f591e000640 1 -- 172.21.15.100:0/934346911 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f592006d350 msgr2=0x7f5920073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:02.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.368+0000 7f591ea00640 1 --2- 172.21.15.100:0/934346911 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f592006d350 0x7f5920073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:02.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.368+0000 7f591f400640 1 --2- 172.21.15.100:0/934346911 >> v2:172.21.15.151:3300/0 conn(0x7f5920162390 0x7f59201627f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:02.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.368+0000 7f591e000640 1 --2- 172.21.15.100:0/934346911 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f592006d350 0x7f5920073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:02.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.368+0000 7f591e000640 1 -- 172.21.15.100:0/934346911 >> v2:172.21.15.151:3300/0 conn(0x7f5920162390 msgr2=0x7f59201627f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:02.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.368+0000 7f591e000640 1 --2- 172.21.15.100:0/934346911 >> v2:172.21.15.151:3300/0 conn(0x7f5920162390 0x7f59201627f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:02.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.368+0000 7f591e000640 1 -- 172.21.15.100:0/934346911 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592006dc70 con 0x7f592015fa90 2024-04-23T14:34:02.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.368+0000 7f591ea00640 1 --2- 172.21.15.100:0/934346911 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f592006d350 0x7f5920073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:02.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.368+0000 7f591f400640 1 --2- 172.21.15.100:0/934346911 >> v2:172.21.15.151:3300/0 conn(0x7f5920162390 0x7f59201627f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:02.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.372+0000 7f591e000640 1 --2- 172.21.15.100:0/934346911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f592015fa90 0x7f592006d730 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f59200cd2b0 tx=0x7f590c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6aa7094850e65f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:02.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.372+0000 7f591d600640 1 -- 172.21.15.100:0/934346911 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c0ab620 con 0x7f592015fa90 2024-04-23T14:34:02.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.372+0000 7f591d600640 1 -- 172.21.15.100:0/934346911 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f590c0b0070 con 0x7f592015fa90 2024-04-23T14:34:02.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.372+0000 7f591d600640 1 -- 172.21.15.100:0/934346911 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c0ab9e0 con 0x7f592015fa90 2024-04-23T14:34:02.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.372+0000 7f5925a00640 1 -- 172.21.15.100:0/934346911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f592015fa90 msgr2=0x7f592006d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:02.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.372+0000 7f5925a00640 1 --2- 172.21.15.100:0/934346911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f592015fa90 0x7f592006d730 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f59200cd2b0 tx=0x7f590c01c460 comp rx=0 tx=0).stop 2024-04-23T14:34:02.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.372+0000 7f5925a00640 1 -- 172.21.15.100:0/934346911 shutdown_connections 2024-04-23T14:34:02.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.372+0000 7f5925a00640 1 --2- 172.21.15.100:0/934346911 >> v2:172.21.15.151:3300/0 conn(0x7f5920162390 0x7f59201627f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:02.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.372+0000 7f5925a00640 1 --2- 172.21.15.100:0/934346911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f592015fa90 0x7f592006d730 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:02.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.372+0000 7f5925a00640 1 --2- 172.21.15.100:0/934346911 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f592006d350 0x7f5920073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:02.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.372+0000 7f5925a00640 1 -- 172.21.15.100:0/934346911 >> 172.21.15.100:0/934346911 conn(0x7f5920072600 msgr2=0x7f5920072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:02.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.372+0000 7f5925a00640 1 -- 172.21.15.100:0/934346911 shutdown_connections 2024-04-23T14:34:02.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.372+0000 7f5925a00640 1 -- 172.21.15.100:0/934346911 wait complete. 2024-04-23T14:34:02.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f5925a00640 1 Processor -- start 2024-04-23T14:34:02.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f5925a00640 1 -- start start 2024-04-23T14:34:02.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f5925a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f592006d350 0x7f5920168e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:02.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f5925a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f592015fa90 0x7f5920142790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:02.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f5925a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5920162390 0x7f5920153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:02.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f5925a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f592006f770 con 0x7f592015fa90 2024-04-23T14:34:02.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f5925a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5920163140 con 0x7f5920162390 2024-04-23T14:34:02.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f5925a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f592006f8f0 con 0x7f592006d350 2024-04-23T14:34:02.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f591e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f592015fa90 0x7f5920142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:02.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f591e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f592015fa90 0x7f5920142790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42922/0 (socket says 172.21.15.100:42922) 2024-04-23T14:34:02.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f591e000640 1 -- 172.21.15.100:0/990576382 learned_addr learned my addr 172.21.15.100:0/990576382 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:02.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f591ea00640 1 --2- 172.21.15.100:0/990576382 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f592006d350 0x7f5920168e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:02.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f591f400640 1 --2- 172.21.15.100:0/990576382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5920162390 0x7f5920153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:02.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f591e000640 1 -- 172.21.15.100:0/990576382 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f592006d350 msgr2=0x7f5920168e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:02.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f591e000640 1 --2- 172.21.15.100:0/990576382 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f592006d350 0x7f5920168e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:02.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f591e000640 1 -- 172.21.15.100:0/990576382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5920162390 msgr2=0x7f5920153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:02.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f591e000640 1 --2- 172.21.15.100:0/990576382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5920162390 0x7f5920153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:02.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f591e000640 1 -- 172.21.15.100:0/990576382 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592013f340 con 0x7f592015fa90 2024-04-23T14:34:02.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f591ea00640 1 --2- 172.21.15.100:0/990576382 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f592006d350 0x7f5920168e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:02.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f591e000640 1 --2- 172.21.15.100:0/990576382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f592015fa90 0x7f5920142790 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f590c01ab10 tx=0x7f590c01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:02.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f5916a00640 1 -- 172.21.15.100:0/990576382 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c0abd70 con 0x7f592015fa90 2024-04-23T14:34:02.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f5916a00640 1 -- 172.21.15.100:0/990576382 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f590c0be3f0 con 0x7f592015fa90 2024-04-23T14:34:02.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f5916a00640 1 -- 172.21.15.100:0/990576382 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c0be7e0 con 0x7f592015fa90 2024-04-23T14:34:02.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f5925a00640 1 -- 172.21.15.100:0/990576382 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f592013f620 con 0x7f592015fa90 2024-04-23T14:34:02.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.376+0000 7f5925a00640 1 -- 172.21.15.100:0/990576382 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f592013fbe0 con 0x7f592015fa90 2024-04-23T14:34:02.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.380+0000 7f5925a00640 1 -- 172.21.15.100:0/990576382 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5920068ad0 con 0x7f592015fa90 2024-04-23T14:34:02.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.380+0000 7f5916a00640 1 -- 172.21.15.100:0/990576382 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f590c0b43f0 con 0x7f592015fa90 2024-04-23T14:34:02.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.380+0000 7f5916a00640 1 --2- 172.21.15.100:0/990576382 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5900068380 0x7f590006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:02.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.380+0000 7f5916a00640 1 -- 172.21.15.100:0/990576382 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f590c120e00 con 0x7f592015fa90 2024-04-23T14:34:02.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.380+0000 7f591ea00640 1 --2- 172.21.15.100:0/990576382 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5900068380 0x7f590006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:02.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.380+0000 7f591ea00640 1 --2- 172.21.15.100:0/990576382 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5900068380 0x7f590006a840 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5910007130 tx=0x7f5910007670 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:02.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.384+0000 7f5916a00640 1 -- 172.21.15.100:0/990576382 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f590c016110 con 0x7f592015fa90 2024-04-23T14:34:02.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.540+0000 7f5925a00640 1 -- 172.21.15.100:0/990576382 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5920073120 con 0x7f592015fa90 2024-04-23T14:34:02.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.540+0000 7f5916a00640 1 -- 172.21.15.100:0/990576382 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+18766 (secure 0 0 0) 0x7f590c0f09e0 con 0x7f592015fa90 2024-04-23T14:34:02.547 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:02.547 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":41,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:33:49.609841+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:33:36.592437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-23T14:33:37.882390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-04-23T14:33:47.299378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:02.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.544+0000 7f5925a00640 1 -- 172.21.15.100:0/990576382 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5900068380 msgr2=0x7f590006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:02.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.544+0000 7f5925a00640 1 --2- 172.21.15.100:0/990576382 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5900068380 0x7f590006a840 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5910007130 tx=0x7f5910007670 comp rx=0 tx=0).stop 2024-04-23T14:34:02.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.544+0000 7f5925a00640 1 -- 172.21.15.100:0/990576382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f592015fa90 msgr2=0x7f5920142790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:02.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.544+0000 7f5925a00640 1 --2- 172.21.15.100:0/990576382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f592015fa90 0x7f5920142790 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f590c01ab10 tx=0x7f590c01aca0 comp rx=0 tx=0).stop 2024-04-23T14:34:02.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.544+0000 7f5925a00640 1 -- 172.21.15.100:0/990576382 shutdown_connections 2024-04-23T14:34:02.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.544+0000 7f5925a00640 1 --2- 172.21.15.100:0/990576382 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5900068380 0x7f590006a840 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:02.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.544+0000 7f5925a00640 1 --2- 172.21.15.100:0/990576382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5920162390 0x7f5920153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:02.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.544+0000 7f5925a00640 1 --2- 172.21.15.100:0/990576382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f592015fa90 0x7f5920142790 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:02.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.544+0000 7f5925a00640 1 --2- 172.21.15.100:0/990576382 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f592006d350 0x7f5920168e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:02.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.544+0000 7f5925a00640 1 -- 172.21.15.100:0/990576382 >> 172.21.15.100:0/990576382 conn(0x7f5920072600 msgr2=0x7f59200aeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:02.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.544+0000 7f5925a00640 1 -- 172.21.15.100:0/990576382 shutdown_connections 2024-04-23T14:34:02.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.544+0000 7f5925a00640 1 -- 172.21.15.100:0/990576382 wait complete. 2024-04-23T14:34:02.569 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:33:36.592437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:33:37.882390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-04-23T14:33:47.299378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:34:02.569 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:34:02.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.704+0000 7f9192400640 1 Processor -- start 2024-04-23T14:34:02.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.704+0000 7f9192400640 1 -- start start 2024-04-23T14:34:02.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.704+0000 7f9192400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f918c15c7a0 0x7f918c0af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:02.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.704+0000 7f9192400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f918c06bb20 0x7f918c0ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:02.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.704+0000 7f9192400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f918c06cbb0 con 0x7f918c0cd130 2024-04-23T14:34:02.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.704+0000 7f9192400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f918c0cd5a0 con 0x7f918c15c7a0 2024-04-23T14:34:02.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.704+0000 7f9192400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f918c06d450 con 0x7f918c06bb20 2024-04-23T14:34:02.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.704+0000 7f918b400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f918c15c7a0 0x7f918c0af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:02.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.704+0000 7f918b400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f918c15c7a0 0x7f918c0af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42924/0 (socket says 172.21.15.100:42924) 2024-04-23T14:34:02.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.704+0000 7f918b400640 1 -- 172.21.15.100:0/3306481716 learned_addr learned my addr 172.21.15.100:0/3306481716 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:02.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.704+0000 7f918aa00640 1 --2- 172.21.15.100:0/3306481716 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f918c06bb20 0x7f918c0ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:02.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.704+0000 7f918b400640 1 -- 172.21.15.100:0/3306481716 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f918c06bb20 msgr2=0x7f918c0ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:02.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.704+0000 7f918b400640 1 --2- 172.21.15.100:0/3306481716 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f918c06bb20 0x7f918c0ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:02.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.704+0000 7f918b400640 1 -- 172.21.15.100:0/3306481716 >> v1:172.21.15.151:6789/0 conn(0x7f918c0cd130 legacy=0x7f918c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:02.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.704+0000 7f918b400640 1 -- 172.21.15.100:0/3306481716 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918c06ce90 con 0x7f918c15c7a0 2024-04-23T14:34:02.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.708+0000 7f918b400640 1 --2- 172.21.15.100:0/3306481716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918c15c7a0 0x7f918c0af190 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f917c009e50 tx=0x7f917c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4362cced099a6e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:02.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.708+0000 7f918a000640 1 -- 172.21.15.100:0/3306481716 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c0a5660 con 0x7f918c15c7a0 2024-04-23T14:34:02.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.708+0000 7f918a000640 1 -- 172.21.15.100:0/3306481716 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f917c0a5800 con 0x7f918c15c7a0 2024-04-23T14:34:02.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.708+0000 7f918a000640 1 -- 172.21.15.100:0/3306481716 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c0a5bd0 con 0x7f918c15c7a0 2024-04-23T14:34:02.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.708+0000 7f9192400640 1 -- 172.21.15.100:0/3306481716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918c15c7a0 msgr2=0x7f918c0af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:02.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.708+0000 7f9192400640 1 --2- 172.21.15.100:0/3306481716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918c15c7a0 0x7f918c0af190 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f917c009e50 tx=0x7f917c01c9a0 comp rx=0 tx=0).stop 2024-04-23T14:34:02.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.708+0000 7f9192400640 1 -- 172.21.15.100:0/3306481716 shutdown_connections 2024-04-23T14:34:02.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.708+0000 7f9192400640 1 --2- 172.21.15.100:0/3306481716 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f918c06bb20 0x7f918c0ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:02.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.708+0000 7f9192400640 1 --2- 172.21.15.100:0/3306481716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918c15c7a0 0x7f918c0af190 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:02.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.708+0000 7f9192400640 1 -- 172.21.15.100:0/3306481716 >> 172.21.15.100:0/3306481716 conn(0x7f918c0b5200 msgr2=0x7f918c0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:02.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.708+0000 7f9192400640 1 -- 172.21.15.100:0/3306481716 shutdown_connections 2024-04-23T14:34:02.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.708+0000 7f9192400640 1 -- 172.21.15.100:0/3306481716 wait complete. 2024-04-23T14:34:02.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f9192400640 1 Processor -- start 2024-04-23T14:34:02.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f9192400640 1 -- start start 2024-04-23T14:34:02.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f9192400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f918c06bb20 0x7f918c0cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:02.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f9192400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f918c0cd130 0x7f918c0afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:02.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f9192400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918c15c7a0 0x7f918c0998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:02.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f9192400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f918c169590 con 0x7f918c15c7a0 2024-04-23T14:34:02.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f9192400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f918c169410 con 0x7f918c0cd130 2024-04-23T14:34:02.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f9192400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f918c169710 con 0x7f918c06bb20 2024-04-23T14:34:02.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f918be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918c15c7a0 0x7f918c0998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:02.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f918be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918c15c7a0 0x7f918c0998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42932/0 (socket says 172.21.15.100:42932) 2024-04-23T14:34:02.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f918be00640 1 -- 172.21.15.100:0/144241780 learned_addr learned my addr 172.21.15.100:0/144241780 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:02.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f918b400640 1 --2- 172.21.15.100:0/144241780 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f918c06bb20 0x7f918c0cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:02.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f918aa00640 1 --2- 172.21.15.100:0/144241780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f918c0cd130 0x7f918c0afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:02.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f918be00640 1 -- 172.21.15.100:0/144241780 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f918c06bb20 msgr2=0x7f918c0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:02.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f918be00640 1 --2- 172.21.15.100:0/144241780 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f918c06bb20 0x7f918c0cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:02.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f918be00640 1 -- 172.21.15.100:0/144241780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f918c0cd130 msgr2=0x7f918c0afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:02.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f918be00640 1 --2- 172.21.15.100:0/144241780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f918c0cd130 0x7f918c0afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:02.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f918be00640 1 -- 172.21.15.100:0/144241780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918c0c6520 con 0x7f918c15c7a0 2024-04-23T14:34:02.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f918be00640 1 --2- 172.21.15.100:0/144241780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918c15c7a0 0x7f918c0998c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f918400c370 tx=0x7f918400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:02.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f9183400640 1 -- 172.21.15.100:0/144241780 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9184005220 con 0x7f918c15c7a0 2024-04-23T14:34:02.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f9183400640 1 -- 172.21.15.100:0/144241780 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91840053c0 con 0x7f918c15c7a0 2024-04-23T14:34:02.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f9183400640 1 -- 172.21.15.100:0/144241780 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9184007270 con 0x7f918c15c7a0 2024-04-23T14:34:02.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f9192400640 1 -- 172.21.15.100:0/144241780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f918c0c67c0 con 0x7f918c15c7a0 2024-04-23T14:34:02.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f9192400640 1 -- 172.21.15.100:0/144241780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f918c15ea90 con 0x7f918c15c7a0 2024-04-23T14:34:02.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.712+0000 7f9192400640 1 -- 172.21.15.100:0/144241780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f918c15ec90 con 0x7f918c15c7a0 2024-04-23T14:34:02.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.716+0000 7f9183400640 1 -- 172.21.15.100:0/144241780 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9184019020 con 0x7f918c15c7a0 2024-04-23T14:34:02.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.716+0000 7f9183400640 1 --2- 172.21.15.100:0/144241780 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f916c068380 0x7f916c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:02.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.716+0000 7f9183400640 1 -- 172.21.15.100:0/144241780 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f9184085f40 con 0x7f918c15c7a0 2024-04-23T14:34:02.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.716+0000 7f918b400640 1 --2- 172.21.15.100:0/144241780 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f916c068380 0x7f916c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:02.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.716+0000 7f918b400640 1 --2- 172.21.15.100:0/144241780 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f916c068380 0x7f916c06a840 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f917c0ae000 tx=0x7f917c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:02.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.720+0000 7f9183400640 1 -- 172.21.15.100:0/144241780 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f918c15ec90 con 0x7f918c15c7a0 2024-04-23T14:34:02.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.876+0000 7f9192400640 1 -- 172.21.15.100:0/144241780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f918c068ad0 con 0x7f918c15c7a0 2024-04-23T14:34:02.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.876+0000 7f9183400640 1 -- 172.21.15.100:0/144241780 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7f9184055a90 con 0x7f918c15c7a0 2024-04-23T14:34:02.882 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:34:02.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.880+0000 7f9192400640 1 -- 172.21.15.100:0/144241780 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f916c068380 msgr2=0x7f916c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:02.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.880+0000 7f9192400640 1 --2- 172.21.15.100:0/144241780 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f916c068380 0x7f916c06a840 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f917c0ae000 tx=0x7f917c0a7070 comp rx=0 tx=0).stop 2024-04-23T14:34:02.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.880+0000 7f9192400640 1 -- 172.21.15.100:0/144241780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918c15c7a0 msgr2=0x7f918c0998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:02.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.880+0000 7f9192400640 1 --2- 172.21.15.100:0/144241780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918c15c7a0 0x7f918c0998c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f918400c370 tx=0x7f918400c830 comp rx=0 tx=0).stop 2024-04-23T14:34:02.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.880+0000 7f9192400640 1 -- 172.21.15.100:0/144241780 shutdown_connections 2024-04-23T14:34:02.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.880+0000 7f9192400640 1 --2- 172.21.15.100:0/144241780 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f916c068380 0x7f916c06a840 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:02.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.880+0000 7f9192400640 1 --2- 172.21.15.100:0/144241780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918c15c7a0 0x7f918c0998c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:02.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.880+0000 7f9192400640 1 --2- 172.21.15.100:0/144241780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f918c0cd130 0x7f918c0afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:02.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.880+0000 7f9192400640 1 --2- 172.21.15.100:0/144241780 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f918c06bb20 0x7f918c0cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:02.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.880+0000 7f9192400640 1 -- 172.21.15.100:0/144241780 >> 172.21.15.100:0/144241780 conn(0x7f918c0b5200 msgr2=0x7f918c0b55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:02.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.880+0000 7f9192400640 1 -- 172.21.15.100:0/144241780 shutdown_connections 2024-04-23T14:34:02.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:02.880+0000 7f9192400640 1 -- 172.21.15.100:0/144241780 wait complete. 2024-04-23T14:34:02.903 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:34:03.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.036+0000 7fe001800640 1 Processor -- start 2024-04-23T14:34:03.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.036+0000 7fe001800640 1 -- start start 2024-04-23T14:34:03.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.036+0000 7fe001800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdffc0cca50 0x7fdffc0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:03.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.036+0000 7fe001800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fdffc15c7e0 0x7fdffc160170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:03.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.036+0000 7fe001800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdffc0cd370 con 0x7fdffc15d600 2024-04-23T14:34:03.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.036+0000 7fe001800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fdffc0d2270 con 0x7fdffc15c7e0 2024-04-23T14:34:03.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.036+0000 7fe001800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fdffc15db00 con 0x7fdffc0cca50 2024-04-23T14:34:03.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.036+0000 7fdffaa00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdffc0cca50 0x7fdffc0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:03.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.036+0000 7fdff2000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fdffc15c7e0 0x7fdffc160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:03.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.036+0000 7fdffaa00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdffc0cca50 0x7fdffc0cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:33888/0 (socket says 172.21.15.100:33888) 2024-04-23T14:34:03.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.036+0000 7fdff2000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fdffc15c7e0 0x7fdffc160170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:37496/0 (socket says 172.21.15.100:37496) 2024-04-23T14:34:03.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.036+0000 7fdffaa00640 1 -- 172.21.15.100:0/3153076425 learned_addr learned my addr 172.21.15.100:0/3153076425 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:03.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.036+0000 7fdffaa00640 1 -- 172.21.15.100:0/3153076425 >> v2:172.21.15.151:3300/0 conn(0x7fdffc15c7e0 msgr2=0x7fdffc160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:03.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.036+0000 7fdffaa00640 1 --2- 172.21.15.100:0/3153076425 >> v2:172.21.15.151:3300/0 conn(0x7fdffc15c7e0 0x7fdffc160170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.036+0000 7fdffaa00640 1 -- 172.21.15.100:0/3153076425 >> v1:172.21.15.151:6789/0 conn(0x7fdffc15d600 legacy=0x7fdffc1606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:03.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.036+0000 7fdffa000640 1 -- 172.21.15.100:0/3153076425 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1717425460 0 0) 0x7fdffc0cd370 con 0x7fdffc15d600 2024-04-23T14:34:03.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.036+0000 7fdffaa00640 1 -- 172.21.15.100:0/3153076425 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdffc0d04a0 con 0x7fdffc0cca50 2024-04-23T14:34:03.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.040+0000 7fdffaa00640 1 --2- 172.21.15.100:0/3153076425 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdffc0cca50 0x7fdffc0cce30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009870 tx=0x7fdfec01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac7ea3bc2a674678 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:03.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.040+0000 7fdffa000640 1 -- 172.21.15.100:0/3153076425 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec0aa600 con 0x7fdffc0cca50 2024-04-23T14:34:03.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.040+0000 7fdffa000640 1 -- 172.21.15.100:0/3153076425 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfec0af070 con 0x7fdffc0cca50 2024-04-23T14:34:03.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.040+0000 7fdffa000640 1 -- 172.21.15.100:0/3153076425 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec0aa9c0 con 0x7fdffc0cca50 2024-04-23T14:34:03.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.040+0000 7fe001800640 1 -- 172.21.15.100:0/3153076425 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdffc0cca50 msgr2=0x7fdffc0cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:03.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.040+0000 7fe001800640 1 --2- 172.21.15.100:0/3153076425 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdffc0cca50 0x7fdffc0cce30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009870 tx=0x7fdfec01c4a0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.040+0000 7fe001800640 1 -- 172.21.15.100:0/3153076425 shutdown_connections 2024-04-23T14:34:03.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.040+0000 7fe001800640 1 --2- 172.21.15.100:0/3153076425 >> v2:172.21.15.151:3300/0 conn(0x7fdffc15c7e0 0x7fdffc160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.040+0000 7fe001800640 1 --2- 172.21.15.100:0/3153076425 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdffc0cca50 0x7fdffc0cce30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.040+0000 7fe001800640 1 -- 172.21.15.100:0/3153076425 >> 172.21.15.100:0/3153076425 conn(0x7fdffc0cee80 msgr2=0x7fdffc0cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:03.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.040+0000 7fe001800640 1 -- 172.21.15.100:0/3153076425 shutdown_connections 2024-04-23T14:34:03.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.040+0000 7fe001800640 1 -- 172.21.15.100:0/3153076425 wait complete. 2024-04-23T14:34:03.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fe001800640 1 Processor -- start 2024-04-23T14:34:03.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fe001800640 1 -- start start 2024-04-23T14:34:03.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fe001800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdffc0cca50 0x7fdffc1505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:03.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fe001800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdffc15c7e0 0x7fdffc161280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:03.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fe001800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdffc15d600 0x7fdffc0cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:03.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fe001800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc0b5430 con 0x7fdffc15c7e0 2024-04-23T14:34:03.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fe001800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc0b52b0 con 0x7fdffc15d600 2024-04-23T14:34:03.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fe001800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fdffc0b3bf0 con 0x7fdffc0cca50 2024-04-23T14:34:03.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fdff2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdffc15c7e0 0x7fdffc161280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:03.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fdff2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdffc15c7e0 0x7fdffc161280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42934/0 (socket says 172.21.15.100:42934) 2024-04-23T14:34:03.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fdff2000640 1 -- 172.21.15.100:0/3663671479 learned_addr learned my addr 172.21.15.100:0/3663671479 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fdffb400640 1 --2- 172.21.15.100:0/3663671479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdffc15d600 0x7fdffc0cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fdffaa00640 1 --2- 172.21.15.100:0/3663671479 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdffc0cca50 0x7fdffc1505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fdff2000640 1 -- 172.21.15.100:0/3663671479 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdffc0cca50 msgr2=0x7fdffc1505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fdff2000640 1 --2- 172.21.15.100:0/3663671479 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdffc0cca50 0x7fdffc1505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fdff2000640 1 -- 172.21.15.100:0/3663671479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdffc15d600 msgr2=0x7fdffc0cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fdff2000640 1 --2- 172.21.15.100:0/3663671479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdffc15d600 0x7fdffc0cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fdff2000640 1 -- 172.21.15.100:0/3663671479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdffc150ae0 con 0x7fdffc15c7e0 2024-04-23T14:34:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fdffaa00640 1 --2- 172.21.15.100:0/3663671479 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdffc0cca50 0x7fdffc1505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:03.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fdff2000640 1 --2- 172.21.15.100:0/3663671479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdffc15c7e0 0x7fdffc161280 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400e900 tx=0x7fdfe400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:03.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fdff3400640 1 -- 172.21.15.100:0/3663671479 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4017070 con 0x7fdffc15c7e0 2024-04-23T14:34:03.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fdff3400640 1 -- 172.21.15.100:0/3663671479 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfe400ca00 con 0x7fdffc15c7e0 2024-04-23T14:34:03.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fdff3400640 1 -- 172.21.15.100:0/3663671479 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe400cdf0 con 0x7fdffc15c7e0 2024-04-23T14:34:03.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fe001800640 1 -- 172.21.15.100:0/3663671479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdffc161d40 con 0x7fdffc15c7e0 2024-04-23T14:34:03.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.044+0000 7fe001800640 1 -- 172.21.15.100:0/3663671479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdffc1f6810 con 0x7fdffc15c7e0 2024-04-23T14:34:03.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.048+0000 7fe001800640 1 -- 172.21.15.100:0/3663671479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdffc068ad0 con 0x7fdffc15c7e0 2024-04-23T14:34:03.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.048+0000 7fdff3400640 1 -- 172.21.15.100:0/3663671479 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdfe401a070 con 0x7fdffc15c7e0 2024-04-23T14:34:03.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.048+0000 7fdff3400640 1 --2- 172.21.15.100:0/3663671479 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdfd8068380 0x7fdfd806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:03.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.048+0000 7fdff3400640 1 -- 172.21.15.100:0/3663671479 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fdfe4086de0 con 0x7fdffc15c7e0 2024-04-23T14:34:03.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.048+0000 7fdffaa00640 1 --2- 172.21.15.100:0/3663671479 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdfd8068380 0x7fdfd806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:03.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.048+0000 7fdffaa00640 1 --2- 172.21.15.100:0/3663671479 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdfd8068380 0x7fdfd806a840 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0ae000 tx=0x7fdfec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:03.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.052+0000 7fdff3400640 1 -- 172.21.15.100:0/3663671479 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdffc150ae0 con 0x7fdffc15c7e0 2024-04-23T14:34:03.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.208+0000 7fe001800640 1 -- 172.21.15.100:0/3663671479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdffc1f60c0 con 0x7fdffc15c7e0 2024-04-23T14:34:03.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.208+0000 7fdff3400640 1 -- 172.21.15.100:0/3663671479 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) ==== 104+0+11 (secure 0 0 0) 0x7fdfe4057010 con 0x7fdffc15c7e0 2024-04-23T14:34:03.214 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:34:03.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.212+0000 7fe001800640 1 -- 172.21.15.100:0/3663671479 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdfd8068380 msgr2=0x7fdfd806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:03.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.212+0000 7fe001800640 1 --2- 172.21.15.100:0/3663671479 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdfd8068380 0x7fdfd806a840 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0ae000 tx=0x7fdfec008040 comp rx=0 tx=0).stop 2024-04-23T14:34:03.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.212+0000 7fe001800640 1 -- 172.21.15.100:0/3663671479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdffc15c7e0 msgr2=0x7fdffc161280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:03.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.212+0000 7fe001800640 1 --2- 172.21.15.100:0/3663671479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdffc15c7e0 0x7fdffc161280 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400e900 tx=0x7fdfe400edc0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.212+0000 7fe001800640 1 -- 172.21.15.100:0/3663671479 shutdown_connections 2024-04-23T14:34:03.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.212+0000 7fe001800640 1 --2- 172.21.15.100:0/3663671479 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdfd8068380 0x7fdfd806a840 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.212+0000 7fe001800640 1 --2- 172.21.15.100:0/3663671479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdffc15d600 0x7fdffc0cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.212+0000 7fe001800640 1 --2- 172.21.15.100:0/3663671479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdffc15c7e0 0x7fdffc161280 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.212+0000 7fe001800640 1 --2- 172.21.15.100:0/3663671479 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdffc0cca50 0x7fdffc1505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.212+0000 7fe001800640 1 -- 172.21.15.100:0/3663671479 >> 172.21.15.100:0/3663671479 conn(0x7fdffc0cee80 msgr2=0x7fdffc1667b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:03.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.212+0000 7fe001800640 1 -- 172.21.15.100:0/3663671479 shutdown_connections 2024-04-23T14:34:03.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.212+0000 7fe001800640 1 -- 172.21.15.100:0/3663671479 wait complete. 2024-04-23T14:34:03.234 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:34:03.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4d0c00640 1 Processor -- start 2024-04-23T14:34:03.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4d0c00640 1 -- start start 2024-04-23T14:34:03.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4d0c00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fa4cc06f8c0 0x7fa4cc06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:03.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4d0c00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4cc06dd60 con 0x7fa4cc0ce7f0 2024-04-23T14:34:03.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4d0c00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4cc0d09c0 con 0x7fa4cc0d0170 2024-04-23T14:34:03.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4d0c00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fa4cc0ad050 con 0x7fa4cc06f8c0 2024-04-23T14:34:03.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4ca800640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fa4cc0ce7f0 0x7fa4cc0add80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:41968/0 (socket says 172.21.15.100:41968) 2024-04-23T14:34:03.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4ca800640 1 -- 172.21.15.100:0/1725540887 learned_addr learned my addr 172.21.15.100:0/1725540887 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:03.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c9e00640 1 --2- 172.21.15.100:0/1725540887 >> v2:172.21.15.151:3300/0 conn(0x7fa4cc06f8c0 0x7fa4cc06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:03.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c9400640 1 -- 172.21.15.100:0/1725540887 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1277899193 0 0) 0x7fa4c4001120 con 0x7fa4cc0ce7f0 2024-04-23T14:34:03.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c9400640 1 -- 172.21.15.100:0/1725540887 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4b0005740 con 0x7fa4cc0ce7f0 2024-04-23T14:34:03.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c9400640 1 -- 172.21.15.100:0/1725540887 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1617242811 0 0) 0x7fa4cc0d09c0 con 0x7fa4cc0d0170 2024-04-23T14:34:03.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c9400640 1 -- 172.21.15.100:0/1725540887 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4c4001120 con 0x7fa4cc0d0170 2024-04-23T14:34:03.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c9400640 1 -- 172.21.15.100:0/1725540887 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3661347805 0 0) 0x7fa4b0005740 con 0x7fa4cc0ce7f0 2024-04-23T14:34:03.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c9400640 1 -- 172.21.15.100:0/1725540887 >> v2:172.21.15.151:3300/0 conn(0x7fa4cc06f8c0 msgr2=0x7fa4cc06c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:03.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c9400640 1 --2- 172.21.15.100:0/1725540887 >> v2:172.21.15.151:3300/0 conn(0x7fa4cc06f8c0 0x7fa4cc06c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c9400640 1 -- 172.21.15.100:0/1725540887 >> v1:172.21.15.151:6789/0 conn(0x7fa4cc0d0170 legacy=0x7fa4cc0ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:03.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c9400640 1 -- 172.21.15.100:0/1725540887 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc0ad2a0 con 0x7fa4cc0ce7f0 2024-04-23T14:34:03.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c9400640 1 -- 172.21.15.100:0/1725540887 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7fa4c4001850 con 0x7fa4cc0ce7f0 2024-04-23T14:34:03.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c9400640 1 -- 172.21.15.100:0/1725540887 >> v1:172.21.15.100:6789/0 conn(0x7fa4cc0ce7f0 legacy=0x7fa4cc0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:03.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c9400640 1 --2- 172.21.15.100:0/1725540887 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4b0005d60 0x7fa4b0006160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:03.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c9400640 1 --2- 172.21.15.100:0/1725540887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4b0006a20 0x7fa4b0006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:03.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c9400640 1 --2- 172.21.15.100:0/1725540887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4b00073e0 0x7fa4b000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:03.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c9400640 1 -- 172.21.15.100:0/1725540887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc0ad050 con 0x7fa4b0006a20 2024-04-23T14:34:03.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c9400640 1 -- 172.21.15.100:0/1725540887 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b000e260 con 0x7fa4b00073e0 2024-04-23T14:34:03.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c9400640 1 -- 172.21.15.100:0/1725540887 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b000e460 con 0x7fa4b0005d60 2024-04-23T14:34:03.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c1400640 1 --2- 172.21.15.100:0/1725540887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4b0006a20 0x7fa4b0006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:03.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c9e00640 1 --2- 172.21.15.100:0/1725540887 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4b0005d60 0x7fa4b0006160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:03.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4ca800640 1 --2- 172.21.15.100:0/1725540887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4b00073e0 0x7fa4b000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:03.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c1400640 1 -- 172.21.15.100:0/1725540887 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4b0005d60 msgr2=0x7fa4b0006160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:03.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c1400640 1 --2- 172.21.15.100:0/1725540887 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4b0005d60 0x7fa4b0006160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c1400640 1 -- 172.21.15.100:0/1725540887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4b00073e0 msgr2=0x7fa4b000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:03.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c1400640 1 --2- 172.21.15.100:0/1725540887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4b00073e0 0x7fa4b000dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4c1400640 1 -- 172.21.15.100:0/1725540887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b0005740 con 0x7fa4b0006a20 2024-04-23T14:34:03.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4ca800640 1 --2- 172.21.15.100:0/1725540887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4b00073e0 0x7fa4b000dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:03.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.368+0000 7fa4ca800640 1 -- 172.21.15.100:0/1725540887 reap_dead start 2024-04-23T14:34:03.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.372+0000 7fa4c1400640 1 --2- 172.21.15.100:0/1725540887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4b0006a20 0x7fa4b0006ea0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa4b40018d0 tx=0x7fa4b401ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8bfd1326f8310fa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:03.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.372+0000 7fa4c9400640 1 -- 172.21.15.100:0/1725540887 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4001520 con 0x7fa4b0006a20 2024-04-23T14:34:03.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.372+0000 7fa4c9400640 1 -- 172.21.15.100:0/1725540887 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4b40052f0 con 0x7fa4b0006a20 2024-04-23T14:34:03.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.372+0000 7fa4c9400640 1 -- 172.21.15.100:0/1725540887 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b40a8930 con 0x7fa4b0006a20 2024-04-23T14:34:03.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.372+0000 7fa4d0c00640 1 -- 172.21.15.100:0/1725540887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4b0006a20 msgr2=0x7fa4b0006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:03.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.372+0000 7fa4d0c00640 1 --2- 172.21.15.100:0/1725540887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4b0006a20 0x7fa4b0006ea0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa4b40018d0 tx=0x7fa4b401ecd0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.372+0000 7fa4d0c00640 1 -- 172.21.15.100:0/1725540887 shutdown_connections 2024-04-23T14:34:03.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.372+0000 7fa4d0c00640 1 --2- 172.21.15.100:0/1725540887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4b0006a20 0x7fa4b0006ea0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.372+0000 7fa4d0c00640 1 -- 172.21.15.100:0/1725540887 >> 172.21.15.100:0/1725540887 conn(0x7fa4cc0b5200 msgr2=0x7fa4cc0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:03.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.372+0000 7fa4d0c00640 1 -- 172.21.15.100:0/1725540887 shutdown_connections 2024-04-23T14:34:03.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.372+0000 7fa4d0c00640 1 -- 172.21.15.100:0/1725540887 wait complete. 2024-04-23T14:34:03.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4d0c00640 1 Processor -- start 2024-04-23T14:34:03.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4d0c00640 1 -- start start 2024-04-23T14:34:03.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4d0c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4b0006a20 0x7fa4cc06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:03.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4d0c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4cc0c9dd0 0x7fa4cc06bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:03.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4d0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4cc1f47c0 0x7fa4cc1f4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:03.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4d0c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc0d05b0 con 0x7fa4cc1f47c0 2024-04-23T14:34:03.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4d0c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc0ad8f0 con 0x7fa4cc0c9dd0 2024-04-23T14:34:03.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4d0c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa4cc0d0730 con 0x7fa4b0006a20 2024-04-23T14:34:03.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4ca800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4cc1f47c0 0x7fa4cc1f4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:03.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4ca800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4cc1f47c0 0x7fa4cc1f4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42954/0 (socket says 172.21.15.100:42954) 2024-04-23T14:34:03.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4ca800640 1 -- 172.21.15.100:0/175858566 learned_addr learned my addr 172.21.15.100:0/175858566 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:03.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4c9e00640 1 --2- 172.21.15.100:0/175858566 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4b0006a20 0x7fa4cc06bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:03.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4c1400640 1 --2- 172.21.15.100:0/175858566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4cc0c9dd0 0x7fa4cc06bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:03.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4ca800640 1 -- 172.21.15.100:0/175858566 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4b0006a20 msgr2=0x7fa4cc06bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:03.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4ca800640 1 --2- 172.21.15.100:0/175858566 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4b0006a20 0x7fa4cc06bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4ca800640 1 -- 172.21.15.100:0/175858566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4cc0c9dd0 msgr2=0x7fa4cc06bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:03.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4ca800640 1 --2- 172.21.15.100:0/175858566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4cc0c9dd0 0x7fa4cc06bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4ca800640 1 -- 172.21.15.100:0/175858566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc0722d0 con 0x7fa4cc1f47c0 2024-04-23T14:34:03.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4c1400640 1 --2- 172.21.15.100:0/175858566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4cc0c9dd0 0x7fa4cc06bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:03.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4ca800640 1 --2- 172.21.15.100:0/175858566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4cc1f47c0 0x7fa4cc1f4c50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00069a0 tx=0x7fa4c4007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:03.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4c2a00640 1 -- 172.21.15.100:0/175858566 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c400e690 con 0x7fa4cc1f47c0 2024-04-23T14:34:03.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4c2a00640 1 -- 172.21.15.100:0/175858566 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4c4018070 con 0x7fa4cc1f47c0 2024-04-23T14:34:03.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4c2a00640 1 -- 172.21.15.100:0/175858566 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c400ea50 con 0x7fa4cc1f47c0 2024-04-23T14:34:03.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4d0c00640 1 -- 172.21.15.100:0/175858566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4cc0725b0 con 0x7fa4cc1f47c0 2024-04-23T14:34:03.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4d0c00640 1 -- 172.21.15.100:0/175858566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4cc072b70 con 0x7fa4cc1f47c0 2024-04-23T14:34:03.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.376+0000 7fa4d0c00640 1 -- 172.21.15.100:0/175858566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4cc068ad0 con 0x7fa4cc1f47c0 2024-04-23T14:34:03.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.380+0000 7fa4c2a00640 1 -- 172.21.15.100:0/175858566 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa4c400ebf0 con 0x7fa4cc1f47c0 2024-04-23T14:34:03.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.380+0000 7fa4c2a00640 1 --2- 172.21.15.100:0/175858566 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa4a80683b0 0x7fa4a806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:03.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.380+0000 7fa4c2a00640 1 -- 172.21.15.100:0/175858566 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fa4c4086b40 con 0x7fa4cc1f47c0 2024-04-23T14:34:03.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.380+0000 7fa4c9e00640 1 --2- 172.21.15.100:0/175858566 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa4a80683b0 0x7fa4a806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:03.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.380+0000 7fa4c9e00640 1 --2- 172.21.15.100:0/175858566 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa4a80683b0 0x7fa4a806a870 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0b0be0 tx=0x7fa4bc00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:03.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.384+0000 7fa4c2a00640 1 -- 172.21.15.100:0/175858566 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa4cc06dd60 con 0x7fa4cc1f47c0 2024-04-23T14:34:03.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.540+0000 7fa4d0c00640 1 -- 172.21.15.100:0/175858566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa4cc06cc00 con 0x7fa4cc1f47c0 2024-04-23T14:34:03.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.540+0000 7fa4c2a00640 1 -- 172.21.15.100:0/175858566 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) ==== 100+0+11 (secure 0 0 0) 0x7fa4c4056e00 con 0x7fa4cc1f47c0 2024-04-23T14:34:03.545 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:34:03.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.540+0000 7fa4d0c00640 1 -- 172.21.15.100:0/175858566 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa4a80683b0 msgr2=0x7fa4a806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:03.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.540+0000 7fa4d0c00640 1 --2- 172.21.15.100:0/175858566 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa4a80683b0 0x7fa4a806a870 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0b0be0 tx=0x7fa4bc00b040 comp rx=0 tx=0).stop 2024-04-23T14:34:03.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.540+0000 7fa4d0c00640 1 -- 172.21.15.100:0/175858566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4cc1f47c0 msgr2=0x7fa4cc1f4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:03.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.540+0000 7fa4d0c00640 1 --2- 172.21.15.100:0/175858566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4cc1f47c0 0x7fa4cc1f4c50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00069a0 tx=0x7fa4c4007420 comp rx=0 tx=0).stop 2024-04-23T14:34:03.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.540+0000 7fa4d0c00640 1 -- 172.21.15.100:0/175858566 shutdown_connections 2024-04-23T14:34:03.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.540+0000 7fa4d0c00640 1 --2- 172.21.15.100:0/175858566 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa4a80683b0 0x7fa4a806a870 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.540+0000 7fa4d0c00640 1 --2- 172.21.15.100:0/175858566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4cc1f47c0 0x7fa4cc1f4c50 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.540+0000 7fa4d0c00640 1 --2- 172.21.15.100:0/175858566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4cc0c9dd0 0x7fa4cc06bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.540+0000 7fa4d0c00640 1 --2- 172.21.15.100:0/175858566 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa4b0006a20 0x7fa4cc06bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.540+0000 7fa4d0c00640 1 -- 172.21.15.100:0/175858566 >> 172.21.15.100:0/175858566 conn(0x7fa4cc0b5200 msgr2=0x7fa4cc15f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:03.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.540+0000 7fa4d0c00640 1 -- 172.21.15.100:0/175858566 shutdown_connections 2024-04-23T14:34:03.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.540+0000 7fa4d0c00640 1 -- 172.21.15.100:0/175858566 wait complete. 2024-04-23T14:34:03.567 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-04-23T14:34:03.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.704+0000 7f0cb9000640 1 Processor -- start 2024-04-23T14:34:03.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.704+0000 7f0cb9000640 1 -- start start 2024-04-23T14:34:03.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.704+0000 7f0cb9000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f0cb40ce7f0 0x7f0cb41663f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:03.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.704+0000 7f0cb9000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cb406dd60 con 0x7f0cb406f8c0 2024-04-23T14:34:03.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.704+0000 7f0cb9000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cb40d09c0 con 0x7f0cb406c6c0 2024-04-23T14:34:03.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.704+0000 7f0cb9000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f0cb40ad050 con 0x7f0cb40ce7f0 2024-04-23T14:34:03.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.704+0000 7f0cb1e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f0cb406f8c0 0x7f0cb40d0170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:41970/0 (socket says 172.21.15.100:41970) 2024-04-23T14:34:03.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.704+0000 7f0cb1e00640 1 -- 172.21.15.100:0/3368805841 learned_addr learned my addr 172.21.15.100:0/3368805841 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:03.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.704+0000 7f0cb2800640 1 --2- 172.21.15.100:0/3368805841 >> v2:172.21.15.151:3300/0 conn(0x7f0cb40ce7f0 0x7f0cb41663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:03.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.708+0000 7f0cb1400640 1 -- 172.21.15.100:0/3368805841 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1258567007 0 0) 0x7f0ca40014a0 con 0x7f0cb406f8c0 2024-04-23T14:34:03.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.708+0000 7f0cb1400640 1 -- 172.21.15.100:0/3368805841 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c98003530 con 0x7f0cb406f8c0 2024-04-23T14:34:03.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.708+0000 7f0cb2800640 1 -- 172.21.15.100:0/3368805841 >> v1:172.21.15.151:6789/0 conn(0x7f0cb406c6c0 legacy=0x7f0cb40ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:03.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.708+0000 7f0cb2800640 1 -- 172.21.15.100:0/3368805841 >> v1:172.21.15.100:6789/0 conn(0x7f0cb406f8c0 legacy=0x7f0cb40d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:03.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.708+0000 7f0cb2800640 1 -- 172.21.15.100:0/3368805841 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb40ad2a0 con 0x7f0cb40ce7f0 2024-04-23T14:34:03.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.708+0000 7f0cb2800640 1 --2- 172.21.15.100:0/3368805841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cb40ce7f0 0x7f0cb41663f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0097c0 tx=0x7f0cac01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3abb4580cee930c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:03.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.708+0000 7f0cb1400640 1 -- 172.21.15.100:0/3368805841 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0b1070 con 0x7f0cb40ce7f0 2024-04-23T14:34:03.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.708+0000 7f0cb1400640 1 -- 172.21.15.100:0/3368805841 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cac0ac410 con 0x7f0cb40ce7f0 2024-04-23T14:34:03.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.708+0000 7f0cb1400640 1 -- 172.21.15.100:0/3368805841 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0ac7c0 con 0x7f0cb40ce7f0 2024-04-23T14:34:03.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0cb9000640 1 -- 172.21.15.100:0/3368805841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cb40ce7f0 msgr2=0x7f0cb41663f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:03.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0cb9000640 1 --2- 172.21.15.100:0/3368805841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cb40ce7f0 0x7f0cb41663f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0097c0 tx=0x7f0cac01c4a0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0cb9000640 1 -- 172.21.15.100:0/3368805841 shutdown_connections 2024-04-23T14:34:03.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0cb9000640 1 --2- 172.21.15.100:0/3368805841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cb40ce7f0 0x7f0cb41663f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0cb9000640 1 -- 172.21.15.100:0/3368805841 >> 172.21.15.100:0/3368805841 conn(0x7f0cb40b5200 msgr2=0x7f0cb40b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:03.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0cb9000640 1 -- 172.21.15.100:0/3368805841 shutdown_connections 2024-04-23T14:34:03.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0cb9000640 1 -- 172.21.15.100:0/3368805841 wait complete. 2024-04-23T14:34:03.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0cb9000640 1 Processor -- start 2024-04-23T14:34:03.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0cb9000640 1 -- start start 2024-04-23T14:34:03.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0cb9000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0cb406c6c0 0x7f0cb40c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:03.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0cb9000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cb406f8c0 0x7f0cb41589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:03.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0cb9000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cb40ce7f0 0x7f0cb40c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:03.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0cb9000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb4169770 con 0x7f0cb406f8c0 2024-04-23T14:34:03.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0cb9000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb40ad8f0 con 0x7f0cb40ce7f0 2024-04-23T14:34:03.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0cb9000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0cb41698f0 con 0x7f0cb406c6c0 2024-04-23T14:34:03.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0ca9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cb406f8c0 0x7f0cb41589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:03.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0ca9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cb406f8c0 0x7f0cb41589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42964/0 (socket says 172.21.15.100:42964) 2024-04-23T14:34:03.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0ca9400640 1 -- 172.21.15.100:0/2888927412 learned_addr learned my addr 172.21.15.100:0/2888927412 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:03.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0cb2800640 1 --2- 172.21.15.100:0/2888927412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cb40ce7f0 0x7f0cb40c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:03.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0cb1e00640 1 --2- 172.21.15.100:0/2888927412 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0cb406c6c0 0x7f0cb40c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:03.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0ca9400640 1 -- 172.21.15.100:0/2888927412 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0cb406c6c0 msgr2=0x7f0cb40c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:03.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0ca9400640 1 --2- 172.21.15.100:0/2888927412 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0cb406c6c0 0x7f0cb40c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0ca9400640 1 -- 172.21.15.100:0/2888927412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cb40ce7f0 msgr2=0x7f0cb40c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:03.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0ca9400640 1 --2- 172.21.15.100:0/2888927412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cb40ce7f0 0x7f0cb40c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0ca9400640 1 -- 172.21.15.100:0/2888927412 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb415ca50 con 0x7f0cb406f8c0 2024-04-23T14:34:03.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0cb1e00640 1 --2- 172.21.15.100:0/2888927412 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0cb406c6c0 0x7f0cb40c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:03.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0ca9400640 1 --2- 172.21.15.100:0/2888927412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cb406f8c0 0x7f0cb41589c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00c450 tx=0x7f0c9c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:03.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.712+0000 7f0caaa00640 1 -- 172.21.15.100:0/2888927412 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c016020 con 0x7f0cb406f8c0 2024-04-23T14:34:03.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.716+0000 7f0caaa00640 1 -- 172.21.15.100:0/2888927412 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c9c0052d0 con 0x7f0cb406f8c0 2024-04-23T14:34:03.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.716+0000 7f0caaa00640 1 -- 172.21.15.100:0/2888927412 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c007260 con 0x7f0cb406f8c0 2024-04-23T14:34:03.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.716+0000 7f0cb9000640 1 -- 172.21.15.100:0/2888927412 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb415a0d0 con 0x7f0cb406f8c0 2024-04-23T14:34:03.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.716+0000 7f0cb9000640 1 -- 172.21.15.100:0/2888927412 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb4159310 con 0x7f0cb406f8c0 2024-04-23T14:34:03.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.716+0000 7f0cb9000640 1 -- 172.21.15.100:0/2888927412 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb4068ad0 con 0x7f0cb406f8c0 2024-04-23T14:34:03.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.716+0000 7f0caaa00640 1 -- 172.21.15.100:0/2888927412 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0c9c007400 con 0x7f0cb406f8c0 2024-04-23T14:34:03.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.716+0000 7f0caaa00640 1 --2- 172.21.15.100:0/2888927412 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0c90068380 0x7f0c9006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:03.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.716+0000 7f0caaa00640 1 -- 172.21.15.100:0/2888927412 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f0c9c086600 con 0x7f0cb406f8c0 2024-04-23T14:34:03.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.716+0000 7f0cb1e00640 1 --2- 172.21.15.100:0/2888927412 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0c90068380 0x7f0c9006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:03.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.720+0000 7f0cb1e00640 1 --2- 172.21.15.100:0/2888927412 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0c90068380 0x7f0c9006a840 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4001640 tx=0x7f0ca4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:03.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.720+0000 7f0caaa00640 1 -- 172.21.15.100:0/2888927412 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0c9c020020 con 0x7f0cb406f8c0 2024-04-23T14:34:03.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.880+0000 7f0cb9000640 1 -- 172.21.15.100:0/2888927412 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) -- 0x7f0cb4074090 con 0x7f0cb406f8c0 2024-04-23T14:34:03.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.880+0000 7f0caaa00640 1 -- 172.21.15.100:0/2888927412 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v41) ==== 97+0+11 (secure 0 0 0) 0x7f0c9c005480 con 0x7f0cb406f8c0 2024-04-23T14:34:03.884 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:34:03.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.880+0000 7f0cb9000640 1 -- 172.21.15.100:0/2888927412 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0c90068380 msgr2=0x7f0c9006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:03.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.880+0000 7f0cb9000640 1 --2- 172.21.15.100:0/2888927412 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0c90068380 0x7f0c9006a840 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4001640 tx=0x7f0ca4009040 comp rx=0 tx=0).stop 2024-04-23T14:34:03.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.880+0000 7f0cb9000640 1 -- 172.21.15.100:0/2888927412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cb406f8c0 msgr2=0x7f0cb41589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:03.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.880+0000 7f0cb9000640 1 --2- 172.21.15.100:0/2888927412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cb406f8c0 0x7f0cb41589c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00c450 tx=0x7f0c9c00c910 comp rx=0 tx=0).stop 2024-04-23T14:34:03.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.880+0000 7f0cb9000640 1 -- 172.21.15.100:0/2888927412 shutdown_connections 2024-04-23T14:34:03.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.880+0000 7f0cb9000640 1 --2- 172.21.15.100:0/2888927412 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0c90068380 0x7f0c9006a840 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.880+0000 7f0cb9000640 1 --2- 172.21.15.100:0/2888927412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cb40ce7f0 0x7f0cb40c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.880+0000 7f0cb9000640 1 --2- 172.21.15.100:0/2888927412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cb406f8c0 0x7f0cb41589c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.880+0000 7f0cb9000640 1 --2- 172.21.15.100:0/2888927412 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0cb406c6c0 0x7f0cb40c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:03.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.880+0000 7f0cb9000640 1 -- 172.21.15.100:0/2888927412 >> 172.21.15.100:0/2888927412 conn(0x7f0cb40b5200 msgr2=0x7f0cb415f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:03.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.880+0000 7f0cb9000640 1 -- 172.21.15.100:0/2888927412 shutdown_connections 2024-04-23T14:34:03.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:03.880+0000 7f0cb9000640 1 -- 172.21.15.100:0/2888927412 wait complete. 2024-04-23T14:34:03.906 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:34:04.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.036+0000 7f939ce00640 1 Processor -- start 2024-04-23T14:34:04.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.036+0000 7f939ce00640 1 -- start start 2024-04-23T14:34:04.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.036+0000 7f939ce00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f939815e720 0x7f93980c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:04.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.036+0000 7f939ce00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f93980c7610 0x7f93980ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:04.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.036+0000 7f939ce00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f939815eca0 con 0x7f93980cf0a0 2024-04-23T14:34:04.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.036+0000 7f939ce00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f939806d450 con 0x7f939815e720 2024-04-23T14:34:04.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.036+0000 7f939ce00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f939815f8b0 con 0x7f93980c7610 2024-04-23T14:34:04.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.036+0000 7f9397400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f939815e720 0x7f93980c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:04.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.036+0000 7f9397400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f939815e720 0x7f93980c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42978/0 (socket says 172.21.15.100:42978) 2024-04-23T14:34:04.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.036+0000 7f9397400640 1 -- 172.21.15.100:0/2344664011 learned_addr learned my addr 172.21.15.100:0/2344664011 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:04.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.036+0000 7f938ea00640 1 --2- 172.21.15.100:0/2344664011 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f93980c7610 0x7f93980ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:04.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.040+0000 7f9397400640 1 -- 172.21.15.100:0/2344664011 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f93980c7610 msgr2=0x7f93980ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:04.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.040+0000 7f9397400640 1 --2- 172.21.15.100:0/2344664011 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f93980c7610 0x7f93980ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.040+0000 7f9397400640 1 -- 172.21.15.100:0/2344664011 >> v1:172.21.15.151:6789/0 conn(0x7f93980cf0a0 legacy=0x7f93981620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:04.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.040+0000 7f9397400640 1 -- 172.21.15.100:0/2344664011 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939815ef80 con 0x7f939815e720 2024-04-23T14:34:04.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.040+0000 7f9397400640 1 --2- 172.21.15.100:0/2344664011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f939815e720 0x7f93980c70d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9388009e50 tx=0x7f938801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119536b9f0235374 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:04.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.040+0000 7f9396a00640 1 -- 172.21.15.100:0/2344664011 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93880a5670 con 0x7f939815e720 2024-04-23T14:34:04.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.040+0000 7f9396a00640 1 -- 172.21.15.100:0/2344664011 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93880a5810 con 0x7f939815e720 2024-04-23T14:34:04.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.040+0000 7f9396a00640 1 -- 172.21.15.100:0/2344664011 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93880a5be0 con 0x7f939815e720 2024-04-23T14:34:04.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.040+0000 7f939ce00640 1 -- 172.21.15.100:0/2344664011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f939815e720 msgr2=0x7f93980c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:04.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.040+0000 7f939ce00640 1 --2- 172.21.15.100:0/2344664011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f939815e720 0x7f93980c70d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9388009e50 tx=0x7f938801c9a0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.040+0000 7f939ce00640 1 -- 172.21.15.100:0/2344664011 shutdown_connections 2024-04-23T14:34:04.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.040+0000 7f939ce00640 1 --2- 172.21.15.100:0/2344664011 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f93980c7610 0x7f93980ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.040+0000 7f939ce00640 1 --2- 172.21.15.100:0/2344664011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f939815e720 0x7f93980c70d0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.040+0000 7f939ce00640 1 -- 172.21.15.100:0/2344664011 >> 172.21.15.100:0/2344664011 conn(0x7f93980b3e60 msgr2=0x7f93980b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:04.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.040+0000 7f939ce00640 1 -- 172.21.15.100:0/2344664011 shutdown_connections 2024-04-23T14:34:04.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.040+0000 7f939ce00640 1 -- 172.21.15.100:0/2344664011 wait complete. 2024-04-23T14:34:04.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f939ce00640 1 Processor -- start 2024-04-23T14:34:04.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f939ce00640 1 -- start start 2024-04-23T14:34:04.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f939ce00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f93980c7610 0x7f939814f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:04.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f939ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93980cf0a0 0x7f93980ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:04.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f939ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f939815e720 0x7f93980ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:04.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f939ce00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f93980cf630 con 0x7f93980cf0a0 2024-04-23T14:34:04.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f939ce00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f93980cf4b0 con 0x7f939815e720 2024-04-23T14:34:04.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f939ce00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f939816b490 con 0x7f93980c7610 2024-04-23T14:34:04.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f938ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93980cf0a0 0x7f93980ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:04.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f938ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93980cf0a0 0x7f93980ca110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42984/0 (socket says 172.21.15.100:42984) 2024-04-23T14:34:04.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f938ea00640 1 -- 172.21.15.100:0/1296226552 learned_addr learned my addr 172.21.15.100:0/1296226552 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:04.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f9397400640 1 --2- 172.21.15.100:0/1296226552 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f93980c7610 0x7f939814f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:04.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f9397e00640 1 --2- 172.21.15.100:0/1296226552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f939815e720 0x7f93980ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:04.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f938ea00640 1 -- 172.21.15.100:0/1296226552 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f93980c7610 msgr2=0x7f939814f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:04.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f938ea00640 1 --2- 172.21.15.100:0/1296226552 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f93980c7610 0x7f939814f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f938ea00640 1 -- 172.21.15.100:0/1296226552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f939815e720 msgr2=0x7f93980ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:04.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f938ea00640 1 --2- 172.21.15.100:0/1296226552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f939815e720 0x7f93980ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f938ea00640 1 -- 172.21.15.100:0/1296226552 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93980cac90 con 0x7f93980cf0a0 2024-04-23T14:34:04.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f9397e00640 1 --2- 172.21.15.100:0/1296226552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f939815e720 0x7f93980ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:04.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f938ea00640 1 --2- 172.21.15.100:0/1296226552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93980cf0a0 0x7f93980ca110 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f938000e8a0 tx=0x7f938000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:04.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f938fe00640 1 -- 172.21.15.100:0/1296226552 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380017070 con 0x7f93980cf0a0 2024-04-23T14:34:04.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f938fe00640 1 -- 172.21.15.100:0/1296226552 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f938000c940 con 0x7f93980cf0a0 2024-04-23T14:34:04.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f938fe00640 1 -- 172.21.15.100:0/1296226552 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938000ccf0 con 0x7f93980cf0a0 2024-04-23T14:34:04.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f939ce00640 1 -- 172.21.15.100:0/1296226552 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93980caf70 con 0x7f93980cf0a0 2024-04-23T14:34:04.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.044+0000 7f939ce00640 1 -- 172.21.15.100:0/1296226552 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9398154e30 con 0x7f93980cf0a0 2024-04-23T14:34:04.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.048+0000 7f939ce00640 1 -- 172.21.15.100:0/1296226552 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9398075c20 con 0x7f93980cf0a0 2024-04-23T14:34:04.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.048+0000 7f938fe00640 1 -- 172.21.15.100:0/1296226552 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f938001a070 con 0x7f93980cf0a0 2024-04-23T14:34:04.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.048+0000 7f938fe00640 1 --2- 172.21.15.100:0/1296226552 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9374068380 0x7f937406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:04.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.048+0000 7f938fe00640 1 -- 172.21.15.100:0/1296226552 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f9380086f30 con 0x7f93980cf0a0 2024-04-23T14:34:04.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.048+0000 7f9397400640 1 --2- 172.21.15.100:0/1296226552 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9374068380 0x7f937406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:04.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.048+0000 7f9397400640 1 --2- 172.21.15.100:0/1296226552 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9374068380 0x7f937406a840 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f93880ae000 tx=0x7f93880a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:04.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.052+0000 7f938fe00640 1 -- 172.21.15.100:0/1296226552 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f93800571f0 con 0x7f93980cf0a0 2024-04-23T14:34:04.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.220+0000 7f939ce00640 1 -- 172.21.15.100:0/1296226552 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f93980b36f0 con 0x7f93980cf0a0 2024-04-23T14:34:04.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.220+0000 7f938fe00640 1 -- 172.21.15.100:0/1296226552 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) ==== 80+0+0 (secure 0 0 0) 0x7f9380056af0 con 0x7f93980cf0a0 2024-04-23T14:34:04.225 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-04-23T14:34:04.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.220+0000 7f939ce00640 1 -- 172.21.15.100:0/1296226552 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9374068380 msgr2=0x7f937406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:04.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.220+0000 7f939ce00640 1 --2- 172.21.15.100:0/1296226552 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9374068380 0x7f937406a840 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f93880ae000 tx=0x7f93880a7070 comp rx=0 tx=0).stop 2024-04-23T14:34:04.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.220+0000 7f939ce00640 1 -- 172.21.15.100:0/1296226552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93980cf0a0 msgr2=0x7f93980ca110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:04.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.220+0000 7f939ce00640 1 --2- 172.21.15.100:0/1296226552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93980cf0a0 0x7f93980ca110 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f938000e8a0 tx=0x7f938000eca0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.220+0000 7f939ce00640 1 -- 172.21.15.100:0/1296226552 shutdown_connections 2024-04-23T14:34:04.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.220+0000 7f939ce00640 1 --2- 172.21.15.100:0/1296226552 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9374068380 0x7f937406a840 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.220+0000 7f939ce00640 1 --2- 172.21.15.100:0/1296226552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f939815e720 0x7f93980ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.220+0000 7f939ce00640 1 --2- 172.21.15.100:0/1296226552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93980cf0a0 0x7f93980ca110 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.220+0000 7f939ce00640 1 --2- 172.21.15.100:0/1296226552 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f93980c7610 0x7f939814f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.220+0000 7f939ce00640 1 -- 172.21.15.100:0/1296226552 >> 172.21.15.100:0/1296226552 conn(0x7f93980b3e60 msgr2=0x7f939806de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:04.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.220+0000 7f939ce00640 1 -- 172.21.15.100:0/1296226552 shutdown_connections 2024-04-23T14:34:04.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.220+0000 7f939ce00640 1 -- 172.21.15.100:0/1296226552 wait complete. 2024-04-23T14:34:04.246 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:34:04.247 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:04.255 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.432+0000 7f5882200640 1 Processor -- start 2024-04-23T14:34:04.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.432+0000 7f5882200640 1 -- start start 2024-04-23T14:34:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f5882200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f587c15b640 0x7f587c0adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f5882200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f587c0af080 con 0x7f587c0b2a10 2024-04-23T14:34:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f5882200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f587c0afbc0 con 0x7f587c073610 2024-04-23T14:34:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f5882200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f587c15ba20 con 0x7f587c15b640 2024-04-23T14:34:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f5872a00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f587c0b2a10 0x7f587c072f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:41980/0 (socket says 172.21.15.100:41980) 2024-04-23T14:34:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f5872a00640 1 -- 172.21.15.100:0/2772991933 learned_addr learned my addr 172.21.15.100:0/2772991933 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:04.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587b400640 1 --2- 172.21.15.100:0/2772991933 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f587c15b640 0x7f587c0adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:04.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587aa00640 1 -- 172.21.15.100:0/2772991933 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3187351260 0 0) 0x7f587c0af080 con 0x7f587c0b2a10 2024-04-23T14:34:04.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587aa00640 1 -- 172.21.15.100:0/2772991933 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5860003430 con 0x7f587c0b2a10 2024-04-23T14:34:04.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587aa00640 1 -- 172.21.15.100:0/2772991933 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3454505197 0 0) 0x7f5860003430 con 0x7f587c0b2a10 2024-04-23T14:34:04.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587aa00640 1 -- 172.21.15.100:0/2772991933 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f587c15b640 msgr2=0x7f587c0adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:04.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587aa00640 1 --2- 172.21.15.100:0/2772991933 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f587c15b640 0x7f587c0adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587aa00640 1 -- 172.21.15.100:0/2772991933 >> v1:172.21.15.151:6789/0 conn(0x7f587c073610 legacy=0x7f587c06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587aa00640 1 -- 172.21.15.100:0/2772991933 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587c0af320 con 0x7f587c0b2a10 2024-04-23T14:34:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587aa00640 1 -- 172.21.15.100:0/2772991933 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f5864001680 con 0x7f587c0b2a10 2024-04-23T14:34:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587aa00640 1 -- 172.21.15.100:0/2772991933 >> v1:172.21.15.100:6789/0 conn(0x7f587c0b2a10 legacy=0x7f587c072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587aa00640 1 --2- 172.21.15.100:0/2772991933 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5860004ad0 0x7f5860004ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587aa00640 1 --2- 172.21.15.100:0/2772991933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58600036b0 0x7f5860003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587aa00640 1 --2- 172.21.15.100:0/2772991933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5860004270 0x7f586000fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587aa00640 1 -- 172.21.15.100:0/2772991933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f587c15ba20 con 0x7f5860004270 2024-04-23T14:34:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587aa00640 1 -- 172.21.15.100:0/2772991933 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f58600103f0 con 0x7f58600036b0 2024-04-23T14:34:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587aa00640 1 -- 172.21.15.100:0/2772991933 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f58600105f0 con 0x7f5860004ad0 2024-04-23T14:34:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587be00640 1 --2- 172.21.15.100:0/2772991933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5860004270 0x7f586000fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587b400640 1 --2- 172.21.15.100:0/2772991933 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5860004ad0 0x7f5860004ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f5872a00640 1 --2- 172.21.15.100:0/2772991933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58600036b0 0x7f5860003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587be00640 1 -- 172.21.15.100:0/2772991933 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5860004ad0 msgr2=0x7f5860004ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587be00640 1 --2- 172.21.15.100:0/2772991933 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5860004ad0 0x7f5860004ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587be00640 1 -- 172.21.15.100:0/2772991933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58600036b0 msgr2=0x7f5860003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:04.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587be00640 1 --2- 172.21.15.100:0/2772991933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58600036b0 0x7f5860003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.436+0000 7f587be00640 1 -- 172.21.15.100:0/2772991933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5860003430 con 0x7f5860004270 2024-04-23T14:34:04.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.440+0000 7f587be00640 1 --2- 172.21.15.100:0/2772991933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5860004270 0x7f586000fb80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f587c06be60 tx=0x7f587401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f563edf311a9ef16 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:04.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.440+0000 7f587aa00640 1 -- 172.21.15.100:0/2772991933 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58740013a0 con 0x7f5860004270 2024-04-23T14:34:04.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.440+0000 7f587aa00640 1 -- 172.21.15.100:0/2772991933 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58740052f0 con 0x7f5860004270 2024-04-23T14:34:04.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.440+0000 7f587aa00640 1 -- 172.21.15.100:0/2772991933 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874003840 con 0x7f5860004270 2024-04-23T14:34:04.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.440+0000 7f5882200640 1 -- 172.21.15.100:0/2772991933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5860004270 msgr2=0x7f586000fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:04.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.440+0000 7f5882200640 1 --2- 172.21.15.100:0/2772991933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5860004270 0x7f586000fb80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f587c06be60 tx=0x7f587401c3c0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.440+0000 7f587be00640 1 -- 172.21.15.100:0/2772991933 reap_dead start 2024-04-23T14:34:04.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.440+0000 7f5882200640 1 -- 172.21.15.100:0/2772991933 shutdown_connections 2024-04-23T14:34:04.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.440+0000 7f5882200640 1 -- 172.21.15.100:0/2772991933 >> 172.21.15.100:0/2772991933 conn(0x7f587c0b1e80 msgr2=0x7f587c0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:04.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.440+0000 7f5882200640 1 -- 172.21.15.100:0/2772991933 shutdown_connections 2024-04-23T14:34:04.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.440+0000 7f5882200640 1 -- 172.21.15.100:0/2772991933 wait complete. 2024-04-23T14:34:04.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.440+0000 7f5882200640 1 Processor -- start 2024-04-23T14:34:04.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f5882200640 1 -- start start 2024-04-23T14:34:04.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f5882200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5860004270 0x7f587c14a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:04.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f5882200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f587c1558a0 0x7f587c155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:04.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f5882200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f587c15d9e0 0x7f587c15de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:04.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f5882200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f587c0cc750 con 0x7f587c15d9e0 2024-04-23T14:34:04.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f5882200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f587c0af8e0 con 0x7f587c1558a0 2024-04-23T14:34:04.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f5882200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f587c0cc8d0 con 0x7f5860004270 2024-04-23T14:34:04.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f587be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f587c15d9e0 0x7f587c15de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f587be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f587c15d9e0 0x7f587c15de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42996/0 (socket says 172.21.15.100:42996) 2024-04-23T14:34:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f587be00640 1 -- 172.21.15.100:0/3825249029 learned_addr learned my addr 172.21.15.100:0/3825249029 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f587b400640 1 --2- 172.21.15.100:0/3825249029 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5860004270 0x7f587c14a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f5872a00640 1 --2- 172.21.15.100:0/3825249029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f587c1558a0 0x7f587c155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f587be00640 1 -- 172.21.15.100:0/3825249029 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5860004270 msgr2=0x7f587c14a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f587be00640 1 --2- 172.21.15.100:0/3825249029 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5860004270 0x7f587c14a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f587be00640 1 -- 172.21.15.100:0/3825249029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f587c1558a0 msgr2=0x7f587c155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f587be00640 1 --2- 172.21.15.100:0/3825249029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f587c1558a0 0x7f587c155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f587be00640 1 -- 172.21.15.100:0/3825249029 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587c15be00 con 0x7f587c15d9e0 2024-04-23T14:34:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f587be00640 1 --2- 172.21.15.100:0/3825249029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f587c15d9e0 0x7f587c15de60 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5874005040 tx=0x7f5860004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f5873e00640 1 -- 172.21.15.100:0/3825249029 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58740013a0 con 0x7f587c15d9e0 2024-04-23T14:34:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f5873e00640 1 -- 172.21.15.100:0/3825249029 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58740bb070 con 0x7f587c15d9e0 2024-04-23T14:34:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f5873e00640 1 -- 172.21.15.100:0/3825249029 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874007a40 con 0x7f587c15d9e0 2024-04-23T14:34:04.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f5882200640 1 -- 172.21.15.100:0/3825249029 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587c15c0a0 con 0x7f587c15d9e0 2024-04-23T14:34:04.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f5882200640 1 -- 172.21.15.100:0/3825249029 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f587c15c660 con 0x7f587c15d9e0 2024-04-23T14:34:04.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.444+0000 7f5882200640 1 -- 172.21.15.100:0/3825249029 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f587c068ad0 con 0x7f587c15d9e0 2024-04-23T14:34:04.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.448+0000 7f5873e00640 1 -- 172.21.15.100:0/3825249029 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f58740b4040 con 0x7f587c15d9e0 2024-04-23T14:34:04.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.448+0000 7f5873e00640 1 --2- 172.21.15.100:0/3825249029 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5858068380 0x7f585806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:04.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.448+0000 7f5873e00640 1 -- 172.21.15.100:0/3825249029 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f5874120c40 con 0x7f587c15d9e0 2024-04-23T14:34:04.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.448+0000 7f587b400640 1 --2- 172.21.15.100:0/3825249029 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5858068380 0x7f585806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:04.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.448+0000 7f587b400640 1 --2- 172.21.15.100:0/3825249029 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5858068380 0x7f585806a840 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f587c068100 tx=0x7f586c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:04.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.452+0000 7f5873e00640 1 -- 172.21.15.100:0/3825249029 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f58740160d0 con 0x7f587c15d9e0 2024-04-23T14:34:04.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.608+0000 7f5882200640 1 -- 172.21.15.100:0/3825249029 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f587c157390 con 0x7f587c15d9e0 2024-04-23T14:34:04.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.612+0000 7f5873e00640 1 -- 172.21.15.100:0/3825249029 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+18766 (secure 0 0 0) 0x7f58740f0820 con 0x7f587c15d9e0 2024-04-23T14:34:04.615 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:04.615 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":41,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:33:49.609841+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:33:36.592437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-23T14:33:37.882390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-04-23T14:33:47.299378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:04.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.612+0000 7f5882200640 1 -- 172.21.15.100:0/3825249029 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5858068380 msgr2=0x7f585806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:04.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.612+0000 7f5882200640 1 --2- 172.21.15.100:0/3825249029 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5858068380 0x7f585806a840 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f587c068100 tx=0x7f586c00b040 comp rx=0 tx=0).stop 2024-04-23T14:34:04.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.612+0000 7f5882200640 1 -- 172.21.15.100:0/3825249029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f587c15d9e0 msgr2=0x7f587c15de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:04.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.612+0000 7f5882200640 1 --2- 172.21.15.100:0/3825249029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f587c15d9e0 0x7f587c15de60 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5874005040 tx=0x7f5860004730 comp rx=0 tx=0).stop 2024-04-23T14:34:04.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.612+0000 7f5882200640 1 -- 172.21.15.100:0/3825249029 shutdown_connections 2024-04-23T14:34:04.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.612+0000 7f5882200640 1 --2- 172.21.15.100:0/3825249029 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5858068380 0x7f585806a840 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.612+0000 7f5882200640 1 --2- 172.21.15.100:0/3825249029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f587c15d9e0 0x7f587c15de60 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.612+0000 7f5882200640 1 --2- 172.21.15.100:0/3825249029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f587c1558a0 0x7f587c155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.612+0000 7f5882200640 1 --2- 172.21.15.100:0/3825249029 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5860004270 0x7f587c14a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.612+0000 7f5882200640 1 -- 172.21.15.100:0/3825249029 >> 172.21.15.100:0/3825249029 conn(0x7f587c0b1e80 msgr2=0x7f587c158320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:04.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.612+0000 7f5882200640 1 -- 172.21.15.100:0/3825249029 shutdown_connections 2024-04-23T14:34:04.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.612+0000 7f5882200640 1 -- 172.21.15.100:0/3825249029 wait complete. 2024-04-23T14:34:04.638 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:33:36.592437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:33:37.882390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-04-23T14:33:47.299378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:34:04.638 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:34:04.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.772+0000 7fb10e800640 1 Processor -- start 2024-04-23T14:34:04.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.772+0000 7fb10e800640 1 -- start start 2024-04-23T14:34:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.776+0000 7fb10e800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb108156930 0x7fb10815d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.776+0000 7fb10e800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb1080d1d00 0x7fb1080b3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.776+0000 7fb10e800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb10815d9b0 con 0x7fb1080b19a0 2024-04-23T14:34:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.776+0000 7fb10e800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fb1080d0790 con 0x7fb108156930 2024-04-23T14:34:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.776+0000 7fb10e800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb1080d0a70 con 0x7fb1080d1d00 2024-04-23T14:34:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.776+0000 7fb10d400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb108156930 0x7fb10815d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.776+0000 7fb10d400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb108156930 0x7fb10815d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43002/0 (socket says 172.21.15.100:43002) 2024-04-23T14:34:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.776+0000 7fb10d400640 1 -- 172.21.15.100:0/3428901176 learned_addr learned my addr 172.21.15.100:0/3428901176 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.776+0000 7fb10ca00640 1 --1- 172.21.15.100:0/3428901176 >> v1:172.21.15.100:6789/0 conn(0x7fb1080b19a0 0x7fb1080d15f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:41988/0 (socket says 172.21.15.100:41988) 2024-04-23T14:34:04.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.776+0000 7fb10de00640 1 --2- 172.21.15.100:0/3428901176 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb1080d1d00 0x7fb1080b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:04.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.776+0000 7fb10d400640 1 -- 172.21.15.100:0/3428901176 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb1080d1d00 msgr2=0x7fb1080b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:04.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.776+0000 7fb10d400640 1 --2- 172.21.15.100:0/3428901176 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb1080d1d00 0x7fb1080b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.776+0000 7fb10d400640 1 -- 172.21.15.100:0/3428901176 >> v1:172.21.15.100:6789/0 conn(0x7fb1080b19a0 legacy=0x7fb1080d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:04.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.776+0000 7fb10d400640 1 -- 172.21.15.100:0/3428901176 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1080d0d50 con 0x7fb108156930 2024-04-23T14:34:04.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb10d400640 1 --2- 172.21.15.100:0/3428901176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb108156930 0x7fb10815d470 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009870 tx=0x7fb0fc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7137b71d3cd0c368 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:04.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb107e00640 1 -- 172.21.15.100:0/3428901176 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc0a7070 con 0x7fb108156930 2024-04-23T14:34:04.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb107e00640 1 -- 172.21.15.100:0/3428901176 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0fc0b0070 con 0x7fb108156930 2024-04-23T14:34:04.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb107e00640 1 -- 172.21.15.100:0/3428901176 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc0ab890 con 0x7fb108156930 2024-04-23T14:34:04.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb10e800640 1 -- 172.21.15.100:0/3428901176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb108156930 msgr2=0x7fb10815d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:04.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb10e800640 1 --2- 172.21.15.100:0/3428901176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb108156930 0x7fb10815d470 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009870 tx=0x7fb0fc01c560 comp rx=0 tx=0).stop 2024-04-23T14:34:04.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb10e800640 1 -- 172.21.15.100:0/3428901176 shutdown_connections 2024-04-23T14:34:04.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb10e800640 1 --2- 172.21.15.100:0/3428901176 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb1080d1d00 0x7fb1080b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb10e800640 1 --2- 172.21.15.100:0/3428901176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb108156930 0x7fb10815d470 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb10e800640 1 -- 172.21.15.100:0/3428901176 >> 172.21.15.100:0/3428901176 conn(0x7fb108072420 msgr2=0x7fb1080ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:04.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb10e800640 1 -- 172.21.15.100:0/3428901176 shutdown_connections 2024-04-23T14:34:04.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb10e800640 1 -- 172.21.15.100:0/3428901176 wait complete. 2024-04-23T14:34:04.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb10e800640 1 Processor -- start 2024-04-23T14:34:04.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb10e800640 1 -- start start 2024-04-23T14:34:04.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb10e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1080b19a0 0x7fb10814fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:04.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb10e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1080d1d00 0x7fb1080c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:04.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb10e800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb108156930 0x7fb1080c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:04.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb10e800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb1081694c0 con 0x7fb1080b19a0 2024-04-23T14:34:04.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb10e800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb108169340 con 0x7fb1080d1d00 2024-04-23T14:34:04.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb10e800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb108169640 con 0x7fb108156930 2024-04-23T14:34:04.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb10d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1080b19a0 0x7fb10814fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:04.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb10d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1080b19a0 0x7fb10814fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43018/0 (socket says 172.21.15.100:43018) 2024-04-23T14:34:04.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb10d400640 1 -- 172.21.15.100:0/2910264889 learned_addr learned my addr 172.21.15.100:0/2910264889 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:04.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb10d400640 1 -- 172.21.15.100:0/2910264889 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb108156930 msgr2=0x7fb1080c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:04.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.780+0000 7fb10d400640 1 --2- 172.21.15.100:0/2910264889 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb108156930 0x7fb1080c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.784+0000 7fb10d400640 1 -- 172.21.15.100:0/2910264889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1080d1d00 msgr2=0x7fb1080c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:04.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.784+0000 7fb10ca00640 1 --2- 172.21.15.100:0/2910264889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1080d1d00 0x7fb1080c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:04.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.784+0000 7fb10d400640 1 --2- 172.21.15.100:0/2910264889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1080d1d00 0x7fb1080c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.784+0000 7fb10d400640 1 -- 172.21.15.100:0/2910264889 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1081f5b10 con 0x7fb1080b19a0 2024-04-23T14:34:04.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.784+0000 7fb10ca00640 1 --2- 172.21.15.100:0/2910264889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1080d1d00 0x7fb1080c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:04.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.784+0000 7fb10d400640 1 --2- 172.21.15.100:0/2910264889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1080b19a0 0x7fb10814fe80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc0ad040 tx=0x7fb0fc01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:04.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.784+0000 7fb105600640 1 -- 172.21.15.100:0/2910264889 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc0be5f0 con 0x7fb1080b19a0 2024-04-23T14:34:04.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.784+0000 7fb105600640 1 -- 172.21.15.100:0/2910264889 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0fc0be790 con 0x7fb1080b19a0 2024-04-23T14:34:04.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.784+0000 7fb10e800640 1 -- 172.21.15.100:0/2910264889 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb10815e800 con 0x7fb1080b19a0 2024-04-23T14:34:04.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.784+0000 7fb105600640 1 -- 172.21.15.100:0/2910264889 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc0beb80 con 0x7fb1080b19a0 2024-04-23T14:34:04.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.784+0000 7fb10e800640 1 -- 172.21.15.100:0/2910264889 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1081f60b0 con 0x7fb1080b19a0 2024-04-23T14:34:04.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.784+0000 7fb10e800640 1 -- 172.21.15.100:0/2910264889 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb108068ad0 con 0x7fb1080b19a0 2024-04-23T14:34:04.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.784+0000 7fb105600640 1 -- 172.21.15.100:0/2910264889 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb0fc0b43f0 con 0x7fb1080b19a0 2024-04-23T14:34:04.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.784+0000 7fb105600640 1 --2- 172.21.15.100:0/2910264889 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb0e8068380 0x7fb0e806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:04.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.784+0000 7fb105600640 1 -- 172.21.15.100:0/2910264889 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fb0fc121a50 con 0x7fb1080b19a0 2024-04-23T14:34:04.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.784+0000 7fb10ca00640 1 --2- 172.21.15.100:0/2910264889 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb0e8068380 0x7fb0e806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:04.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.788+0000 7fb10ca00640 1 --2- 172.21.15.100:0/2910264889 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb0e8068380 0x7fb0e806a840 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb108166660 tx=0x7fb0f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:04.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.788+0000 7fb105600640 1 -- 172.21.15.100:0/2910264889 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb0fc0c4060 con 0x7fb1080b19a0 2024-04-23T14:34:04.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.944+0000 7fb10e800640 1 -- 172.21.15.100:0/2910264889 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb1080b7f10 con 0x7fb1080b19a0 2024-04-23T14:34:04.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.948+0000 7fb105600640 1 -- 172.21.15.100:0/2910264889 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7fb0fc016150 con 0x7fb1080b19a0 2024-04-23T14:34:04.951 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:34:04.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.948+0000 7fb10e800640 1 -- 172.21.15.100:0/2910264889 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb0e8068380 msgr2=0x7fb0e806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:04.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.948+0000 7fb10e800640 1 --2- 172.21.15.100:0/2910264889 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb0e8068380 0x7fb0e806a840 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb108166660 tx=0x7fb0f4009040 comp rx=0 tx=0).stop 2024-04-23T14:34:04.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.948+0000 7fb10e800640 1 -- 172.21.15.100:0/2910264889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1080b19a0 msgr2=0x7fb10814fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:04.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.948+0000 7fb10e800640 1 --2- 172.21.15.100:0/2910264889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1080b19a0 0x7fb10814fe80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc0ad040 tx=0x7fb0fc01ada0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.948+0000 7fb10e800640 1 -- 172.21.15.100:0/2910264889 shutdown_connections 2024-04-23T14:34:04.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.948+0000 7fb10e800640 1 --2- 172.21.15.100:0/2910264889 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb0e8068380 0x7fb0e806a840 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.948+0000 7fb10e800640 1 --2- 172.21.15.100:0/2910264889 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb108156930 0x7fb1080c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.948+0000 7fb10e800640 1 --2- 172.21.15.100:0/2910264889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1080d1d00 0x7fb1080c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.948+0000 7fb10e800640 1 --2- 172.21.15.100:0/2910264889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1080b19a0 0x7fb10814fe80 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:04.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.948+0000 7fb10e800640 1 -- 172.21.15.100:0/2910264889 >> 172.21.15.100:0/2910264889 conn(0x7fb108072420 msgr2=0x7fb1081554b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:04.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.948+0000 7fb10e800640 1 -- 172.21.15.100:0/2910264889 shutdown_connections 2024-04-23T14:34:04.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:04.948+0000 7fb10e800640 1 -- 172.21.15.100:0/2910264889 wait complete. 2024-04-23T14:34:04.972 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:34:05.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.104+0000 7f0fee800640 1 Processor -- start 2024-04-23T14:34:05.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.104+0000 7f0fee800640 1 -- start start 2024-04-23T14:34:05.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.104+0000 7f0fee800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fe806bb20 0x7f0fe80ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:05.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.104+0000 7f0fee800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0fe80cd130 0x7f0fe80ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:05.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.104+0000 7f0fee800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fe806cbb0 con 0x7f0fe815c7a0 2024-04-23T14:34:05.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.108+0000 7f0fee800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f0fe80cd5a0 con 0x7f0fe80cd130 2024-04-23T14:34:05.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.108+0000 7f0fee800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0fe806d450 con 0x7f0fe806bb20 2024-04-23T14:34:05.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.108+0000 7f0fe7e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0fe80cd130 0x7f0fe80ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:05.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.108+0000 7f0fe7e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0fe80cd130 0x7f0fe80ad430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43032/0 (socket says 172.21.15.100:43032) 2024-04-23T14:34:05.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.108+0000 7f0fe7e00640 1 -- 172.21.15.100:0/4256481536 learned_addr learned my addr 172.21.15.100:0/4256481536 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:05.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.108+0000 7f0fdea00640 1 --2- 172.21.15.100:0/4256481536 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fe806bb20 0x7f0fe80ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:05.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.108+0000 7f0fe7e00640 1 -- 172.21.15.100:0/4256481536 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fe806bb20 msgr2=0x7f0fe80ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:05.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.108+0000 7f0fe7e00640 1 --2- 172.21.15.100:0/4256481536 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fe806bb20 0x7f0fe80ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.108+0000 7f0fe7e00640 1 -- 172.21.15.100:0/4256481536 >> v1:172.21.15.151:6789/0 conn(0x7f0fe815c7a0 legacy=0x7f0fe80af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:05.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.108+0000 7f0fe7e00640 1 -- 172.21.15.100:0/4256481536 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe806ce90 con 0x7f0fe80cd130 2024-04-23T14:34:05.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.108+0000 7f0fe7e00640 1 --2- 172.21.15.100:0/4256481536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fe80cd130 0x7f0fe80ad430 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0009870 tx=0x7f0fe001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e1c3836e4f28a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:05.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.108+0000 7f0fe6a00640 1 -- 172.21.15.100:0/4256481536 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe00b0070 con 0x7f0fe80cd130 2024-04-23T14:34:05.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fe6a00640 1 -- 172.21.15.100:0/4256481536 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fe00ab600 con 0x7f0fe80cd130 2024-04-23T14:34:05.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fe6a00640 1 -- 172.21.15.100:0/4256481536 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe00ab9d0 con 0x7f0fe80cd130 2024-04-23T14:34:05.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fee800640 1 -- 172.21.15.100:0/4256481536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fe80cd130 msgr2=0x7f0fe80ad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:05.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fee800640 1 --2- 172.21.15.100:0/4256481536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fe80cd130 0x7f0fe80ad430 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0009870 tx=0x7f0fe001c4d0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fee800640 1 -- 172.21.15.100:0/4256481536 shutdown_connections 2024-04-23T14:34:05.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fee800640 1 --2- 172.21.15.100:0/4256481536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fe80cd130 0x7f0fe80ad430 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fee800640 1 --2- 172.21.15.100:0/4256481536 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fe806bb20 0x7f0fe80ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fee800640 1 -- 172.21.15.100:0/4256481536 >> 172.21.15.100:0/4256481536 conn(0x7f0fe80b5200 msgr2=0x7f0fe80b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:05.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fee800640 1 -- 172.21.15.100:0/4256481536 shutdown_connections 2024-04-23T14:34:05.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fee800640 1 -- 172.21.15.100:0/4256481536 wait complete. 2024-04-23T14:34:05.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fee800640 1 Processor -- start 2024-04-23T14:34:05.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fee800640 1 -- start start 2024-04-23T14:34:05.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fee800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fe806bb20 0x7f0fe80995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:05.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fee800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fe80cd130 0x7f0fe8158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:05.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fee800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fe815c7a0 0x7f0fe8159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:05.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fee800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe80ae510 con 0x7f0fe806bb20 2024-04-23T14:34:05.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fee800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe80ae390 con 0x7f0fe815c7a0 2024-04-23T14:34:05.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fee800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0fe80ae690 con 0x7f0fe80cd130 2024-04-23T14:34:05.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fe7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fe806bb20 0x7f0fe80995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:05.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fe7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fe806bb20 0x7f0fe80995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43038/0 (socket says 172.21.15.100:43038) 2024-04-23T14:34:05.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fe7400640 1 -- 172.21.15.100:0/246598513 learned_addr learned my addr 172.21.15.100:0/246598513 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:05.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fdea00640 1 --2- 172.21.15.100:0/246598513 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fe80cd130 0x7f0fe8158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:05.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fe7e00640 1 --2- 172.21.15.100:0/246598513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fe815c7a0 0x7f0fe8159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:05.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fe7400640 1 -- 172.21.15.100:0/246598513 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fe80cd130 msgr2=0x7f0fe8158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:05.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fe7400640 1 --2- 172.21.15.100:0/246598513 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fe80cd130 0x7f0fe8158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fe7400640 1 -- 172.21.15.100:0/246598513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fe815c7a0 msgr2=0x7f0fe8159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:05.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.112+0000 7f0fe7400640 1 --2- 172.21.15.100:0/246598513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fe815c7a0 0x7f0fe8159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.116+0000 7f0fe7400640 1 -- 172.21.15.100:0/246598513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe8150940 con 0x7f0fe806bb20 2024-04-23T14:34:05.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.116+0000 7f0fe7e00640 1 --2- 172.21.15.100:0/246598513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fe815c7a0 0x7f0fe8159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:05.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.116+0000 7f0fdea00640 1 --2- 172.21.15.100:0/246598513 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fe80cd130 0x7f0fe8158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:05.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.116+0000 7f0fe7400640 1 --2- 172.21.15.100:0/246598513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fe806bb20 0x7f0fe80995e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800c790 tx=0x7f0fd800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:05.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.116+0000 7f0fdfe00640 1 -- 172.21.15.100:0/246598513 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8016020 con 0x7f0fe806bb20 2024-04-23T14:34:05.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.116+0000 7f0fdfe00640 1 -- 172.21.15.100:0/246598513 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd8006d60 con 0x7f0fe806bb20 2024-04-23T14:34:05.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.116+0000 7f0fdfe00640 1 -- 172.21.15.100:0/246598513 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8003bc0 con 0x7f0fe806bb20 2024-04-23T14:34:05.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.116+0000 7f0fee800640 1 -- 172.21.15.100:0/246598513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe8150c20 con 0x7f0fe806bb20 2024-04-23T14:34:05.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.116+0000 7f0fee800640 1 -- 172.21.15.100:0/246598513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe81511e0 con 0x7f0fe806bb20 2024-04-23T14:34:05.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.116+0000 7f0fee800640 1 -- 172.21.15.100:0/246598513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fe8068ad0 con 0x7f0fe806bb20 2024-04-23T14:34:05.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.116+0000 7f0fdfe00640 1 -- 172.21.15.100:0/246598513 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0fd800a7f0 con 0x7f0fe806bb20 2024-04-23T14:34:05.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.116+0000 7f0fdfe00640 1 --2- 172.21.15.100:0/246598513 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0fc4068380 0x7f0fc406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:05.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.116+0000 7f0fdfe00640 1 -- 172.21.15.100:0/246598513 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f0fd80869a0 con 0x7f0fe806bb20 2024-04-23T14:34:05.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.116+0000 7f0fdea00640 1 --2- 172.21.15.100:0/246598513 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0fc4068380 0x7f0fc406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:05.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.120+0000 7f0fdea00640 1 --2- 172.21.15.100:0/246598513 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0fc4068380 0x7f0fc406a840 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8149d50 tx=0x7f0fd0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:05.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.120+0000 7f0fdfe00640 1 -- 172.21.15.100:0/246598513 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0fd8020020 con 0x7f0fe806bb20 2024-04-23T14:34:05.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.276+0000 7f0fee800640 1 -- 172.21.15.100:0/246598513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0fe80aeb20 con 0x7f0fe806bb20 2024-04-23T14:34:05.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.276+0000 7f0fdfe00640 1 -- 172.21.15.100:0/246598513 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) ==== 104+0+11 (secure 0 0 0) 0x7f0fd8056c60 con 0x7f0fe806bb20 2024-04-23T14:34:05.283 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:34:05.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.280+0000 7f0fee800640 1 -- 172.21.15.100:0/246598513 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0fc4068380 msgr2=0x7f0fc406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:05.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.280+0000 7f0fee800640 1 --2- 172.21.15.100:0/246598513 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0fc4068380 0x7f0fc406a840 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8149d50 tx=0x7f0fd0008040 comp rx=0 tx=0).stop 2024-04-23T14:34:05.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.280+0000 7f0fee800640 1 -- 172.21.15.100:0/246598513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fe806bb20 msgr2=0x7f0fe80995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:05.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.280+0000 7f0fee800640 1 --2- 172.21.15.100:0/246598513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fe806bb20 0x7f0fe80995e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800c790 tx=0x7f0fd800cc50 comp rx=0 tx=0).stop 2024-04-23T14:34:05.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.280+0000 7f0fee800640 1 -- 172.21.15.100:0/246598513 shutdown_connections 2024-04-23T14:34:05.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.280+0000 7f0fee800640 1 --2- 172.21.15.100:0/246598513 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0fc4068380 0x7f0fc406a840 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.280+0000 7f0fee800640 1 --2- 172.21.15.100:0/246598513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fe815c7a0 0x7f0fe8159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.280+0000 7f0fee800640 1 --2- 172.21.15.100:0/246598513 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fe80cd130 0x7f0fe8158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.280+0000 7f0fee800640 1 --2- 172.21.15.100:0/246598513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fe806bb20 0x7f0fe80995e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.280+0000 7f0fee800640 1 -- 172.21.15.100:0/246598513 >> 172.21.15.100:0/246598513 conn(0x7f0fe80b5200 msgr2=0x7f0fe80b55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:05.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.280+0000 7f0fee800640 1 -- 172.21.15.100:0/246598513 shutdown_connections 2024-04-23T14:34:05.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.280+0000 7f0fee800640 1 -- 172.21.15.100:0/246598513 wait complete. 2024-04-23T14:34:05.303 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:34:05.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.436+0000 7f1078200640 1 Processor -- start 2024-04-23T14:34:05.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.436+0000 7f1078200640 1 -- start start 2024-04-23T14:34:05.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.436+0000 7f1078200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f107006f8c0 0x7f107006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:05.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.436+0000 7f1078200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f10700ce7f0 0x7f10700add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:05.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.436+0000 7f1078200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f107006dd60 con 0x7f10700d0170 2024-04-23T14:34:05.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.436+0000 7f1078200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f10700ad050 con 0x7f107006f8c0 2024-04-23T14:34:05.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.436+0000 7f1078200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f10700ad8f0 con 0x7f10700ce7f0 2024-04-23T14:34:05.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.436+0000 7f1075800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f107006f8c0 0x7f107006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:05.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.436+0000 7f1076200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f10700ce7f0 0x7f10700add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:05.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.436+0000 7f1075800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f107006f8c0 0x7f107006c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:37618/0 (socket says 172.21.15.100:37618) 2024-04-23T14:34:05.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.436+0000 7f1075800640 1 -- 172.21.15.100:0/112074312 learned_addr learned my addr 172.21.15.100:0/112074312 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:05.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.440+0000 7f1075800640 1 -- 172.21.15.100:0/112074312 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f10700ce7f0 msgr2=0x7f10700add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:05.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.440+0000 7f1075800640 1 --2- 172.21.15.100:0/112074312 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f10700ce7f0 0x7f10700add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.440+0000 7f1075800640 1 -- 172.21.15.100:0/112074312 >> v1:172.21.15.151:6789/0 conn(0x7f10700d0170 legacy=0x7f10700ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:05.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.440+0000 7f1075800640 1 -- 172.21.15.100:0/112074312 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10700d09c0 con 0x7f107006f8c0 2024-04-23T14:34:05.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.440+0000 7f1076200640 1 --2- 172.21.15.100:0/112074312 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f10700ce7f0 0x7f10700add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:34:05.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.440+0000 7f1075800640 1 --2- 172.21.15.100:0/112074312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f107006f8c0 0x7f107006c6c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f106400d150 tx=0x7f106401ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b91f74a69bd5b70 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:05.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.440+0000 7f106fe00640 1 -- 172.21.15.100:0/112074312 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10640a7d00 con 0x7f107006f8c0 2024-04-23T14:34:05.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.440+0000 7f106fe00640 1 -- 172.21.15.100:0/112074312 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f106400b0d0 con 0x7f107006f8c0 2024-04-23T14:34:05.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.440+0000 7f106fe00640 1 -- 172.21.15.100:0/112074312 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1064009600 con 0x7f107006f8c0 2024-04-23T14:34:05.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1078200640 1 -- 172.21.15.100:0/112074312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f107006f8c0 msgr2=0x7f107006c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:05.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1078200640 1 --2- 172.21.15.100:0/112074312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f107006f8c0 0x7f107006c6c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f106400d150 tx=0x7f106401ec30 comp rx=0 tx=0).stop 2024-04-23T14:34:05.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1078200640 1 -- 172.21.15.100:0/112074312 shutdown_connections 2024-04-23T14:34:05.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1078200640 1 --2- 172.21.15.100:0/112074312 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f10700ce7f0 0x7f10700add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1078200640 1 --2- 172.21.15.100:0/112074312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f107006f8c0 0x7f107006c6c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1078200640 1 -- 172.21.15.100:0/112074312 >> 172.21.15.100:0/112074312 conn(0x7f10700b5200 msgr2=0x7f10700b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:05.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1078200640 1 -- 172.21.15.100:0/112074312 shutdown_connections 2024-04-23T14:34:05.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1078200640 1 -- 172.21.15.100:0/112074312 wait complete. 2024-04-23T14:34:05.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1078200640 1 Processor -- start 2024-04-23T14:34:05.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1078200640 1 -- start start 2024-04-23T14:34:05.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1078200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f107006f8c0 0x7f10700c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:05.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1078200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f10700ce7f0 0x7f1070155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:05.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1078200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10700d0170 0x7f10701561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:05.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1078200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f10700aead0 con 0x7f10700d0170 2024-04-23T14:34:05.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1078200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f10700d0760 con 0x7f107006f8c0 2024-04-23T14:34:05.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1078200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f10700aec50 con 0x7f10700ce7f0 2024-04-23T14:34:05.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1076200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10700d0170 0x7f10701561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:05.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1076200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10700d0170 0x7f10701561e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43050/0 (socket says 172.21.15.100:43050) 2024-04-23T14:34:05.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1076200640 1 -- 172.21.15.100:0/2351311526 learned_addr learned my addr 172.21.15.100:0/2351311526 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:05.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1075800640 1 --2- 172.21.15.100:0/2351311526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f107006f8c0 0x7f10700c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:05.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1074e00640 1 --2- 172.21.15.100:0/2351311526 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f10700ce7f0 0x7f1070155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:05.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1076200640 1 -- 172.21.15.100:0/2351311526 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f10700ce7f0 msgr2=0x7f1070155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:05.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1076200640 1 --2- 172.21.15.100:0/2351311526 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f10700ce7f0 0x7f1070155ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1076200640 1 -- 172.21.15.100:0/2351311526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f107006f8c0 msgr2=0x7f10700c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:05.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1076200640 1 --2- 172.21.15.100:0/2351311526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f107006f8c0 0x7f10700c66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.444+0000 7f1076200640 1 -- 172.21.15.100:0/2351311526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1070164a20 con 0x7f10700d0170 2024-04-23T14:34:05.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.448+0000 7f1076200640 1 --2- 172.21.15.100:0/2351311526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10700d0170 0x7f10701561e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f106800e950 tx=0x7f106800ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:05.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.448+0000 7f106d600640 1 -- 172.21.15.100:0/2351311526 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1068017070 con 0x7f10700d0170 2024-04-23T14:34:05.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.448+0000 7f106d600640 1 -- 172.21.15.100:0/2351311526 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f106800ca00 con 0x7f10700d0170 2024-04-23T14:34:05.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.448+0000 7f106d600640 1 -- 172.21.15.100:0/2351311526 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106800cdd0 con 0x7f10700d0170 2024-04-23T14:34:05.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.448+0000 7f1078200640 1 -- 172.21.15.100:0/2351311526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1070164d00 con 0x7f10700d0170 2024-04-23T14:34:05.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.448+0000 7f1078200640 1 -- 172.21.15.100:0/2351311526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10701652c0 con 0x7f10700d0170 2024-04-23T14:34:05.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.448+0000 7f1078200640 1 -- 172.21.15.100:0/2351311526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1070068ad0 con 0x7f10700d0170 2024-04-23T14:34:05.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.448+0000 7f106d600640 1 -- 172.21.15.100:0/2351311526 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1068013470 con 0x7f10700d0170 2024-04-23T14:34:05.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.448+0000 7f106d600640 1 --2- 172.21.15.100:0/2351311526 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1050068380 0x7f105006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:05.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.448+0000 7f106d600640 1 -- 172.21.15.100:0/2351311526 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f1068086b40 con 0x7f10700d0170 2024-04-23T14:34:05.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.452+0000 7f1075800640 1 --2- 172.21.15.100:0/2351311526 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1050068380 0x7f105006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:05.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.452+0000 7f1075800640 1 --2- 172.21.15.100:0/2351311526 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1050068380 0x7f105006a840 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f10640b0000 tx=0x7f10640a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:05.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.456+0000 7f106d600640 1 -- 172.21.15.100:0/2351311526 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1068056e80 con 0x7f10700d0170 2024-04-23T14:34:05.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.608+0000 7f1078200640 1 -- 172.21.15.100:0/2351311526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f107006cc00 con 0x7f10700d0170 2024-04-23T14:34:05.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.612+0000 7f106d600640 1 -- 172.21.15.100:0/2351311526 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) ==== 100+0+11 (secure 0 0 0) 0x7f1068056780 con 0x7f10700d0170 2024-04-23T14:34:05.615 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:34:05.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.612+0000 7f1078200640 1 -- 172.21.15.100:0/2351311526 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1050068380 msgr2=0x7f105006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:05.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.612+0000 7f1078200640 1 --2- 172.21.15.100:0/2351311526 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1050068380 0x7f105006a840 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f10640b0000 tx=0x7f10640a9070 comp rx=0 tx=0).stop 2024-04-23T14:34:05.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.612+0000 7f1078200640 1 -- 172.21.15.100:0/2351311526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10700d0170 msgr2=0x7f10701561e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:05.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.612+0000 7f1078200640 1 --2- 172.21.15.100:0/2351311526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10700d0170 0x7f10701561e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f106800e950 tx=0x7f106800ee10 comp rx=0 tx=0).stop 2024-04-23T14:34:05.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.612+0000 7f1078200640 1 -- 172.21.15.100:0/2351311526 shutdown_connections 2024-04-23T14:34:05.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.612+0000 7f1078200640 1 --2- 172.21.15.100:0/2351311526 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1050068380 0x7f105006a840 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.612+0000 7f1078200640 1 --2- 172.21.15.100:0/2351311526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10700d0170 0x7f10701561e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.612+0000 7f1078200640 1 --2- 172.21.15.100:0/2351311526 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f10700ce7f0 0x7f1070155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.612+0000 7f1078200640 1 --2- 172.21.15.100:0/2351311526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f107006f8c0 0x7f10700c66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.612+0000 7f1078200640 1 -- 172.21.15.100:0/2351311526 >> 172.21.15.100:0/2351311526 conn(0x7f10700b5200 msgr2=0x7f107006ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:05.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.612+0000 7f1078200640 1 -- 172.21.15.100:0/2351311526 shutdown_connections 2024-04-23T14:34:05.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.612+0000 7f1078200640 1 -- 172.21.15.100:0/2351311526 wait complete. 2024-04-23T14:34:05.637 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-04-23T14:34:05.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.772+0000 7f1e66600640 1 Processor -- start 2024-04-23T14:34:05.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.772+0000 7f1e66600640 1 -- start start 2024-04-23T14:34:05.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.772+0000 7f1e66600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e600cd300 0x7f1e601671f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:05.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.772+0000 7f1e66600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e6006cbb0 con 0x7f1e6015c7a0 2024-04-23T14:34:05.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.772+0000 7f1e66600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e6006ce90 con 0x7f1e6006bb20 2024-04-23T14:34:05.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.772+0000 7f1e66600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1e6006d450 con 0x7f1e600cd300 2024-04-23T14:34:05.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.772+0000 7f1e5f400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1e6015c7a0 0x7f1e600af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42004/0 (socket says 172.21.15.100:42004) 2024-04-23T14:34:05.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.772+0000 7f1e5f400640 1 -- 172.21.15.100:0/3646483720 learned_addr learned my addr 172.21.15.100:0/3646483720 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:05.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.772+0000 7f1e5fe00640 1 --2- 172.21.15.100:0/3646483720 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e600cd300 0x7f1e601671f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:05.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.772+0000 7f1e5e000640 1 -- 172.21.15.100:0/3646483720 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1198700397 0 0) 0x7f1e6006cbb0 con 0x7f1e6015c7a0 2024-04-23T14:34:05.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.772+0000 7f1e5e000640 1 -- 172.21.15.100:0/3646483720 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e44003530 con 0x7f1e6015c7a0 2024-04-23T14:34:05.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.772+0000 7f1e5fe00640 1 -- 172.21.15.100:0/3646483720 >> v1:172.21.15.151:6789/0 conn(0x7f1e6006bb20 legacy=0x7f1e600ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:05.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.772+0000 7f1e5e000640 1 -- 172.21.15.100:0/3646483720 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2396167316 0 0) 0x7f1e44003530 con 0x7f1e6015c7a0 2024-04-23T14:34:05.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.772+0000 7f1e5fe00640 1 -- 172.21.15.100:0/3646483720 >> v1:172.21.15.100:6789/0 conn(0x7f1e6015c7a0 legacy=0x7f1e600af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:05.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.772+0000 7f1e5fe00640 1 -- 172.21.15.100:0/3646483720 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6006d170 con 0x7f1e600cd300 2024-04-23T14:34:05.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.776+0000 7f1e5fe00640 1 --2- 172.21.15.100:0/3646483720 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e600cd300 0x7f1e601671f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1e580097f0 tx=0x7f1e5801c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d24995abf0dfafe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:05.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.776+0000 7f1e5e000640 1 -- 172.21.15.100:0/3646483720 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e580b0070 con 0x7f1e600cd300 2024-04-23T14:34:05.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.776+0000 7f1e5e000640 1 -- 172.21.15.100:0/3646483720 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e580ab400 con 0x7f1e600cd300 2024-04-23T14:34:05.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.776+0000 7f1e5e000640 1 -- 172.21.15.100:0/3646483720 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50001a00 con 0x7f1e600cd300 2024-04-23T14:34:05.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.776+0000 7f1e66600640 1 -- 172.21.15.100:0/3646483720 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e600cd300 msgr2=0x7f1e601671f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:05.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.776+0000 7f1e66600640 1 --2- 172.21.15.100:0/3646483720 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e600cd300 0x7f1e601671f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1e580097f0 tx=0x7f1e5801c370 comp rx=0 tx=0).stop 2024-04-23T14:34:05.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.776+0000 7f1e66600640 1 -- 172.21.15.100:0/3646483720 shutdown_connections 2024-04-23T14:34:05.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.776+0000 7f1e66600640 1 --2- 172.21.15.100:0/3646483720 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e600cd300 0x7f1e601671f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.776+0000 7f1e66600640 1 -- 172.21.15.100:0/3646483720 >> 172.21.15.100:0/3646483720 conn(0x7f1e600b5200 msgr2=0x7f1e600b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:05.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.776+0000 7f1e66600640 1 -- 172.21.15.100:0/3646483720 shutdown_connections 2024-04-23T14:34:05.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.776+0000 7f1e66600640 1 -- 172.21.15.100:0/3646483720 wait complete. 2024-04-23T14:34:05.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e66600640 1 Processor -- start 2024-04-23T14:34:05.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e66600640 1 -- start start 2024-04-23T14:34:05.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e66600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e6006bb20 0x7f1e6015ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:05.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e66600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e600cd300 0x7f1e601595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:05.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e66600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e6015c7a0 0x7f1e60099510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:05.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e66600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6016a2e0 con 0x7f1e6015c7a0 2024-04-23T14:34:05.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e66600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6016a160 con 0x7f1e6006bb20 2024-04-23T14:34:05.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e66600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1e6016a460 con 0x7f1e600cd300 2024-04-23T14:34:05.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e5fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e6015c7a0 0x7f1e60099510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:05.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e5fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e6015c7a0 0x7f1e60099510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43062/0 (socket says 172.21.15.100:43062) 2024-04-23T14:34:05.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e5fe00640 1 -- 172.21.15.100:0/3525697948 learned_addr learned my addr 172.21.15.100:0/3525697948 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:05.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e5f400640 1 --2- 172.21.15.100:0/3525697948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e6006bb20 0x7f1e6015ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:05.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e5ea00640 1 --2- 172.21.15.100:0/3525697948 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e600cd300 0x7f1e601595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:05.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e5fe00640 1 -- 172.21.15.100:0/3525697948 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e600cd300 msgr2=0x7f1e601595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:05.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e5fe00640 1 --2- 172.21.15.100:0/3525697948 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e600cd300 0x7f1e601595c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e5fe00640 1 -- 172.21.15.100:0/3525697948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e6006bb20 msgr2=0x7f1e6015ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:05.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e5fe00640 1 --2- 172.21.15.100:0/3525697948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e6006bb20 0x7f1e6015ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e5fe00640 1 -- 172.21.15.100:0/3525697948 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6015fb10 con 0x7f1e6015c7a0 2024-04-23T14:34:05.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e5f400640 1 --2- 172.21.15.100:0/3525697948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e6006bb20 0x7f1e6015ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:05.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e5ea00640 1 --2- 172.21.15.100:0/3525697948 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e600cd300 0x7f1e601595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:05.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e5fe00640 1 --2- 172.21.15.100:0/3525697948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e6015c7a0 0x7f1e60099510 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1e580097f0 tx=0x7f1e5801abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:05.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e57400640 1 -- 172.21.15.100:0/3525697948 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e580bb020 con 0x7f1e6015c7a0 2024-04-23T14:34:05.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e57400640 1 -- 172.21.15.100:0/3525697948 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e580b1040 con 0x7f1e6015c7a0 2024-04-23T14:34:05.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e66600640 1 -- 172.21.15.100:0/3525697948 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e600c6490 con 0x7f1e6015c7a0 2024-04-23T14:34:05.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e57400640 1 -- 172.21.15.100:0/3525697948 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e580b0050 con 0x7f1e6015c7a0 2024-04-23T14:34:05.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e66600640 1 -- 172.21.15.100:0/3525697948 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e600c6a50 con 0x7f1e6015c7a0 2024-04-23T14:34:05.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.780+0000 7f1e66600640 1 -- 172.21.15.100:0/3525697948 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e60068ad0 con 0x7f1e6015c7a0 2024-04-23T14:34:05.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.784+0000 7f1e57400640 1 -- 172.21.15.100:0/3525697948 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1e580be070 con 0x7f1e6015c7a0 2024-04-23T14:34:05.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.784+0000 7f1e57400640 1 --2- 172.21.15.100:0/3525697948 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e40068380 0x7f1e4006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:05.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.784+0000 7f1e57400640 1 -- 172.21.15.100:0/3525697948 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f1e581206d0 con 0x7f1e6015c7a0 2024-04-23T14:34:05.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.784+0000 7f1e5f400640 1 --2- 172.21.15.100:0/3525697948 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e40068380 0x7f1e4006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:05.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.784+0000 7f1e5f400640 1 --2- 172.21.15.100:0/3525697948 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e40068380 0x7f1e4006a840 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1e50001570 tx=0x7f1e50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:05.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.788+0000 7f1e57400640 1 -- 172.21.15.100:0/3525697948 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1e58015ff0 con 0x7f1e6015c7a0 2024-04-23T14:34:05.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.940+0000 7f1e66600640 1 -- 172.21.15.100:0/3525697948 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) -- 0x7f1e60074f20 con 0x7f1e6015c7a0 2024-04-23T14:34:05.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.940+0000 7f1e57400640 1 -- 172.21.15.100:0/3525697948 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v41) ==== 97+0+11 (secure 0 0 0) 0x7f1e580f0990 con 0x7f1e6015c7a0 2024-04-23T14:34:05.947 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:34:05.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.944+0000 7f1e66600640 1 -- 172.21.15.100:0/3525697948 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e40068380 msgr2=0x7f1e4006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:05.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.944+0000 7f1e66600640 1 --2- 172.21.15.100:0/3525697948 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e40068380 0x7f1e4006a840 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1e50001570 tx=0x7f1e50009040 comp rx=0 tx=0).stop 2024-04-23T14:34:05.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.944+0000 7f1e66600640 1 -- 172.21.15.100:0/3525697948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e6015c7a0 msgr2=0x7f1e60099510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:05.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.944+0000 7f1e66600640 1 --2- 172.21.15.100:0/3525697948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e6015c7a0 0x7f1e60099510 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1e580097f0 tx=0x7f1e5801abb0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.944+0000 7f1e66600640 1 -- 172.21.15.100:0/3525697948 shutdown_connections 2024-04-23T14:34:05.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.944+0000 7f1e66600640 1 --2- 172.21.15.100:0/3525697948 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1e40068380 0x7f1e4006a840 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.944+0000 7f1e66600640 1 --2- 172.21.15.100:0/3525697948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e6015c7a0 0x7f1e60099510 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.944+0000 7f1e66600640 1 --2- 172.21.15.100:0/3525697948 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e600cd300 0x7f1e601595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.944+0000 7f1e66600640 1 --2- 172.21.15.100:0/3525697948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e6006bb20 0x7f1e6015ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:05.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.944+0000 7f1e66600640 1 -- 172.21.15.100:0/3525697948 >> 172.21.15.100:0/3525697948 conn(0x7f1e600b5200 msgr2=0x7f1e600cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:05.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.944+0000 7f1e66600640 1 -- 172.21.15.100:0/3525697948 shutdown_connections 2024-04-23T14:34:05.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:05.944+0000 7f1e66600640 1 -- 172.21.15.100:0/3525697948 wait complete. 2024-04-23T14:34:05.967 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:34:06.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.100+0000 7fdcf0a00640 1 Processor -- start 2024-04-23T14:34:06.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.100+0000 7fdcf0a00640 1 -- start start 2024-04-23T14:34:06.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.100+0000 7fdcf0a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdcec06d450 0x7fdcec06d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:06.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.100+0000 7fdcf0a00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fdcec0d0790 0x7fdcec0d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:06.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.100+0000 7fdcf0a00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcec0cecc0 con 0x7fdcec15e950 2024-04-23T14:34:06.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.100+0000 7fdcf0a00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fdcec06ddf0 con 0x7fdcec06d450 2024-04-23T14:34:06.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.100+0000 7fdcf0a00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fdcec0d0300 con 0x7fdcec0d0790 2024-04-23T14:34:06.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.100+0000 7fdce9400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdcec06d450 0x7fdcec06d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:06.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.100+0000 7fdce9e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fdcec15e950 0x7fdcec0cd460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42014/0 (socket says 172.21.15.100:42014) 2024-04-23T14:34:06.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.100+0000 7fdce9e00640 1 -- 172.21.15.100:0/482340944 learned_addr learned my addr 172.21.15.100:0/482340944 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:06.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.100+0000 7fdcea800640 1 --2- 172.21.15.100:0/482340944 >> v2:172.21.15.151:3300/0 conn(0x7fdcec0d0790 0x7fdcec0d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:06.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.100+0000 7fdce9400640 1 -- 172.21.15.100:0/482340944 >> v2:172.21.15.151:3300/0 conn(0x7fdcec0d0790 msgr2=0x7fdcec0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:06.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.100+0000 7fdce9400640 1 --2- 172.21.15.100:0/482340944 >> v2:172.21.15.151:3300/0 conn(0x7fdcec0d0790 0x7fdcec0d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:06.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.100+0000 7fdce9400640 1 -- 172.21.15.100:0/482340944 >> v1:172.21.15.100:6789/0 conn(0x7fdcec15e950 legacy=0x7fdcec0cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:06.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.100+0000 7fdce9400640 1 -- 172.21.15.100:0/482340944 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcec0cef60 con 0x7fdcec06d450 2024-04-23T14:34:06.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.104+0000 7fdce9400640 1 --2- 172.21.15.100:0/482340944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcec06d450 0x7fdcec06d8b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0cc620 tx=0x7fdcd801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea3028ee570aaa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:06.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.104+0000 7fdce8a00640 1 -- 172.21.15.100:0/482340944 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80b0070 con 0x7fdcec06d450 2024-04-23T14:34:06.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.104+0000 7fdce8a00640 1 -- 172.21.15.100:0/482340944 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcd80ab600 con 0x7fdcec06d450 2024-04-23T14:34:06.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.104+0000 7fdce8a00640 1 -- 172.21.15.100:0/482340944 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80ab9d0 con 0x7fdcec06d450 2024-04-23T14:34:06.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.104+0000 7fdcf0a00640 1 -- 172.21.15.100:0/482340944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcec06d450 msgr2=0x7fdcec06d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:06.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.104+0000 7fdcf0a00640 1 --2- 172.21.15.100:0/482340944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcec06d450 0x7fdcec06d8b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0cc620 tx=0x7fdcd801c3f0 comp rx=0 tx=0).stop 2024-04-23T14:34:06.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.104+0000 7fdcf0a00640 1 -- 172.21.15.100:0/482340944 shutdown_connections 2024-04-23T14:34:06.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.104+0000 7fdcf0a00640 1 --2- 172.21.15.100:0/482340944 >> v2:172.21.15.151:3300/0 conn(0x7fdcec0d0790 0x7fdcec0d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:06.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.104+0000 7fdcf0a00640 1 --2- 172.21.15.100:0/482340944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcec06d450 0x7fdcec06d8b0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:06.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.104+0000 7fdcf0a00640 1 -- 172.21.15.100:0/482340944 >> 172.21.15.100:0/482340944 conn(0x7fdcec0719d0 msgr2=0x7fdcec071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:06.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.104+0000 7fdcf0a00640 1 -- 172.21.15.100:0/482340944 shutdown_connections 2024-04-23T14:34:06.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.104+0000 7fdcf0a00640 1 -- 172.21.15.100:0/482340944 wait complete. 2024-04-23T14:34:06.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdcf0a00640 1 Processor -- start 2024-04-23T14:34:06.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdcf0a00640 1 -- start start 2024-04-23T14:34:06.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdcf0a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdcec06d450 0x7fdcec15ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:06.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdcf0a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcec0d0790 0x7fdcec1513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:06.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdcf0a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcec15e950 0x7fdcec151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:06.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdcf0a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec16b780 con 0x7fdcec0d0790 2024-04-23T14:34:06.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdcf0a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec0cdb70 con 0x7fdcec15e950 2024-04-23T14:34:06.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdcf0a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fdcec16b900 con 0x7fdcec06d450 2024-04-23T14:34:06.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdce9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcec0d0790 0x7fdcec1513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:06.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdce9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcec0d0790 0x7fdcec1513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43078/0 (socket says 172.21.15.100:43078) 2024-04-23T14:34:06.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdce9400640 1 -- 172.21.15.100:0/3794424295 learned_addr learned my addr 172.21.15.100:0/3794424295 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:06.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdce9400640 1 -- 172.21.15.100:0/3794424295 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdcec06d450 msgr2=0x7fdcec15ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:06.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdce9400640 1 --2- 172.21.15.100:0/3794424295 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdcec06d450 0x7fdcec15ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:06.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdce9400640 1 -- 172.21.15.100:0/3794424295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcec15e950 msgr2=0x7fdcec151900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:06.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdce9400640 1 --2- 172.21.15.100:0/3794424295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcec15e950 0x7fdcec151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:06.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdce9400640 1 -- 172.21.15.100:0/3794424295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcec15b320 con 0x7fdcec0d0790 2024-04-23T14:34:06.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdce9400640 1 --2- 172.21.15.100:0/3794424295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcec0d0790 0x7fdcec1513c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80ae040 tx=0x7fdcd801ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:06.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdce2000640 1 -- 172.21.15.100:0/3794424295 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80b0050 con 0x7fdcec0d0790 2024-04-23T14:34:06.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdce2000640 1 -- 172.21.15.100:0/3794424295 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcd80bc070 con 0x7fdcec0d0790 2024-04-23T14:34:06.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdcf0a00640 1 -- 172.21.15.100:0/3794424295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcec160f80 con 0x7fdcec0d0790 2024-04-23T14:34:06.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdce2000640 1 -- 172.21.15.100:0/3794424295 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80b5a70 con 0x7fdcec0d0790 2024-04-23T14:34:06.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdcf0a00640 1 -- 172.21.15.100:0/3794424295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcec161540 con 0x7fdcec0d0790 2024-04-23T14:34:06.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.108+0000 7fdcf0a00640 1 -- 172.21.15.100:0/3794424295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcec068ad0 con 0x7fdcec0d0790 2024-04-23T14:34:06.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.112+0000 7fdce2000640 1 -- 172.21.15.100:0/3794424295 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdcec0d1970 con 0x7fdcec0d0790 2024-04-23T14:34:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.112+0000 7fdce2000640 1 --2- 172.21.15.100:0/3794424295 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdccc068380 0x7fdccc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.112+0000 7fdce2000640 1 -- 172.21.15.100:0/3794424295 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fdcd81219f0 con 0x7fdcec0d0790 2024-04-23T14:34:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.112+0000 7fdce9e00640 1 --2- 172.21.15.100:0/3794424295 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdccc068380 0x7fdccc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:06.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.112+0000 7fdce9e00640 1 --2- 172.21.15.100:0/3794424295 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdccc068380 0x7fdccc06a840 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc000f70 tx=0x7fdcdc007a90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.116+0000 7fdce2000640 1 -- 172.21.15.100:0/3794424295 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdcd8016070 con 0x7fdcec0d0790 2024-04-23T14:34:06.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.284+0000 7fdcf0a00640 1 -- 172.21.15.100:0/3794424295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdcec0b36f0 con 0x7fdcec0d0790 2024-04-23T14:34:06.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.284+0000 7fdce2000640 1 -- 172.21.15.100:0/3794424295 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) ==== 80+0+0 (secure 0 0 0) 0x7fdcd811fe20 con 0x7fdcec0d0790 2024-04-23T14:34:06.288 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-04-23T14:34:06.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.284+0000 7fdcf0a00640 1 -- 172.21.15.100:0/3794424295 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdccc068380 msgr2=0x7fdccc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:06.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.284+0000 7fdcf0a00640 1 --2- 172.21.15.100:0/3794424295 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdccc068380 0x7fdccc06a840 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc000f70 tx=0x7fdcdc007a90 comp rx=0 tx=0).stop 2024-04-23T14:34:06.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.284+0000 7fdcf0a00640 1 -- 172.21.15.100:0/3794424295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcec0d0790 msgr2=0x7fdcec1513c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:06.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.284+0000 7fdcf0a00640 1 --2- 172.21.15.100:0/3794424295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcec0d0790 0x7fdcec1513c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80ae040 tx=0x7fdcd801ac30 comp rx=0 tx=0).stop 2024-04-23T14:34:06.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.284+0000 7fdcf0a00640 1 -- 172.21.15.100:0/3794424295 shutdown_connections 2024-04-23T14:34:06.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.284+0000 7fdcf0a00640 1 --2- 172.21.15.100:0/3794424295 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdccc068380 0x7fdccc06a840 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:06.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.284+0000 7fdcf0a00640 1 --2- 172.21.15.100:0/3794424295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcec15e950 0x7fdcec151900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:06.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.284+0000 7fdcf0a00640 1 --2- 172.21.15.100:0/3794424295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcec0d0790 0x7fdcec1513c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:06.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.284+0000 7fdcf0a00640 1 --2- 172.21.15.100:0/3794424295 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdcec06d450 0x7fdcec15ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:06.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.284+0000 7fdcf0a00640 1 -- 172.21.15.100:0/3794424295 >> 172.21.15.100:0/3794424295 conn(0x7fdcec0719d0 msgr2=0x7fdcec0c9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:06.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.284+0000 7fdcf0a00640 1 -- 172.21.15.100:0/3794424295 shutdown_connections 2024-04-23T14:34:06.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.284+0000 7fdcf0a00640 1 -- 172.21.15.100:0/3794424295 wait complete. 2024-04-23T14:34:06.310 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi100.front.sepia.ceph.com... 2024-04-23T14:34:06.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:06.310 DEBUG:teuthology.orchestra.run.smithi100:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:06.328 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[40322]: fuse finished with error 0 and tester_r 0 2024-04-23T14:34:06.331 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:34:06.421 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:34:06.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:06.421 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:34:06.430 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2024-04-23T14:34:06.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.612+0000 7fa2ee000640 1 Processor -- start 2024-04-23T14:34:06.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.612+0000 7fa2ee000640 1 -- start start 2024-04-23T14:34:06.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.612+0000 7fa2ee000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa2e80b3910 0x7fa2e806d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:06.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.612+0000 7fa2ee000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2e80cd230 con 0x7fa2e815e700 2024-04-23T14:34:06.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.612+0000 7fa2ee000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2e80cd510 con 0x7fa2e806d990 2024-04-23T14:34:06.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.612+0000 7fa2ee000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fa2e80cd7f0 con 0x7fa2e80b3910 2024-04-23T14:34:06.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.612+0000 7fa2dea00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa2e80b3910 0x7fa2e806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:06.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.612+0000 7fa2e7400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fa2e815e700 0x7fa2e80cca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42016/0 (socket says 172.21.15.100:42016) 2024-04-23T14:34:06.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.612+0000 7fa2e7400640 1 -- 172.21.15.100:0/947054484 learned_addr learned my addr 172.21.15.100:0/947054484 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:06.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.612+0000 7fa2dea00640 1 -- 172.21.15.100:0/947054484 >> v1:172.21.15.151:6789/0 conn(0x7fa2e806d990 legacy=0x7fa2e80d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:06.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.612+0000 7fa2dea00640 1 -- 172.21.15.100:0/947054484 >> v1:172.21.15.100:6789/0 conn(0x7fa2e815e700 legacy=0x7fa2e80cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:06.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.612+0000 7fa2dea00640 1 -- 172.21.15.100:0/947054484 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e80cdad0 con 0x7fa2e80b3910 2024-04-23T14:34:06.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.612+0000 7fa2e6a00640 1 -- 172.21.15.100:0/947054484 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 854725652 0 0) 0x7fa2e80cd230 con 0x7fa2e815e700 2024-04-23T14:34:06.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.616+0000 7fa2dea00640 1 --2- 172.21.15.100:0/947054484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e80b3910 0x7fa2e806d450 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00097c0 tx=0x7fa2d001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d34a3d4c145d2d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:06.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.616+0000 7fa2e6a00640 1 -- 172.21.15.100:0/947054484 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00b1070 con 0x7fa2e80b3910 2024-04-23T14:34:06.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.616+0000 7fa2e6a00640 1 -- 172.21.15.100:0/947054484 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d00ac410 con 0x7fa2e80b3910 2024-04-23T14:34:06.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.616+0000 7fa2e6a00640 1 -- 172.21.15.100:0/947054484 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00ac7e0 con 0x7fa2e80b3910 2024-04-23T14:34:06.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.616+0000 7fa2ee000640 1 -- 172.21.15.100:0/947054484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e80b3910 msgr2=0x7fa2e806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:06.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.616+0000 7fa2ee000640 1 --2- 172.21.15.100:0/947054484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e80b3910 0x7fa2e806d450 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00097c0 tx=0x7fa2d001c4a0 comp rx=0 tx=0).stop 2024-04-23T14:34:06.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.616+0000 7fa2ee000640 1 -- 172.21.15.100:0/947054484 shutdown_connections 2024-04-23T14:34:06.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.616+0000 7fa2ee000640 1 --2- 172.21.15.100:0/947054484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e80b3910 0x7fa2e806d450 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:06.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.616+0000 7fa2ee000640 1 -- 172.21.15.100:0/947054484 >> 172.21.15.100:0/947054484 conn(0x7fa2e80719d0 msgr2=0x7fa2e8071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:06.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.616+0000 7fa2ee000640 1 -- 172.21.15.100:0/947054484 shutdown_connections 2024-04-23T14:34:06.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.616+0000 7fa2ee000640 1 -- 172.21.15.100:0/947054484 wait complete. 2024-04-23T14:34:06.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.616+0000 7fa2ee000640 1 Processor -- start 2024-04-23T14:34:06.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2ee000640 1 -- start start 2024-04-23T14:34:06.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2ee000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2e806d990 0x7fa2e80b6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:06.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2ee000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa2e80b3910 0x7fa2e8152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:06.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2ee000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e815e700 0x7fa2e80b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:06.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2ee000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e80d1ad0 con 0x7fa2e815e700 2024-04-23T14:34:06.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2ee000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e80d1950 con 0x7fa2e806d990 2024-04-23T14:34:06.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2ee000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e816c7b0 con 0x7fa2e80b3910 2024-04-23T14:34:06.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2e7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e815e700 0x7fa2e80b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:06.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2e7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e815e700 0x7fa2e80b5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43104/0 (socket says 172.21.15.100:43104) 2024-04-23T14:34:06.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2e7e00640 1 -- 172.21.15.100:0/4167399106 learned_addr learned my addr 172.21.15.100:0/4167399106 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:06.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2dea00640 1 --2- 172.21.15.100:0/4167399106 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa2e80b3910 0x7fa2e8152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:06.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2e7400640 1 --2- 172.21.15.100:0/4167399106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2e806d990 0x7fa2e80b6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:06.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2e7e00640 1 -- 172.21.15.100:0/4167399106 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa2e80b3910 msgr2=0x7fa2e8152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:06.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2e7e00640 1 --2- 172.21.15.100:0/4167399106 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa2e80b3910 0x7fa2e8152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:06.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2e7e00640 1 -- 172.21.15.100:0/4167399106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2e806d990 msgr2=0x7fa2e80b6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:06.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2e7e00640 1 --2- 172.21.15.100:0/4167399106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2e806d990 0x7fa2e80b6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:06.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2e7e00640 1 -- 172.21.15.100:0/4167399106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e80b59b0 con 0x7fa2e815e700 2024-04-23T14:34:06.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2e7400640 1 --2- 172.21.15.100:0/4167399106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2e806d990 0x7fa2e80b6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:06.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2dea00640 1 --2- 172.21.15.100:0/4167399106 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa2e80b3910 0x7fa2e8152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:06.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2e7e00640 1 --2- 172.21.15.100:0/4167399106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e815e700 0x7fa2e80b5060 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000c730 tx=0x7fa2e000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:06.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2dfe00640 1 -- 172.21.15.100:0/4167399106 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0003b80 con 0x7fa2e815e700 2024-04-23T14:34:06.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2dfe00640 1 -- 172.21.15.100:0/4167399106 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2e0003d20 con 0x7fa2e815e700 2024-04-23T14:34:06.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2ee000640 1 -- 172.21.15.100:0/4167399106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e80b5c90 con 0x7fa2e815e700 2024-04-23T14:34:06.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2dfe00640 1 -- 172.21.15.100:0/4167399106 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e000aa30 con 0x7fa2e815e700 2024-04-23T14:34:06.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2ee000640 1 -- 172.21.15.100:0/4167399106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e80b6250 con 0x7fa2e815e700 2024-04-23T14:34:06.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.620+0000 7fa2ee000640 1 -- 172.21.15.100:0/4167399106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2e8068ad0 con 0x7fa2e815e700 2024-04-23T14:34:06.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.624+0000 7fa2dfe00640 1 -- 172.21.15.100:0/4167399106 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa2e000abd0 con 0x7fa2e815e700 2024-04-23T14:34:06.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.624+0000 7fa2dfe00640 1 --2- 172.21.15.100:0/4167399106 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa2c4068380 0x7fa2c406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:06.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.624+0000 7fa2dfe00640 1 -- 172.21.15.100:0/4167399106 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fa2e0086c10 con 0x7fa2e815e700 2024-04-23T14:34:06.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.624+0000 7fa2e7400640 1 --2- 172.21.15.100:0/4167399106 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa2c4068380 0x7fa2c406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:06.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.624+0000 7fa2e7400640 1 --2- 172.21.15.100:0/4167399106 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa2c4068380 0x7fa2c406a840 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8007fd0 tx=0x7fa2d8007c80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:06.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.628+0000 7fa2dfe00640 1 -- 172.21.15.100:0/4167399106 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa2e0056e40 con 0x7fa2e815e700 2024-04-23T14:34:06.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.840+0000 7fa2ee000640 1 -- 172.21.15.100:0/4167399106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) -- 0x7fa2e80b7b60 con 0x7fa2e815e700 2024-04-23T14:34:06.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.844+0000 7fa2dfe00640 1 -- 172.21.15.100:0/4167399106 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v14) ==== 184+0+141 (secure 0 0 0) 0x7fa2e0056740 con 0x7fa2e815e700 2024-04-23T14:34:06.849 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-04-23T14:34:06.849 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAtxidmyhImMBAAa49ApQEF2Lng6rY94F1HAQ== 2024-04-23T14:34:06.850 INFO:teuthology.orchestra.run.smithi100.stdout: caps mds = "allow" 2024-04-23T14:34:06.850 INFO:teuthology.orchestra.run.smithi100.stdout: caps mon = "allow r" 2024-04-23T14:34:06.850 INFO:teuthology.orchestra.run.smithi100.stdout: caps osd = "allow rw pool=cephfs_data" 2024-04-23T14:34:06.850 INFO:teuthology.orchestra.run.smithi100.stderr:updated caps for client.0 2024-04-23T14:34:06.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.848+0000 7fa2ee000640 1 -- 172.21.15.100:0/4167399106 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa2c4068380 msgr2=0x7fa2c406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:06.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.848+0000 7fa2ee000640 1 --2- 172.21.15.100:0/4167399106 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa2c4068380 0x7fa2c406a840 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8007fd0 tx=0x7fa2d8007c80 comp rx=0 tx=0).stop 2024-04-23T14:34:06.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.848+0000 7fa2ee000640 1 -- 172.21.15.100:0/4167399106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e815e700 msgr2=0x7fa2e80b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:06.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.848+0000 7fa2ee000640 1 --2- 172.21.15.100:0/4167399106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e815e700 0x7fa2e80b5060 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000c730 tx=0x7fa2e000cbf0 comp rx=0 tx=0).stop 2024-04-23T14:34:06.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.848+0000 7fa2ee000640 1 -- 172.21.15.100:0/4167399106 shutdown_connections 2024-04-23T14:34:06.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.848+0000 7fa2ee000640 1 --2- 172.21.15.100:0/4167399106 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa2c4068380 0x7fa2c406a840 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:06.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.848+0000 7fa2ee000640 1 --2- 172.21.15.100:0/4167399106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e815e700 0x7fa2e80b5060 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:06.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.848+0000 7fa2ee000640 1 --2- 172.21.15.100:0/4167399106 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa2e80b3910 0x7fa2e8152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:06.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.848+0000 7fa2ee000640 1 --2- 172.21.15.100:0/4167399106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2e806d990 0x7fa2e80b6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:06.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.848+0000 7fa2ee000640 1 -- 172.21.15.100:0/4167399106 >> 172.21.15.100:0/4167399106 conn(0x7fa2e80719d0 msgr2=0x7fa2e80d2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:06.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.848+0000 7fa2ee000640 1 -- 172.21.15.100:0/4167399106 shutdown_connections 2024-04-23T14:34:06.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:06.848+0000 7fa2ee000640 1 -- 172.21.15.100:0/4167399106 wait complete. 2024-04-23T14:34:06.871 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-23T14:34:06.872 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:34:06.953 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:34:07.038 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-23T14:34:07.038 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:34:07.038 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:34:07.038 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:07.047 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350140k,nr_inodes=4087535,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274960k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=820 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274956k,nr_inodes=818739,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274960k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:34:07.047 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:34:07.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:07.047 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:34:07.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:34:07.099 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-23T14:34:07.099 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:34:07.099 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:34:07.099 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:07.108 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3578 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:34:07.108 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-04-23T14:34:07.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:07.108 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-23T14:34:07.163 INFO:teuthology.orchestra.run.smithi151.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-23T14:34:07.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:34:07.164 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:34:07.164 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:34:07.164 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:07.173 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350140k,nr_inodes=4087535,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274960k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=820 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274956k,nr_inodes=818739,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274960k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:34:07.173 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:34:07.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:07.173 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-23T14:34:07.199 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:34:07.219 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:34:07.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:34:07.223 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-23T14:34:07.223 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:34:07.223 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:34:07.223 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:07.276 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3578 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:34:07.276 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-04-23T14:34:07.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:07.276 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-23T14:34:07.327 INFO:teuthology.orchestra.run.smithi151.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-23T14:34:07.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:34:07.328 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:34:07.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.460+0000 7faee5600640 1 Processor -- start 2024-04-23T14:34:07.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.460+0000 7faee5600640 1 -- start start 2024-04-23T14:34:07.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.460+0000 7faee5600640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7faee015b640 0x7faee00adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:07.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.460+0000 7faee5600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faee00b2a10 0x7faee0072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:07.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.460+0000 7faee5600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faee00cc750 0x7faee0073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:07.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.460+0000 7faee5600640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7faee00af080 con 0x7faee00cc750 2024-04-23T14:34:07.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.460+0000 7faee5600640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7faee00afbc0 con 0x7faee015b640 2024-04-23T14:34:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.460+0000 7faee5600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7faee015ba20 con 0x7faee00b2a10 2024-04-23T14:34:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.460+0000 7faedf400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faee00cc750 0x7faee0073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.460+0000 7faedf400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faee00cc750 0x7faee0073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43116/0 (socket says 172.21.15.100:43116) 2024-04-23T14:34:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.460+0000 7faedf400640 1 -- 172.21.15.100:0/612056914 learned_addr learned my addr 172.21.15.100:0/612056914 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.460+0000 7faedea00640 1 --2- 172.21.15.100:0/612056914 >> v2:172.21.15.151:3300/0 conn(0x7faee015b640 0x7faee00adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.460+0000 7faed7e00640 1 --2- 172.21.15.100:0/612056914 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faee00b2a10 0x7faee0072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.460+0000 7faedf400640 1 -- 172.21.15.100:0/612056914 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faee00b2a10 msgr2=0x7faee0072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.460+0000 7faedf400640 1 --2- 172.21.15.100:0/612056914 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faee00b2a10 0x7faee0072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.460+0000 7faedf400640 1 -- 172.21.15.100:0/612056914 >> v2:172.21.15.151:3300/0 conn(0x7faee015b640 msgr2=0x7faee00adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.460+0000 7faedf400640 1 --2- 172.21.15.100:0/612056914 >> v2:172.21.15.151:3300/0 conn(0x7faee015b640 0x7faee00adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.460+0000 7faedf400640 1 -- 172.21.15.100:0/612056914 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee00af320 con 0x7faee00cc750 2024-04-23T14:34:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.460+0000 7faedea00640 1 --2- 172.21.15.100:0/612056914 >> v2:172.21.15.151:3300/0 conn(0x7faee015b640 0x7faee00adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.460+0000 7faed7e00640 1 --2- 172.21.15.100:0/612056914 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faee00b2a10 0x7faee0072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:07.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.464+0000 7faedf400640 1 --2- 172.21.15.100:0/612056914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faee00cc750 0x7faee0073440 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7faee0068100 tx=0x7faed801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1972d46521c3a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:07.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.464+0000 7faede000640 1 -- 172.21.15.100:0/612056914 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed80a5840 con 0x7faee00cc750 2024-04-23T14:34:07.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.464+0000 7faede000640 1 -- 172.21.15.100:0/612056914 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed80a59e0 con 0x7faee00cc750 2024-04-23T14:34:07.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.464+0000 7faede000640 1 -- 172.21.15.100:0/612056914 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed80a5db0 con 0x7faee00cc750 2024-04-23T14:34:07.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.464+0000 7faee5600640 1 -- 172.21.15.100:0/612056914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faee00cc750 msgr2=0x7faee0073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:07.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.464+0000 7faee5600640 1 --2- 172.21.15.100:0/612056914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faee00cc750 0x7faee0073440 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7faee0068100 tx=0x7faed801c9c0 comp rx=0 tx=0).stop 2024-04-23T14:34:07.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.464+0000 7faee5600640 1 -- 172.21.15.100:0/612056914 shutdown_connections 2024-04-23T14:34:07.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.464+0000 7faee5600640 1 --2- 172.21.15.100:0/612056914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faee00cc750 0x7faee0073440 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:07.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.464+0000 7faee5600640 1 --2- 172.21.15.100:0/612056914 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faee00b2a10 0x7faee0072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:07.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.464+0000 7faee5600640 1 --2- 172.21.15.100:0/612056914 >> v2:172.21.15.151:3300/0 conn(0x7faee015b640 0x7faee00adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:07.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.464+0000 7faee5600640 1 -- 172.21.15.100:0/612056914 >> 172.21.15.100:0/612056914 conn(0x7faee00b1e80 msgr2=0x7faee00b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:07.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.464+0000 7faee5600640 1 -- 172.21.15.100:0/612056914 shutdown_connections 2024-04-23T14:34:07.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.464+0000 7faee5600640 1 -- 172.21.15.100:0/612056914 wait complete. 2024-04-23T14:34:07.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.464+0000 7faee5600640 1 Processor -- start 2024-04-23T14:34:07.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.464+0000 7faee5600640 1 -- start start 2024-04-23T14:34:07.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faee5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faee00b2a10 0x7faee01535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:07.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faee5600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faee00cc750 0x7faee014b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:07.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faee5600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faee015b640 0x7faee006c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faee5600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faee0072880 con 0x7faee00b2a10 2024-04-23T14:34:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faee5600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faee0072700 con 0x7faee015b640 2024-04-23T14:34:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faee5600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7faee0072a00 con 0x7faee00cc750 2024-04-23T14:34:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faedea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faee00b2a10 0x7faee01535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faedea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faee00b2a10 0x7faee01535f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43122/0 (socket says 172.21.15.100:43122) 2024-04-23T14:34:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faedea00640 1 -- 172.21.15.100:0/1989946689 learned_addr learned my addr 172.21.15.100:0/1989946689 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faedf400640 1 --2- 172.21.15.100:0/1989946689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faee015b640 0x7faee006c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faed7e00640 1 --2- 172.21.15.100:0/1989946689 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faee00cc750 0x7faee014b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faedea00640 1 -- 172.21.15.100:0/1989946689 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faee00cc750 msgr2=0x7faee014b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faedea00640 1 --2- 172.21.15.100:0/1989946689 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faee00cc750 0x7faee014b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faedea00640 1 -- 172.21.15.100:0/1989946689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faee015b640 msgr2=0x7faee006c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faedea00640 1 --2- 172.21.15.100:0/1989946689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faee015b640 0x7faee006c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faedea00640 1 -- 172.21.15.100:0/1989946689 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee00679a0 con 0x7faee00b2a10 2024-04-23T14:34:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faedea00640 1 --2- 172.21.15.100:0/1989946689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faee00b2a10 0x7faee01535f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7faed000c7f0 tx=0x7faed000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:07.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faed6a00640 1 -- 172.21.15.100:0/1989946689 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0015070 con 0x7faee00b2a10 2024-04-23T14:34:07.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faed6a00640 1 -- 172.21.15.100:0/1989946689 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed0007820 con 0x7faee00b2a10 2024-04-23T14:34:07.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faed6a00640 1 -- 172.21.15.100:0/1989946689 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0007c10 con 0x7faee00b2a10 2024-04-23T14:34:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faee5600640 1 -- 172.21.15.100:0/1989946689 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee0153c80 con 0x7faee00b2a10 2024-04-23T14:34:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faee5600640 1 -- 172.21.15.100:0/1989946689 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee0154240 con 0x7faee00b2a10 2024-04-23T14:34:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faee5600640 1 -- 172.21.15.100:0/1989946689 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee0154480 con 0x7faee00b2a10 2024-04-23T14:34:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.468+0000 7faed6a00640 1 -- 172.21.15.100:0/1989946689 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7faed0018070 con 0x7faee00b2a10 2024-04-23T14:34:07.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.472+0000 7faed6a00640 1 --2- 172.21.15.100:0/1989946689 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faebc068380 0x7faebc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:07.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.472+0000 7faed6a00640 1 -- 172.21.15.100:0/1989946689 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7faed0086de0 con 0x7faee00b2a10 2024-04-23T14:34:07.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.472+0000 7faed7e00640 1 --2- 172.21.15.100:0/1989946689 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faebc068380 0x7faebc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:07.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.472+0000 7faed7e00640 1 --2- 172.21.15.100:0/1989946689 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faebc068380 0x7faebc06a840 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faee014c450 tx=0x7faec8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:07.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.476+0000 7faed6a00640 1 -- 172.21.15.100:0/1989946689 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faed0056a50 con 0x7faee00b2a10 2024-04-23T14:34:07.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.664+0000 7faee5600640 1 -- 172.21.15.100:0/1989946689 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faee0068ad0 con 0x7faee00b2a10 2024-04-23T14:34:07.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.668+0000 7faed6a00640 1 -- 172.21.15.100:0/1989946689 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5195 (secure 0 0 0) 0x7faed005ad50 con 0x7faee00b2a10 2024-04-23T14:34:07.671 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:07.671 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2601071793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2601071793},{"type":"v1","addr":"172.21.15.100:6837","nonce":2601071793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2930424296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2930424296},{"type":"v1","addr":"172.21.15.151:6833","nonce":2930424296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1406976216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1406976216},{"type":"v1","addr":"172.21.15.151:6835","nonce":1406976216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:33:38.928245+0000","modified":"2024-04-23T14:33:47.983305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5466":{"gid":5466,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/2062210445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2062210445},{"type":"v1","addr":"172.21.15.100:6836","nonce":2062210445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5466,"qdb_cluster":[5466]},"id":2}]} 2024-04-23T14:34:07.672 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 19 2024-04-23T14:34:07.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.668+0000 7faee5600640 1 -- 172.21.15.100:0/1989946689 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faebc068380 msgr2=0x7faebc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:07.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.668+0000 7faee5600640 1 --2- 172.21.15.100:0/1989946689 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faebc068380 0x7faebc06a840 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faee014c450 tx=0x7faec8008040 comp rx=0 tx=0).stop 2024-04-23T14:34:07.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.668+0000 7faee5600640 1 -- 172.21.15.100:0/1989946689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faee00b2a10 msgr2=0x7faee01535f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:07.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.668+0000 7faee5600640 1 --2- 172.21.15.100:0/1989946689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faee00b2a10 0x7faee01535f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7faed000c7f0 tx=0x7faed000ccb0 comp rx=0 tx=0).stop 2024-04-23T14:34:07.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.668+0000 7faee5600640 1 -- 172.21.15.100:0/1989946689 shutdown_connections 2024-04-23T14:34:07.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.668+0000 7faee5600640 1 --2- 172.21.15.100:0/1989946689 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faebc068380 0x7faebc06a840 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:07.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.668+0000 7faee5600640 1 --2- 172.21.15.100:0/1989946689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faee015b640 0x7faee006c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:07.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.668+0000 7faee5600640 1 --2- 172.21.15.100:0/1989946689 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faee00cc750 0x7faee014b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:07.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.668+0000 7faee5600640 1 --2- 172.21.15.100:0/1989946689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faee00b2a10 0x7faee01535f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:07.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.668+0000 7faee5600640 1 -- 172.21.15.100:0/1989946689 >> 172.21.15.100:0/1989946689 conn(0x7faee00b1e80 msgr2=0x7faee015a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:07.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.668+0000 7faee5600640 1 -- 172.21.15.100:0/1989946689 shutdown_connections 2024-04-23T14:34:07.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.668+0000 7faee5600640 1 -- 172.21.15.100:0/1989946689 wait complete. 2024-04-23T14:34:07.694 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:07.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.824+0000 7f366a800640 1 Processor -- start 2024-04-23T14:34:07.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.824+0000 7f366a800640 1 -- start start 2024-04-23T14:34:07.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.824+0000 7f366a800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f366415b640 0x7f36640adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:07.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.824+0000 7f366a800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36640af080 con 0x7f3664073610 2024-04-23T14:34:07.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.824+0000 7f366a800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36640afbc0 con 0x7f36640b2a10 2024-04-23T14:34:07.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.824+0000 7f366a800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f366415ba20 con 0x7f366415b640 2024-04-23T14:34:07.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.824+0000 7f3663e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f3664073610 0x7f366406df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42030/0 (socket says 172.21.15.100:42030) 2024-04-23T14:34:07.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.824+0000 7f3663e00640 1 -- 172.21.15.100:0/1480520788 learned_addr learned my addr 172.21.15.100:0/1480520788 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:07.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.828+0000 7f3663400640 1 --2- 172.21.15.100:0/1480520788 >> v2:172.21.15.151:3300/0 conn(0x7f366415b640 0x7f36640adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:07.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.828+0000 7f3662a00640 1 -- 172.21.15.100:0/1480520788 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2525333556 0 0) 0x7f36640af080 con 0x7f3664073610 2024-04-23T14:34:07.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.828+0000 7f3662a00640 1 -- 172.21.15.100:0/1480520788 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3648003530 con 0x7f3664073610 2024-04-23T14:34:07.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.828+0000 7f3662a00640 1 -- 172.21.15.100:0/1480520788 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1166457356 0 0) 0x7f36640afbc0 con 0x7f36640b2a10 2024-04-23T14:34:07.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.828+0000 7f3662a00640 1 -- 172.21.15.100:0/1480520788 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36640af080 con 0x7f36640b2a10 2024-04-23T14:34:07.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.828+0000 7f3663400640 1 -- 172.21.15.100:0/1480520788 >> v1:172.21.15.151:6789/0 conn(0x7f36640b2a10 legacy=0x7f3664072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:07.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.828+0000 7f3663400640 1 -- 172.21.15.100:0/1480520788 >> v1:172.21.15.100:6789/0 conn(0x7f3664073610 legacy=0x7f366406df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:07.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.828+0000 7f3663400640 1 -- 172.21.15.100:0/1480520788 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36640af320 con 0x7f366415b640 2024-04-23T14:34:07.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.828+0000 7f3663400640 1 --2- 172.21.15.100:0/1480520788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f366415b640 0x7f36640adfc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f36540097c0 tx=0x7f365401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a370fb268f300178 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:07.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f3662a00640 1 -- 172.21.15.100:0/1480520788 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36540b1070 con 0x7f366415b640 2024-04-23T14:34:07.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f3662a00640 1 -- 172.21.15.100:0/1480520788 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36540a8050 con 0x7f366415b640 2024-04-23T14:34:07.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f3662a00640 1 -- 172.21.15.100:0/1480520788 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36540ac840 con 0x7f366415b640 2024-04-23T14:34:07.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f366a800640 1 -- 172.21.15.100:0/1480520788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f366415b640 msgr2=0x7f36640adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:07.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f366a800640 1 --2- 172.21.15.100:0/1480520788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f366415b640 0x7f36640adfc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f36540097c0 tx=0x7f365401c3e0 comp rx=0 tx=0).stop 2024-04-23T14:34:07.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f366a800640 1 -- 172.21.15.100:0/1480520788 shutdown_connections 2024-04-23T14:34:07.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f366a800640 1 --2- 172.21.15.100:0/1480520788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f366415b640 0x7f36640adfc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:07.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f366a800640 1 -- 172.21.15.100:0/1480520788 >> 172.21.15.100:0/1480520788 conn(0x7f36640b1e80 msgr2=0x7f36640b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:07.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f366a800640 1 -- 172.21.15.100:0/1480520788 shutdown_connections 2024-04-23T14:34:07.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f366a800640 1 -- 172.21.15.100:0/1480520788 wait complete. 2024-04-23T14:34:07.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f366a800640 1 Processor -- start 2024-04-23T14:34:07.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f366a800640 1 -- start start 2024-04-23T14:34:07.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f366a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3664073610 0x7f366415e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:07.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f366a800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f36640b2a10 0x7f3664157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:07.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f366a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f366415b640 0x7f36641faa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:07.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f366a800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f366406c340 con 0x7f366415b640 2024-04-23T14:34:07.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f366a800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f366406f160 con 0x7f3664073610 2024-04-23T14:34:07.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f366a800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f366406c4c0 con 0x7f36640b2a10 2024-04-23T14:34:07.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f3663e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f366415b640 0x7f36641faa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:07.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f3663e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f366415b640 0x7f36641faa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43138/0 (socket says 172.21.15.100:43138) 2024-04-23T14:34:07.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f3663e00640 1 -- 172.21.15.100:0/3220264586 learned_addr learned my addr 172.21.15.100:0/3220264586 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:07.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f3663400640 1 --2- 172.21.15.100:0/3220264586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3664073610 0x7f366415e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:07.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f365aa00640 1 --2- 172.21.15.100:0/3220264586 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f36640b2a10 0x7f3664157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:07.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f3663e00640 1 -- 172.21.15.100:0/3220264586 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f36640b2a10 msgr2=0x7f3664157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:07.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f3663e00640 1 --2- 172.21.15.100:0/3220264586 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f36640b2a10 0x7f3664157a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:07.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f3663e00640 1 -- 172.21.15.100:0/3220264586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3664073610 msgr2=0x7f366415e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:07.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f3663e00640 1 --2- 172.21.15.100:0/3220264586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3664073610 0x7f366415e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:07.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.832+0000 7f3663e00640 1 -- 172.21.15.100:0/3220264586 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366414af40 con 0x7f366415b640 2024-04-23T14:34:07.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.836+0000 7f3663e00640 1 --2- 172.21.15.100:0/3220264586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f366415b640 0x7f36641faa80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f365c00caf0 tx=0x7f365c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:07.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.836+0000 7f365be00640 1 -- 172.21.15.100:0/3220264586 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c017070 con 0x7f366415b640 2024-04-23T14:34:07.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.836+0000 7f365be00640 1 -- 172.21.15.100:0/3220264586 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f365c00ac40 con 0x7f366415b640 2024-04-23T14:34:07.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.836+0000 7f366a800640 1 -- 172.21.15.100:0/3220264586 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366414b220 con 0x7f366415b640 2024-04-23T14:34:07.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.836+0000 7f365be00640 1 -- 172.21.15.100:0/3220264586 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c012600 con 0x7f366415b640 2024-04-23T14:34:07.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.836+0000 7f366a800640 1 -- 172.21.15.100:0/3220264586 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f366414bac0 con 0x7f366415b640 2024-04-23T14:34:07.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.836+0000 7f366a800640 1 -- 172.21.15.100:0/3220264586 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3664068ad0 con 0x7f366415b640 2024-04-23T14:34:07.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.836+0000 7f365be00640 1 -- 172.21.15.100:0/3220264586 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f365c01a070 con 0x7f366415b640 2024-04-23T14:34:07.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.836+0000 7f365be00640 1 --2- 172.21.15.100:0/3220264586 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3640068380 0x7f364006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:07.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.840+0000 7f365be00640 1 -- 172.21.15.100:0/3220264586 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f365c0868d0 con 0x7f366415b640 2024-04-23T14:34:07.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.840+0000 7f3663400640 1 --2- 172.21.15.100:0/3220264586 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3640068380 0x7f364006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:07.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.840+0000 7f3663400640 1 --2- 172.21.15.100:0/3220264586 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3640068380 0x7f364006a840 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f36540ae000 tx=0x7f36540a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:07.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:07.844+0000 7f365be00640 1 -- 172.21.15.100:0/3220264586 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f365c0565c0 con 0x7f366415b640 2024-04-23T14:34:08.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.000+0000 7f366a800640 1 -- 172.21.15.100:0/3220264586 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3664154c20 con 0x7f366415b640 2024-04-23T14:34:08.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.000+0000 7f365be00640 1 -- 172.21.15.100:0/3220264586 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+18766 (secure 0 0 0) 0x7f365c05a8c0 con 0x7f366415b640 2024-04-23T14:34:08.006 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:08.006 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":41,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:33:49.609841+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:33:36.592437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-23T14:33:37.882390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-04-23T14:33:47.299378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:08.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.004+0000 7f366a800640 1 -- 172.21.15.100:0/3220264586 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3640068380 msgr2=0x7f364006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:08.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.004+0000 7f366a800640 1 --2- 172.21.15.100:0/3220264586 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3640068380 0x7f364006a840 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f36540ae000 tx=0x7f36540a7000 comp rx=0 tx=0).stop 2024-04-23T14:34:08.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.004+0000 7f366a800640 1 -- 172.21.15.100:0/3220264586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f366415b640 msgr2=0x7f36641faa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:08.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.004+0000 7f366a800640 1 --2- 172.21.15.100:0/3220264586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f366415b640 0x7f36641faa80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f365c00caf0 tx=0x7f365c00a060 comp rx=0 tx=0).stop 2024-04-23T14:34:08.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.004+0000 7f366a800640 1 -- 172.21.15.100:0/3220264586 shutdown_connections 2024-04-23T14:34:08.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.004+0000 7f366a800640 1 --2- 172.21.15.100:0/3220264586 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3640068380 0x7f364006a840 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.004+0000 7f366a800640 1 --2- 172.21.15.100:0/3220264586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f366415b640 0x7f36641faa80 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.004+0000 7f366a800640 1 --2- 172.21.15.100:0/3220264586 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f36640b2a10 0x7f3664157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.004+0000 7f366a800640 1 --2- 172.21.15.100:0/3220264586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3664073610 0x7f366415e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.004+0000 7f366a800640 1 -- 172.21.15.100:0/3220264586 >> 172.21.15.100:0/3220264586 conn(0x7f36640b1e80 msgr2=0x7f36640b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:08.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.004+0000 7f366a800640 1 -- 172.21.15.100:0/3220264586 shutdown_connections 2024-04-23T14:34:08.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.004+0000 7f366a800640 1 -- 172.21.15.100:0/3220264586 wait complete. 2024-04-23T14:34:08.027 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:33:36.592437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:33:37.882390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-04-23T14:33:47.299378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:34:08.028 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:34:08.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.160+0000 7f8992200640 1 Processor -- start 2024-04-23T14:34:08.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.160+0000 7f8992200640 1 -- start start 2024-04-23T14:34:08.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.160+0000 7f8992200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f898c0adb90 0x7f898c0adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:08.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.160+0000 7f8992200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f898c06f950 0x7f898c06fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:08.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.160+0000 7f8992200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f898c06e5f0 con 0x7f898c15e860 2024-04-23T14:34:08.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.160+0000 7f8992200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f898c06e8b0 con 0x7f898c0adb90 2024-04-23T14:34:08.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.160+0000 7f8992200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f898c06eb90 con 0x7f898c06f950 2024-04-23T14:34:08.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.160+0000 7f898be00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f898c06f950 0x7f898c06fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:08.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.160+0000 7f8982a00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f898c0adb90 0x7f898c0adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:08.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.160+0000 7f898be00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f898c06f950 0x7f898c06fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:34080/0 (socket says 172.21.15.100:34080) 2024-04-23T14:34:08.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.160+0000 7f898be00640 1 -- 172.21.15.100:0/2306481204 learned_addr learned my addr 172.21.15.100:0/2306481204 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:08.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.164+0000 7f898be00640 1 -- 172.21.15.100:0/2306481204 >> v2:172.21.15.151:3300/0 conn(0x7f898c0adb90 msgr2=0x7f898c0adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:08.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.164+0000 7f898be00640 1 --2- 172.21.15.100:0/2306481204 >> v2:172.21.15.151:3300/0 conn(0x7f898c0adb90 0x7f898c0adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.164+0000 7f898be00640 1 -- 172.21.15.100:0/2306481204 >> v1:172.21.15.151:6789/0 conn(0x7f898c15e860 legacy=0x7f898c0b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:08.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.164+0000 7f898be00640 1 -- 172.21.15.100:0/2306481204 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898c06ee70 con 0x7f898c06f950 2024-04-23T14:34:08.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.164+0000 7f8982a00640 1 --2- 172.21.15.100:0/2306481204 >> v2:172.21.15.151:3300/0 conn(0x7f898c0adb90 0x7f898c0adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:34:08.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.164+0000 7f898be00640 1 --2- 172.21.15.100:0/2306481204 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f898c06f950 0x7f898c06fd80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8984009870 tx=0x7f898401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cc048c6b4d37f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:08.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.164+0000 7f898aa00640 1 -- 172.21.15.100:0/2306481204 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89840ab650 con 0x7f898c06f950 2024-04-23T14:34:08.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.164+0000 7f898aa00640 1 -- 172.21.15.100:0/2306481204 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89840b0070 con 0x7f898c06f950 2024-04-23T14:34:08.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.164+0000 7f898aa00640 1 -- 172.21.15.100:0/2306481204 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89840aba10 con 0x7f898c06f950 2024-04-23T14:34:08.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.164+0000 7f8992200640 1 -- 172.21.15.100:0/2306481204 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f898c06f950 msgr2=0x7f898c06fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:08.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.164+0000 7f8992200640 1 --2- 172.21.15.100:0/2306481204 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f898c06f950 0x7f898c06fd80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8984009870 tx=0x7f898401c560 comp rx=0 tx=0).stop 2024-04-23T14:34:08.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.164+0000 7f8992200640 1 -- 172.21.15.100:0/2306481204 shutdown_connections 2024-04-23T14:34:08.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.164+0000 7f8992200640 1 --2- 172.21.15.100:0/2306481204 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f898c06f950 0x7f898c06fd80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.164+0000 7f8992200640 1 --2- 172.21.15.100:0/2306481204 >> v2:172.21.15.151:3300/0 conn(0x7f898c0adb90 0x7f898c0adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.164+0000 7f8992200640 1 -- 172.21.15.100:0/2306481204 >> 172.21.15.100:0/2306481204 conn(0x7f898c072590 msgr2=0x7f898c0729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:08.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.168+0000 7f8992200640 1 -- 172.21.15.100:0/2306481204 shutdown_connections 2024-04-23T14:34:08.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.168+0000 7f8992200640 1 -- 172.21.15.100:0/2306481204 wait complete. 2024-04-23T14:34:08.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.168+0000 7f8992200640 1 Processor -- start 2024-04-23T14:34:08.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.168+0000 7f8992200640 1 -- start start 2024-04-23T14:34:08.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.168+0000 7f8992200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f898c06f950 0x7f898c14f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:08.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.168+0000 7f8992200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f898c0adb90 0x7f898c144c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:08.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.168+0000 7f8992200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f898c15e860 0x7f898c160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:08.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.168+0000 7f8992200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f898c16b740 con 0x7f898c0adb90 2024-04-23T14:34:08.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.168+0000 7f8992200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f898c15ec40 con 0x7f898c06f950 2024-04-23T14:34:08.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.168+0000 7f8992200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f898c16b8c0 con 0x7f898c15e860 2024-04-23T14:34:08.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.168+0000 7f8982a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f898c0adb90 0x7f898c144c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:08.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.168+0000 7f8982a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f898c0adb90 0x7f898c144c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43146/0 (socket says 172.21.15.100:43146) 2024-04-23T14:34:08.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.168+0000 7f8982a00640 1 -- 172.21.15.100:0/3809630668 learned_addr learned my addr 172.21.15.100:0/3809630668 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:08.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.168+0000 7f898be00640 1 --2- 172.21.15.100:0/3809630668 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f898c15e860 0x7f898c160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:08.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.168+0000 7f898b400640 1 --2- 172.21.15.100:0/3809630668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f898c06f950 0x7f898c14f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:08.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.168+0000 7f8982a00640 1 -- 172.21.15.100:0/3809630668 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f898c15e860 msgr2=0x7f898c160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:08.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.168+0000 7f8982a00640 1 --2- 172.21.15.100:0/3809630668 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f898c15e860 0x7f898c160060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.168+0000 7f8982a00640 1 -- 172.21.15.100:0/3809630668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f898c06f950 msgr2=0x7f898c14f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:08.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.168+0000 7f8982a00640 1 --2- 172.21.15.100:0/3809630668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f898c06f950 0x7f898c14f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.168+0000 7f8982a00640 1 -- 172.21.15.100:0/3809630668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898c1500f0 con 0x7f898c0adb90 2024-04-23T14:34:08.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.168+0000 7f8982a00640 1 --2- 172.21.15.100:0/3809630668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f898c0adb90 0x7f898c144c80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f897400ebe0 tx=0x7f897400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:08.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.172+0000 7f8983e00640 1 -- 172.21.15.100:0/3809630668 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8974017070 con 0x7f898c0adb90 2024-04-23T14:34:08.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.172+0000 7f8983e00640 1 -- 172.21.15.100:0/3809630668 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f897400cc30 con 0x7f898c0adb90 2024-04-23T14:34:08.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.172+0000 7f8983e00640 1 -- 172.21.15.100:0/3809630668 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8974013470 con 0x7f898c0adb90 2024-04-23T14:34:08.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.172+0000 7f8992200640 1 -- 172.21.15.100:0/3809630668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898c1503d0 con 0x7f898c0adb90 2024-04-23T14:34:08.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.172+0000 7f8992200640 1 -- 172.21.15.100:0/3809630668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f898c150990 con 0x7f898c0adb90 2024-04-23T14:34:08.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.172+0000 7f8992200640 1 -- 172.21.15.100:0/3809630668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f898c068ad0 con 0x7f898c0adb90 2024-04-23T14:34:08.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.172+0000 7f8983e00640 1 -- 172.21.15.100:0/3809630668 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f897401a070 con 0x7f898c0adb90 2024-04-23T14:34:08.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.172+0000 7f8983e00640 1 --2- 172.21.15.100:0/3809630668 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8968068380 0x7f896806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:08.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.172+0000 7f8983e00640 1 -- 172.21.15.100:0/3809630668 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f8974086ef0 con 0x7f898c0adb90 2024-04-23T14:34:08.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.172+0000 7f898b400640 1 --2- 172.21.15.100:0/3809630668 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8968068380 0x7f896806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:08.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.172+0000 7f898b400640 1 --2- 172.21.15.100:0/3809630668 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8968068380 0x7f896806a840 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f897c001e00 tx=0x7f897c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:08.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.176+0000 7f8983e00640 1 -- 172.21.15.100:0/3809630668 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8974056b60 con 0x7f898c0adb90 2024-04-23T14:34:08.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.332+0000 7f8992200640 1 -- 172.21.15.100:0/3809630668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f898c1621e0 con 0x7f898c0adb90 2024-04-23T14:34:08.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.336+0000 7f8983e00640 1 -- 172.21.15.100:0/3809630668 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7f897405ae60 con 0x7f898c0adb90 2024-04-23T14:34:08.339 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:34:08.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.336+0000 7f8992200640 1 -- 172.21.15.100:0/3809630668 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8968068380 msgr2=0x7f896806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:08.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.336+0000 7f8992200640 1 --2- 172.21.15.100:0/3809630668 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8968068380 0x7f896806a840 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f897c001e00 tx=0x7f897c009040 comp rx=0 tx=0).stop 2024-04-23T14:34:08.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.336+0000 7f8992200640 1 -- 172.21.15.100:0/3809630668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f898c0adb90 msgr2=0x7f898c144c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:08.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.336+0000 7f8992200640 1 --2- 172.21.15.100:0/3809630668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f898c0adb90 0x7f898c144c80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f897400ebe0 tx=0x7f897400c520 comp rx=0 tx=0).stop 2024-04-23T14:34:08.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.336+0000 7f8992200640 1 -- 172.21.15.100:0/3809630668 shutdown_connections 2024-04-23T14:34:08.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.336+0000 7f8992200640 1 --2- 172.21.15.100:0/3809630668 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8968068380 0x7f896806a840 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.336+0000 7f8992200640 1 --2- 172.21.15.100:0/3809630668 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f898c15e860 0x7f898c160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.336+0000 7f8992200640 1 --2- 172.21.15.100:0/3809630668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f898c0adb90 0x7f898c144c80 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.336+0000 7f8992200640 1 --2- 172.21.15.100:0/3809630668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f898c06f950 0x7f898c14f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.336+0000 7f8992200640 1 -- 172.21.15.100:0/3809630668 >> 172.21.15.100:0/3809630668 conn(0x7f898c072590 msgr2=0x7f898c15a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:08.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.336+0000 7f8992200640 1 -- 172.21.15.100:0/3809630668 shutdown_connections 2024-04-23T14:34:08.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.336+0000 7f8992200640 1 -- 172.21.15.100:0/3809630668 wait complete. 2024-04-23T14:34:08.360 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:34:08.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.492+0000 7f6f90a00640 1 Processor -- start 2024-04-23T14:34:08.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.492+0000 7f6f90a00640 1 -- start start 2024-04-23T14:34:08.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.496+0000 7f6f90a00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f6f8c15c600 0x7f6f8c0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:08.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.496+0000 7f6f90a00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f8c0cf570 con 0x7f6f8c0d1eb0 2024-04-23T14:34:08.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.496+0000 7f6f90a00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f8c0d0e90 con 0x7f6f8c0cea20 2024-04-23T14:34:08.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.496+0000 7f6f90a00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f6f8c0d1170 con 0x7f6f8c15c600 2024-04-23T14:34:08.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.496+0000 7f6f81400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f6f8c0d1eb0 0x7f6f8c0b52b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42032/0 (socket says 172.21.15.100:42032) 2024-04-23T14:34:08.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.496+0000 7f6f81400640 1 -- 172.21.15.100:0/3445407830 learned_addr learned my addr 172.21.15.100:0/3445407830 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:08.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.496+0000 7f6f89e00640 1 --2- 172.21.15.100:0/3445407830 >> v2:172.21.15.151:3300/0 conn(0x7f6f8c15c600 0x7f6f8c0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:08.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.496+0000 7f6f89400640 1 -- 172.21.15.100:0/3445407830 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2869954669 0 0) 0x7f6f8c0cf570 con 0x7f6f8c0d1eb0 2024-04-23T14:34:08.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.496+0000 7f6f89400640 1 -- 172.21.15.100:0/3445407830 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f70003530 con 0x7f6f8c0d1eb0 2024-04-23T14:34:08.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.496+0000 7f6f89e00640 1 -- 172.21.15.100:0/3445407830 >> v1:172.21.15.151:6789/0 conn(0x7f6f8c0cea20 legacy=0x7f6f8c0cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:08.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.496+0000 7f6f89e00640 1 -- 172.21.15.100:0/3445407830 >> v1:172.21.15.100:6789/0 conn(0x7f6f8c0d1eb0 legacy=0x7f6f8c0b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:08.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.496+0000 7f6f89e00640 1 -- 172.21.15.100:0/3445407830 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c0d1450 con 0x7f6f8c15c600 2024-04-23T14:34:08.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.500+0000 7f6f89e00640 1 --2- 172.21.15.100:0/3445407830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f8c15c600 0x7f6f8c0d1970 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009d90 tx=0x7f6f7c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4efd32cbe6306a97 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:08.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.500+0000 7f6f89400640 1 -- 172.21.15.100:0/3445407830 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c0b2070 con 0x7f6f8c15c600 2024-04-23T14:34:08.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.500+0000 7f6f89400640 1 -- 172.21.15.100:0/3445407830 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f7c0a5940 con 0x7f6f8c15c600 2024-04-23T14:34:08.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.500+0000 7f6f89400640 1 -- 172.21.15.100:0/3445407830 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c0a5d10 con 0x7f6f8c15c600 2024-04-23T14:34:08.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.500+0000 7f6f90a00640 1 -- 172.21.15.100:0/3445407830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f8c15c600 msgr2=0x7f6f8c0d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:08.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.500+0000 7f6f90a00640 1 --2- 172.21.15.100:0/3445407830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f8c15c600 0x7f6f8c0d1970 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009d90 tx=0x7f6f7c01c970 comp rx=0 tx=0).stop 2024-04-23T14:34:08.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.500+0000 7f6f90a00640 1 -- 172.21.15.100:0/3445407830 shutdown_connections 2024-04-23T14:34:08.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.500+0000 7f6f90a00640 1 --2- 172.21.15.100:0/3445407830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f8c15c600 0x7f6f8c0d1970 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.500+0000 7f6f90a00640 1 -- 172.21.15.100:0/3445407830 >> 172.21.15.100:0/3445407830 conn(0x7f6f8c0719d0 msgr2=0x7f6f8c071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:08.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.500+0000 7f6f90a00640 1 -- 172.21.15.100:0/3445407830 shutdown_connections 2024-04-23T14:34:08.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.500+0000 7f6f90a00640 1 -- 172.21.15.100:0/3445407830 wait complete. 2024-04-23T14:34:08.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.500+0000 7f6f90a00640 1 Processor -- start 2024-04-23T14:34:08.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.500+0000 7f6f90a00640 1 -- start start 2024-04-23T14:34:08.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f90a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f8c0cea20 0x7f6f8c0cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:08.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f90a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f8c0d1eb0 0x7f6f8c0c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:08.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f90a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6f8c15c600 0x7f6f8c14d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:08.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f90a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c0cd2e0 con 0x7f6f8c0cea20 2024-04-23T14:34:08.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f90a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c0cd160 con 0x7f6f8c0d1eb0 2024-04-23T14:34:08.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f90a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f6f8c0cee30 con 0x7f6f8c15c600 2024-04-23T14:34:08.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f89e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f8c0cea20 0x7f6f8c0cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:08.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f89e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f8c0cea20 0x7f6f8c0cbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43154/0 (socket says 172.21.15.100:43154) 2024-04-23T14:34:08.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f89e00640 1 -- 172.21.15.100:0/839140229 learned_addr learned my addr 172.21.15.100:0/839140229 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:08.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f89e00640 1 -- 172.21.15.100:0/839140229 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6f8c15c600 msgr2=0x7f6f8c14d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:08.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f8a800640 1 --2- 172.21.15.100:0/839140229 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6f8c15c600 0x7f6f8c14d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:08.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f81400640 1 --2- 172.21.15.100:0/839140229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f8c0d1eb0 0x7f6f8c0c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:08.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f89e00640 1 --2- 172.21.15.100:0/839140229 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6f8c15c600 0x7f6f8c14d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f89e00640 1 -- 172.21.15.100:0/839140229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f8c0d1eb0 msgr2=0x7f6f8c0c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:08.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f89e00640 1 --2- 172.21.15.100:0/839140229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f8c0d1eb0 0x7f6f8c0c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f89e00640 1 -- 172.21.15.100:0/839140229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c14e7d0 con 0x7f6f8c0cea20 2024-04-23T14:34:08.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f8a800640 1 --2- 172.21.15.100:0/839140229 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6f8c15c600 0x7f6f8c14d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:08.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f81400640 1 --2- 172.21.15.100:0/839140229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f8c0d1eb0 0x7f6f8c0c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:08.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f89e00640 1 --2- 172.21.15.100:0/839140229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f8c0cea20 0x7f6f8c0cbce0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0b0040 tx=0x7f6f7c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:08.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f82a00640 1 -- 172.21.15.100:0/839140229 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c0b2050 con 0x7f6f8c0cea20 2024-04-23T14:34:08.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f82a00640 1 -- 172.21.15.100:0/839140229 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f7c0be070 con 0x7f6f8c0cea20 2024-04-23T14:34:08.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f82a00640 1 -- 172.21.15.100:0/839140229 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c0ada50 con 0x7f6f8c0cea20 2024-04-23T14:34:08.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f90a00640 1 -- 172.21.15.100:0/839140229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8c0cb590 con 0x7f6f8c0cea20 2024-04-23T14:34:08.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f90a00640 1 -- 172.21.15.100:0/839140229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f8c0cb870 con 0x7f6f8c0cea20 2024-04-23T14:34:08.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f90a00640 1 -- 172.21.15.100:0/839140229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f8c068ad0 con 0x7f6f8c0cea20 2024-04-23T14:34:08.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.504+0000 7f6f82a00640 1 -- 172.21.15.100:0/839140229 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6f7c0adbf0 con 0x7f6f8c0cea20 2024-04-23T14:34:08.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.508+0000 7f6f82a00640 1 --2- 172.21.15.100:0/839140229 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6f68068380 0x7f6f6806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:08.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.508+0000 7f6f82a00640 1 -- 172.21.15.100:0/839140229 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f6f7c122050 con 0x7f6f8c0cea20 2024-04-23T14:34:08.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.508+0000 7f6f81400640 1 --2- 172.21.15.100:0/839140229 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6f68068380 0x7f6f6806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:08.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.508+0000 7f6f81400640 1 --2- 172.21.15.100:0/839140229 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6f68068380 0x7f6f6806a840 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0d1b90 tx=0x7f6f74009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:08.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.512+0000 7f6f82a00640 1 -- 172.21.15.100:0/839140229 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6f7c016560 con 0x7f6f8c0cea20 2024-04-23T14:34:08.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.668+0000 7f6f90a00640 1 -- 172.21.15.100:0/839140229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6f8c0b3720 con 0x7f6f8c0cea20 2024-04-23T14:34:08.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.668+0000 7f6f82a00640 1 -- 172.21.15.100:0/839140229 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) ==== 104+0+11 (secure 0 0 0) 0x7f6f7c0f2310 con 0x7f6f8c0cea20 2024-04-23T14:34:08.673 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:34:08.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.668+0000 7f6f90a00640 1 -- 172.21.15.100:0/839140229 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6f68068380 msgr2=0x7f6f6806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:08.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.668+0000 7f6f90a00640 1 --2- 172.21.15.100:0/839140229 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6f68068380 0x7f6f6806a840 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0d1b90 tx=0x7f6f74009040 comp rx=0 tx=0).stop 2024-04-23T14:34:08.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.668+0000 7f6f90a00640 1 -- 172.21.15.100:0/839140229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f8c0cea20 msgr2=0x7f6f8c0cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:08.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.668+0000 7f6f90a00640 1 --2- 172.21.15.100:0/839140229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f8c0cea20 0x7f6f8c0cbce0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0b0040 tx=0x7f6f7c019710 comp rx=0 tx=0).stop 2024-04-23T14:34:08.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.668+0000 7f6f90a00640 1 -- 172.21.15.100:0/839140229 shutdown_connections 2024-04-23T14:34:08.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.668+0000 7f6f90a00640 1 --2- 172.21.15.100:0/839140229 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6f68068380 0x7f6f6806a840 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.668+0000 7f6f90a00640 1 --2- 172.21.15.100:0/839140229 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6f8c15c600 0x7f6f8c14d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.668+0000 7f6f90a00640 1 --2- 172.21.15.100:0/839140229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f8c0d1eb0 0x7f6f8c0c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.668+0000 7f6f90a00640 1 --2- 172.21.15.100:0/839140229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f8c0cea20 0x7f6f8c0cbce0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.668+0000 7f6f90a00640 1 -- 172.21.15.100:0/839140229 >> 172.21.15.100:0/839140229 conn(0x7f6f8c0719d0 msgr2=0x7f6f8c1577d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:08.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.668+0000 7f6f90a00640 1 -- 172.21.15.100:0/839140229 shutdown_connections 2024-04-23T14:34:08.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.668+0000 7f6f90a00640 1 -- 172.21.15.100:0/839140229 wait complete. 2024-04-23T14:34:08.695 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:34:08.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.828+0000 7f2124200640 1 Processor -- start 2024-04-23T14:34:08.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.828+0000 7f2124200640 1 -- start start 2024-04-23T14:34:08.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.828+0000 7f2124200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f211c06f8c0 0x7f211c06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:08.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.828+0000 7f2124200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f211c06dd60 con 0x7f211c0ce7f0 2024-04-23T14:34:08.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.828+0000 7f2124200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f211c0d09c0 con 0x7f211c0d0170 2024-04-23T14:34:08.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.828+0000 7f2124200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f211c0ad050 con 0x7f211c06f8c0 2024-04-23T14:34:08.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.828+0000 7f2121800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f211c06f8c0 0x7f211c06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:08.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.828+0000 7f2122200640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f211c0ce7f0 0x7f211c0add80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46496/0 (socket says 172.21.15.100:46496) 2024-04-23T14:34:08.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.828+0000 7f2122200640 1 -- 172.21.15.100:0/223882674 learned_addr learned my addr 172.21.15.100:0/223882674 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:08.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.832+0000 7f2121800640 1 -- 172.21.15.100:0/223882674 >> v1:172.21.15.151:6789/0 conn(0x7f211c0d0170 legacy=0x7f211c0ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:08.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.832+0000 7f2121800640 1 -- 172.21.15.100:0/223882674 >> v1:172.21.15.100:6789/0 conn(0x7f211c0ce7f0 legacy=0x7f211c0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:08.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.832+0000 7f2121800640 1 -- 172.21.15.100:0/223882674 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211c0ad2a0 con 0x7f211c06f8c0 2024-04-23T14:34:08.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.832+0000 7f2121800640 1 --2- 172.21.15.100:0/223882674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c06f8c0 0x7f211c06c6c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f21100097c0 tx=0x7f211001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aaf5cb7b324bb47 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:08.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.832+0000 7f211be00640 1 -- 172.21.15.100:0/223882674 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21100b0070 con 0x7f211c06f8c0 2024-04-23T14:34:08.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.832+0000 7f211be00640 1 -- 172.21.15.100:0/223882674 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21100ab600 con 0x7f211c06f8c0 2024-04-23T14:34:08.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.832+0000 7f211be00640 1 -- 172.21.15.100:0/223882674 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21100ab9d0 con 0x7f211c06f8c0 2024-04-23T14:34:08.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.832+0000 7f2124200640 1 -- 172.21.15.100:0/223882674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c06f8c0 msgr2=0x7f211c06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:08.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.832+0000 7f2124200640 1 --2- 172.21.15.100:0/223882674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c06f8c0 0x7f211c06c6c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f21100097c0 tx=0x7f211001c460 comp rx=0 tx=0).stop 2024-04-23T14:34:08.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2124200640 1 -- 172.21.15.100:0/223882674 shutdown_connections 2024-04-23T14:34:08.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2124200640 1 --2- 172.21.15.100:0/223882674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c06f8c0 0x7f211c06c6c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2124200640 1 -- 172.21.15.100:0/223882674 >> 172.21.15.100:0/223882674 conn(0x7f211c0b5200 msgr2=0x7f211c0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:08.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2124200640 1 -- 172.21.15.100:0/223882674 shutdown_connections 2024-04-23T14:34:08.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2124200640 1 -- 172.21.15.100:0/223882674 wait complete. 2024-04-23T14:34:08.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2124200640 1 Processor -- start 2024-04-23T14:34:08.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2124200640 1 -- start start 2024-04-23T14:34:08.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2124200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f211c06f8c0 0x7f211c099800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:08.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2124200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f211c0ce7f0 0x7f211c14aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:08.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2124200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c0d0170 0x7f211c14b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:08.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2124200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f211c1697a0 con 0x7f211c0d0170 2024-04-23T14:34:08.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2124200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f211c169620 con 0x7f211c0ce7f0 2024-04-23T14:34:08.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2124200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f211c169920 con 0x7f211c06f8c0 2024-04-23T14:34:08.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2122200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c0d0170 0x7f211c14b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:08.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2122200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c0d0170 0x7f211c14b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35828/0 (socket says 172.21.15.100:35828) 2024-04-23T14:34:08.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2122200640 1 -- 172.21.15.100:0/3195506473 learned_addr learned my addr 172.21.15.100:0/3195506473 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:08.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2121800640 1 --2- 172.21.15.100:0/3195506473 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f211c06f8c0 0x7f211c099800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:08.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2120e00640 1 --2- 172.21.15.100:0/3195506473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f211c0ce7f0 0x7f211c14aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:08.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2122200640 1 -- 172.21.15.100:0/3195506473 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f211c06f8c0 msgr2=0x7f211c099800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:08.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2122200640 1 --2- 172.21.15.100:0/3195506473 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f211c06f8c0 0x7f211c099800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2122200640 1 -- 172.21.15.100:0/3195506473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f211c0ce7f0 msgr2=0x7f211c14aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:08.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2122200640 1 --2- 172.21.15.100:0/3195506473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f211c0ce7f0 0x7f211c14aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:08.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2122200640 1 -- 172.21.15.100:0/3195506473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211c0c6340 con 0x7f211c0d0170 2024-04-23T14:34:08.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2122200640 1 --2- 172.21.15.100:0/3195506473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c0d0170 0x7f211c14b1e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f211400cb10 tx=0x7f211400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:08.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2119600640 1 -- 172.21.15.100:0/3195506473 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2114017070 con 0x7f211c0d0170 2024-04-23T14:34:08.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2119600640 1 -- 172.21.15.100:0/3195506473 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f211400ac70 con 0x7f211c0d0170 2024-04-23T14:34:08.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2119600640 1 -- 172.21.15.100:0/3195506473 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211400ae60 con 0x7f211c0d0170 2024-04-23T14:34:08.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.836+0000 7f2124200640 1 -- 172.21.15.100:0/3195506473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f211c0728f0 con 0x7f211c0d0170 2024-04-23T14:34:08.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.840+0000 7f2124200640 1 -- 172.21.15.100:0/3195506473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f211c0b11e0 con 0x7f211c0d0170 2024-04-23T14:34:08.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.840+0000 7f2124200640 1 -- 172.21.15.100:0/3195506473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f211c068ad0 con 0x7f211c0d0170 2024-04-23T14:34:08.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.840+0000 7f2119600640 1 -- 172.21.15.100:0/3195506473 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f211401a070 con 0x7f211c0d0170 2024-04-23T14:34:08.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.840+0000 7f2119600640 1 --2- 172.21.15.100:0/3195506473 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f20fc068380 0x7f20fc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:08.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.840+0000 7f2119600640 1 -- 172.21.15.100:0/3195506473 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f21140868b0 con 0x7f211c0d0170 2024-04-23T14:34:08.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.840+0000 7f2121800640 1 --2- 172.21.15.100:0/3195506473 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f20fc068380 0x7f20fc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:08.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.840+0000 7f2121800640 1 --2- 172.21.15.100:0/3195506473 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f20fc068380 0x7f20fc06a840 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f21100ad000 tx=0x7f21100a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:08.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:08.844+0000 7f2119600640 1 -- 172.21.15.100:0/3195506473 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f211c06dd60 con 0x7f211c0d0170 2024-04-23T14:34:09.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.000+0000 7f2124200640 1 -- 172.21.15.100:0/3195506473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f211c06cc00 con 0x7f211c0d0170 2024-04-23T14:34:09.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.000+0000 7f2119600640 1 -- 172.21.15.100:0/3195506473 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) ==== 100+0+11 (secure 0 0 0) 0x7f2114056520 con 0x7f211c0d0170 2024-04-23T14:34:09.004 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:34:09.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.000+0000 7f2124200640 1 -- 172.21.15.100:0/3195506473 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f20fc068380 msgr2=0x7f20fc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:09.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.000+0000 7f2124200640 1 --2- 172.21.15.100:0/3195506473 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f20fc068380 0x7f20fc06a840 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f21100ad000 tx=0x7f21100a6000 comp rx=0 tx=0).stop 2024-04-23T14:34:09.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.000+0000 7f2124200640 1 -- 172.21.15.100:0/3195506473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c0d0170 msgr2=0x7f211c14b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:09.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.000+0000 7f2124200640 1 --2- 172.21.15.100:0/3195506473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c0d0170 0x7f211c14b1e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f211400cb10 tx=0x7f211400a060 comp rx=0 tx=0).stop 2024-04-23T14:34:09.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.000+0000 7f2124200640 1 -- 172.21.15.100:0/3195506473 shutdown_connections 2024-04-23T14:34:09.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.000+0000 7f2124200640 1 --2- 172.21.15.100:0/3195506473 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f20fc068380 0x7f20fc06a840 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.000+0000 7f2124200640 1 --2- 172.21.15.100:0/3195506473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c0d0170 0x7f211c14b1e0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.000+0000 7f2124200640 1 --2- 172.21.15.100:0/3195506473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f211c0ce7f0 0x7f211c14aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.000+0000 7f2124200640 1 --2- 172.21.15.100:0/3195506473 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f211c06f8c0 0x7f211c099800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.000+0000 7f2124200640 1 -- 172.21.15.100:0/3195506473 >> 172.21.15.100:0/3195506473 conn(0x7f211c0b5200 msgr2=0x7f211c0ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:09.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.000+0000 7f2124200640 1 -- 172.21.15.100:0/3195506473 shutdown_connections 2024-04-23T14:34:09.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.000+0000 7f2124200640 1 -- 172.21.15.100:0/3195506473 wait complete. 2024-04-23T14:34:09.024 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-04-23T14:34:09.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.156+0000 7f3663e00640 1 Processor -- start 2024-04-23T14:34:09.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.156+0000 7f3663e00640 1 -- start start 2024-04-23T14:34:09.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.160+0000 7f3663e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f36640d03e0 0x7f36640b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:09.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.160+0000 7f3663e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f366406d6f0 0x7f366406db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:09.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.160+0000 7f3663e00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f366415f420 con 0x7f366415ec40 2024-04-23T14:34:09.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.160+0000 7f3663e00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f366406e0a0 con 0x7f36640d03e0 2024-04-23T14:34:09.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.160+0000 7f3663e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f36640b1c50 con 0x7f366406d6f0 2024-04-23T14:34:09.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.160+0000 7f3662a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f36640d03e0 0x7f36640b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:09.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.160+0000 7f3662a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f36640d03e0 0x7f36640b36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35844/0 (socket says 172.21.15.100:35844) 2024-04-23T14:34:09.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.160+0000 7f3662a00640 1 -- 172.21.15.100:0/1334082754 learned_addr learned my addr 172.21.15.100:0/1334082754 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:09.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.160+0000 7f3663400640 1 --2- 172.21.15.100:0/1334082754 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f366406d6f0 0x7f366406db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:09.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.160+0000 7f3662a00640 1 -- 172.21.15.100:0/1334082754 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f366406d6f0 msgr2=0x7f366406db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:09.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.160+0000 7f3662a00640 1 --2- 172.21.15.100:0/1334082754 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f366406d6f0 0x7f366406db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.160+0000 7f3662a00640 1 -- 172.21.15.100:0/1334082754 >> v1:172.21.15.151:6789/0 conn(0x7f366415ec40 legacy=0x7f36640b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:09.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.160+0000 7f3662a00640 1 -- 172.21.15.100:0/1334082754 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366415f700 con 0x7f36640d03e0 2024-04-23T14:34:09.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.160+0000 7f3662a00640 1 --2- 172.21.15.100:0/1334082754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36640d03e0 0x7f36640b36c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3654009e50 tx=0x7f365401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3775788d24a5fd97 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:09.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3661600640 1 -- 172.21.15.100:0/1334082754 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36540a58d0 con 0x7f36640d03e0 2024-04-23T14:34:09.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3661600640 1 -- 172.21.15.100:0/1334082754 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36540a5a70 con 0x7f36640d03e0 2024-04-23T14:34:09.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3661600640 1 -- 172.21.15.100:0/1334082754 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36540a5e40 con 0x7f36640d03e0 2024-04-23T14:34:09.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3663e00640 1 -- 172.21.15.100:0/1334082754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36640d03e0 msgr2=0x7f36640b36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:09.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3663e00640 1 --2- 172.21.15.100:0/1334082754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36640d03e0 0x7f36640b36c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3654009e50 tx=0x7f365401ca00 comp rx=0 tx=0).stop 2024-04-23T14:34:09.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3663e00640 1 -- 172.21.15.100:0/1334082754 shutdown_connections 2024-04-23T14:34:09.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3663e00640 1 --2- 172.21.15.100:0/1334082754 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f366406d6f0 0x7f366406db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3663e00640 1 --2- 172.21.15.100:0/1334082754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36640d03e0 0x7f36640b36c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3663e00640 1 -- 172.21.15.100:0/1334082754 >> 172.21.15.100:0/1334082754 conn(0x7f3664072420 msgr2=0x7f36640ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:09.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3663e00640 1 -- 172.21.15.100:0/1334082754 shutdown_connections 2024-04-23T14:34:09.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3663e00640 1 -- 172.21.15.100:0/1334082754 wait complete. 2024-04-23T14:34:09.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3663e00640 1 Processor -- start 2024-04-23T14:34:09.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3663e00640 1 -- start start 2024-04-23T14:34:09.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3663e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f366406d6f0 0x7f36641523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:09.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3663e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36640d03e0 0x7f3664153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:09.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3663e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f366415ec40 0x7f36640b6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:09.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3663e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f366415f1c0 con 0x7f36640d03e0 2024-04-23T14:34:09.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3663e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f366415f040 con 0x7f366415ec40 2024-04-23T14:34:09.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3663e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f36640d07c0 con 0x7f366406d6f0 2024-04-23T14:34:09.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3662000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36640d03e0 0x7f3664153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:09.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3662000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36640d03e0 0x7f3664153d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35846/0 (socket says 172.21.15.100:35846) 2024-04-23T14:34:09.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3662000640 1 -- 172.21.15.100:0/3010395115 learned_addr learned my addr 172.21.15.100:0/3010395115 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:09.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3662a00640 1 --2- 172.21.15.100:0/3010395115 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f366406d6f0 0x7f36641523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:09.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.164+0000 7f3663400640 1 --2- 172.21.15.100:0/3010395115 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f366415ec40 0x7f36640b6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:09.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.168+0000 7f3662000640 1 -- 172.21.15.100:0/3010395115 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f366406d6f0 msgr2=0x7f36641523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:09.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.168+0000 7f3662000640 1 --2- 172.21.15.100:0/3010395115 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f366406d6f0 0x7f36641523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.168+0000 7f3662000640 1 -- 172.21.15.100:0/3010395115 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f366415ec40 msgr2=0x7f36640b6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:09.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.168+0000 7f3662000640 1 --2- 172.21.15.100:0/3010395115 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f366415ec40 0x7f36640b6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.168+0000 7f3662000640 1 -- 172.21.15.100:0/3010395115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3664152cc0 con 0x7f36640d03e0 2024-04-23T14:34:09.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.168+0000 7f3662000640 1 --2- 172.21.15.100:0/3010395115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36640d03e0 0x7f3664153d60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f365000c420 tx=0x7f365000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:09.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.168+0000 7f365b400640 1 -- 172.21.15.100:0/3010395115 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36500052d0 con 0x7f36640d03e0 2024-04-23T14:34:09.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.168+0000 7f365b400640 1 -- 172.21.15.100:0/3010395115 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3650005470 con 0x7f36640d03e0 2024-04-23T14:34:09.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.168+0000 7f365b400640 1 -- 172.21.15.100:0/3010395115 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3650007260 con 0x7f36640d03e0 2024-04-23T14:34:09.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.168+0000 7f3663e00640 1 -- 172.21.15.100:0/3010395115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3664152fa0 con 0x7f36640d03e0 2024-04-23T14:34:09.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.168+0000 7f3663e00640 1 -- 172.21.15.100:0/3010395115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3664153840 con 0x7f36640d03e0 2024-04-23T14:34:09.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.168+0000 7f3663e00640 1 -- 172.21.15.100:0/3010395115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3664153170 con 0x7f36640d03e0 2024-04-23T14:34:09.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.168+0000 7f365b400640 1 -- 172.21.15.100:0/3010395115 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3650007400 con 0x7f36640d03e0 2024-04-23T14:34:09.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.168+0000 7f365b400640 1 --2- 172.21.15.100:0/3010395115 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3644068380 0x7f364406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:09.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.168+0000 7f365b400640 1 -- 172.21.15.100:0/3010395115 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f3650086020 con 0x7f36640d03e0 2024-04-23T14:34:09.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.172+0000 7f3662a00640 1 --2- 172.21.15.100:0/3010395115 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3644068380 0x7f364406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:09.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.172+0000 7f3662a00640 1 --2- 172.21.15.100:0/3010395115 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3644068380 0x7f364406a840 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f36540ae000 tx=0x7f36540a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:09.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.172+0000 7f365b400640 1 -- 172.21.15.100:0/3010395115 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3650055d10 con 0x7f36640d03e0 2024-04-23T14:34:09.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.328+0000 7f3663e00640 1 -- 172.21.15.100:0/3010395115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) -- 0x7f3664068ad0 con 0x7f36640d03e0 2024-04-23T14:34:09.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.328+0000 7f365b400640 1 -- 172.21.15.100:0/3010395115 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v41) ==== 97+0+11 (secure 0 0 0) 0x7f365005a010 con 0x7f36640d03e0 2024-04-23T14:34:09.334 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:34:09.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.332+0000 7f3663e00640 1 -- 172.21.15.100:0/3010395115 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3644068380 msgr2=0x7f364406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:09.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.332+0000 7f3663e00640 1 --2- 172.21.15.100:0/3010395115 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3644068380 0x7f364406a840 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f36540ae000 tx=0x7f36540a7070 comp rx=0 tx=0).stop 2024-04-23T14:34:09.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.332+0000 7f3663e00640 1 -- 172.21.15.100:0/3010395115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36640d03e0 msgr2=0x7f3664153d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:09.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.332+0000 7f3663e00640 1 --2- 172.21.15.100:0/3010395115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36640d03e0 0x7f3664153d60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f365000c420 tx=0x7f365000c820 comp rx=0 tx=0).stop 2024-04-23T14:34:09.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.332+0000 7f3663e00640 1 -- 172.21.15.100:0/3010395115 shutdown_connections 2024-04-23T14:34:09.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.332+0000 7f3663e00640 1 --2- 172.21.15.100:0/3010395115 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3644068380 0x7f364406a840 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.332+0000 7f3663e00640 1 --2- 172.21.15.100:0/3010395115 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f366415ec40 0x7f36640b6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.332+0000 7f3663e00640 1 --2- 172.21.15.100:0/3010395115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36640d03e0 0x7f3664153d60 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.332+0000 7f3663e00640 1 --2- 172.21.15.100:0/3010395115 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f366406d6f0 0x7f36641523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.332+0000 7f3663e00640 1 -- 172.21.15.100:0/3010395115 >> 172.21.15.100:0/3010395115 conn(0x7f3664072420 msgr2=0x7f3664155a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:09.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.332+0000 7f3663e00640 1 -- 172.21.15.100:0/3010395115 shutdown_connections 2024-04-23T14:34:09.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.332+0000 7f3663e00640 1 -- 172.21.15.100:0/3010395115 wait complete. 2024-04-23T14:34:09.355 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:34:09.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.484+0000 7fc13be00640 1 Processor -- start 2024-04-23T14:34:09.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.484+0000 7fc13be00640 1 -- start start 2024-04-23T14:34:09.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.484+0000 7fc13be00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc1340ad870 0x7fc1340b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:09.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.484+0000 7fc13be00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc134160de0 0x7fc13406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:09.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.484+0000 7fc13be00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc134160320 con 0x7fc13415c470 2024-04-23T14:34:09.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.484+0000 7fc13be00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc1340cc7e0 con 0x7fc134160de0 2024-04-23T14:34:09.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.484+0000 7fc13be00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fc1341605e0 con 0x7fc1340ad870 2024-04-23T14:34:09.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.484+0000 7fc139e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc134160de0 0x7fc13406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:09.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.484+0000 7fc139400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fc13415c470 0x7fc1340ad160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46512/0 (socket says 172.21.15.100:46512) 2024-04-23T14:34:09.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.484+0000 7fc139400640 1 -- 172.21.15.100:0/2679734715 learned_addr learned my addr 172.21.15.100:0/2679734715 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:09.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.484+0000 7fc138a00640 1 --2- 172.21.15.100:0/2679734715 >> v2:172.21.15.151:3300/0 conn(0x7fc1340ad870 0x7fc1340b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:09.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.488+0000 7fc133e00640 1 -- 172.21.15.100:0/2679734715 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3319045543 0 0) 0x7fc134160320 con 0x7fc13415c470 2024-04-23T14:34:09.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.488+0000 7fc133e00640 1 -- 172.21.15.100:0/2679734715 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc11c003530 con 0x7fc13415c470 2024-04-23T14:34:09.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.488+0000 7fc139e00640 1 -- 172.21.15.100:0/2679734715 >> v2:172.21.15.151:3300/0 conn(0x7fc1340ad870 msgr2=0x7fc1340b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:09.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.488+0000 7fc139e00640 1 --2- 172.21.15.100:0/2679734715 >> v2:172.21.15.151:3300/0 conn(0x7fc1340ad870 0x7fc1340b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.488+0000 7fc139e00640 1 -- 172.21.15.100:0/2679734715 >> v1:172.21.15.100:6789/0 conn(0x7fc13415c470 legacy=0x7fc1340ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:09.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.488+0000 7fc139e00640 1 -- 172.21.15.100:0/2679734715 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1341608c0 con 0x7fc134160de0 2024-04-23T14:34:09.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.488+0000 7fc138a00640 1 --2- 172.21.15.100:0/2679734715 >> v2:172.21.15.151:3300/0 conn(0x7fc1340ad870 0x7fc1340b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:34:09.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.488+0000 7fc139e00640 1 --2- 172.21.15.100:0/2679734715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc134160de0 0x7fc13406bab0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc13406d590 tx=0x7fc12c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0189155597b32fa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:09.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.488+0000 7fc133e00640 1 -- 172.21.15.100:0/2679734715 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c0b2070 con 0x7fc134160de0 2024-04-23T14:34:09.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.488+0000 7fc133e00640 1 -- 172.21.15.100:0/2679734715 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc12c0a5870 con 0x7fc134160de0 2024-04-23T14:34:09.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.488+0000 7fc133e00640 1 -- 172.21.15.100:0/2679734715 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c0a5c20 con 0x7fc134160de0 2024-04-23T14:34:09.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc13be00640 1 -- 172.21.15.100:0/2679734715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc134160de0 msgr2=0x7fc13406bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:09.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc13be00640 1 --2- 172.21.15.100:0/2679734715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc134160de0 0x7fc13406bab0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc13406d590 tx=0x7fc12c01c8a0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc13be00640 1 -- 172.21.15.100:0/2679734715 shutdown_connections 2024-04-23T14:34:09.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc13be00640 1 --2- 172.21.15.100:0/2679734715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc134160de0 0x7fc13406bab0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc13be00640 1 --2- 172.21.15.100:0/2679734715 >> v2:172.21.15.151:3300/0 conn(0x7fc1340ad870 0x7fc1340b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc13be00640 1 -- 172.21.15.100:0/2679734715 >> 172.21.15.100:0/2679734715 conn(0x7fc1340b1e80 msgr2=0x7fc1340b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:09.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc13be00640 1 -- 172.21.15.100:0/2679734715 shutdown_connections 2024-04-23T14:34:09.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc13be00640 1 -- 172.21.15.100:0/2679734715 wait complete. 2024-04-23T14:34:09.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc13be00640 1 Processor -- start 2024-04-23T14:34:09.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc13be00640 1 -- start start 2024-04-23T14:34:09.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc13be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1340ad870 0x7fc134150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:09.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc13be00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc13415c470 0x7fc13415e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:09.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc13be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc134160de0 0x7fc1341f5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:09.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc13be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc13415ca60 con 0x7fc1340ad870 2024-04-23T14:34:09.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc13be00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc1340cc580 con 0x7fc134160de0 2024-04-23T14:34:09.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc13be00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc134169540 con 0x7fc13415c470 2024-04-23T14:34:09.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc139400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1340ad870 0x7fc134150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:09.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc139400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1340ad870 0x7fc134150eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35862/0 (socket says 172.21.15.100:35862) 2024-04-23T14:34:09.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc139400640 1 -- 172.21.15.100:0/410509991 learned_addr learned my addr 172.21.15.100:0/410509991 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:09.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc138a00640 1 --2- 172.21.15.100:0/410509991 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc13415c470 0x7fc13415e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:09.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc139e00640 1 --2- 172.21.15.100:0/410509991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc134160de0 0x7fc1341f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:09.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc139400640 1 -- 172.21.15.100:0/410509991 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc13415c470 msgr2=0x7fc13415e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:09.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc139400640 1 --2- 172.21.15.100:0/410509991 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc13415c470 0x7fc13415e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc139400640 1 -- 172.21.15.100:0/410509991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc134160de0 msgr2=0x7fc1341f5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:09.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc139400640 1 --2- 172.21.15.100:0/410509991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc134160de0 0x7fc1341f5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.492+0000 7fc139400640 1 -- 172.21.15.100:0/410509991 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13415aee0 con 0x7fc1340ad870 2024-04-23T14:34:09.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.496+0000 7fc139400640 1 --2- 172.21.15.100:0/410509991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1340ad870 0x7fc134150eb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc12800c450 tx=0x7fc12800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:09.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.496+0000 7fc131600640 1 -- 172.21.15.100:0/410509991 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128017070 con 0x7fc1340ad870 2024-04-23T14:34:09.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.496+0000 7fc131600640 1 -- 172.21.15.100:0/410509991 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc128005cf0 con 0x7fc1340ad870 2024-04-23T14:34:09.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.496+0000 7fc131600640 1 -- 172.21.15.100:0/410509991 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128007290 con 0x7fc1340ad870 2024-04-23T14:34:09.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.496+0000 7fc13be00640 1 -- 172.21.15.100:0/410509991 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc134151490 con 0x7fc1340ad870 2024-04-23T14:34:09.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.496+0000 7fc13be00640 1 -- 172.21.15.100:0/410509991 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc134151a50 con 0x7fc1340ad870 2024-04-23T14:34:09.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.496+0000 7fc13be00640 1 -- 172.21.15.100:0/410509991 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc134068ad0 con 0x7fc1340ad870 2024-04-23T14:34:09.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.496+0000 7fc131600640 1 -- 172.21.15.100:0/410509991 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc12801a070 con 0x7fc1340ad870 2024-04-23T14:34:09.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.496+0000 7fc131600640 1 --2- 172.21.15.100:0/410509991 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc114068380 0x7fc11406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:09.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.496+0000 7fc131600640 1 -- 172.21.15.100:0/410509991 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fc128086070 con 0x7fc1340ad870 2024-04-23T14:34:09.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.496+0000 7fc138a00640 1 --2- 172.21.15.100:0/410509991 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc114068380 0x7fc11406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:09.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.500+0000 7fc138a00640 1 --2- 172.21.15.100:0/410509991 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc114068380 0x7fc11406a840 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc120009580 tx=0x7fc120009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:09.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.500+0000 7fc131600640 1 -- 172.21.15.100:0/410509991 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc128055c50 con 0x7fc1340ad870 2024-04-23T14:34:09.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.692+0000 7fc13be00640 1 -- 172.21.15.100:0/410509991 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc134167310 con 0x7fc1340ad870 2024-04-23T14:34:09.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.692+0000 7fc131600640 1 -- 172.21.15.100:0/410509991 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5195 (secure 0 0 0) 0x7fc128059f50 con 0x7fc1340ad870 2024-04-23T14:34:09.699 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:09.699 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2601071793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2601071793},{"type":"v1","addr":"172.21.15.100:6837","nonce":2601071793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2930424296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2930424296},{"type":"v1","addr":"172.21.15.151:6833","nonce":2930424296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1406976216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1406976216},{"type":"v1","addr":"172.21.15.151:6835","nonce":1406976216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:33:38.928245+0000","modified":"2024-04-23T14:33:47.983305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5466":{"gid":5466,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/2062210445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2062210445},{"type":"v1","addr":"172.21.15.100:6836","nonce":2062210445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5466,"qdb_cluster":[5466]},"id":2}]} 2024-04-23T14:34:09.699 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 19 2024-04-23T14:34:09.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.696+0000 7fc13be00640 1 -- 172.21.15.100:0/410509991 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc114068380 msgr2=0x7fc11406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:09.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.696+0000 7fc13be00640 1 --2- 172.21.15.100:0/410509991 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc114068380 0x7fc11406a840 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc120009580 tx=0x7fc120009190 comp rx=0 tx=0).stop 2024-04-23T14:34:09.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.696+0000 7fc13be00640 1 -- 172.21.15.100:0/410509991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1340ad870 msgr2=0x7fc134150eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:09.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.696+0000 7fc13be00640 1 --2- 172.21.15.100:0/410509991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1340ad870 0x7fc134150eb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc12800c450 tx=0x7fc12800c910 comp rx=0 tx=0).stop 2024-04-23T14:34:09.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.696+0000 7fc13be00640 1 -- 172.21.15.100:0/410509991 shutdown_connections 2024-04-23T14:34:09.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.696+0000 7fc13be00640 1 --2- 172.21.15.100:0/410509991 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc114068380 0x7fc11406a840 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.696+0000 7fc13be00640 1 --2- 172.21.15.100:0/410509991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc134160de0 0x7fc1341f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.696+0000 7fc13be00640 1 --2- 172.21.15.100:0/410509991 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc13415c470 0x7fc13415e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.696+0000 7fc13be00640 1 --2- 172.21.15.100:0/410509991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1340ad870 0x7fc134150eb0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.696+0000 7fc13be00640 1 -- 172.21.15.100:0/410509991 >> 172.21.15.100:0/410509991 conn(0x7fc1340b1e80 msgr2=0x7fc1340b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:09.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.696+0000 7fc13be00640 1 -- 172.21.15.100:0/410509991 shutdown_connections 2024-04-23T14:34:09.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.696+0000 7fc13be00640 1 -- 172.21.15.100:0/410509991 wait complete. 2024-04-23T14:34:09.720 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:09.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.848+0000 7fb104400640 1 Processor -- start 2024-04-23T14:34:09.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.848+0000 7fb104400640 1 -- start start 2024-04-23T14:34:09.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.852+0000 7fb104400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb0fc06d350 0x7fb0fc073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:09.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.852+0000 7fb104400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fb0fc15fa90 0x7fb0fc06d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:09.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.852+0000 7fb104400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0fc073880 con 0x7fb0fc162390 2024-04-23T14:34:09.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.852+0000 7fb104400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fb0fc06eaf0 con 0x7fb0fc06d350 2024-04-23T14:34:09.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.852+0000 7fb104400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fb0fc06e530 con 0x7fb0fc15fa90 2024-04-23T14:34:09.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.852+0000 7fb101a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb0fc06d350 0x7fb0fc073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:09.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.852+0000 7fb101a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb0fc06d350 0x7fb0fc073340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35866/0 (socket says 172.21.15.100:35866) 2024-04-23T14:34:09.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.852+0000 7fb101a00640 1 -- 172.21.15.100:0/1477050028 learned_addr learned my addr 172.21.15.100:0/1477050028 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:09.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.852+0000 7fb101000640 1 --2- 172.21.15.100:0/1477050028 >> v2:172.21.15.151:3300/0 conn(0x7fb0fc15fa90 0x7fb0fc06d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:09.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.852+0000 7fb101a00640 1 -- 172.21.15.100:0/1477050028 >> v2:172.21.15.151:3300/0 conn(0x7fb0fc15fa90 msgr2=0x7fb0fc06d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:09.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.852+0000 7fb101a00640 1 --2- 172.21.15.100:0/1477050028 >> v2:172.21.15.151:3300/0 conn(0x7fb0fc15fa90 0x7fb0fc06d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.852+0000 7fb101a00640 1 -- 172.21.15.100:0/1477050028 >> v1:172.21.15.151:6789/0 conn(0x7fb0fc162390 legacy=0x7fb0fc162840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:09.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.852+0000 7fb101a00640 1 -- 172.21.15.100:0/1477050028 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0fc06dc70 con 0x7fb0fc06d350 2024-04-23T14:34:09.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.852+0000 7fb101000640 1 --2- 172.21.15.100:0/1477050028 >> v2:172.21.15.151:3300/0 conn(0x7fb0fc15fa90 0x7fb0fc06d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:09.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.856+0000 7fb101a00640 1 --2- 172.21.15.100:0/1477050028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0fc06d350 0x7fb0fc073340 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc06df10 tx=0x7fb0f001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8da5b7e6bf5ebe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:09.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.856+0000 7fb0fbe00640 1 -- 172.21.15.100:0/1477050028 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f00ab680 con 0x7fb0fc06d350 2024-04-23T14:34:09.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.856+0000 7fb0fbe00640 1 -- 172.21.15.100:0/1477050028 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f00b0070 con 0x7fb0fc06d350 2024-04-23T14:34:09.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.856+0000 7fb0fbe00640 1 -- 172.21.15.100:0/1477050028 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f00aba40 con 0x7fb0fc06d350 2024-04-23T14:34:09.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.856+0000 7fb104400640 1 -- 172.21.15.100:0/1477050028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0fc06d350 msgr2=0x7fb0fc073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:09.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.856+0000 7fb104400640 1 --2- 172.21.15.100:0/1477050028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0fc06d350 0x7fb0fc073340 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc06df10 tx=0x7fb0f001c460 comp rx=0 tx=0).stop 2024-04-23T14:34:09.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.856+0000 7fb104400640 1 -- 172.21.15.100:0/1477050028 shutdown_connections 2024-04-23T14:34:09.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.856+0000 7fb104400640 1 --2- 172.21.15.100:0/1477050028 >> v2:172.21.15.151:3300/0 conn(0x7fb0fc15fa90 0x7fb0fc06d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.856+0000 7fb104400640 1 --2- 172.21.15.100:0/1477050028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0fc06d350 0x7fb0fc073340 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.856+0000 7fb104400640 1 -- 172.21.15.100:0/1477050028 >> 172.21.15.100:0/1477050028 conn(0x7fb0fc072600 msgr2=0x7fb0fc072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:09.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.856+0000 7fb104400640 1 -- 172.21.15.100:0/1477050028 shutdown_connections 2024-04-23T14:34:09.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.856+0000 7fb104400640 1 -- 172.21.15.100:0/1477050028 wait complete. 2024-04-23T14:34:09.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.856+0000 7fb104400640 1 Processor -- start 2024-04-23T14:34:09.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.856+0000 7fb104400640 1 -- start start 2024-04-23T14:34:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb104400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0fc06d350 0x7fb0fc1fb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb104400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0fc15fa90 0x7fb0fc14ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb104400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb0fc162390 0x7fb0fc14cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb104400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc16c110 con 0x7fb0fc15fa90 2024-04-23T14:34:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb104400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc16bf90 con 0x7fb0fc06d350 2024-04-23T14:34:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb104400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb0fc16c290 con 0x7fb0fc162390 2024-04-23T14:34:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb101000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0fc15fa90 0x7fb0fc14ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb101000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0fc15fa90 0x7fb0fc14ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35874/0 (socket says 172.21.15.100:35874) 2024-04-23T14:34:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb101000640 1 -- 172.21.15.100:0/4232782653 learned_addr learned my addr 172.21.15.100:0/4232782653 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb101a00640 1 --2- 172.21.15.100:0/4232782653 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0fc06d350 0x7fb0fc1fb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb102400640 1 --2- 172.21.15.100:0/4232782653 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb0fc162390 0x7fb0fc14cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb101000640 1 -- 172.21.15.100:0/4232782653 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb0fc162390 msgr2=0x7fb0fc14cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb101000640 1 --2- 172.21.15.100:0/4232782653 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb0fc162390 0x7fb0fc14cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb101000640 1 -- 172.21.15.100:0/4232782653 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0fc06d350 msgr2=0x7fb0fc1fb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb101000640 1 --2- 172.21.15.100:0/4232782653 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0fc06d350 0x7fb0fc1fb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb101000640 1 -- 172.21.15.100:0/4232782653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0fc150fd0 con 0x7fb0fc15fa90 2024-04-23T14:34:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb101000640 1 --2- 172.21.15.100:0/4232782653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0fc15fa90 0x7fb0fc14ca40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800e7e0 tx=0x7fb0e800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb0f9600640 1 -- 172.21.15.100:0/4232782653 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8017070 con 0x7fb0fc15fa90 2024-04-23T14:34:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb0f9600640 1 -- 172.21.15.100:0/4232782653 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0e800c6d0 con 0x7fb0fc15fa90 2024-04-23T14:34:09.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb104400640 1 -- 172.21.15.100:0/4232782653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0fc1512b0 con 0x7fb0fc15fa90 2024-04-23T14:34:09.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb0f9600640 1 -- 172.21.15.100:0/4232782653 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e800f040 con 0x7fb0fc15fa90 2024-04-23T14:34:09.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb104400640 1 -- 172.21.15.100:0/4232782653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0fc1fbce0 con 0x7fb0fc15fa90 2024-04-23T14:34:09.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb104400640 1 -- 172.21.15.100:0/4232782653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0fc068ad0 con 0x7fb0fc15fa90 2024-04-23T14:34:09.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.860+0000 7fb0f9600640 1 -- 172.21.15.100:0/4232782653 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb0e801a070 con 0x7fb0fc15fa90 2024-04-23T14:34:09.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.864+0000 7fb0f9600640 1 --2- 172.21.15.100:0/4232782653 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb0dc068380 0x7fb0dc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:09.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.864+0000 7fb0f9600640 1 -- 172.21.15.100:0/4232782653 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fb0e8086920 con 0x7fb0fc15fa90 2024-04-23T14:34:09.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.864+0000 7fb101a00640 1 --2- 172.21.15.100:0/4232782653 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb0dc068380 0x7fb0dc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:09.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.864+0000 7fb101a00640 1 --2- 172.21.15.100:0/4232782653 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb0dc068380 0x7fb0dc06a840 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00af000 tx=0x7fb0f00a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:09.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:09.868+0000 7fb0f9600640 1 -- 172.21.15.100:0/4232782653 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb0e8056590 con 0x7fb0fc15fa90 2024-04-23T14:34:10.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.024+0000 7fb104400640 1 -- 172.21.15.100:0/4232782653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0fc073120 con 0x7fb0fc15fa90 2024-04-23T14:34:10.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.028+0000 7fb0f9600640 1 -- 172.21.15.100:0/4232782653 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+18766 (secure 0 0 0) 0x7fb0e805a890 con 0x7fb0fc15fa90 2024-04-23T14:34:10.032 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:10.032 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":41,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:33:49.609841+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:33:36.592437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-23T14:33:37.882390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-04-23T14:33:47.299378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:10.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.028+0000 7fb104400640 1 -- 172.21.15.100:0/4232782653 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb0dc068380 msgr2=0x7fb0dc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:10.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.028+0000 7fb104400640 1 --2- 172.21.15.100:0/4232782653 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb0dc068380 0x7fb0dc06a840 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00af000 tx=0x7fb0f00a7000 comp rx=0 tx=0).stop 2024-04-23T14:34:10.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.028+0000 7fb104400640 1 -- 172.21.15.100:0/4232782653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0fc15fa90 msgr2=0x7fb0fc14ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:10.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.028+0000 7fb104400640 1 --2- 172.21.15.100:0/4232782653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0fc15fa90 0x7fb0fc14ca40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800e7e0 tx=0x7fb0e800eca0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.028+0000 7fb104400640 1 -- 172.21.15.100:0/4232782653 shutdown_connections 2024-04-23T14:34:10.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.028+0000 7fb104400640 1 --2- 172.21.15.100:0/4232782653 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb0dc068380 0x7fb0dc06a840 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.028+0000 7fb104400640 1 --2- 172.21.15.100:0/4232782653 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb0fc162390 0x7fb0fc14cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.028+0000 7fb104400640 1 --2- 172.21.15.100:0/4232782653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0fc15fa90 0x7fb0fc14ca40 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.028+0000 7fb104400640 1 --2- 172.21.15.100:0/4232782653 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0fc06d350 0x7fb0fc1fb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.028+0000 7fb104400640 1 -- 172.21.15.100:0/4232782653 >> 172.21.15.100:0/4232782653 conn(0x7fb0fc072600 msgr2=0x7fb0fc0d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:10.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.028+0000 7fb104400640 1 -- 172.21.15.100:0/4232782653 shutdown_connections 2024-04-23T14:34:10.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.028+0000 7fb104400640 1 -- 172.21.15.100:0/4232782653 wait complete. 2024-04-23T14:34:10.052 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-23T14:34:10.053 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:34:10.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.184+0000 7f0203000640 1 Processor -- start 2024-04-23T14:34:10.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.184+0000 7f0203000640 1 -- start start 2024-04-23T14:34:10.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.184+0000 7f0203000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f01fc06d450 0x7f01fc06d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:10.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.184+0000 7f0203000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01fc06dd70 con 0x7f01fc0cea60 2024-04-23T14:34:10.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.184+0000 7f0203000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01fc0b1f90 con 0x7f01fc0cd460 2024-04-23T14:34:10.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.184+0000 7f0203000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f01fc0b2270 con 0x7f01fc06d450 2024-04-23T14:34:10.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.184+0000 7f01fb400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f01fc0cea60 0x7f01fc0d0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46522/0 (socket says 172.21.15.100:46522) 2024-04-23T14:34:10.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.184+0000 7f01fb400640 1 -- 172.21.15.100:0/2575154301 learned_addr learned my addr 172.21.15.100:0/2575154301 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:10.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.184+0000 7f01fbe00640 1 --2- 172.21.15.100:0/2575154301 >> v2:172.21.15.151:3300/0 conn(0x7f01fc06d450 0x7f01fc06d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:10.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.184+0000 7f01faa00640 1 -- 172.21.15.100:0/2575154301 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2106179101 0 0) 0x7f01fc06dd70 con 0x7f01fc0cea60 2024-04-23T14:34:10.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.184+0000 7f01faa00640 1 -- 172.21.15.100:0/2575154301 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01e0003530 con 0x7f01fc0cea60 2024-04-23T14:34:10.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.184+0000 7f01faa00640 1 -- 172.21.15.100:0/2575154301 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3026658544 0 0) 0x7f01fc0b1f90 con 0x7f01fc0cd460 2024-04-23T14:34:10.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.184+0000 7f01fbe00640 1 -- 172.21.15.100:0/2575154301 >> v1:172.21.15.151:6789/0 conn(0x7f01fc0cd460 legacy=0x7f01fc0b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:10.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.184+0000 7f01fbe00640 1 -- 172.21.15.100:0/2575154301 >> v1:172.21.15.100:6789/0 conn(0x7f01fc0cea60 legacy=0x7f01fc0d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:10.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.184+0000 7f01fbe00640 1 -- 172.21.15.100:0/2575154301 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01fc0b2550 con 0x7f01fc06d450 2024-04-23T14:34:10.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.188+0000 7f01fbe00640 1 --2- 172.21.15.100:0/2575154301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01fc06d450 0x7f01fc06d830 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0097c0 tx=0x7f01ec01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d3ecdacf769576f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:10.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.188+0000 7f01faa00640 1 -- 172.21.15.100:0/2575154301 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec0b0070 con 0x7f01fc06d450 2024-04-23T14:34:10.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.188+0000 7f01faa00640 1 -- 172.21.15.100:0/2575154301 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01ec0ab600 con 0x7f01fc06d450 2024-04-23T14:34:10.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.188+0000 7f01faa00640 1 -- 172.21.15.100:0/2575154301 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec0ab9d0 con 0x7f01fc06d450 2024-04-23T14:34:10.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.188+0000 7f0203000640 1 -- 172.21.15.100:0/2575154301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01fc06d450 msgr2=0x7f01fc06d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:10.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.188+0000 7f0203000640 1 --2- 172.21.15.100:0/2575154301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01fc06d450 0x7f01fc06d830 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0097c0 tx=0x7f01ec01c410 comp rx=0 tx=0).stop 2024-04-23T14:34:10.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.188+0000 7f0203000640 1 -- 172.21.15.100:0/2575154301 shutdown_connections 2024-04-23T14:34:10.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.188+0000 7f0203000640 1 --2- 172.21.15.100:0/2575154301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01fc06d450 0x7f01fc06d830 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.188+0000 7f0203000640 1 -- 172.21.15.100:0/2575154301 >> 172.21.15.100:0/2575154301 conn(0x7f01fc0cee80 msgr2=0x7f01fc0cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:10.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.188+0000 7f0203000640 1 -- 172.21.15.100:0/2575154301 shutdown_connections 2024-04-23T14:34:10.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.188+0000 7f0203000640 1 -- 172.21.15.100:0/2575154301 wait complete. 2024-04-23T14:34:10.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f0203000640 1 Processor -- start 2024-04-23T14:34:10.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f0203000640 1 -- start start 2024-04-23T14:34:10.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f0203000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f01fc06d450 0x7f01fc0c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:10.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f0203000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01fc0cd460 0x7f01fc158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:10.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f0203000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01fc0cea60 0x7f01fc155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:10.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f0203000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f01fc16cd50 con 0x7f01fc0cd460 2024-04-23T14:34:10.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f0203000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f01fc16cbd0 con 0x7f01fc0cea60 2024-04-23T14:34:10.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f0203000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f01fc16ced0 con 0x7f01fc06d450 2024-04-23T14:34:10.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f01fb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01fc0cd460 0x7f01fc158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:10.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f01fb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01fc0cd460 0x7f01fc158940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35888/0 (socket says 172.21.15.100:35888) 2024-04-23T14:34:10.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f01fb400640 1 -- 172.21.15.100:0/3999317032 learned_addr learned my addr 172.21.15.100:0/3999317032 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:10.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f01fbe00640 1 --2- 172.21.15.100:0/3999317032 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f01fc06d450 0x7f01fc0c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:10.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f01fb400640 1 -- 172.21.15.100:0/3999317032 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f01fc06d450 msgr2=0x7f01fc0c88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:10.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f01fb400640 1 --2- 172.21.15.100:0/3999317032 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f01fc06d450 0x7f01fc0c88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f0201000640 1 --2- 172.21.15.100:0/3999317032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01fc0cea60 0x7f01fc155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:10.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f01fb400640 1 -- 172.21.15.100:0/3999317032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01fc0cea60 msgr2=0x7f01fc155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:10.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f01fb400640 1 --2- 172.21.15.100:0/3999317032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01fc0cea60 0x7f01fc155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f01fb400640 1 -- 172.21.15.100:0/3999317032 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01fc0cb9c0 con 0x7f01fc0cd460 2024-04-23T14:34:10.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f01fbe00640 1 --2- 172.21.15.100:0/3999317032 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f01fc06d450 0x7f01fc0c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:10.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f0201000640 1 --2- 172.21.15.100:0/3999317032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01fc0cea60 0x7f01fc155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:10.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f01fb400640 1 --2- 172.21.15.100:0/3999317032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01fc0cd460 0x7f01fc158940 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f01e800c420 tx=0x7f01e800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:10.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f01f3e00640 1 -- 172.21.15.100:0/3999317032 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8017070 con 0x7f01fc0cd460 2024-04-23T14:34:10.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f01f3e00640 1 -- 172.21.15.100:0/3999317032 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e8005d10 con 0x7f01fc0cd460 2024-04-23T14:34:10.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f0203000640 1 -- 172.21.15.100:0/3999317032 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01fc0cbca0 con 0x7f01fc0cd460 2024-04-23T14:34:10.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f01f3e00640 1 -- 172.21.15.100:0/3999317032 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8007310 con 0x7f01fc0cd460 2024-04-23T14:34:10.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f0203000640 1 -- 172.21.15.100:0/3999317032 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01fc15a720 con 0x7f01fc0cd460 2024-04-23T14:34:10.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.192+0000 7f0203000640 1 -- 172.21.15.100:0/3999317032 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01fc068ad0 con 0x7f01fc0cd460 2024-04-23T14:34:10.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.196+0000 7f01f3e00640 1 -- 172.21.15.100:0/3999317032 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f01e801a070 con 0x7f01fc0cd460 2024-04-23T14:34:10.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.196+0000 7f01f3e00640 1 --2- 172.21.15.100:0/3999317032 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f01dc068380 0x7f01dc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:10.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.196+0000 7f01f3e00640 1 -- 172.21.15.100:0/3999317032 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f01e8086080 con 0x7f01fc0cd460 2024-04-23T14:34:10.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.196+0000 7f01fbe00640 1 --2- 172.21.15.100:0/3999317032 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f01dc068380 0x7f01dc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:10.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.196+0000 7f01fbe00640 1 --2- 172.21.15.100:0/3999317032 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f01dc068380 0x7f01dc06a840 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0ad000 tx=0x7f01ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:10.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.200+0000 7f01f3e00640 1 -- 172.21.15.100:0/3999317032 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f01e8056480 con 0x7f01fc0cd460 2024-04-23T14:34:10.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.388+0000 7f0203000640 1 -- 172.21.15.100:0/3999317032 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01fc0ce640 con 0x7f01fc0cd460 2024-04-23T14:34:10.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.388+0000 7f01f3e00640 1 -- 172.21.15.100:0/3999317032 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5195 (secure 0 0 0) 0x7f01e8055d80 con 0x7f01fc0cd460 2024-04-23T14:34:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:10.393 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2601071793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2601071793},{"type":"v1","addr":"172.21.15.100:6837","nonce":2601071793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2930424296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2930424296},{"type":"v1","addr":"172.21.15.151:6833","nonce":2930424296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1406976216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1406976216},{"type":"v1","addr":"172.21.15.151:6835","nonce":1406976216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:33:38.928245+0000","modified":"2024-04-23T14:33:47.983305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5466":{"gid":5466,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/2062210445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2062210445},{"type":"v1","addr":"172.21.15.100:6836","nonce":2062210445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5466,"qdb_cluster":[5466]},"id":2}]} 2024-04-23T14:34:10.393 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 19 2024-04-23T14:34:10.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.388+0000 7f0203000640 1 -- 172.21.15.100:0/3999317032 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f01dc068380 msgr2=0x7f01dc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:10.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.388+0000 7f0203000640 1 --2- 172.21.15.100:0/3999317032 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f01dc068380 0x7f01dc06a840 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0ad000 tx=0x7f01ec008040 comp rx=0 tx=0).stop 2024-04-23T14:34:10.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.388+0000 7f0203000640 1 -- 172.21.15.100:0/3999317032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01fc0cd460 msgr2=0x7f01fc158940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:10.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.388+0000 7f0203000640 1 --2- 172.21.15.100:0/3999317032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01fc0cd460 0x7f01fc158940 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f01e800c420 tx=0x7f01e800c8e0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.388+0000 7f0203000640 1 -- 172.21.15.100:0/3999317032 shutdown_connections 2024-04-23T14:34:10.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.388+0000 7f0203000640 1 --2- 172.21.15.100:0/3999317032 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f01dc068380 0x7f01dc06a840 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.388+0000 7f0203000640 1 --2- 172.21.15.100:0/3999317032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01fc0cea60 0x7f01fc155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.388+0000 7f0203000640 1 --2- 172.21.15.100:0/3999317032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01fc0cd460 0x7f01fc158940 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.388+0000 7f0203000640 1 --2- 172.21.15.100:0/3999317032 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f01fc06d450 0x7f01fc0c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.388+0000 7f0203000640 1 -- 172.21.15.100:0/3999317032 >> 172.21.15.100:0/3999317032 conn(0x7f01fc0cee80 msgr2=0x7f01fc161280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:10.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.392+0000 7f0203000640 1 -- 172.21.15.100:0/3999317032 shutdown_connections 2024-04-23T14:34:10.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.392+0000 7f0203000640 1 -- 172.21.15.100:0/3999317032 wait complete. 2024-04-23T14:34:10.414 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:34:10.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.544+0000 7f7a7cc00640 1 Processor -- start 2024-04-23T14:34:10.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.544+0000 7f7a7cc00640 1 -- start start 2024-04-23T14:34:10.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.544+0000 7f7a7cc00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7a7815c470 0x7f7a780b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:10.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.544+0000 7f7a7cc00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7a780ad870 0x7f7a7806bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:10.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.544+0000 7f7a7cc00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a78160320 con 0x7f7a78160de0 2024-04-23T14:34:10.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.544+0000 7f7a7cc00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f7a780cc7e0 con 0x7f7a7815c470 2024-04-23T14:34:10.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.544+0000 7f7a7cc00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7a781605e0 con 0x7f7a780ad870 2024-04-23T14:34:10.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.544+0000 7f7a75e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7a7815c470 0x7f7a780b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:10.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.544+0000 7f7a75e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7a7815c470 0x7f7a780b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35898/0 (socket says 172.21.15.100:35898) 2024-04-23T14:34:10.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.544+0000 7f7a75e00640 1 -- 172.21.15.100:0/789160522 learned_addr learned my addr 172.21.15.100:0/789160522 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:10.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.544+0000 7f7a6fe00640 1 --1- 172.21.15.100:0/789160522 >> v1:172.21.15.100:6789/0 conn(0x7f7a78160de0 0x7f7a780ad160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46538/0 (socket says 172.21.15.100:46538) 2024-04-23T14:34:10.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.544+0000 7f7a76800640 1 --2- 172.21.15.100:0/789160522 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7a780ad870 0x7f7a7806bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:10.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.548+0000 7f7a75e00640 1 -- 172.21.15.100:0/789160522 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7a780ad870 msgr2=0x7f7a7806bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:10.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.548+0000 7f7a75e00640 1 --2- 172.21.15.100:0/789160522 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7a780ad870 0x7f7a7806bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.548+0000 7f7a75e00640 1 -- 172.21.15.100:0/789160522 >> v1:172.21.15.100:6789/0 conn(0x7f7a78160de0 legacy=0x7f7a780ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:10.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.548+0000 7f7a75e00640 1 -- 172.21.15.100:0/789160522 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a781608c0 con 0x7f7a7815c470 2024-04-23T14:34:10.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.548+0000 7f7a75400640 1 -- 172.21.15.100:0/789160522 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3160099369 0 0) 0x7f7a78160320 con 0x7f7a78160de0 2024-04-23T14:34:10.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.548+0000 7f7a75e00640 1 --2- 172.21.15.100:0/789160522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7815c470 0x7f7a780b29a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009870 tx=0x7f7a6801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc495bfbf749592b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:10.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.548+0000 7f7a75400640 1 -- 172.21.15.100:0/789160522 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a680b1070 con 0x7f7a7815c470 2024-04-23T14:34:10.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.548+0000 7f7a75400640 1 -- 172.21.15.100:0/789160522 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a680a8050 con 0x7f7a7815c470 2024-04-23T14:34:10.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.548+0000 7f7a75400640 1 -- 172.21.15.100:0/789160522 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a680ac860 con 0x7f7a7815c470 2024-04-23T14:34:10.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.548+0000 7f7a7cc00640 1 -- 172.21.15.100:0/789160522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7815c470 msgr2=0x7f7a780b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:10.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.548+0000 7f7a7cc00640 1 --2- 172.21.15.100:0/789160522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7815c470 0x7f7a780b29a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009870 tx=0x7f7a6801c4d0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a7cc00640 1 -- 172.21.15.100:0/789160522 shutdown_connections 2024-04-23T14:34:10.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a7cc00640 1 --2- 172.21.15.100:0/789160522 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7a780ad870 0x7f7a7806bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a7cc00640 1 --2- 172.21.15.100:0/789160522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7815c470 0x7f7a780b29a0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a7cc00640 1 -- 172.21.15.100:0/789160522 >> 172.21.15.100:0/789160522 conn(0x7f7a780b1e80 msgr2=0x7f7a780b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:10.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a7cc00640 1 -- 172.21.15.100:0/789160522 shutdown_connections 2024-04-23T14:34:10.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a7cc00640 1 -- 172.21.15.100:0/789160522 wait complete. 2024-04-23T14:34:10.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a7cc00640 1 Processor -- start 2024-04-23T14:34:10.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a7cc00640 1 -- start start 2024-04-23T14:34:10.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a7cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a780ad870 0x7f7a78143910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:10.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a7cc00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7a7815c470 0x7f7a78097030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:10.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a7cc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a78160de0 0x7f7a78097570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:10.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a7cc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7815ca60 con 0x7f7a780ad870 2024-04-23T14:34:10.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a7cc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7a780cc580 con 0x7f7a78160de0 2024-04-23T14:34:10.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a7cc00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7a78169540 con 0x7f7a7815c470 2024-04-23T14:34:10.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a75e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a780ad870 0x7f7a78143910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:10.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a75e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a780ad870 0x7f7a78143910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35906/0 (socket says 172.21.15.100:35906) 2024-04-23T14:34:10.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a75e00640 1 -- 172.21.15.100:0/4200002724 learned_addr learned my addr 172.21.15.100:0/4200002724 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:10.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a6fe00640 1 --2- 172.21.15.100:0/4200002724 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7a7815c470 0x7f7a78097030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:10.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a76800640 1 --2- 172.21.15.100:0/4200002724 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a78160de0 0x7f7a78097570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:10.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a75e00640 1 -- 172.21.15.100:0/4200002724 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7a7815c470 msgr2=0x7f7a78097030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:10.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a75e00640 1 --2- 172.21.15.100:0/4200002724 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7a7815c470 0x7f7a78097030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a75e00640 1 -- 172.21.15.100:0/4200002724 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a78160de0 msgr2=0x7f7a78097570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:10.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a75e00640 1 --2- 172.21.15.100:0/4200002724 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a78160de0 0x7f7a78097570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a75e00640 1 -- 172.21.15.100:0/4200002724 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a78155a10 con 0x7f7a780ad870 2024-04-23T14:34:10.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.552+0000 7f7a75e00640 1 --2- 172.21.15.100:0/4200002724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a780ad870 0x7f7a78143910 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7a680af040 tx=0x7f7a68019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:10.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.556+0000 7f7a6e000640 1 -- 172.21.15.100:0/4200002724 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a680bc070 con 0x7f7a780ad870 2024-04-23T14:34:10.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.556+0000 7f7a6e000640 1 -- 172.21.15.100:0/4200002724 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a680acd70 con 0x7f7a780ad870 2024-04-23T14:34:10.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.556+0000 7f7a6e000640 1 -- 172.21.15.100:0/4200002724 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a680b1040 con 0x7f7a780ad870 2024-04-23T14:34:10.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.556+0000 7f7a7cc00640 1 -- 172.21.15.100:0/4200002724 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a781555f0 con 0x7f7a780ad870 2024-04-23T14:34:10.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.556+0000 7f7a7cc00640 1 -- 172.21.15.100:0/4200002724 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a780ce360 con 0x7f7a780ad870 2024-04-23T14:34:10.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.556+0000 7f7a7cc00640 1 -- 172.21.15.100:0/4200002724 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a78068ad0 con 0x7f7a780ad870 2024-04-23T14:34:10.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.556+0000 7f7a6e000640 1 -- 172.21.15.100:0/4200002724 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7a680ab3e0 con 0x7f7a780ad870 2024-04-23T14:34:10.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.556+0000 7f7a6e000640 1 --2- 172.21.15.100:0/4200002724 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7a540683b0 0x7f7a5406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:10.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.556+0000 7f7a6e000640 1 -- 172.21.15.100:0/4200002724 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f7a68120370 con 0x7f7a780ad870 2024-04-23T14:34:10.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.556+0000 7f7a6fe00640 1 --2- 172.21.15.100:0/4200002724 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7a540683b0 0x7f7a5406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:10.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.556+0000 7f7a6fe00640 1 --2- 172.21.15.100:0/4200002724 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7a540683b0 0x7f7a5406a870 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7a78151e30 tx=0x7f7a60007c30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:10.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.560+0000 7f7a6e000640 1 -- 172.21.15.100:0/4200002724 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7a68016150 con 0x7f7a780ad870 2024-04-23T14:34:10.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.756+0000 7f7a7cc00640 1 -- 172.21.15.100:0/4200002724 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a780b5460 con 0x7f7a780ad870 2024-04-23T14:34:10.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.756+0000 7f7a6e000640 1 -- 172.21.15.100:0/4200002724 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5195 (secure 0 0 0) 0x7f7a680f0070 con 0x7f7a780ad870 2024-04-23T14:34:10.761 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:10.761 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2601071793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2601071793},{"type":"v1","addr":"172.21.15.100:6837","nonce":2601071793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2930424296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2930424296},{"type":"v1","addr":"172.21.15.151:6833","nonce":2930424296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1406976216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1406976216},{"type":"v1","addr":"172.21.15.151:6835","nonce":1406976216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:33:38.928245+0000","modified":"2024-04-23T14:33:47.983305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5466":{"gid":5466,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/2062210445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2062210445},{"type":"v1","addr":"172.21.15.100:6836","nonce":2062210445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5466,"qdb_cluster":[5466]},"id":2}]} 2024-04-23T14:34:10.762 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 19 2024-04-23T14:34:10.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.760+0000 7f7a7cc00640 1 -- 172.21.15.100:0/4200002724 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7a540683b0 msgr2=0x7f7a5406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:10.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.760+0000 7f7a7cc00640 1 --2- 172.21.15.100:0/4200002724 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7a540683b0 0x7f7a5406a870 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7a78151e30 tx=0x7f7a60007c30 comp rx=0 tx=0).stop 2024-04-23T14:34:10.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.760+0000 7f7a7cc00640 1 -- 172.21.15.100:0/4200002724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a780ad870 msgr2=0x7f7a78143910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.760+0000 7f7a7cc00640 1 --2- 172.21.15.100:0/4200002724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a780ad870 0x7f7a78143910 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7a680af040 tx=0x7f7a68019270 comp rx=0 tx=0).stop 2024-04-23T14:34:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.760+0000 7f7a7cc00640 1 -- 172.21.15.100:0/4200002724 shutdown_connections 2024-04-23T14:34:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.760+0000 7f7a7cc00640 1 --2- 172.21.15.100:0/4200002724 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7a540683b0 0x7f7a5406a870 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.760+0000 7f7a7cc00640 1 --2- 172.21.15.100:0/4200002724 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a78160de0 0x7f7a78097570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.760+0000 7f7a7cc00640 1 --2- 172.21.15.100:0/4200002724 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7a7815c470 0x7f7a78097030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.760+0000 7f7a7cc00640 1 --2- 172.21.15.100:0/4200002724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a780ad870 0x7f7a78143910 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.760+0000 7f7a7cc00640 1 -- 172.21.15.100:0/4200002724 >> 172.21.15.100:0/4200002724 conn(0x7f7a780b1e80 msgr2=0x7f7a78143070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.760+0000 7f7a7cc00640 1 -- 172.21.15.100:0/4200002724 shutdown_connections 2024-04-23T14:34:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.760+0000 7f7a7cc00640 1 -- 172.21.15.100:0/4200002724 wait complete. 2024-04-23T14:34:10.783 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:10.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.912+0000 7f8e02a00640 1 Processor -- start 2024-04-23T14:34:10.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.912+0000 7f8e02a00640 1 -- start start 2024-04-23T14:34:10.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8e02a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8dfc15e830 0x7f8dfc15ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:10.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8e02a00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dfc15f320 con 0x7f8dfc0d00b0 2024-04-23T14:34:10.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8e02a00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dfc15f5e0 con 0x7f8dfc0cdd60 2024-04-23T14:34:10.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8e02a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8dfc15f8c0 con 0x7f8dfc15e830 2024-04-23T14:34:10.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8e00a00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f8dfc0d00b0 0x7f8dfc0b45c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46550/0 (socket says 172.21.15.100:46550) 2024-04-23T14:34:10.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8e00a00640 1 -- 172.21.15.100:0/2386446646 learned_addr learned my addr 172.21.15.100:0/2386446646 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8dfbe00640 1 --2- 172.21.15.100:0/2386446646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8dfc15e830 0x7f8dfc15ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8dfaa00640 1 -- 172.21.15.100:0/2386446646 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 9335767 0 0) 0x7f8dfc15f320 con 0x7f8dfc0d00b0 2024-04-23T14:34:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8dfaa00640 1 -- 172.21.15.100:0/2386446646 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8de0003530 con 0x7f8dfc0d00b0 2024-04-23T14:34:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8dfaa00640 1 -- 172.21.15.100:0/2386446646 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1738029533 0 0) 0x7f8de0003530 con 0x7f8dfc0d00b0 2024-04-23T14:34:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8dfaa00640 1 -- 172.21.15.100:0/2386446646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8dfc15e830 msgr2=0x7f8dfc15ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8dfaa00640 1 --2- 172.21.15.100:0/2386446646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8dfc15e830 0x7f8dfc15ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8dfaa00640 1 -- 172.21.15.100:0/2386446646 >> v1:172.21.15.151:6789/0 conn(0x7f8dfc0cdd60 legacy=0x7f8dfc0cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8dfaa00640 1 -- 172.21.15.100:0/2386446646 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dfc15fba0 con 0x7f8dfc0d00b0 2024-04-23T14:34:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8dfaa00640 1 -- 172.21.15.100:0/2386446646 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f8df4001a00 con 0x7f8dfc0d00b0 2024-04-23T14:34:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8dfaa00640 1 -- 172.21.15.100:0/2386446646 >> v1:172.21.15.100:6789/0 conn(0x7f8dfc0d00b0 legacy=0x7f8dfc0b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8dfaa00640 1 --2- 172.21.15.100:0/2386446646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8de00039f0 0x7f8de0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8dfaa00640 1 --2- 172.21.15.100:0/2386446646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8dfc15e830 0x7f8de0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:10.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8dfaa00640 1 --2- 172.21.15.100:0/2386446646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8de0004d50 0x7f8de00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:10.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8dfaa00640 1 -- 172.21.15.100:0/2386446646 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc15f8c0 con 0x7f8de0004d50 2024-04-23T14:34:10.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8dfaa00640 1 -- 172.21.15.100:0/2386446646 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8de000ba80 con 0x7f8de00039f0 2024-04-23T14:34:10.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8dfaa00640 1 -- 172.21.15.100:0/2386446646 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8de000bc80 con 0x7f8dfc15e830 2024-04-23T14:34:10.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8e00a00640 1 --2- 172.21.15.100:0/2386446646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8de0004d50 0x7f8de00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:10.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8dfbe00640 1 --2- 172.21.15.100:0/2386446646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8de00039f0 0x7f8de0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:10.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8dfb400640 1 --2- 172.21.15.100:0/2386446646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8dfc15e830 0x7f8de0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:10.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8e00a00640 1 -- 172.21.15.100:0/2386446646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8dfc15e830 msgr2=0x7f8de0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:10.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8e00a00640 1 --2- 172.21.15.100:0/2386446646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8dfc15e830 0x7f8de0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8e00a00640 1 -- 172.21.15.100:0/2386446646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8de00039f0 msgr2=0x7f8de0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:10.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8e00a00640 1 --2- 172.21.15.100:0/2386446646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8de00039f0 0x7f8de0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.916+0000 7f8e00a00640 1 -- 172.21.15.100:0/2386446646 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de0003530 con 0x7f8de0004d50 2024-04-23T14:34:10.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.920+0000 7f8e00a00640 1 --2- 172.21.15.100:0/2386446646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8de0004d50 0x7f8de00051d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc0b6950 tx=0x7f8df401d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8941cc3058bda12 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:10.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.920+0000 7f8dfaa00640 1 -- 172.21.15.100:0/2386446646 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df400a480 con 0x7f8de0004d50 2024-04-23T14:34:10.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.920+0000 7f8dfaa00640 1 -- 172.21.15.100:0/2386446646 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df40ad630 con 0x7f8de0004d50 2024-04-23T14:34:10.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.920+0000 7f8dfaa00640 1 -- 172.21.15.100:0/2386446646 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df40ada40 con 0x7f8de0004d50 2024-04-23T14:34:10.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.920+0000 7f8e02a00640 1 -- 172.21.15.100:0/2386446646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8de0004d50 msgr2=0x7f8de00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:10.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.920+0000 7f8e02a00640 1 --2- 172.21.15.100:0/2386446646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8de0004d50 0x7f8de00051d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc0b6950 tx=0x7f8df401d320 comp rx=0 tx=0).stop 2024-04-23T14:34:10.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.920+0000 7f8e00a00640 1 -- 172.21.15.100:0/2386446646 reap_dead start 2024-04-23T14:34:10.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.920+0000 7f8e02a00640 1 -- 172.21.15.100:0/2386446646 shutdown_connections 2024-04-23T14:34:10.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.920+0000 7f8e02a00640 1 -- 172.21.15.100:0/2386446646 >> 172.21.15.100:0/2386446646 conn(0x7f8dfc0704d0 msgr2=0x7f8dfc0708e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:10.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.920+0000 7f8e02a00640 1 -- 172.21.15.100:0/2386446646 shutdown_connections 2024-04-23T14:34:10.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.920+0000 7f8e02a00640 1 -- 172.21.15.100:0/2386446646 wait complete. 2024-04-23T14:34:10.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.920+0000 7f8e02a00640 1 Processor -- start 2024-04-23T14:34:10.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.920+0000 7f8e02a00640 1 -- start start 2024-04-23T14:34:10.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8e02a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8de0004d50 0x7f8dfc14bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:10.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8e02a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8dfc06df40 0x7f8dfc06cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:10.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8e02a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8dfc06e380 0x7f8dfc06d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:10.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8e02a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc15ec10 con 0x7f8de0004d50 2024-04-23T14:34:10.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8e02a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc0ced60 con 0x7f8dfc06df40 2024-04-23T14:34:10.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8e02a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8dfc15ed90 con 0x7f8dfc06e380 2024-04-23T14:34:10.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8dfbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8de0004d50 0x7f8dfc14bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:10.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8dfbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8de0004d50 0x7f8dfc14bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35934/0 (socket says 172.21.15.100:35934) 2024-04-23T14:34:10.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8dfbe00640 1 -- 172.21.15.100:0/3356193003 learned_addr learned my addr 172.21.15.100:0/3356193003 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:10.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8dfb400640 1 --2- 172.21.15.100:0/3356193003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8dfc06df40 0x7f8dfc06cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:10.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8e00a00640 1 --2- 172.21.15.100:0/3356193003 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8dfc06e380 0x7f8dfc06d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:10.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8dfbe00640 1 -- 172.21.15.100:0/3356193003 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8dfc06e380 msgr2=0x7f8dfc06d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:10.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8dfbe00640 1 --2- 172.21.15.100:0/3356193003 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8dfc06e380 0x7f8dfc06d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8dfbe00640 1 -- 172.21.15.100:0/3356193003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8dfc06df40 msgr2=0x7f8dfc06cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:10.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8dfbe00640 1 --2- 172.21.15.100:0/3356193003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8dfc06df40 0x7f8dfc06cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:10.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8dfbe00640 1 -- 172.21.15.100:0/3356193003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dfc14c1f0 con 0x7f8de0004d50 2024-04-23T14:34:10.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8dfbe00640 1 --2- 172.21.15.100:0/3356193003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8de0004d50 0x7f8dfc14bcb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8dec0119d0 tx=0x7f8dec011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:10.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8df3e00640 1 -- 172.21.15.100:0/3356193003 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec00fb50 con 0x7f8de0004d50 2024-04-23T14:34:10.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8df3e00640 1 -- 172.21.15.100:0/3356193003 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dec00fcf0 con 0x7f8de0004d50 2024-04-23T14:34:10.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8df3e00640 1 -- 172.21.15.100:0/3356193003 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec00fea0 con 0x7f8de0004d50 2024-04-23T14:34:10.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8e02a00640 1 -- 172.21.15.100:0/3356193003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dfc06d7d0 con 0x7f8de0004d50 2024-04-23T14:34:10.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8e02a00640 1 -- 172.21.15.100:0/3356193003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dfc160630 con 0x7f8de0004d50 2024-04-23T14:34:10.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.924+0000 7f8e02a00640 1 -- 172.21.15.100:0/3356193003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dfc068ad0 con 0x7f8de0004d50 2024-04-23T14:34:10.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.928+0000 7f8df3e00640 1 -- 172.21.15.100:0/3356193003 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8dec012040 con 0x7f8de0004d50 2024-04-23T14:34:10.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.928+0000 7f8df3e00640 1 --2- 172.21.15.100:0/3356193003 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8ddc0683b0 0x7f8ddc06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:10.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.928+0000 7f8df3e00640 1 -- 172.21.15.100:0/3356193003 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f8dec0871f0 con 0x7f8de0004d50 2024-04-23T14:34:10.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.928+0000 7f8dfb400640 1 --2- 172.21.15.100:0/3356193003 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8ddc0683b0 0x7f8ddc06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:10.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.928+0000 7f8dfb400640 1 --2- 172.21.15.100:0/3356193003 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8ddc0683b0 0x7f8ddc06a870 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8de8000c10 tx=0x7f8de8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:10.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:10.932+0000 7f8df3e00640 1 -- 172.21.15.100:0/3356193003 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8dec019030 con 0x7f8de0004d50 2024-04-23T14:34:11.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.088+0000 7f8e02a00640 1 -- 172.21.15.100:0/3356193003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8dfc0d0a00 con 0x7f8de0004d50 2024-04-23T14:34:11.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.088+0000 7f8df3e00640 1 -- 172.21.15.100:0/3356193003 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+18766 (secure 0 0 0) 0x7f8dec056f20 con 0x7f8de0004d50 2024-04-23T14:34:11.094 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:11.094 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":41,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:33:49.609841+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:33:36.592437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-23T14:33:37.882390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-04-23T14:33:47.299378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:11.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.092+0000 7f8e02a00640 1 -- 172.21.15.100:0/3356193003 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8ddc0683b0 msgr2=0x7f8ddc06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:11.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.092+0000 7f8e02a00640 1 --2- 172.21.15.100:0/3356193003 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8ddc0683b0 0x7f8ddc06a870 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8de8000c10 tx=0x7f8de8009040 comp rx=0 tx=0).stop 2024-04-23T14:34:11.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.092+0000 7f8e02a00640 1 -- 172.21.15.100:0/3356193003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8de0004d50 msgr2=0x7f8dfc14bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:11.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.092+0000 7f8e02a00640 1 --2- 172.21.15.100:0/3356193003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8de0004d50 0x7f8dfc14bcb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8dec0119d0 tx=0x7f8dec011e90 comp rx=0 tx=0).stop 2024-04-23T14:34:11.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.092+0000 7f8e02a00640 1 -- 172.21.15.100:0/3356193003 shutdown_connections 2024-04-23T14:34:11.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.092+0000 7f8e02a00640 1 --2- 172.21.15.100:0/3356193003 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8ddc0683b0 0x7f8ddc06a870 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:11.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.092+0000 7f8e02a00640 1 --2- 172.21.15.100:0/3356193003 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8dfc06e380 0x7f8dfc06d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:11.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.092+0000 7f8e02a00640 1 --2- 172.21.15.100:0/3356193003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8dfc06df40 0x7f8dfc06cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:11.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.092+0000 7f8e02a00640 1 --2- 172.21.15.100:0/3356193003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8de0004d50 0x7f8dfc14bcb0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:11.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.092+0000 7f8e02a00640 1 -- 172.21.15.100:0/3356193003 >> 172.21.15.100:0/3356193003 conn(0x7f8dfc0704d0 msgr2=0x7f8dfc0ae050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:11.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.092+0000 7f8e02a00640 1 -- 172.21.15.100:0/3356193003 shutdown_connections 2024-04-23T14:34:11.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.092+0000 7f8e02a00640 1 -- 172.21.15.100:0/3356193003 wait complete. 2024-04-23T14:34:11.115 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T14:34:11.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd52600640 1 Processor -- start 2024-04-23T14:34:11.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd52600640 1 -- start start 2024-04-23T14:34:11.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd52600640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fdd4c072430 0x7fdd4c0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:11.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd52600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd4c0cdfc0 con 0x7fdd4c15ae60 2024-04-23T14:34:11.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd52600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd4c0cfdf0 con 0x7fdd4c0c6520 2024-04-23T14:34:11.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd52600640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fdd4c0d00d0 con 0x7fdd4c072430 2024-04-23T14:34:11.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4b400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fdd4c15ae60 0x7fdd4c06c880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46556/0 (socket says 172.21.15.100:46556) 2024-04-23T14:34:11.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4b400640 1 -- 172.21.15.100:0/2564657149 learned_addr learned my addr 172.21.15.100:0/2564657149 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:11.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4be00640 1 --2- 172.21.15.100:0/2564657149 >> v2:172.21.15.151:3300/0 conn(0x7fdd4c072430 0x7fdd4c0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:11.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4aa00640 1 -- 172.21.15.100:0/2564657149 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 173350923 0 0) 0x7fdd4c0cdfc0 con 0x7fdd4c15ae60 2024-04-23T14:34:11.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4aa00640 1 -- 172.21.15.100:0/2564657149 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd30003530 con 0x7fdd4c15ae60 2024-04-23T14:34:11.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4aa00640 1 -- 172.21.15.100:0/2564657149 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 902029914 0 0) 0x7fdd30003530 con 0x7fdd4c15ae60 2024-04-23T14:34:11.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4aa00640 1 -- 172.21.15.100:0/2564657149 >> v2:172.21.15.151:3300/0 conn(0x7fdd4c072430 msgr2=0x7fdd4c0ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:11.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4aa00640 1 --2- 172.21.15.100:0/2564657149 >> v2:172.21.15.151:3300/0 conn(0x7fdd4c072430 0x7fdd4c0ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:11.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4aa00640 1 -- 172.21.15.100:0/2564657149 >> v1:172.21.15.151:6789/0 conn(0x7fdd4c0c6520 legacy=0x7fdd4c071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:11.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4aa00640 1 -- 172.21.15.100:0/2564657149 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd4c0ce260 con 0x7fdd4c15ae60 2024-04-23T14:34:11.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4aa00640 1 -- 172.21.15.100:0/2564657149 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7fdd3c001680 con 0x7fdd4c15ae60 2024-04-23T14:34:11.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4be00640 1 --2- 172.21.15.100:0/2564657149 >> v2:172.21.15.151:3300/0 conn(0x7fdd4c072430 0x7fdd4c0ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:34:11.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4aa00640 1 -- 172.21.15.100:0/2564657149 >> v1:172.21.15.100:6789/0 conn(0x7fdd4c15ae60 legacy=0x7fdd4c06c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:11.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4aa00640 1 --2- 172.21.15.100:0/2564657149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd300039c0 0x7fdd30003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:11.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4aa00640 1 --2- 172.21.15.100:0/2564657149 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd30004810 0x7fdd30004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:11.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4aa00640 1 --2- 172.21.15.100:0/2564657149 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdd300051d0 0x7fdd3000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:11.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4aa00640 1 -- 172.21.15.100:0/2564657149 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c0d00d0 con 0x7fdd300039c0 2024-04-23T14:34:11.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4aa00640 1 -- 172.21.15.100:0/2564657149 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3000c070 con 0x7fdd30004810 2024-04-23T14:34:11.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4aa00640 1 -- 172.21.15.100:0/2564657149 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fdd3000c270 con 0x7fdd300051d0 2024-04-23T14:34:11.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4b400640 1 --2- 172.21.15.100:0/2564657149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd300039c0 0x7fdd30003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:11.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd43e00640 1 --2- 172.21.15.100:0/2564657149 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd30004810 0x7fdd30004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:11.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4be00640 1 --2- 172.21.15.100:0/2564657149 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdd300051d0 0x7fdd3000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:11.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4b400640 1 -- 172.21.15.100:0/2564657149 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdd300051d0 msgr2=0x7fdd3000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:11.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4b400640 1 --2- 172.21.15.100:0/2564657149 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdd300051d0 0x7fdd3000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:11.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4b400640 1 -- 172.21.15.100:0/2564657149 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd30004810 msgr2=0x7fdd30004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:11.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4b400640 1 --2- 172.21.15.100:0/2564657149 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd30004810 0x7fdd30004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:11.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4b400640 1 -- 172.21.15.100:0/2564657149 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd30003530 con 0x7fdd300039c0 2024-04-23T14:34:11.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4be00640 1 --2- 172.21.15.100:0/2564657149 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdd300051d0 0x7fdd3000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:11.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.248+0000 7fdd4be00640 1 -- 172.21.15.100:0/2564657149 reap_dead start 2024-04-23T14:34:11.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.252+0000 7fdd4b400640 1 --2- 172.21.15.100:0/2564657149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd300039c0 0x7fdd30003de0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00c1c0 tx=0x7fdd3c01cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b40ea2518a20f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:11.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.252+0000 7fdd4aa00640 1 -- 172.21.15.100:0/2564657149 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c007d00 con 0x7fdd300039c0 2024-04-23T14:34:11.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.252+0000 7fdd4aa00640 1 -- 172.21.15.100:0/2564657149 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd3c0a9020 con 0x7fdd300039c0 2024-04-23T14:34:11.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.252+0000 7fdd4aa00640 1 -- 172.21.15.100:0/2564657149 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c005ae0 con 0x7fdd300039c0 2024-04-23T14:34:11.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.252+0000 7fdd52600640 1 -- 172.21.15.100:0/2564657149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd300039c0 msgr2=0x7fdd30003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:11.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.252+0000 7fdd52600640 1 --2- 172.21.15.100:0/2564657149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd300039c0 0x7fdd30003de0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00c1c0 tx=0x7fdd3c01cbe0 comp rx=0 tx=0).stop 2024-04-23T14:34:11.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.252+0000 7fdd52600640 1 -- 172.21.15.100:0/2564657149 shutdown_connections 2024-04-23T14:34:11.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.252+0000 7fdd52600640 1 --2- 172.21.15.100:0/2564657149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd300039c0 0x7fdd30003de0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:11.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.252+0000 7fdd52600640 1 -- 172.21.15.100:0/2564657149 >> 172.21.15.100:0/2564657149 conn(0x7fdd4c0b5310 msgr2=0x7fdd4c0b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:11.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.252+0000 7fdd52600640 1 -- 172.21.15.100:0/2564657149 shutdown_connections 2024-04-23T14:34:11.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.252+0000 7fdd52600640 1 -- 172.21.15.100:0/2564657149 wait complete. 2024-04-23T14:34:11.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd52600640 1 Processor -- start 2024-04-23T14:34:11.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd52600640 1 -- start start 2024-04-23T14:34:11.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd52600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd300039c0 0x7fdd4c147650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:11.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd52600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdd4c15b350 0x7fdd4c15cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:11.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd52600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd4c15b7d0 0x7fdd4c15d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:11.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd52600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c06cc10 con 0x7fdd300039c0 2024-04-23T14:34:11.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd52600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c06ca90 con 0x7fdd4c15b7d0 2024-04-23T14:34:11.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd52600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fdd4c06cd90 con 0x7fdd4c15b350 2024-04-23T14:34:11.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd4b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd300039c0 0x7fdd4c147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:11.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd4b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd300039c0 0x7fdd4c147650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35942/0 (socket says 172.21.15.100:35942) 2024-04-23T14:34:11.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd4b400640 1 -- 172.21.15.100:0/2315592523 learned_addr learned my addr 172.21.15.100:0/2315592523 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:11.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd43e00640 1 --2- 172.21.15.100:0/2315592523 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdd4c15b350 0x7fdd4c15cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:11.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd4be00640 1 --2- 172.21.15.100:0/2315592523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd4c15b7d0 0x7fdd4c15d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:11.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd4b400640 1 -- 172.21.15.100:0/2315592523 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdd4c15b350 msgr2=0x7fdd4c15cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:11.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd4b400640 1 --2- 172.21.15.100:0/2315592523 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdd4c15b350 0x7fdd4c15cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:11.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd4b400640 1 -- 172.21.15.100:0/2315592523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd4c15b7d0 msgr2=0x7fdd4c15d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:11.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd4b400640 1 --2- 172.21.15.100:0/2315592523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd4c15b7d0 0x7fdd4c15d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:11.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd4b400640 1 -- 172.21.15.100:0/2315592523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd4c147b90 con 0x7fdd300039c0 2024-04-23T14:34:11.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd4b400640 1 --2- 172.21.15.100:0/2315592523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd300039c0 0x7fdd4c147650 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0ab040 tx=0x7fdd30005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:11.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd43400640 1 -- 172.21.15.100:0/2315592523 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c0b4070 con 0x7fdd300039c0 2024-04-23T14:34:11.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd43400640 1 -- 172.21.15.100:0/2315592523 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd3c000ee0 con 0x7fdd300039c0 2024-04-23T14:34:11.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd43400640 1 -- 172.21.15.100:0/2315592523 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c007d00 con 0x7fdd300039c0 2024-04-23T14:34:11.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd52600640 1 -- 172.21.15.100:0/2315592523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd4c15deb0 con 0x7fdd300039c0 2024-04-23T14:34:11.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.256+0000 7fdd52600640 1 -- 172.21.15.100:0/2315592523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd4c0b00c0 con 0x7fdd300039c0 2024-04-23T14:34:11.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.260+0000 7fdd52600640 1 -- 172.21.15.100:0/2315592523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd4c068ad0 con 0x7fdd300039c0 2024-04-23T14:34:11.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.260+0000 7fdd43400640 1 -- 172.21.15.100:0/2315592523 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdd3c0bf060 con 0x7fdd300039c0 2024-04-23T14:34:11.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.260+0000 7fdd43400640 1 --2- 172.21.15.100:0/2315592523 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdd280683b0 0x7fdd2806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:11.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.260+0000 7fdd43400640 1 -- 172.21.15.100:0/2315592523 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fdd3c11fe90 con 0x7fdd300039c0 2024-04-23T14:34:11.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.260+0000 7fdd43e00640 1 --2- 172.21.15.100:0/2315592523 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdd280683b0 0x7fdd2806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:11.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.260+0000 7fdd43e00640 1 --2- 172.21.15.100:0/2315592523 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdd280683b0 0x7fdd2806a870 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdd34000c10 tx=0x7fdd34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:11.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.264+0000 7fdd43400640 1 -- 172.21.15.100:0/2315592523 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdd3c0168f0 con 0x7fdd300039c0 2024-04-23T14:34:11.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.455+0000 7fdd52600640 1 -- 172.21.15.100:0/2315592523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdd4c06c340 con 0x7fdd300039c0 2024-04-23T14:34:11.462 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-04-23T14:34:11.455+0000 7f7a7e400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T14:34:11.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.459+0000 7fdd43400640 1 -- 172.21.15.100:0/2315592523 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v20) ==== 161+0+0 (secure 0 0 0) 0x7fdd3c0f00c0 con 0x7fdd300039c0 2024-04-23T14:34:11.464 INFO:teuthology.orchestra.run.smithi100.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T14:34:11.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.463+0000 7fdd52600640 1 -- 172.21.15.100:0/2315592523 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdd280683b0 msgr2=0x7fdd2806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:11.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.463+0000 7fdd52600640 1 --2- 172.21.15.100:0/2315592523 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdd280683b0 0x7fdd2806a870 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdd34000c10 tx=0x7fdd34009040 comp rx=0 tx=0).stop 2024-04-23T14:34:11.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.463+0000 7fdd52600640 1 -- 172.21.15.100:0/2315592523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd300039c0 msgr2=0x7fdd4c147650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:11.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.463+0000 7fdd52600640 1 --2- 172.21.15.100:0/2315592523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd300039c0 0x7fdd4c147650 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0ab040 tx=0x7fdd30005840 comp rx=0 tx=0).stop 2024-04-23T14:34:11.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.463+0000 7fdd52600640 1 -- 172.21.15.100:0/2315592523 shutdown_connections 2024-04-23T14:34:11.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.463+0000 7fdd52600640 1 --2- 172.21.15.100:0/2315592523 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdd280683b0 0x7fdd2806a870 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:11.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.463+0000 7fdd52600640 1 --2- 172.21.15.100:0/2315592523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd4c15b7d0 0x7fdd4c15d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:11.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.463+0000 7fdd52600640 1 --2- 172.21.15.100:0/2315592523 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdd4c15b350 0x7fdd4c15cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:11.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.463+0000 7fdd52600640 1 --2- 172.21.15.100:0/2315592523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd300039c0 0x7fdd4c147650 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:11.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.463+0000 7fdd52600640 1 -- 172.21.15.100:0/2315592523 >> 172.21.15.100:0/2315592523 conn(0x7fdd4c0b5310 msgr2=0x7fdd4c0b56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:11.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.463+0000 7fdd52600640 1 -- 172.21.15.100:0/2315592523 shutdown_connections 2024-04-23T14:34:11.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.463+0000 7fdd52600640 1 -- 172.21.15.100:0/2315592523 wait complete. 2024-04-23T14:34:11.474 INFO:tasks.ceph.mds.c.smithi100.stderr: -3597> 2024-04-23T14:33:42.500+0000 7f3075600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:34:11.490 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-23T14:34:11.527 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-04-23T14:34:11.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f506ae00640 1 Processor -- start 2024-04-23T14:34:11.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f506ae00640 1 -- start start 2024-04-23T14:34:11.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f506ae00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f50640cc960 0x7f50640ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:11.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f506ae00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f5064160080 0x7f5064160510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:11.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f506ae00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50640b1e00 con 0x7f5064156a00 2024-04-23T14:34:11.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f506ae00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f50640b2a10 con 0x7f5064160080 2024-04-23T14:34:11.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f506ae00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f50640b20e0 con 0x7f50640cc960 2024-04-23T14:34:11.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f5069a00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f5064156a00 0x7f506415d370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46574/0 (socket says 172.21.15.100:46574) 2024-04-23T14:34:11.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f5069a00640 1 -- 172.21.15.100:0/4083816017 learned_addr learned my addr 172.21.15.100:0/4083816017 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:11.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f506a400640 1 --2- 172.21.15.100:0/4083816017 >> v2:172.21.15.151:3300/0 conn(0x7f5064160080 0x7f5064160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:11.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f5069000640 1 --2- 172.21.15.100:0/4083816017 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f50640cc960 0x7f50640ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:11.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f5063e00640 1 -- 172.21.15.100:0/4083816017 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 954702700 0 0) 0x7f50640b1e00 con 0x7f5064156a00 2024-04-23T14:34:11.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f5063e00640 1 -- 172.21.15.100:0/4083816017 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f504c003530 con 0x7f5064156a00 2024-04-23T14:34:11.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f5063e00640 1 -- 172.21.15.100:0/4083816017 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3133946886 0 0) 0x7f504c003530 con 0x7f5064156a00 2024-04-23T14:34:11.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f5063e00640 1 -- 172.21.15.100:0/4083816017 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f50640cc960 msgr2=0x7f50640ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:11.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f5063e00640 1 --2- 172.21.15.100:0/4083816017 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f50640cc960 0x7f50640ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:11.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f5063e00640 1 -- 172.21.15.100:0/4083816017 >> v2:172.21.15.151:3300/0 conn(0x7f5064160080 msgr2=0x7f5064160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:11.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f5063e00640 1 --2- 172.21.15.100:0/4083816017 >> v2:172.21.15.151:3300/0 conn(0x7f5064160080 0x7f5064160510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:11.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f5063e00640 1 -- 172.21.15.100:0/4083816017 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50640b23c0 con 0x7f5064156a00 2024-04-23T14:34:11.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f5063e00640 1 -- 172.21.15.100:0/4083816017 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f5058001680 con 0x7f5064156a00 2024-04-23T14:34:11.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f5063e00640 1 -- 172.21.15.100:0/4083816017 >> v1:172.21.15.100:6789/0 conn(0x7f5064156a00 legacy=0x7f506415d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:11.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f5063e00640 1 --2- 172.21.15.100:0/4083816017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f504c0039c0 0x7f504c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:11.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f5063e00640 1 --2- 172.21.15.100:0/4083816017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f504c004810 0x7f504c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:11.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f5063e00640 1 --2- 172.21.15.100:0/4083816017 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f50640cc960 0x7f504c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:11.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f5063e00640 1 -- 172.21.15.100:0/4083816017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f50640b2a10 con 0x7f504c004810 2024-04-23T14:34:11.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f5063e00640 1 -- 172.21.15.100:0/4083816017 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f50640b20e0 con 0x7f504c0039c0 2024-04-23T14:34:11.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f5063e00640 1 -- 172.21.15.100:0/4083816017 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f504c00fde0 con 0x7f50640cc960 2024-04-23T14:34:11.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f506a400640 1 --2- 172.21.15.100:0/4083816017 >> v2:172.21.15.151:3300/0 conn(0x7f5064160080 0x7f5064160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:34:11.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f5069000640 1 --2- 172.21.15.100:0/4083816017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f504c004810 0x7f504c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:11.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f5069a00640 1 --2- 172.21.15.100:0/4083816017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f504c0039c0 0x7f504c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:11.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.631+0000 7f506a400640 1 --2- 172.21.15.100:0/4083816017 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f50640cc960 0x7f504c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:11.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.635+0000 7f5069000640 1 -- 172.21.15.100:0/4083816017 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f50640cc960 msgr2=0x7f504c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:11.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.635+0000 7f5069000640 1 --2- 172.21.15.100:0/4083816017 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f50640cc960 0x7f504c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:11.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.635+0000 7f5069000640 1 -- 172.21.15.100:0/4083816017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f504c0039c0 msgr2=0x7f504c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:11.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.635+0000 7f5069000640 1 --2- 172.21.15.100:0/4083816017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f504c0039c0 0x7f504c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:11.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.635+0000 7f5069000640 1 -- 172.21.15.100:0/4083816017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504c003530 con 0x7f504c004810 2024-04-23T14:34:11.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.635+0000 7f506a400640 1 --2- 172.21.15.100:0/4083816017 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f50640cc960 0x7f504c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:11.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.635+0000 7f5069000640 1 --2- 172.21.15.100:0/4083816017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f504c004810 0x7f504c004c90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5050009b10 tx=0x7f505001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e9b21effb3b424 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:11.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.635+0000 7f5063e00640 1 -- 172.21.15.100:0/4083816017 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50500096b0 con 0x7f504c004810 2024-04-23T14:34:11.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.635+0000 7f5063e00640 1 -- 172.21.15.100:0/4083816017 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50500a9470 con 0x7f504c004810 2024-04-23T14:34:11.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.635+0000 7f5063e00640 1 -- 172.21.15.100:0/4083816017 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50500a9860 con 0x7f504c004810 2024-04-23T14:34:11.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.635+0000 7f506ae00640 1 -- 172.21.15.100:0/4083816017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f504c004810 msgr2=0x7f504c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:11.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.635+0000 7f506ae00640 1 --2- 172.21.15.100:0/4083816017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f504c004810 0x7f504c004c90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5050009b10 tx=0x7f505001ca70 comp rx=0 tx=0).stop 2024-04-23T14:34:11.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.635+0000 7f506a400640 1 -- 172.21.15.100:0/4083816017 reap_dead start 2024-04-23T14:34:11.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.635+0000 7f506ae00640 1 -- 172.21.15.100:0/4083816017 shutdown_connections 2024-04-23T14:34:11.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.635+0000 7f506ae00640 1 -- 172.21.15.100:0/4083816017 >> 172.21.15.100:0/4083816017 conn(0x7f5064072140 msgr2=0x7f50640b3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:11.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f506ae00640 1 -- 172.21.15.100:0/4083816017 shutdown_connections 2024-04-23T14:34:11.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f506ae00640 1 -- 172.21.15.100:0/4083816017 wait complete. 2024-04-23T14:34:11.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f506ae00640 1 Processor -- start 2024-04-23T14:34:11.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f506ae00640 1 -- start start 2024-04-23T14:34:11.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f506ae00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f504c004810 0x7f50640c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:11.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f506ae00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50640c93e0 0x7f50640b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:11.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f506ae00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5064152770 0x7f50641615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:11.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f506ae00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5064160fe0 con 0x7f504c004810 2024-04-23T14:34:11.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f506ae00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5064160e60 con 0x7f50640c93e0 2024-04-23T14:34:11.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f506ae00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f50640d0a20 con 0x7f5064152770 2024-04-23T14:34:11.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f5069a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f504c004810 0x7f50640c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:11.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f5069a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f504c004810 0x7f50640c4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35988/0 (socket says 172.21.15.100:35988) 2024-04-23T14:34:11.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f5069a00640 1 -- 172.21.15.100:0/2145043576 learned_addr learned my addr 172.21.15.100:0/2145043576 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:11.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f506a400640 1 --2- 172.21.15.100:0/2145043576 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5064152770 0x7f50641615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:11.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f5069000640 1 --2- 172.21.15.100:0/2145043576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50640c93e0 0x7f50640b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:11.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f5069a00640 1 -- 172.21.15.100:0/2145043576 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5064152770 msgr2=0x7f50641615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:11.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f5069a00640 1 --2- 172.21.15.100:0/2145043576 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5064152770 0x7f50641615e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:11.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f5069a00640 1 -- 172.21.15.100:0/2145043576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50640c93e0 msgr2=0x7f50640b7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:11.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f5069a00640 1 --2- 172.21.15.100:0/2145043576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50640c93e0 0x7f50640b7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:11.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f5069a00640 1 -- 172.21.15.100:0/2145043576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50640c5240 con 0x7f504c004810 2024-04-23T14:34:11.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f5069000640 1 --2- 172.21.15.100:0/2145043576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50640c93e0 0x7f50640b7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:11.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f5069a00640 1 --2- 172.21.15.100:0/2145043576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f504c004810 0x7f50640c4d00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5058007310 tx=0x7f50580077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:11.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f5061600640 1 -- 172.21.15.100:0/2145043576 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505800f9c0 con 0x7f504c004810 2024-04-23T14:34:11.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f5061600640 1 -- 172.21.15.100:0/2145043576 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5058018070 con 0x7f504c004810 2024-04-23T14:34:11.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f5061600640 1 -- 172.21.15.100:0/2145043576 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505800fda0 con 0x7f504c004810 2024-04-23T14:34:11.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f506ae00640 1 -- 172.21.15.100:0/2145043576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50640c99a0 con 0x7f504c004810 2024-04-23T14:34:11.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.639+0000 7f506ae00640 1 -- 172.21.15.100:0/2145043576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50640c9f60 con 0x7f504c004810 2024-04-23T14:34:11.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.643+0000 7f506ae00640 1 -- 172.21.15.100:0/2145043576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5064068ad0 con 0x7f504c004810 2024-04-23T14:34:11.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.643+0000 7f5061600640 1 -- 172.21.15.100:0/2145043576 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f505801d460 con 0x7f504c004810 2024-04-23T14:34:11.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.643+0000 7f5061600640 1 --2- 172.21.15.100:0/2145043576 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5044068380 0x7f504406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:11.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.643+0000 7f5061600640 1 -- 172.21.15.100:0/2145043576 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6582+0+0 (secure 0 0 0) 0x7f5058086e30 con 0x7f504c004810 2024-04-23T14:34:11.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.643+0000 7f5069000640 1 --2- 172.21.15.100:0/2145043576 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5044068380 0x7f504406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:11.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.643+0000 7f5069000640 1 --2- 172.21.15.100:0/2145043576 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5044068380 0x7f504406a840 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f50500ae000 tx=0x7f50500b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:11.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.647+0000 7f5061600640 1 -- 172.21.15.100:0/2145043576 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5058056fe0 con 0x7f504c004810 2024-04-23T14:34:11.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:11.843+0000 7f506ae00640 1 -- 172.21.15.100:0/2145043576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f50640b3480 con 0x7f504c004810 2024-04-23T14:34:12.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.459+0000 7f5061600640 1 -- 172.21.15.100:0/2145043576 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f505805be20 con 0x7f504c004810 2024-04-23T14:34:12.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.463+0000 7f506ae00640 1 -- 172.21.15.100:0/2145043576 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5044068380 msgr2=0x7f504406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:12.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.463+0000 7f506ae00640 1 --2- 172.21.15.100:0/2145043576 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5044068380 0x7f504406a840 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f50500ae000 tx=0x7f50500b3040 comp rx=0 tx=0).stop 2024-04-23T14:34:12.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.463+0000 7f506ae00640 1 -- 172.21.15.100:0/2145043576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f504c004810 msgr2=0x7f50640c4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:12.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.463+0000 7f506ae00640 1 --2- 172.21.15.100:0/2145043576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f504c004810 0x7f50640c4d00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5058007310 tx=0x7f50580077d0 comp rx=0 tx=0).stop 2024-04-23T14:34:12.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.463+0000 7f506ae00640 1 -- 172.21.15.100:0/2145043576 shutdown_connections 2024-04-23T14:34:12.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.463+0000 7f506ae00640 1 --2- 172.21.15.100:0/2145043576 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5044068380 0x7f504406a840 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:12.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.463+0000 7f506ae00640 1 --2- 172.21.15.100:0/2145043576 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5064152770 0x7f50641615e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:12.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.463+0000 7f506ae00640 1 --2- 172.21.15.100:0/2145043576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50640c93e0 0x7f50640b7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:12.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.463+0000 7f506ae00640 1 --2- 172.21.15.100:0/2145043576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f504c004810 0x7f50640c4d00 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:12.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.463+0000 7f506ae00640 1 -- 172.21.15.100:0/2145043576 >> 172.21.15.100:0/2145043576 conn(0x7f5064072140 msgr2=0x7f50640ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:12.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.463+0000 7f506ae00640 1 -- 172.21.15.100:0/2145043576 shutdown_connections 2024-04-23T14:34:12.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.463+0000 7f506ae00640 1 -- 172.21.15.100:0/2145043576 wait complete. 2024-04-23T14:34:12.493 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-23T14:34:12.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.671+0000 7f623ac00640 1 Processor -- start 2024-04-23T14:34:12.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.671+0000 7f623ac00640 1 -- start start 2024-04-23T14:34:12.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.671+0000 7f623ac00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f623415c7a0 0x7f62340af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:12.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.671+0000 7f623ac00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f623406cbb0 con 0x7f62340cd300 2024-04-23T14:34:12.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.671+0000 7f623ac00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f623406ce90 con 0x7f623406bb20 2024-04-23T14:34:12.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.671+0000 7f623ac00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f623406d450 con 0x7f623415c7a0 2024-04-23T14:34:12.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.675+0000 7f6233e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f623415c7a0 0x7f62340af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:12.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.675+0000 7f6233e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f623415c7a0 0x7f62340af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35990/0 (socket says 172.21.15.100:35990) 2024-04-23T14:34:12.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.675+0000 7f6233e00640 1 -- 172.21.15.100:0/2763272765 learned_addr learned my addr 172.21.15.100:0/2763272765 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:12.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.675+0000 7f6238c00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f62340cd300 0x7f62340ad430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46582/0 (socket says 172.21.15.100:46582) 2024-04-23T14:34:12.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.675+0000 7f6233e00640 1 -- 172.21.15.100:0/2763272765 >> v1:172.21.15.151:6789/0 conn(0x7f623406bb20 legacy=0x7f62340ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:12.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.675+0000 7f6233e00640 1 -- 172.21.15.100:0/2763272765 >> v1:172.21.15.100:6789/0 conn(0x7f62340cd300 legacy=0x7f62340ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:12.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.675+0000 7f6233e00640 1 -- 172.21.15.100:0/2763272765 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623406d170 con 0x7f623415c7a0 2024-04-23T14:34:12.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.675+0000 7f6233e00640 1 --2- 172.21.15.100:0/2763272765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f623415c7a0 0x7f62340af190 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6224009e50 tx=0x7f622401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183ca6e88af19338 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:12.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.675+0000 7f6232a00640 1 -- 172.21.15.100:0/2763272765 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62240a5660 con 0x7f623415c7a0 2024-04-23T14:34:12.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.675+0000 7f6232a00640 1 -- 172.21.15.100:0/2763272765 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62240a5800 con 0x7f623415c7a0 2024-04-23T14:34:12.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.675+0000 7f6232a00640 1 -- 172.21.15.100:0/2763272765 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62240a5bd0 con 0x7f623415c7a0 2024-04-23T14:34:12.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f623ac00640 1 -- 172.21.15.100:0/2763272765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f623415c7a0 msgr2=0x7f62340af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:12.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f623ac00640 1 --2- 172.21.15.100:0/2763272765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f623415c7a0 0x7f62340af190 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6224009e50 tx=0x7f622401c9a0 comp rx=0 tx=0).stop 2024-04-23T14:34:12.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f623ac00640 1 -- 172.21.15.100:0/2763272765 shutdown_connections 2024-04-23T14:34:12.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f623ac00640 1 --2- 172.21.15.100:0/2763272765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f623415c7a0 0x7f62340af190 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:12.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f623ac00640 1 -- 172.21.15.100:0/2763272765 >> 172.21.15.100:0/2763272765 conn(0x7f62340b5200 msgr2=0x7f62340b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:12.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f623ac00640 1 -- 172.21.15.100:0/2763272765 shutdown_connections 2024-04-23T14:34:12.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f623ac00640 1 -- 172.21.15.100:0/2763272765 wait complete. 2024-04-23T14:34:12.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f623ac00640 1 Processor -- start 2024-04-23T14:34:12.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f623ac00640 1 -- start start 2024-04-23T14:34:12.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f623ac00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f623406bb20 0x7f6234099ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:12.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f623ac00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f62340cd300 0x7f62340c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:12.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f623ac00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f623415c7a0 0x7f62340c5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:12.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f623ac00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f623416a210 con 0x7f623406bb20 2024-04-23T14:34:12.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f623ac00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f62340ae620 con 0x7f623415c7a0 2024-04-23T14:34:12.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f623ac00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f623416a390 con 0x7f62340cd300 2024-04-23T14:34:12.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f6233e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f623406bb20 0x7f6234099ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:12.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f6233e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f623406bb20 0x7f6234099ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36002/0 (socket says 172.21.15.100:36002) 2024-04-23T14:34:12.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f6233e00640 1 -- 172.21.15.100:0/1238036148 learned_addr learned my addr 172.21.15.100:0/1238036148 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:12.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f6238c00640 1 --2- 172.21.15.100:0/1238036148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f623415c7a0 0x7f62340c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:12.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f6233400640 1 --2- 172.21.15.100:0/1238036148 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f62340cd300 0x7f62340c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:12.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f6233e00640 1 -- 172.21.15.100:0/1238036148 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f62340cd300 msgr2=0x7f62340c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:12.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f6233e00640 1 --2- 172.21.15.100:0/1238036148 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f62340cd300 0x7f62340c5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:12.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f6233e00640 1 -- 172.21.15.100:0/1238036148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f623415c7a0 msgr2=0x7f62340c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:12.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f6233e00640 1 --2- 172.21.15.100:0/1238036148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f623415c7a0 0x7f62340c5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:12.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.679+0000 7f6233e00640 1 -- 172.21.15.100:0/1238036148 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62340afb30 con 0x7f623406bb20 2024-04-23T14:34:12.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.683+0000 7f6233e00640 1 --2- 172.21.15.100:0/1238036148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f623406bb20 0x7f6234099ac0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f62240195b0 tx=0x7f6224019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:12.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.683+0000 7f622be00640 1 -- 172.21.15.100:0/1238036148 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62240bc020 con 0x7f623406bb20 2024-04-23T14:34:12.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.683+0000 7f622be00640 1 -- 172.21.15.100:0/1238036148 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62240b8880 con 0x7f623406bb20 2024-04-23T14:34:12.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.683+0000 7f623ac00640 1 -- 172.21.15.100:0/1238036148 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62340c6800 con 0x7f623406bb20 2024-04-23T14:34:12.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.683+0000 7f622be00640 1 -- 172.21.15.100:0/1238036148 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224009510 con 0x7f623406bb20 2024-04-23T14:34:12.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.683+0000 7f623ac00640 1 -- 172.21.15.100:0/1238036148 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62340b00b0 con 0x7f623406bb20 2024-04-23T14:34:12.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.683+0000 7f623ac00640 1 -- 172.21.15.100:0/1238036148 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62340c6550 con 0x7f623406bb20 2024-04-23T14:34:12.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.683+0000 7f622be00640 1 -- 172.21.15.100:0/1238036148 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f62240bf020 con 0x7f623406bb20 2024-04-23T14:34:12.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.683+0000 7f622be00640 1 --2- 172.21.15.100:0/1238036148 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6214068380 0x7f621406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:12.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.683+0000 7f622be00640 1 -- 172.21.15.100:0/1238036148 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6582+0+0 (secure 0 0 0) 0x7f6224121fe0 con 0x7f623406bb20 2024-04-23T14:34:12.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.683+0000 7f6233400640 1 --2- 172.21.15.100:0/1238036148 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6214068380 0x7f621406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:12.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.683+0000 7f6233400640 1 --2- 172.21.15.100:0/1238036148 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6214068380 0x7f621406a840 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f623415fa80 tx=0x7f6220009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:12.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.687+0000 7f622be00640 1 -- 172.21.15.100:0/1238036148 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f62240c6030 con 0x7f623406bb20 2024-04-23T14:34:12.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:12.847+0000 7f623ac00640 1 -- 172.21.15.100:0/1238036148 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6234068ad0 con 0x7f623406bb20 2024-04-23T14:34:13.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.699+0000 7f622be00640 1 -- 172.21.15.100:0/1238036148 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v43) ==== 181+0+0 (secure 0 0 0) 0x7f6224016620 con 0x7f623406bb20 2024-04-23T14:34:13.711 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' removed 2024-04-23T14:34:13.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.703+0000 7f623ac00640 1 -- 172.21.15.100:0/1238036148 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6214068380 msgr2=0x7f621406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:13.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.703+0000 7f623ac00640 1 --2- 172.21.15.100:0/1238036148 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6214068380 0x7f621406a840 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f623415fa80 tx=0x7f6220009040 comp rx=0 tx=0).stop 2024-04-23T14:34:13.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.703+0000 7f623ac00640 1 -- 172.21.15.100:0/1238036148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f623406bb20 msgr2=0x7f6234099ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:13.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.703+0000 7f623ac00640 1 --2- 172.21.15.100:0/1238036148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f623406bb20 0x7f6234099ac0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f62240195b0 tx=0x7f6224019740 comp rx=0 tx=0).stop 2024-04-23T14:34:13.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.703+0000 7f623ac00640 1 -- 172.21.15.100:0/1238036148 shutdown_connections 2024-04-23T14:34:13.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.703+0000 7f623ac00640 1 --2- 172.21.15.100:0/1238036148 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6214068380 0x7f621406a840 secure :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f623415fa80 tx=0x7f6220009040 comp rx=0 tx=0).stop 2024-04-23T14:34:13.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.703+0000 7f623ac00640 1 --2- 172.21.15.100:0/1238036148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f623415c7a0 0x7f62340c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:13.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.703+0000 7f623ac00640 1 --2- 172.21.15.100:0/1238036148 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f62340cd300 0x7f62340c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:13.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.703+0000 7f623ac00640 1 --2- 172.21.15.100:0/1238036148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f623406bb20 0x7f6234099ac0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:13.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.703+0000 7f623ac00640 1 -- 172.21.15.100:0/1238036148 >> 172.21.15.100:0/1238036148 conn(0x7f62340b5200 msgr2=0x7f6234151890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:13.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.703+0000 7f623ac00640 1 -- 172.21.15.100:0/1238036148 shutdown_connections 2024-04-23T14:34:13.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.703+0000 7f623ac00640 1 -- 172.21.15.100:0/1238036148 wait complete. 2024-04-23T14:34:13.738 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-23T14:34:13.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.875+0000 7fc5f5600640 1 Processor -- start 2024-04-23T14:34:13.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.875+0000 7fc5f5600640 1 -- start start 2024-04-23T14:34:13.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.875+0000 7fc5f5600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc5f015b250 0x7fc5f00ca720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:13.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.875+0000 7fc5f5600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5f015b630 0x7fc5f00acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:13.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.875+0000 7fc5f5600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5f006e330 con 0x7fc5f00cac60 2024-04-23T14:34:13.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.875+0000 7fc5f5600640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc5f00cecb0 con 0x7fc5f015b250 2024-04-23T14:34:13.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.875+0000 7fc5f5600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc5f006e5b0 con 0x7fc5f015b630 2024-04-23T14:34:13.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.875+0000 7fc5efe00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc5f015b250 0x7fc5f00ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.875+0000 7fc5efe00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc5f015b250 0x7fc5f00ca720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36014/0 (socket says 172.21.15.100:36014) 2024-04-23T14:34:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.875+0000 7fc5efe00640 1 -- 172.21.15.100:0/3691636180 learned_addr learned my addr 172.21.15.100:0/3691636180 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.875+0000 7fc5f4c00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fc5f00cac60 0x7fc5f015ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46598/0 (socket says 172.21.15.100:46598) 2024-04-23T14:34:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.875+0000 7fc5ef400640 1 --2- 172.21.15.100:0/3691636180 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5f015b630 0x7fc5f00acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.875+0000 7fc5efe00640 1 -- 172.21.15.100:0/3691636180 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5f015b630 msgr2=0x7fc5f00acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.875+0000 7fc5efe00640 1 --2- 172.21.15.100:0/3691636180 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5f015b630 0x7fc5f00acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.875+0000 7fc5efe00640 1 -- 172.21.15.100:0/3691636180 >> v1:172.21.15.100:6789/0 conn(0x7fc5f00cac60 legacy=0x7fc5f015ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.875+0000 7fc5efe00640 1 -- 172.21.15.100:0/3691636180 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f006e890 con 0x7fc5f015b250 2024-04-23T14:34:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.875+0000 7fc5eea00640 1 -- 172.21.15.100:0/3691636180 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 518818032 0 0) 0x7fc5e80014a0 con 0x7fc5f00cac60 2024-04-23T14:34:13.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.879+0000 7fc5efe00640 1 --2- 172.21.15.100:0/3691636180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f015b250 0x7fc5f00ca720 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009870 tx=0x7fc5e001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce4ba3a056a341d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:13.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.879+0000 7fc5eea00640 1 -- 172.21.15.100:0/3691636180 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e00ab640 con 0x7fc5f015b250 2024-04-23T14:34:13.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.879+0000 7fc5eea00640 1 -- 172.21.15.100:0/3691636180 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5e00b0070 con 0x7fc5f015b250 2024-04-23T14:34:13.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.879+0000 7fc5eea00640 1 -- 172.21.15.100:0/3691636180 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e00aba00 con 0x7fc5f015b250 2024-04-23T14:34:13.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.879+0000 7fc5f5600640 1 -- 172.21.15.100:0/3691636180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f015b250 msgr2=0x7fc5f00ca720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:13.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.879+0000 7fc5f5600640 1 --2- 172.21.15.100:0/3691636180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f015b250 0x7fc5f00ca720 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009870 tx=0x7fc5e001c530 comp rx=0 tx=0).stop 2024-04-23T14:34:13.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.879+0000 7fc5f5600640 1 -- 172.21.15.100:0/3691636180 shutdown_connections 2024-04-23T14:34:13.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.879+0000 7fc5f5600640 1 --2- 172.21.15.100:0/3691636180 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5f015b630 0x7fc5f00acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:13.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.879+0000 7fc5f5600640 1 --2- 172.21.15.100:0/3691636180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f015b250 0x7fc5f00ca720 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:13.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.879+0000 7fc5f5600640 1 -- 172.21.15.100:0/3691636180 >> 172.21.15.100:0/3691636180 conn(0x7fc5f00d00d0 msgr2=0x7fc5f00d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:13.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.879+0000 7fc5f5600640 1 -- 172.21.15.100:0/3691636180 shutdown_connections 2024-04-23T14:34:13.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.879+0000 7fc5f5600640 1 -- 172.21.15.100:0/3691636180 wait complete. 2024-04-23T14:34:13.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.883+0000 7fc5f5600640 1 Processor -- start 2024-04-23T14:34:13.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.883+0000 7fc5f5600640 1 -- start start 2024-04-23T14:34:13.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.883+0000 7fc5f5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f00cac60 0x7fc5f0158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:13.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.883+0000 7fc5f5600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5f015b250 0x7fc5f015a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:13.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.883+0000 7fc5f5600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5f015b630 0x7fc5f015a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:13.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.883+0000 7fc5f5600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f006f650 con 0x7fc5f00cac60 2024-04-23T14:34:13.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.883+0000 7fc5f5600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f006fd10 con 0x7fc5f015b250 2024-04-23T14:34:13.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.883+0000 7fc5f5600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc5f0168290 con 0x7fc5f015b630 2024-04-23T14:34:13.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.883+0000 7fc5efe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f00cac60 0x7fc5f0158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:13.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.883+0000 7fc5efe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f00cac60 0x7fc5f0158190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36016/0 (socket says 172.21.15.100:36016) 2024-04-23T14:34:13.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.883+0000 7fc5efe00640 1 -- 172.21.15.100:0/318853678 learned_addr learned my addr 172.21.15.100:0/318853678 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:13.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.883+0000 7fc5efe00640 1 -- 172.21.15.100:0/318853678 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5f015b630 msgr2=0x7fc5f015a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:13.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.883+0000 7fc5efe00640 1 --2- 172.21.15.100:0/318853678 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5f015b630 0x7fc5f015a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:13.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.883+0000 7fc5efe00640 1 -- 172.21.15.100:0/318853678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5f015b250 msgr2=0x7fc5f015a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:13.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.883+0000 7fc5efe00640 1 --2- 172.21.15.100:0/318853678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5f015b250 0x7fc5f015a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:13.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.883+0000 7fc5efe00640 1 -- 172.21.15.100:0/318853678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f0164510 con 0x7fc5f00cac60 2024-04-23T14:34:13.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.883+0000 7fc5efe00640 1 --2- 172.21.15.100:0/318853678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f00cac60 0x7fc5f0158190 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00ad040 tx=0x7fc5e001ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:13.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.883+0000 7fc5e7e00640 1 -- 172.21.15.100:0/318853678 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e00be3f0 con 0x7fc5f00cac60 2024-04-23T14:34:13.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.883+0000 7fc5e7e00640 1 -- 172.21.15.100:0/318853678 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5e00be5b0 con 0x7fc5f00cac60 2024-04-23T14:34:13.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.883+0000 7fc5f5600640 1 -- 172.21.15.100:0/318853678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f01647f0 con 0x7fc5f00cac60 2024-04-23T14:34:13.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.883+0000 7fc5e7e00640 1 -- 172.21.15.100:0/318853678 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e00be9a0 con 0x7fc5f00cac60 2024-04-23T14:34:13.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.883+0000 7fc5f5600640 1 -- 172.21.15.100:0/318853678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f0164db0 con 0x7fc5f00cac60 2024-04-23T14:34:13.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.887+0000 7fc5f5600640 1 -- 172.21.15.100:0/318853678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5f0075c20 con 0x7fc5f00cac60 2024-04-23T14:34:13.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.887+0000 7fc5e7e00640 1 -- 172.21.15.100:0/318853678 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc5e00b43f0 con 0x7fc5f00cac60 2024-04-23T14:34:13.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.887+0000 7fc5e7e00640 1 --2- 172.21.15.100:0/318853678 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc5d0068380 0x7fc5d006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:13.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.887+0000 7fc5e7e00640 1 -- 172.21.15.100:0/318853678 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6144+0+0 (secure 0 0 0) 0x7fc5e0121150 con 0x7fc5f00cac60 2024-04-23T14:34:13.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.887+0000 7fc5ef400640 1 --2- 172.21.15.100:0/318853678 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc5d0068380 0x7fc5d006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:13.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.887+0000 7fc5ef400640 1 --2- 172.21.15.100:0/318853678 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc5d0068380 0x7fc5d006a840 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc5f015ad40 tx=0x7fc5dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:13.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:13.891+0000 7fc5e7e00640 1 -- 172.21.15.100:0/318853678 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc5e00c4060 con 0x7fc5f00cac60 2024-04-23T14:34:14.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.047+0000 7fc5f5600640 1 -- 172.21.15.100:0/318853678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc5f00cfa00 con 0x7fc5f00cac60 2024-04-23T14:34:14.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.703+0000 7fc5e7e00640 1 -- 172.21.15.100:0/318853678 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v44) ==== 169+0+0 (secure 0 0 0) 0x7fc5e0016150 con 0x7fc5f00cac60 2024-04-23T14:34:14.708 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' removed 2024-04-23T14:34:14.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.703+0000 7fc5f5600640 1 -- 172.21.15.100:0/318853678 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc5d0068380 msgr2=0x7fc5d006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:14.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.703+0000 7fc5f5600640 1 --2- 172.21.15.100:0/318853678 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc5d0068380 0x7fc5d006a840 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc5f015ad40 tx=0x7fc5dc008040 comp rx=0 tx=0).stop 2024-04-23T14:34:14.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.703+0000 7fc5f5600640 1 -- 172.21.15.100:0/318853678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f00cac60 msgr2=0x7fc5f0158190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:14.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.703+0000 7fc5f5600640 1 --2- 172.21.15.100:0/318853678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f00cac60 0x7fc5f0158190 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00ad040 tx=0x7fc5e001ad70 comp rx=0 tx=0).stop 2024-04-23T14:34:14.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.707+0000 7fc5f5600640 1 -- 172.21.15.100:0/318853678 shutdown_connections 2024-04-23T14:34:14.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.707+0000 7fc5f5600640 1 --2- 172.21.15.100:0/318853678 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc5d0068380 0x7fc5d006a840 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:14.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.707+0000 7fc5f5600640 1 --2- 172.21.15.100:0/318853678 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5f015b630 0x7fc5f015a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:14.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.707+0000 7fc5f5600640 1 --2- 172.21.15.100:0/318853678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5f015b250 0x7fc5f015a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:14.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.707+0000 7fc5f5600640 1 --2- 172.21.15.100:0/318853678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f00cac60 0x7fc5f0158190 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:14.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.707+0000 7fc5f5600640 1 -- 172.21.15.100:0/318853678 >> 172.21.15.100:0/318853678 conn(0x7fc5f00d00d0 msgr2=0x7fc5f00afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:14.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.707+0000 7fc5f5600640 1 -- 172.21.15.100:0/318853678 shutdown_connections 2024-04-23T14:34:14.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.707+0000 7fc5f5600640 1 -- 172.21.15.100:0/318853678 wait complete. 2024-04-23T14:34:14.737 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm data_new data_new --yes-i-really-really-mean-it 2024-04-23T14:34:14.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.871+0000 7fac45000640 1 Processor -- start 2024-04-23T14:34:14.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.871+0000 7fac45000640 1 -- start start 2024-04-23T14:34:14.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.871+0000 7fac45000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fac4015e7a0 0x7fac400aea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:14.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.871+0000 7fac45000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac400ada30 con 0x7fac4006bc70 2024-04-23T14:34:14.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.871+0000 7fac45000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac400aefb0 con 0x7fac400cee40 2024-04-23T14:34:14.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.871+0000 7fac45000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fac4006f6e0 con 0x7fac4015e7a0 2024-04-23T14:34:14.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.871+0000 7fac3e800640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fac4006bc70 0x7fac4006c0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46604/0 (socket says 172.21.15.100:46604) 2024-04-23T14:34:14.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.871+0000 7fac3e800640 1 -- 172.21.15.100:0/3406805884 learned_addr learned my addr 172.21.15.100:0/3406805884 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:14.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3de00640 1 --2- 172.21.15.100:0/3406805884 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fac4015e7a0 0x7fac400aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:14.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3ca00640 1 -- 172.21.15.100:0/3406805884 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 827369918 0 0) 0x7fac380014a0 con 0x7fac4006bc70 2024-04-23T14:34:14.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3ca00640 1 -- 172.21.15.100:0/3406805884 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac24003530 con 0x7fac4006bc70 2024-04-23T14:34:14.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3ca00640 1 -- 172.21.15.100:0/3406805884 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3424141534 0 0) 0x7fac2c001120 con 0x7fac400cee40 2024-04-23T14:34:14.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3ca00640 1 -- 172.21.15.100:0/3406805884 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac380014a0 con 0x7fac400cee40 2024-04-23T14:34:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3ca00640 1 -- 172.21.15.100:0/3406805884 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3212016093 0 0) 0x7fac24003530 con 0x7fac4006bc70 2024-04-23T14:34:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3ca00640 1 -- 172.21.15.100:0/3406805884 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fac4015e7a0 msgr2=0x7fac400aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3ca00640 1 --2- 172.21.15.100:0/3406805884 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fac4015e7a0 0x7fac400aea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3ca00640 1 -- 172.21.15.100:0/3406805884 >> v1:172.21.15.151:6789/0 conn(0x7fac400cee40 legacy=0x7fac400cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3ca00640 1 -- 172.21.15.100:0/3406805884 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac400adcb0 con 0x7fac4006bc70 2024-04-23T14:34:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3ca00640 1 -- 172.21.15.100:0/3406805884 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7fac38001bd0 con 0x7fac4006bc70 2024-04-23T14:34:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3ca00640 1 -- 172.21.15.100:0/3406805884 >> v1:172.21.15.100:6789/0 conn(0x7fac4006bc70 legacy=0x7fac4006c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3ca00640 1 --2- 172.21.15.100:0/3406805884 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fac24003b80 0x7fac24003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3ca00640 1 --2- 172.21.15.100:0/3406805884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac24004810 0x7fac24004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3ca00640 1 --2- 172.21.15.100:0/3406805884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac240051d0 0x7fac2400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3ca00640 1 -- 172.21.15.100:0/3406805884 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fac4006f6e0 con 0x7fac24004810 2024-04-23T14:34:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3ca00640 1 -- 172.21.15.100:0/3406805884 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fac2400bff0 con 0x7fac240051d0 2024-04-23T14:34:14.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3ca00640 1 -- 172.21.15.100:0/3406805884 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fac2400c1f0 con 0x7fac24003b80 2024-04-23T14:34:14.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3de00640 1 --2- 172.21.15.100:0/3406805884 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fac4015e7a0 0x7fac400aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:34:14.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3d400640 1 --2- 172.21.15.100:0/3406805884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac24004810 0x7fac24004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:14.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3e800640 1 --2- 172.21.15.100:0/3406805884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac240051d0 0x7fac2400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:14.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3de00640 1 --2- 172.21.15.100:0/3406805884 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fac24003b80 0x7fac24003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:14.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3d400640 1 -- 172.21.15.100:0/3406805884 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fac24003b80 msgr2=0x7fac24003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:14.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3d400640 1 --2- 172.21.15.100:0/3406805884 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fac24003b80 0x7fac24003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:14.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3d400640 1 -- 172.21.15.100:0/3406805884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac240051d0 msgr2=0x7fac2400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:14.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3d400640 1 --2- 172.21.15.100:0/3406805884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac240051d0 0x7fac2400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:14.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3d400640 1 -- 172.21.15.100:0/3406805884 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac24003530 con 0x7fac24004810 2024-04-23T14:34:14.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.875+0000 7fac3e800640 1 --2- 172.21.15.100:0/3406805884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac240051d0 0x7fac2400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:14.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.879+0000 7fac3d400640 1 --2- 172.21.15.100:0/3406805884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac24004810 0x7fac24004c90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fac2c007fd0 tx=0x7fac2c01ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537a8504d37d5eac server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:14.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.879+0000 7fac3ca00640 1 -- 172.21.15.100:0/3406805884 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c0016e0 con 0x7fac24004810 2024-04-23T14:34:14.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.879+0000 7fac3ca00640 1 -- 172.21.15.100:0/3406805884 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac2c0052d0 con 0x7fac24004810 2024-04-23T14:34:14.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.879+0000 7fac3ca00640 1 -- 172.21.15.100:0/3406805884 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c0054c0 con 0x7fac24004810 2024-04-23T14:34:14.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.879+0000 7fac45000640 1 -- 172.21.15.100:0/3406805884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac24004810 msgr2=0x7fac24004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:14.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.879+0000 7fac45000640 1 --2- 172.21.15.100:0/3406805884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac24004810 0x7fac24004c90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fac2c007fd0 tx=0x7fac2c01ecb0 comp rx=0 tx=0).stop 2024-04-23T14:34:14.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.879+0000 7fac3e800640 1 -- 172.21.15.100:0/3406805884 reap_dead start 2024-04-23T14:34:14.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.879+0000 7fac45000640 1 -- 172.21.15.100:0/3406805884 shutdown_connections 2024-04-23T14:34:14.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.879+0000 7fac45000640 1 -- 172.21.15.100:0/3406805884 >> 172.21.15.100:0/3406805884 conn(0x7fac400b5200 msgr2=0x7fac400b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:14.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.879+0000 7fac45000640 1 -- 172.21.15.100:0/3406805884 shutdown_connections 2024-04-23T14:34:14.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.879+0000 7fac45000640 1 -- 172.21.15.100:0/3406805884 wait complete. 2024-04-23T14:34:14.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.879+0000 7fac45000640 1 Processor -- start 2024-04-23T14:34:14.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.879+0000 7fac45000640 1 -- start start 2024-04-23T14:34:14.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.879+0000 7fac45000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac24004810 0x7fac400cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:14.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.879+0000 7fac45000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fac40158610 0x7fac4015ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:14.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.879+0000 7fac45000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac400b26c0 0x7fac40098a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:14.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.879+0000 7fac45000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fac400ae550 con 0x7fac24004810 2024-04-23T14:34:14.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.879+0000 7fac45000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fac400ae270 con 0x7fac400b26c0 2024-04-23T14:34:14.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.879+0000 7fac45000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fac4015eb80 con 0x7fac40158610 2024-04-23T14:34:14.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.879+0000 7fac3de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac24004810 0x7fac400cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:14.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.879+0000 7fac3de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac24004810 0x7fac400cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36040/0 (socket says 172.21.15.100:36040) 2024-04-23T14:34:14.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.879+0000 7fac3de00640 1 -- 172.21.15.100:0/486822674 learned_addr learned my addr 172.21.15.100:0/486822674 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:14.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.883+0000 7fac3d400640 1 --2- 172.21.15.100:0/486822674 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fac40158610 0x7fac4015ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:14.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.883+0000 7fac3e800640 1 --2- 172.21.15.100:0/486822674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac400b26c0 0x7fac40098a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:14.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.883+0000 7fac3de00640 1 -- 172.21.15.100:0/486822674 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fac40158610 msgr2=0x7fac4015ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:14.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.883+0000 7fac3de00640 1 --2- 172.21.15.100:0/486822674 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fac40158610 0x7fac4015ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:14.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.883+0000 7fac3de00640 1 -- 172.21.15.100:0/486822674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac400b26c0 msgr2=0x7fac40098a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:14.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.883+0000 7fac3de00640 1 --2- 172.21.15.100:0/486822674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac400b26c0 0x7fac40098a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:14.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.883+0000 7fac3de00640 1 -- 172.21.15.100:0/486822674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac400cb950 con 0x7fac24004810 2024-04-23T14:34:14.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.883+0000 7fac3e800640 1 --2- 172.21.15.100:0/486822674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac400b26c0 0x7fac40098a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:14.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.883+0000 7fac3de00640 1 --2- 172.21.15.100:0/486822674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac24004810 0x7fac400cb250 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fac30011800 tx=0x7fac30011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:14.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.883+0000 7fac36000640 1 -- 172.21.15.100:0/486822674 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3000f9c0 con 0x7fac24004810 2024-04-23T14:34:14.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.883+0000 7fac36000640 1 -- 172.21.15.100:0/486822674 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac3000fb60 con 0x7fac24004810 2024-04-23T14:34:14.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.883+0000 7fac36000640 1 -- 172.21.15.100:0/486822674 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac30015460 con 0x7fac24004810 2024-04-23T14:34:14.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.883+0000 7fac45000640 1 -- 172.21.15.100:0/486822674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac400cbc30 con 0x7fac24004810 2024-04-23T14:34:14.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.883+0000 7fac45000640 1 -- 172.21.15.100:0/486822674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac400cc1f0 con 0x7fac24004810 2024-04-23T14:34:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.883+0000 7fac45000640 1 -- 172.21.15.100:0/486822674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac40068ad0 con 0x7fac24004810 2024-04-23T14:34:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.883+0000 7fac36000640 1 -- 172.21.15.100:0/486822674 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fac30015600 con 0x7fac24004810 2024-04-23T14:34:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.883+0000 7fac36000640 1 --2- 172.21.15.100:0/486822674 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fac20068380 0x7fac2006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.883+0000 7fac36000640 1 -- 172.21.15.100:0/486822674 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5730+0+0 (secure 0 0 0) 0x7fac30086f90 con 0x7fac24004810 2024-04-23T14:34:14.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.883+0000 7fac3d400640 1 --2- 172.21.15.100:0/486822674 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fac20068380 0x7fac2006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:14.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.887+0000 7fac3d400640 1 --2- 172.21.15.100:0/486822674 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fac20068380 0x7fac2006a840 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0b0000 tx=0x7fac2c001520 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:14.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:14.887+0000 7fac36000640 1 -- 172.21.15.100:0/486822674 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fac30019030 con 0x7fac24004810 2024-04-23T14:34:15.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.043+0000 7fac45000640 1 -- 172.21.15.100:0/486822674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true} v 0) -- 0x7fac40074f20 con 0x7fac24004810 2024-04-23T14:34:15.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.715+0000 7fac36000640 1 -- 172.21.15.100:0/486822674 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true}]=0 pool 'data_new' removed v45) ==== 160+0+0 (secure 0 0 0) 0x7fac30057520 con 0x7fac24004810 2024-04-23T14:34:15.723 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'data_new' removed 2024-04-23T14:34:15.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.719+0000 7fac45000640 1 -- 172.21.15.100:0/486822674 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fac20068380 msgr2=0x7fac2006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:15.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.719+0000 7fac45000640 1 --2- 172.21.15.100:0/486822674 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fac20068380 0x7fac2006a840 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0b0000 tx=0x7fac2c001520 comp rx=0 tx=0).stop 2024-04-23T14:34:15.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.719+0000 7fac45000640 1 -- 172.21.15.100:0/486822674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac24004810 msgr2=0x7fac400cb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:15.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.719+0000 7fac45000640 1 --2- 172.21.15.100:0/486822674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac24004810 0x7fac400cb250 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fac30011800 tx=0x7fac30011cc0 comp rx=0 tx=0).stop 2024-04-23T14:34:15.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.719+0000 7fac45000640 1 -- 172.21.15.100:0/486822674 shutdown_connections 2024-04-23T14:34:15.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.719+0000 7fac45000640 1 --2- 172.21.15.100:0/486822674 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fac20068380 0x7fac2006a840 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:15.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.719+0000 7fac45000640 1 --2- 172.21.15.100:0/486822674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac400b26c0 0x7fac40098a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:15.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.719+0000 7fac45000640 1 --2- 172.21.15.100:0/486822674 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fac40158610 0x7fac4015ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:15.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.719+0000 7fac45000640 1 --2- 172.21.15.100:0/486822674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac24004810 0x7fac400cb250 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:15.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.719+0000 7fac45000640 1 -- 172.21.15.100:0/486822674 >> 172.21.15.100:0/486822674 conn(0x7fac400b5200 msgr2=0x7fac4006c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:15.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.723+0000 7fac45000640 1 -- 172.21.15.100:0/486822674 shutdown_connections 2024-04-23T14:34:15.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.723+0000 7fac45000640 1 -- 172.21.15.100:0/486822674 wait complete. 2024-04-23T14:34:15.749 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-23T14:34:15.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.891+0000 7f447f400640 1 Processor -- start 2024-04-23T14:34:15.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.891+0000 7f447f400640 1 -- start start 2024-04-23T14:34:15.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.891+0000 7f447f400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f44780ce7f0 0x7f44781663f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:15.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.891+0000 7f447f400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f447806dd60 con 0x7f447806c6c0 2024-04-23T14:34:15.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.891+0000 7f447f400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44780d09c0 con 0x7f447806f8c0 2024-04-23T14:34:15.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.891+0000 7f447f400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f44780ad050 con 0x7f44780ce7f0 2024-04-23T14:34:15.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.891+0000 7f447d400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f44780ce7f0 0x7f44781663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:15.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.891+0000 7f4477e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f447806c6c0 0x7f44780ce0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46608/0 (socket says 172.21.15.100:46608) 2024-04-23T14:34:15.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.891+0000 7f4477e00640 1 -- 172.21.15.100:0/2398676230 learned_addr learned my addr 172.21.15.100:0/2398676230 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:15.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.891+0000 7f447d400640 1 -- 172.21.15.100:0/2398676230 >> v1:172.21.15.151:6789/0 conn(0x7f447806f8c0 legacy=0x7f44780d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:15.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.891+0000 7f447d400640 1 -- 172.21.15.100:0/2398676230 >> v1:172.21.15.100:6789/0 conn(0x7f447806c6c0 legacy=0x7f44780ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:15.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.891+0000 7f447d400640 1 -- 172.21.15.100:0/2398676230 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44780ad2a0 con 0x7f44780ce7f0 2024-04-23T14:34:15.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.891+0000 7f4477400640 1 -- 172.21.15.100:0/2398676230 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2813892300 0 0) 0x7f4464001120 con 0x7f447806c6c0 2024-04-23T14:34:15.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.895+0000 7f447d400640 1 --2- 172.21.15.100:0/2398676230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44780ce7f0 0x7f44781663f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4470009d90 tx=0x7f447001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbed46d97ddb3973 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:15.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.895+0000 7f4477400640 1 -- 172.21.15.100:0/2398676230 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44700b2070 con 0x7f44780ce7f0 2024-04-23T14:34:15.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.895+0000 7f4477400640 1 -- 172.21.15.100:0/2398676230 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44700a5900 con 0x7f44780ce7f0 2024-04-23T14:34:15.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.895+0000 7f4477400640 1 -- 172.21.15.100:0/2398676230 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44700a5cd0 con 0x7f44780ce7f0 2024-04-23T14:34:15.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.895+0000 7f447f400640 1 -- 172.21.15.100:0/2398676230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44780ce7f0 msgr2=0x7f44781663f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:15.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.895+0000 7f447f400640 1 --2- 172.21.15.100:0/2398676230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44780ce7f0 0x7f44781663f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4470009d90 tx=0x7f447001c930 comp rx=0 tx=0).stop 2024-04-23T14:34:15.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.895+0000 7f447f400640 1 -- 172.21.15.100:0/2398676230 shutdown_connections 2024-04-23T14:34:15.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.895+0000 7f447f400640 1 --2- 172.21.15.100:0/2398676230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44780ce7f0 0x7f44781663f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:15.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.895+0000 7f447f400640 1 -- 172.21.15.100:0/2398676230 >> 172.21.15.100:0/2398676230 conn(0x7f44780b5200 msgr2=0x7f44780b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:15.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.895+0000 7f447f400640 1 -- 172.21.15.100:0/2398676230 shutdown_connections 2024-04-23T14:34:15.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.895+0000 7f447f400640 1 -- 172.21.15.100:0/2398676230 wait complete. 2024-04-23T14:34:15.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.895+0000 7f447f400640 1 Processor -- start 2024-04-23T14:34:15.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f447f400640 1 -- start start 2024-04-23T14:34:15.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f447f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f447806c6c0 0x7f447815b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:15.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f447f400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f447806f8c0 0x7f44780c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:15.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f447f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44780ce7f0 0x7f4478157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:15.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f447f400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4478169770 con 0x7f44780ce7f0 2024-04-23T14:34:15.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f447f400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f44780ad8f0 con 0x7f447806c6c0 2024-04-23T14:34:15.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f447f400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f44781698f0 con 0x7f447806f8c0 2024-04-23T14:34:15.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f447d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44780ce7f0 0x7f4478157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:15.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f447d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44780ce7f0 0x7f4478157200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36054/0 (socket says 172.21.15.100:36054) 2024-04-23T14:34:15.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f447d400640 1 -- 172.21.15.100:0/1380611678 learned_addr learned my addr 172.21.15.100:0/1380611678 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:15.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f447ca00640 1 --2- 172.21.15.100:0/1380611678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f447806c6c0 0x7f447815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:15.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f4477e00640 1 --2- 172.21.15.100:0/1380611678 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f447806f8c0 0x7f44780c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:15.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f447d400640 1 -- 172.21.15.100:0/1380611678 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f447806f8c0 msgr2=0x7f44780c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:15.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f447d400640 1 --2- 172.21.15.100:0/1380611678 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f447806f8c0 0x7f44780c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:15.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f447d400640 1 -- 172.21.15.100:0/1380611678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f447806c6c0 msgr2=0x7f447815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:15.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f447d400640 1 --2- 172.21.15.100:0/1380611678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f447806c6c0 0x7f447815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:15.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f447d400640 1 -- 172.21.15.100:0/1380611678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44780ca5d0 con 0x7f44780ce7f0 2024-04-23T14:34:15.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f447d400640 1 --2- 172.21.15.100:0/1380611678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44780ce7f0 0x7f4478157200 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f44700b0040 tx=0x7f44700196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:15.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f4474c00640 1 -- 172.21.15.100:0/1380611678 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44700b2040 con 0x7f44780ce7f0 2024-04-23T14:34:15.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f4474c00640 1 -- 172.21.15.100:0/1380611678 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44700bf070 con 0x7f44780ce7f0 2024-04-23T14:34:15.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f4474c00640 1 -- 172.21.15.100:0/1380611678 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44700b8890 con 0x7f44780ce7f0 2024-04-23T14:34:15.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f447f400640 1 -- 172.21.15.100:0/1380611678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44780c5200 con 0x7f44780ce7f0 2024-04-23T14:34:15.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f447f400640 1 -- 172.21.15.100:0/1380611678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44780c57c0 con 0x7f44780ce7f0 2024-04-23T14:34:15.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.899+0000 7f447f400640 1 -- 172.21.15.100:0/1380611678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4478068ad0 con 0x7f44780ce7f0 2024-04-23T14:34:15.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.903+0000 7f4474c00640 1 -- 172.21.15.100:0/1380611678 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f44700b8a30 con 0x7f44780ce7f0 2024-04-23T14:34:15.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.903+0000 7f4474c00640 1 --2- 172.21.15.100:0/1380611678 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f44580683b0 0x7f445806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:15.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.903+0000 7f4474c00640 1 -- 172.21.15.100:0/1380611678 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f4470121410 con 0x7f44780ce7f0 2024-04-23T14:34:15.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.903+0000 7f447ca00640 1 --2- 172.21.15.100:0/1380611678 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f44580683b0 0x7f445806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:15.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.903+0000 7f447ca00640 1 --2- 172.21.15.100:0/1380611678 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f44580683b0 0x7f445806a870 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f446c004e30 tx=0x7f446c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:15.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:15.907+0000 7f4474c00640 1 -- 172.21.15.100:0/1380611678 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f44700c8020 con 0x7f44780ce7f0 2024-04-23T14:34:16.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.063+0000 7f447f400640 1 -- 172.21.15.100:0/1380611678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f447806e2c0 con 0x7f44780ce7f0 2024-04-23T14:34:16.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.067+0000 7f4474c00640 1 -- 172.21.15.100:0/1380611678 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f4470016520 con 0x7f44780ce7f0 2024-04-23T14:34:16.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.071+0000 7f447f400640 1 -- 172.21.15.100:0/1380611678 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f44580683b0 msgr2=0x7f445806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:16.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.071+0000 7f447f400640 1 --2- 172.21.15.100:0/1380611678 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f44580683b0 0x7f445806a870 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f446c004e30 tx=0x7f446c009040 comp rx=0 tx=0).stop 2024-04-23T14:34:16.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.071+0000 7f447f400640 1 -- 172.21.15.100:0/1380611678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44780ce7f0 msgr2=0x7f4478157200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:16.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.071+0000 7f447f400640 1 --2- 172.21.15.100:0/1380611678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44780ce7f0 0x7f4478157200 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f44700b0040 tx=0x7f44700196d0 comp rx=0 tx=0).stop 2024-04-23T14:34:16.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.071+0000 7f447f400640 1 -- 172.21.15.100:0/1380611678 shutdown_connections 2024-04-23T14:34:16.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.071+0000 7f447f400640 1 --2- 172.21.15.100:0/1380611678 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f44580683b0 0x7f445806a870 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:16.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.071+0000 7f447f400640 1 --2- 172.21.15.100:0/1380611678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44780ce7f0 0x7f4478157200 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:16.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.071+0000 7f447f400640 1 --2- 172.21.15.100:0/1380611678 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f447806f8c0 0x7f44780c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:16.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.071+0000 7f447f400640 1 --2- 172.21.15.100:0/1380611678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f447806c6c0 0x7f447815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:16.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.071+0000 7f447f400640 1 -- 172.21.15.100:0/1380611678 >> 172.21.15.100:0/1380611678 conn(0x7f44780b5200 msgr2=0x7f44780afc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:16.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.071+0000 7f447f400640 1 -- 172.21.15.100:0/1380611678 shutdown_connections 2024-04-23T14:34:16.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.071+0000 7f447f400640 1 -- 172.21.15.100:0/1380611678 wait complete. 2024-04-23T14:34:16.095 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2024-04-23T14:34:16.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.223+0000 7f7ae3e00640 1 Processor -- start 2024-04-23T14:34:16.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.223+0000 7f7ae3e00640 1 -- start start 2024-04-23T14:34:16.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.227+0000 7f7ae3e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f7adc06d350 0x7f7adc073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:16.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.227+0000 7f7ae3e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7adc15fa90 0x7f7adc06d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:16.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.227+0000 7f7ae3e00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7adc073880 con 0x7f7adc162390 2024-04-23T14:34:16.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.227+0000 7f7ae3e00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f7adc06eaf0 con 0x7f7adc15fa90 2024-04-23T14:34:16.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.227+0000 7f7ae3e00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f7adc06e530 con 0x7f7adc06d350 2024-04-23T14:34:16.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.227+0000 7f7ae0a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7adc15fa90 0x7f7adc06d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:16.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.227+0000 7f7ae1e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f7adc162390 0x7f7adc162840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46610/0 (socket says 172.21.15.100:46610) 2024-04-23T14:34:16.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.227+0000 7f7ae1e00640 1 -- 172.21.15.100:0/3411626108 learned_addr learned my addr 172.21.15.100:0/3411626108 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:16.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.227+0000 7f7ae1400640 1 --2- 172.21.15.100:0/3411626108 >> v2:172.21.15.151:3300/0 conn(0x7f7adc06d350 0x7f7adc073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:16.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.227+0000 7f7ae0a00640 1 -- 172.21.15.100:0/3411626108 >> v2:172.21.15.151:3300/0 conn(0x7f7adc06d350 msgr2=0x7f7adc073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:16.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.227+0000 7f7ae0a00640 1 --2- 172.21.15.100:0/3411626108 >> v2:172.21.15.151:3300/0 conn(0x7f7adc06d350 0x7f7adc073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:16.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.227+0000 7f7ae0a00640 1 -- 172.21.15.100:0/3411626108 >> v1:172.21.15.100:6789/0 conn(0x7f7adc162390 legacy=0x7f7adc162840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:16.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.227+0000 7f7ae0a00640 1 -- 172.21.15.100:0/3411626108 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7adc06dc70 con 0x7f7adc15fa90 2024-04-23T14:34:16.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.227+0000 7f7ae1400640 1 --2- 172.21.15.100:0/3411626108 >> v2:172.21.15.151:3300/0 conn(0x7f7adc06d350 0x7f7adc073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:16.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.227+0000 7f7ae0a00640 1 --2- 172.21.15.100:0/3411626108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7adc15fa90 0x7f7adc06d730 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0b5960 tx=0x7f7ac801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca62adc4ffaac0f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:16.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.227+0000 7f7adbe00640 1 -- 172.21.15.100:0/3411626108 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac80b1070 con 0x7f7adc15fa90 2024-04-23T14:34:16.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7adbe00640 1 -- 172.21.15.100:0/3411626108 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac80ac620 con 0x7f7adc15fa90 2024-04-23T14:34:16.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7adbe00640 1 -- 172.21.15.100:0/3411626108 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac80ac9f0 con 0x7f7adc15fa90 2024-04-23T14:34:16.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7ae3e00640 1 -- 172.21.15.100:0/3411626108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7adc15fa90 msgr2=0x7f7adc06d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:16.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7ae3e00640 1 --2- 172.21.15.100:0/3411626108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7adc15fa90 0x7f7adc06d730 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0b5960 tx=0x7f7ac801c3c0 comp rx=0 tx=0).stop 2024-04-23T14:34:16.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7ae3e00640 1 -- 172.21.15.100:0/3411626108 shutdown_connections 2024-04-23T14:34:16.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7ae3e00640 1 --2- 172.21.15.100:0/3411626108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7adc15fa90 0x7f7adc06d730 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:16.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7ae3e00640 1 --2- 172.21.15.100:0/3411626108 >> v2:172.21.15.151:3300/0 conn(0x7f7adc06d350 0x7f7adc073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:16.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7ae3e00640 1 -- 172.21.15.100:0/3411626108 >> 172.21.15.100:0/3411626108 conn(0x7f7adc072600 msgr2=0x7f7adc072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:16.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7ae3e00640 1 -- 172.21.15.100:0/3411626108 shutdown_connections 2024-04-23T14:34:16.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7ae3e00640 1 -- 172.21.15.100:0/3411626108 wait complete. 2024-04-23T14:34:16.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7ae3e00640 1 Processor -- start 2024-04-23T14:34:16.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7ae3e00640 1 -- start start 2024-04-23T14:34:16.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7ae3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7adc06d350 0x7f7adc15d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:16.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7ae3e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7adc15fa90 0x7f7adc140590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:16.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7ae3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7adc162390 0x7f7adc0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:16.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7ae3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc16c110 con 0x7f7adc06d350 2024-04-23T14:34:16.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7ae3e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc16bf90 con 0x7f7adc162390 2024-04-23T14:34:16.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7ae3e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7adc16c290 con 0x7f7adc15fa90 2024-04-23T14:34:16.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7ae1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7adc06d350 0x7f7adc15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:16.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7ae1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7adc06d350 0x7f7adc15d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36072/0 (socket says 172.21.15.100:36072) 2024-04-23T14:34:16.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7ae1400640 1 -- 172.21.15.100:0/2598986488 learned_addr learned my addr 172.21.15.100:0/2598986488 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:16.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7ae1e00640 1 --2- 172.21.15.100:0/2598986488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7adc162390 0x7f7adc0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:16.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.231+0000 7f7ae0a00640 1 --2- 172.21.15.100:0/2598986488 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7adc15fa90 0x7f7adc140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:16.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.235+0000 7f7ae1400640 1 -- 172.21.15.100:0/2598986488 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7adc15fa90 msgr2=0x7f7adc140590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:16.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.235+0000 7f7ae1400640 1 --2- 172.21.15.100:0/2598986488 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7adc15fa90 0x7f7adc140590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:16.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.235+0000 7f7ae1400640 1 -- 172.21.15.100:0/2598986488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7adc162390 msgr2=0x7f7adc0aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:16.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.235+0000 7f7ae1400640 1 --2- 172.21.15.100:0/2598986488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7adc162390 0x7f7adc0aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:16.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.235+0000 7f7ae1400640 1 -- 172.21.15.100:0/2598986488 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7adc15dcf0 con 0x7f7adc06d350 2024-04-23T14:34:16.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.235+0000 7f7ae1400640 1 --2- 172.21.15.100:0/2598986488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7adc06d350 0x7f7adc15d630 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7ad000c820 tx=0x7f7ad000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:16.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.235+0000 7f7ad9600640 1 -- 172.21.15.100:0/2598986488 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0015070 con 0x7f7adc06d350 2024-04-23T14:34:16.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.235+0000 7f7ad9600640 1 -- 172.21.15.100:0/2598986488 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad0007a30 con 0x7f7adc06d350 2024-04-23T14:34:16.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.235+0000 7f7ad9600640 1 -- 172.21.15.100:0/2598986488 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0007e40 con 0x7f7adc06d350 2024-04-23T14:34:16.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.235+0000 7f7ae3e00640 1 -- 172.21.15.100:0/2598986488 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7adc15dfd0 con 0x7f7adc06d350 2024-04-23T14:34:16.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.235+0000 7f7ae3e00640 1 -- 172.21.15.100:0/2598986488 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7adc15e590 con 0x7f7adc06d350 2024-04-23T14:34:16.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.235+0000 7f7ae3e00640 1 -- 172.21.15.100:0/2598986488 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7adc068ad0 con 0x7f7adc06d350 2024-04-23T14:34:16.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.235+0000 7f7ad9600640 1 -- 172.21.15.100:0/2598986488 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7ad0018070 con 0x7f7adc06d350 2024-04-23T14:34:16.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.235+0000 7f7ad9600640 1 --2- 172.21.15.100:0/2598986488 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7abc068380 0x7f7abc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:16.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.235+0000 7f7ad9600640 1 -- 172.21.15.100:0/2598986488 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f7ad0086780 con 0x7f7adc06d350 2024-04-23T14:34:16.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.235+0000 7f7ae0a00640 1 --2- 172.21.15.100:0/2598986488 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7abc068380 0x7f7abc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:16.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.239+0000 7f7ae0a00640 1 --2- 172.21.15.100:0/2598986488 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7abc068380 0x7f7abc06a840 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80ae000 tx=0x7f7ac8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:16.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.239+0000 7f7ad9600640 1 -- 172.21.15.100:0/2598986488 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7ad0056ea0 con 0x7f7adc06d350 2024-04-23T14:34:16.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.451+0000 7f7ae3e00640 1 -- 172.21.15.100:0/2598986488 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) -- 0x7f7adc073120 con 0x7f7adc06d350 2024-04-23T14:34:16.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.719+0000 7f7ad9600640 1 -- 172.21.15.100:0/2598986488 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v144) ==== 147+0+0 (secure 0 0 0) 0x7f7ad00567a0 con 0x7f7adc06d350 2024-04-23T14:34:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.719+0000 7f7ae3e00640 1 -- 172.21.15.100:0/2598986488 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7abc068380 msgr2=0x7f7abc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.719+0000 7f7ae3e00640 1 --2- 172.21.15.100:0/2598986488 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7abc068380 0x7f7abc06a840 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80ae000 tx=0x7f7ac8008040 comp rx=0 tx=0).stop 2024-04-23T14:34:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.719+0000 7f7ae3e00640 1 -- 172.21.15.100:0/2598986488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7adc06d350 msgr2=0x7f7adc15d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.719+0000 7f7ae3e00640 1 --2- 172.21.15.100:0/2598986488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7adc06d350 0x7f7adc15d630 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7ad000c820 tx=0x7f7ad000cce0 comp rx=0 tx=0).stop 2024-04-23T14:34:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.719+0000 7f7ae3e00640 1 -- 172.21.15.100:0/2598986488 shutdown_connections 2024-04-23T14:34:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.719+0000 7f7ae3e00640 1 --2- 172.21.15.100:0/2598986488 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7abc068380 0x7f7abc06a840 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.719+0000 7f7ae3e00640 1 --2- 172.21.15.100:0/2598986488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7adc162390 0x7f7adc0aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.719+0000 7f7ae3e00640 1 --2- 172.21.15.100:0/2598986488 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7adc15fa90 0x7f7adc140590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.719+0000 7f7ae3e00640 1 --2- 172.21.15.100:0/2598986488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7adc06d350 0x7f7adc15d630 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.719+0000 7f7ae3e00640 1 -- 172.21.15.100:0/2598986488 >> 172.21.15.100:0/2598986488 conn(0x7f7adc072600 msgr2=0x7f7adc15f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.719+0000 7f7ae3e00640 1 -- 172.21.15.100:0/2598986488 shutdown_connections 2024-04-23T14:34:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.719+0000 7f7ae3e00640 1 -- 172.21.15.100:0/2598986488 wait complete. 2024-04-23T14:34:16.747 INFO:tasks.cephfs_test_runner:That a client who does not have the capability for setting ... ok 2024-04-23T14:34:16.747 INFO:tasks.cephfs_test_runner:Starting test: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-04-23T14:34:16.748 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:16.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.879+0000 7f0deb000640 1 Processor -- start 2024-04-23T14:34:16.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.879+0000 7f0deb000640 1 -- start start 2024-04-23T14:34:16.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.879+0000 7f0deb000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0de40cea60 0x7f0de40d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:16.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.879+0000 7f0deb000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0de40cd460 0x7f0de40b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:16.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.879+0000 7f0deb000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0de40b1f90 con 0x7f0de406d450 2024-04-23T14:34:16.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.879+0000 7f0deb000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f0de40b2270 con 0x7f0de40cd460 2024-04-23T14:34:16.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.879+0000 7f0deb000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0de40b2550 con 0x7f0de40cea60 2024-04-23T14:34:16.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.879+0000 7f0de9000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0de40cd460 0x7f0de40b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:16.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.879+0000 7f0de9000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0de40cd460 0x7f0de40b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36084/0 (socket says 172.21.15.100:36084) 2024-04-23T14:34:16.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.879+0000 7f0de9000640 1 -- 172.21.15.100:0/4276745246 learned_addr learned my addr 172.21.15.100:0/4276745246 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:16.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.879+0000 7f0de3400640 1 --2- 172.21.15.100:0/4276745246 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0de40cea60 0x7f0de40d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:16.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.879+0000 7f0de9000640 1 -- 172.21.15.100:0/4276745246 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0de40cea60 msgr2=0x7f0de40d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:16.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.879+0000 7f0de9000640 1 --2- 172.21.15.100:0/4276745246 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0de40cea60 0x7f0de40d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:16.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.879+0000 7f0de9000640 1 -- 172.21.15.100:0/4276745246 >> v1:172.21.15.151:6789/0 conn(0x7f0de406d450 legacy=0x7f0de406d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:16.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.879+0000 7f0de9000640 1 -- 172.21.15.100:0/4276745246 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de40b2830 con 0x7f0de40cd460 2024-04-23T14:34:16.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.879+0000 7f0de3400640 1 --2- 172.21.15.100:0/4276745246 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0de40cea60 0x7f0de40d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:16.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.883+0000 7f0de9000640 1 --2- 172.21.15.100:0/4276745246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de40cd460 0x7f0de40b2d50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc009870 tx=0x7f0ddc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee63c472f66168e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:16.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.883+0000 7f0de2a00640 1 -- 172.21.15.100:0/4276745246 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc0b0070 con 0x7f0de40cd460 2024-04-23T14:34:16.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.883+0000 7f0de2a00640 1 -- 172.21.15.100:0/4276745246 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ddc0ab600 con 0x7f0de40cd460 2024-04-23T14:34:16.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.883+0000 7f0de2a00640 1 -- 172.21.15.100:0/4276745246 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc0ab9d0 con 0x7f0de40cd460 2024-04-23T14:34:16.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.883+0000 7f0deb000640 1 -- 172.21.15.100:0/4276745246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de40cd460 msgr2=0x7f0de40b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:16.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.883+0000 7f0deb000640 1 --2- 172.21.15.100:0/4276745246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de40cd460 0x7f0de40b2d50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc009870 tx=0x7f0ddc01c490 comp rx=0 tx=0).stop 2024-04-23T14:34:16.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.883+0000 7f0deb000640 1 -- 172.21.15.100:0/4276745246 shutdown_connections 2024-04-23T14:34:16.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.883+0000 7f0deb000640 1 --2- 172.21.15.100:0/4276745246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de40cd460 0x7f0de40b2d50 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:16.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.883+0000 7f0deb000640 1 --2- 172.21.15.100:0/4276745246 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0de40cea60 0x7f0de40d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:16.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.883+0000 7f0deb000640 1 -- 172.21.15.100:0/4276745246 >> 172.21.15.100:0/4276745246 conn(0x7f0de40cee80 msgr2=0x7f0de40cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:16.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.883+0000 7f0deb000640 1 -- 172.21.15.100:0/4276745246 shutdown_connections 2024-04-23T14:34:16.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.883+0000 7f0deb000640 1 -- 172.21.15.100:0/4276745246 wait complete. 2024-04-23T14:34:16.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0deb000640 1 Processor -- start 2024-04-23T14:34:16.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0deb000640 1 -- start start 2024-04-23T14:34:16.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0deb000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0de406d450 0x7f0de40cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:16.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0deb000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de40cd460 0x7f0de415a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:16.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0deb000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0de40cea60 0x7f0de415b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:16.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0deb000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0de416b740 con 0x7f0de40cd460 2024-04-23T14:34:16.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0deb000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0de416b5c0 con 0x7f0de406d450 2024-04-23T14:34:16.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0deb000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0de416b8c0 con 0x7f0de40cea60 2024-04-23T14:34:16.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0de3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de40cd460 0x7f0de415a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:16.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0de3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de40cd460 0x7f0de415a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36094/0 (socket says 172.21.15.100:36094) 2024-04-23T14:34:16.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0de3400640 1 -- 172.21.15.100:0/491229047 learned_addr learned my addr 172.21.15.100:0/491229047 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:16.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0de3e00640 1 --2- 172.21.15.100:0/491229047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0de406d450 0x7f0de40cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:16.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0de9000640 1 --2- 172.21.15.100:0/491229047 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0de40cea60 0x7f0de415b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0de3400640 1 -- 172.21.15.100:0/491229047 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0de40cea60 msgr2=0x7f0de415b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0de3400640 1 --2- 172.21.15.100:0/491229047 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0de40cea60 0x7f0de415b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0de3400640 1 -- 172.21.15.100:0/491229047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0de406d450 msgr2=0x7f0de40cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0de3400640 1 --2- 172.21.15.100:0/491229047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0de406d450 0x7f0de40cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0de3400640 1 -- 172.21.15.100:0/491229047 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de415a9c0 con 0x7f0de40cd460 2024-04-23T14:34:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0de3e00640 1 --2- 172.21.15.100:0/491229047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0de406d450 0x7f0de40cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0de3400640 1 --2- 172.21.15.100:0/491229047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de40cd460 0x7f0de415a480 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0dd000c7c0 tx=0x7f0dd000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0ddbe00640 1 -- 172.21.15.100:0/491229047 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0007a30 con 0x7f0de40cd460 2024-04-23T14:34:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0ddbe00640 1 -- 172.21.15.100:0/491229047 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dd0007bd0 con 0x7f0de40cd460 2024-04-23T14:34:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0deb000640 1 -- 172.21.15.100:0/491229047 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de41559f0 con 0x7f0de40cd460 2024-04-23T14:34:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0ddbe00640 1 -- 172.21.15.100:0/491229047 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd00113f0 con 0x7f0de40cd460 2024-04-23T14:34:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0deb000640 1 -- 172.21.15.100:0/491229047 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de4155fb0 con 0x7f0de40cd460 2024-04-23T14:34:16.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.887+0000 7f0deb000640 1 -- 172.21.15.100:0/491229047 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0de4068ad0 con 0x7f0de40cd460 2024-04-23T14:34:16.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.891+0000 7f0ddbe00640 1 -- 172.21.15.100:0/491229047 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0dd0011590 con 0x7f0de40cd460 2024-04-23T14:34:16.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.891+0000 7f0ddbe00640 1 --2- 172.21.15.100:0/491229047 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0dc4068380 0x7f0dc406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:16.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.891+0000 7f0ddbe00640 1 -- 172.21.15.100:0/491229047 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f0dd0086980 con 0x7f0de40cd460 2024-04-23T14:34:16.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.891+0000 7f0de3e00640 1 --2- 172.21.15.100:0/491229047 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0dc4068380 0x7f0dc406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:16.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.891+0000 7f0de3e00640 1 --2- 172.21.15.100:0/491229047 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0dc4068380 0x7f0dc406a840 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4009fc0 tx=0x7f0dd4009550 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:16.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:16.895+0000 7f0ddbe00640 1 -- 172.21.15.100:0/491229047 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0dd0015020 con 0x7f0de40cd460 2024-04-23T14:34:17.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.055+0000 7f0deb000640 1 -- 172.21.15.100:0/491229047 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0de40d0430 con 0x7f0de40cd460 2024-04-23T14:34:17.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.059+0000 7f0ddbe00640 1 -- 172.21.15.100:0/491229047 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+13307 (secure 0 0 0) 0x7f0dd0056a50 con 0x7f0de40cd460 2024-04-23T14:34:17.064 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:17.064 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":45,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:34:15.718599+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6836/2062210445":"2024-04-24T14:34:11.463050+0000","172.21.15.100:6834/2062210445":"2024-04-24T14:34:11.463050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:17.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.059+0000 7f0deb000640 1 -- 172.21.15.100:0/491229047 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0dc4068380 msgr2=0x7f0dc406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:17.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.059+0000 7f0deb000640 1 --2- 172.21.15.100:0/491229047 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0dc4068380 0x7f0dc406a840 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4009fc0 tx=0x7f0dd4009550 comp rx=0 tx=0).stop 2024-04-23T14:34:17.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.059+0000 7f0deb000640 1 -- 172.21.15.100:0/491229047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de40cd460 msgr2=0x7f0de415a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:17.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.059+0000 7f0deb000640 1 --2- 172.21.15.100:0/491229047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de40cd460 0x7f0de415a480 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0dd000c7c0 tx=0x7f0dd000cc80 comp rx=0 tx=0).stop 2024-04-23T14:34:17.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.059+0000 7f0deb000640 1 -- 172.21.15.100:0/491229047 shutdown_connections 2024-04-23T14:34:17.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.059+0000 7f0deb000640 1 --2- 172.21.15.100:0/491229047 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0dc4068380 0x7f0dc406a840 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:17.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.059+0000 7f0deb000640 1 --2- 172.21.15.100:0/491229047 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0de40cea60 0x7f0de415b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:17.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.059+0000 7f0deb000640 1 --2- 172.21.15.100:0/491229047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0de40cd460 0x7f0de415a480 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:17.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.059+0000 7f0deb000640 1 --2- 172.21.15.100:0/491229047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0de406d450 0x7f0de40cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:17.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.059+0000 7f0deb000640 1 -- 172.21.15.100:0/491229047 >> 172.21.15.100:0/491229047 conn(0x7f0de40cee80 msgr2=0x7f0de406f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:17.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.059+0000 7f0deb000640 1 -- 172.21.15.100:0/491229047 shutdown_connections 2024-04-23T14:34:17.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.059+0000 7f0deb000640 1 -- 172.21.15.100:0/491229047 wait complete. 2024-04-23T14:34:17.085 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:34:17.085 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:34:17.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.219+0000 7f930f000640 1 Processor -- start 2024-04-23T14:34:17.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.219+0000 7f930f000640 1 -- start start 2024-04-23T14:34:17.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.223+0000 7f930f000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f930815c7a0 0x7f93080af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:17.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.223+0000 7f930f000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f93080cd300 0x7f93080ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:17.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.223+0000 7f930f000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f930806cbb0 con 0x7f930806bb20 2024-04-23T14:34:17.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.223+0000 7f930f000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f930806d450 con 0x7f93080cd300 2024-04-23T14:34:17.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.223+0000 7f930f000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f930806ce90 con 0x7f930815c7a0 2024-04-23T14:34:17.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.223+0000 7f930d000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f93080cd300 0x7f93080ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:17.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.223+0000 7f930d000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f93080cd300 0x7f93080ad430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36096/0 (socket says 172.21.15.100:36096) 2024-04-23T14:34:17.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.223+0000 7f930d000640 1 -- 172.21.15.100:0/926659906 learned_addr learned my addr 172.21.15.100:0/926659906 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:17.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.223+0000 7f9307e00640 1 --2- 172.21.15.100:0/926659906 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f930815c7a0 0x7f93080af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:17.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.223+0000 7f930d000640 1 -- 172.21.15.100:0/926659906 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f930815c7a0 msgr2=0x7f93080af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:17.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.223+0000 7f930d000640 1 --2- 172.21.15.100:0/926659906 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f930815c7a0 0x7f93080af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:17.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.223+0000 7f930d000640 1 -- 172.21.15.100:0/926659906 >> v1:172.21.15.151:6789/0 conn(0x7f930806bb20 legacy=0x7f93080ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:17.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.223+0000 7f930d000640 1 -- 172.21.15.100:0/926659906 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930806d170 con 0x7f93080cd300 2024-04-23T14:34:17.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.223+0000 7f9307e00640 1 --2- 172.21.15.100:0/926659906 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f930815c7a0 0x7f93080af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:17.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.227+0000 7f930d000640 1 --2- 172.21.15.100:0/926659906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93080cd300 0x7f93080ad430 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9300009870 tx=0x7f930001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4d3b81a7c8d571 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:17.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.227+0000 7f9307400640 1 -- 172.21.15.100:0/926659906 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93000b0070 con 0x7f93080cd300 2024-04-23T14:34:17.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.227+0000 7f9307400640 1 -- 172.21.15.100:0/926659906 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93000ab600 con 0x7f93080cd300 2024-04-23T14:34:17.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.227+0000 7f9307400640 1 -- 172.21.15.100:0/926659906 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93000ab9d0 con 0x7f93080cd300 2024-04-23T14:34:17.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.227+0000 7f930f000640 1 -- 172.21.15.100:0/926659906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93080cd300 msgr2=0x7f93080ad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:17.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.227+0000 7f930f000640 1 --2- 172.21.15.100:0/926659906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93080cd300 0x7f93080ad430 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9300009870 tx=0x7f930001c490 comp rx=0 tx=0).stop 2024-04-23T14:34:17.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.227+0000 7f930f000640 1 -- 172.21.15.100:0/926659906 shutdown_connections 2024-04-23T14:34:17.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.227+0000 7f930f000640 1 --2- 172.21.15.100:0/926659906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93080cd300 0x7f93080ad430 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:17.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.227+0000 7f930f000640 1 --2- 172.21.15.100:0/926659906 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f930815c7a0 0x7f93080af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:17.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.227+0000 7f930f000640 1 -- 172.21.15.100:0/926659906 >> 172.21.15.100:0/926659906 conn(0x7f93080b5200 msgr2=0x7f93080b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:17.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.227+0000 7f930f000640 1 -- 172.21.15.100:0/926659906 shutdown_connections 2024-04-23T14:34:17.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.227+0000 7f930f000640 1 -- 172.21.15.100:0/926659906 wait complete. 2024-04-23T14:34:17.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.227+0000 7f930f000640 1 Processor -- start 2024-04-23T14:34:17.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.227+0000 7f930f000640 1 -- start start 2024-04-23T14:34:17.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.227+0000 7f930f000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f930806bb20 0x7f93080c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:17.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.227+0000 7f930f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93080cd300 0x7f9308166a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:17.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.227+0000 7f930f000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f930815c7a0 0x7f930808bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:17.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.227+0000 7f930f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f93080ae550 con 0x7f93080cd300 2024-04-23T14:34:17.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.227+0000 7f930f000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f93080ae3d0 con 0x7f930815c7a0 2024-04-23T14:34:17.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.227+0000 7f930f000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f93080ae6d0 con 0x7f930806bb20 2024-04-23T14:34:17.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.231+0000 7f92ff400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93080cd300 0x7f9308166a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:17.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.231+0000 7f92ff400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93080cd300 0x7f9308166a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36112/0 (socket says 172.21.15.100:36112) 2024-04-23T14:34:17.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.231+0000 7f92ff400640 1 -- 172.21.15.100:0/2705560312 learned_addr learned my addr 172.21.15.100:0/2705560312 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:17.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.231+0000 7f9307e00640 1 --2- 172.21.15.100:0/2705560312 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f930806bb20 0x7f93080c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:17.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.231+0000 7f930d000640 1 --2- 172.21.15.100:0/2705560312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f930815c7a0 0x7f930808bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:17.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.231+0000 7f92ff400640 1 -- 172.21.15.100:0/2705560312 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f930806bb20 msgr2=0x7f93080c2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:17.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.231+0000 7f92ff400640 1 --2- 172.21.15.100:0/2705560312 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f930806bb20 0x7f93080c2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:17.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.231+0000 7f92ff400640 1 -- 172.21.15.100:0/2705560312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f930815c7a0 msgr2=0x7f930808bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:17.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.231+0000 7f92ff400640 1 --2- 172.21.15.100:0/2705560312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f930815c7a0 0x7f930808bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:17.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.231+0000 7f92ff400640 1 -- 172.21.15.100:0/2705560312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93081534e0 con 0x7f93080cd300 2024-04-23T14:34:17.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.231+0000 7f92ff400640 1 --2- 172.21.15.100:0/2705560312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93080cd300 0x7f9308166a40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f92f000b410 tx=0x7f92f000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:17.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.231+0000 7f9304c00640 1 -- 172.21.15.100:0/2705560312 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0015030 con 0x7f93080cd300 2024-04-23T14:34:17.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.231+0000 7f9304c00640 1 -- 172.21.15.100:0/2705560312 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92f0004020 con 0x7f93080cd300 2024-04-23T14:34:17.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.231+0000 7f930f000640 1 -- 172.21.15.100:0/2705560312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9308165c80 con 0x7f93080cd300 2024-04-23T14:34:17.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.231+0000 7f9304c00640 1 -- 172.21.15.100:0/2705560312 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0004410 con 0x7f93080cd300 2024-04-23T14:34:17.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.231+0000 7f930f000640 1 -- 172.21.15.100:0/2705560312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9308166240 con 0x7f93080cd300 2024-04-23T14:34:17.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.231+0000 7f930f000640 1 -- 172.21.15.100:0/2705560312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9308068ad0 con 0x7f93080cd300 2024-04-23T14:34:17.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.231+0000 7f9304c00640 1 -- 172.21.15.100:0/2705560312 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f92f0018070 con 0x7f93080cd300 2024-04-23T14:34:17.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.235+0000 7f9304c00640 1 --2- 172.21.15.100:0/2705560312 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f92e4068380 0x7f92e406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:17.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.235+0000 7f9304c00640 1 -- 172.21.15.100:0/2705560312 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f92f00861e0 con 0x7f93080cd300 2024-04-23T14:34:17.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.235+0000 7f9307e00640 1 --2- 172.21.15.100:0/2705560312 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f92e4068380 0x7f92e406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:17.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.235+0000 7f9307e00640 1 --2- 172.21.15.100:0/2705560312 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f92e4068380 0x7f92e406a840 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f93080b6230 tx=0x7f92f8007810 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:17.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.239+0000 7f9304c00640 1 -- 172.21.15.100:0/2705560312 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f92f00562b0 con 0x7f93080cd300 2024-04-23T14:34:17.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.395+0000 7f930f000640 1 -- 172.21.15.100:0/2705560312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f93080cecb0 con 0x7f93080cd300 2024-04-23T14:34:17.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.395+0000 7f9304c00640 1 -- 172.21.15.100:0/2705560312 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) ==== 93+0+10 (secure 0 0 0) 0x7f92f005a5b0 con 0x7f93080cd300 2024-04-23T14:34:17.403 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:34:17.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.399+0000 7f930f000640 1 -- 172.21.15.100:0/2705560312 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f92e4068380 msgr2=0x7f92e406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:17.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.399+0000 7f930f000640 1 --2- 172.21.15.100:0/2705560312 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f92e4068380 0x7f92e406a840 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f93080b6230 tx=0x7f92f8007810 comp rx=0 tx=0).stop 2024-04-23T14:34:17.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.399+0000 7f930f000640 1 -- 172.21.15.100:0/2705560312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93080cd300 msgr2=0x7f9308166a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:17.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.399+0000 7f930f000640 1 --2- 172.21.15.100:0/2705560312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93080cd300 0x7f9308166a40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f92f000b410 tx=0x7f92f000b8d0 comp rx=0 tx=0).stop 2024-04-23T14:34:17.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.399+0000 7f930f000640 1 -- 172.21.15.100:0/2705560312 shutdown_connections 2024-04-23T14:34:17.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.399+0000 7f930f000640 1 --2- 172.21.15.100:0/2705560312 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f92e4068380 0x7f92e406a840 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:17.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.399+0000 7f930f000640 1 --2- 172.21.15.100:0/2705560312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f930815c7a0 0x7f930808bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:17.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.399+0000 7f930f000640 1 --2- 172.21.15.100:0/2705560312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93080cd300 0x7f9308166a40 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:17.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.399+0000 7f930f000640 1 --2- 172.21.15.100:0/2705560312 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f930806bb20 0x7f93080c2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:17.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.399+0000 7f930f000640 1 -- 172.21.15.100:0/2705560312 >> 172.21.15.100:0/2705560312 conn(0x7f93080b5200 msgr2=0x7f9308071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:17.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.399+0000 7f930f000640 1 -- 172.21.15.100:0/2705560312 shutdown_connections 2024-04-23T14:34:17.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.399+0000 7f930f000640 1 -- 172.21.15.100:0/2705560312 wait complete. 2024-04-23T14:34:17.424 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2024-04-23T14:34:17.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.555+0000 7fc323200640 1 Processor -- start 2024-04-23T14:34:17.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.555+0000 7fc323200640 1 -- start start 2024-04-23T14:34:17.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.555+0000 7fc323200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc3240cd050 0x7fc32415c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:17.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.555+0000 7fc323200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc324153840 con 0x7fc3240d1d60 2024-04-23T14:34:17.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.555+0000 7fc323200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc32415cfe0 con 0x7fc324153460 2024-04-23T14:34:17.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.555+0000 7fc323200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc32415d2c0 con 0x7fc3240cd050 2024-04-23T14:34:17.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.555+0000 7fc322800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc3240cd050 0x7fc32415c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:17.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.555+0000 7fc321400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fc3240d1d60 0x7fc3240cc940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46614/0 (socket says 172.21.15.100:46614) 2024-04-23T14:34:17.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.555+0000 7fc321400640 1 -- 172.21.15.100:0/3145879727 learned_addr learned my addr 172.21.15.100:0/3145879727 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:17.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.555+0000 7fc322800640 1 -- 172.21.15.100:0/3145879727 >> v1:172.21.15.151:6789/0 conn(0x7fc324153460 legacy=0x7fc3240d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:17.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.555+0000 7fc322800640 1 -- 172.21.15.100:0/3145879727 >> v1:172.21.15.100:6789/0 conn(0x7fc3240d1d60 legacy=0x7fc3240cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:17.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.555+0000 7fc322800640 1 -- 172.21.15.100:0/3145879727 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32415d5a0 con 0x7fc3240cd050 2024-04-23T14:34:17.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.555+0000 7fc320a00640 1 -- 172.21.15.100:0/3145879727 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 154937687 0 0) 0x7fc310001120 con 0x7fc3240d1d60 2024-04-23T14:34:17.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.559+0000 7fc322800640 1 --2- 172.21.15.100:0/3145879727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3240cd050 0x7fc32415c940 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc31c009d90 tx=0x7fc31c01c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a2c1ee02497e99 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:17.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.559+0000 7fc320a00640 1 -- 172.21.15.100:0/3145879727 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c0b2070 con 0x7fc3240cd050 2024-04-23T14:34:17.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.559+0000 7fc320a00640 1 -- 172.21.15.100:0/3145879727 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc31c0a5990 con 0x7fc3240cd050 2024-04-23T14:34:17.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.559+0000 7fc320a00640 1 -- 172.21.15.100:0/3145879727 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c0a5d60 con 0x7fc3240cd050 2024-04-23T14:34:17.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.559+0000 7fc323200640 1 -- 172.21.15.100:0/3145879727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3240cd050 msgr2=0x7fc32415c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:17.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.559+0000 7fc323200640 1 --2- 172.21.15.100:0/3145879727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3240cd050 0x7fc32415c940 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc31c009d90 tx=0x7fc31c01c8b0 comp rx=0 tx=0).stop 2024-04-23T14:34:17.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.559+0000 7fc323200640 1 -- 172.21.15.100:0/3145879727 shutdown_connections 2024-04-23T14:34:17.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.559+0000 7fc323200640 1 --2- 172.21.15.100:0/3145879727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3240cd050 0x7fc32415c940 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:17.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.559+0000 7fc323200640 1 -- 172.21.15.100:0/3145879727 >> 172.21.15.100:0/3145879727 conn(0x7fc3240b3e60 msgr2=0x7fc3240b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:17.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.559+0000 7fc323200640 1 -- 172.21.15.100:0/3145879727 shutdown_connections 2024-04-23T14:34:17.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.559+0000 7fc323200640 1 -- 172.21.15.100:0/3145879727 wait complete. 2024-04-23T14:34:17.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.559+0000 7fc323200640 1 Processor -- start 2024-04-23T14:34:17.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.559+0000 7fc323200640 1 -- start start 2024-04-23T14:34:17.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc323200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc3240cd050 0x7fc3240cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:17.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc323200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3240d1d60 0x7fc3240cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:17.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc323200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc324153460 0x7fc3240c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:17.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc323200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc32416a4b0 con 0x7fc3240d1d60 2024-04-23T14:34:17.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc323200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc32416a330 con 0x7fc324153460 2024-04-23T14:34:17.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc323200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc32416a630 con 0x7fc3240cd050 2024-04-23T14:34:17.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc321400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3240d1d60 0x7fc3240cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:17.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc321400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3240d1d60 0x7fc3240cc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36122/0 (socket says 172.21.15.100:36122) 2024-04-23T14:34:17.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc321400640 1 -- 172.21.15.100:0/3092037599 learned_addr learned my addr 172.21.15.100:0/3092037599 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:17.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc322800640 1 --2- 172.21.15.100:0/3092037599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc324153460 0x7fc3240c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:17.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc321e00640 1 --2- 172.21.15.100:0/3092037599 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc3240cd050 0x7fc3240cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:17.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc321400640 1 -- 172.21.15.100:0/3092037599 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc3240cd050 msgr2=0x7fc3240cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:17.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc321400640 1 --2- 172.21.15.100:0/3092037599 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc3240cd050 0x7fc3240cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:17.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc321400640 1 -- 172.21.15.100:0/3092037599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc324153460 msgr2=0x7fc3240c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:17.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc321400640 1 --2- 172.21.15.100:0/3092037599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc324153460 0x7fc3240c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:17.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc321400640 1 -- 172.21.15.100:0/3092037599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3240c8320 con 0x7fc3240d1d60 2024-04-23T14:34:17.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc321400640 1 --2- 172.21.15.100:0/3092037599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3240d1d60 0x7fc3240cc1b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc31000c780 tx=0x7fc31000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:17.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc31a000640 1 -- 172.21.15.100:0/3092037599 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310018070 con 0x7fc3240d1d60 2024-04-23T14:34:17.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc31a000640 1 -- 172.21.15.100:0/3092037599 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3100072d0 con 0x7fc3240d1d60 2024-04-23T14:34:17.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc31a000640 1 -- 172.21.15.100:0/3092037599 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3100047f0 con 0x7fc3240d1d60 2024-04-23T14:34:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc323200640 1 -- 172.21.15.100:0/3092037599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3240c8600 con 0x7fc3240d1d60 2024-04-23T14:34:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc323200640 1 -- 172.21.15.100:0/3092037599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc324139dc0 con 0x7fc3240d1d60 2024-04-23T14:34:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.563+0000 7fc323200640 1 -- 172.21.15.100:0/3092037599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc324075c20 con 0x7fc3240d1d60 2024-04-23T14:34:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.567+0000 7fc31a000640 1 -- 172.21.15.100:0/3092037599 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc310004990 con 0x7fc3240d1d60 2024-04-23T14:34:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.567+0000 7fc31a000640 1 --2- 172.21.15.100:0/3092037599 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc304068380 0x7fc30406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:17.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.567+0000 7fc31a000640 1 -- 172.21.15.100:0/3092037599 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fc310085b90 con 0x7fc3240d1d60 2024-04-23T14:34:17.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.567+0000 7fc321e00640 1 --2- 172.21.15.100:0/3092037599 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc304068380 0x7fc30406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:17.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.567+0000 7fc321e00640 1 --2- 172.21.15.100:0/3092037599 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc304068380 0x7fc30406a840 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc3240c92f0 tx=0x7fc314009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:17.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.571+0000 7fc31a000640 1 -- 172.21.15.100:0/3092037599 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc324153840 con 0x7fc3240d1d60 2024-04-23T14:34:17.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:17.787+0000 7fc323200640 1 -- 172.21.15.100:0/3092037599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) -- 0x7fc32415e830 con 0x7fc3240d1d60 2024-04-23T14:34:18.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.727+0000 7fc31a000640 1 -- 172.21.15.100:0/3092037599 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v146) ==== 137+0+0 (secure 0 0 0) 0x7fc3100076c0 con 0x7fc3240d1d60 2024-04-23T14:34:18.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.727+0000 7fc323200640 1 -- 172.21.15.100:0/3092037599 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc304068380 msgr2=0x7fc30406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.727+0000 7fc323200640 1 --2- 172.21.15.100:0/3092037599 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc304068380 0x7fc30406a840 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc3240c92f0 tx=0x7fc314009040 comp rx=0 tx=0).stop 2024-04-23T14:34:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.727+0000 7fc323200640 1 -- 172.21.15.100:0/3092037599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3240d1d60 msgr2=0x7fc3240cc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.727+0000 7fc323200640 1 --2- 172.21.15.100:0/3092037599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3240d1d60 0x7fc3240cc1b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc31000c780 tx=0x7fc31000cc40 comp rx=0 tx=0).stop 2024-04-23T14:34:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.727+0000 7fc323200640 1 -- 172.21.15.100:0/3092037599 shutdown_connections 2024-04-23T14:34:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.727+0000 7fc323200640 1 --2- 172.21.15.100:0/3092037599 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc304068380 0x7fc30406a840 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.727+0000 7fc323200640 1 --2- 172.21.15.100:0/3092037599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc324153460 0x7fc3240c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.727+0000 7fc323200640 1 --2- 172.21.15.100:0/3092037599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3240d1d60 0x7fc3240cc1b0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.727+0000 7fc323200640 1 --2- 172.21.15.100:0/3092037599 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc3240cd050 0x7fc3240cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.727+0000 7fc323200640 1 -- 172.21.15.100:0/3092037599 >> 172.21.15.100:0/3092037599 conn(0x7fc3240b3e60 msgr2=0x7fc324160810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.727+0000 7fc323200640 1 -- 172.21.15.100:0/3092037599 shutdown_connections 2024-04-23T14:34:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.727+0000 7fc323200640 1 -- 172.21.15.100:0/3092037599 wait complete. 2024-04-23T14:34:18.757 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-23T14:34:18.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.895+0000 7f5d6b200640 1 Processor -- start 2024-04-23T14:34:18.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.895+0000 7f5d6b200640 1 -- start start 2024-04-23T14:34:18.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.895+0000 7f5d6b200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f5d6c15b2f0 0x7f5d6c0701b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:18.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.895+0000 7f5d6b200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d6c0b5910 0x7f5d6c0706f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:18.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.895+0000 7f5d6b200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d6c0d0840 con 0x7f5d6c0d0d90 2024-04-23T14:34:18.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.895+0000 7f5d6b200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f5d6c0d0b20 con 0x7f5d6c15b2f0 2024-04-23T14:34:18.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.895+0000 7f5d6b200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5d6c15b6d0 con 0x7f5d6c0b5910 2024-04-23T14:34:18.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.895+0000 7f5d61400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f5d6c0d0d90 0x7f5d6c0b41e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48664/0 (socket says 172.21.15.100:48664) 2024-04-23T14:34:18.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.895+0000 7f5d61400640 1 -- 172.21.15.100:0/3541596351 learned_addr learned my addr 172.21.15.100:0/3541596351 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:18.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.895+0000 7f5d69e00640 1 --2- 172.21.15.100:0/3541596351 >> v2:172.21.15.151:3300/0 conn(0x7f5d6c15b2f0 0x7f5d6c0701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:18.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.899+0000 7f5d6a800640 1 --2- 172.21.15.100:0/3541596351 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d6c0b5910 0x7f5d6c0706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:18.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.899+0000 7f5d69400640 1 -- 172.21.15.100:0/3541596351 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1674268037 0 0) 0x7f5d6c0d0840 con 0x7f5d6c0d0d90 2024-04-23T14:34:18.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.899+0000 7f5d69400640 1 -- 172.21.15.100:0/3541596351 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d50003530 con 0x7f5d6c0d0d90 2024-04-23T14:34:18.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.899+0000 7f5d69e00640 1 -- 172.21.15.100:0/3541596351 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d6c0b5910 msgr2=0x7f5d6c0706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:18.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.899+0000 7f5d69e00640 1 --2- 172.21.15.100:0/3541596351 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d6c0b5910 0x7f5d6c0706f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:18.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.899+0000 7f5d69e00640 1 -- 172.21.15.100:0/3541596351 >> v1:172.21.15.100:6789/0 conn(0x7f5d6c0d0d90 legacy=0x7f5d6c0b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:18.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.899+0000 7f5d69e00640 1 -- 172.21.15.100:0/3541596351 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6c15eeb0 con 0x7f5d6c15b2f0 2024-04-23T14:34:18.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.899+0000 7f5d69e00640 1 --2- 172.21.15.100:0/3541596351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d6c15b2f0 0x7f5d6c0701b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009d90 tx=0x7f5d5c01c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8be132e32c2539a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:18.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.899+0000 7f5d69400640 1 -- 172.21.15.100:0/3541596351 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c0b2070 con 0x7f5d6c15b2f0 2024-04-23T14:34:18.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.899+0000 7f5d69400640 1 -- 172.21.15.100:0/3541596351 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d5c0a5920 con 0x7f5d6c15b2f0 2024-04-23T14:34:18.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.899+0000 7f5d69400640 1 -- 172.21.15.100:0/3541596351 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c0a5cf0 con 0x7f5d6c15b2f0 2024-04-23T14:34:18.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6b200640 1 -- 172.21.15.100:0/3541596351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d6c15b2f0 msgr2=0x7f5d6c0701b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:18.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6b200640 1 --2- 172.21.15.100:0/3541596351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d6c15b2f0 0x7f5d6c0701b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009d90 tx=0x7f5d5c01c940 comp rx=0 tx=0).stop 2024-04-23T14:34:18.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6b200640 1 -- 172.21.15.100:0/3541596351 shutdown_connections 2024-04-23T14:34:18.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6b200640 1 --2- 172.21.15.100:0/3541596351 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d6c0b5910 0x7f5d6c0706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:18.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6b200640 1 --2- 172.21.15.100:0/3541596351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d6c15b2f0 0x7f5d6c0701b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:18.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6b200640 1 -- 172.21.15.100:0/3541596351 >> 172.21.15.100:0/3541596351 conn(0x7f5d6c0d15f0 msgr2=0x7f5d6c0d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:18.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6b200640 1 -- 172.21.15.100:0/3541596351 shutdown_connections 2024-04-23T14:34:18.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6b200640 1 -- 172.21.15.100:0/3541596351 wait complete. 2024-04-23T14:34:18.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6b200640 1 Processor -- start 2024-04-23T14:34:18.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6b200640 1 -- start start 2024-04-23T14:34:18.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6b200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d6c0b5910 0x7f5d6c165a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:18.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6b200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d6c0d0d90 0x7f5d6c144b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:18.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d6c15b2f0 0x7f5d6c06e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:18.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c15fc70 con 0x7f5d6c15b2f0 2024-04-23T14:34:18.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6b200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c06faa0 con 0x7f5d6c0d0d90 2024-04-23T14:34:18.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6b200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5d6c0cfcc0 con 0x7f5d6c0b5910 2024-04-23T14:34:18.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d6c15b2f0 0x7f5d6c06e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:18.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d6c15b2f0 0x7f5d6c06e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57032/0 (socket says 172.21.15.100:57032) 2024-04-23T14:34:18.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6a800640 1 -- 172.21.15.100:0/3993648925 learned_addr learned my addr 172.21.15.100:0/3993648925 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:18.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d69e00640 1 --2- 172.21.15.100:0/3993648925 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d6c0b5910 0x7f5d6c165a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:18.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d61400640 1 --2- 172.21.15.100:0/3993648925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d6c0d0d90 0x7f5d6c144b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:18.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6a800640 1 -- 172.21.15.100:0/3993648925 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d6c0b5910 msgr2=0x7f5d6c165a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:18.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6a800640 1 --2- 172.21.15.100:0/3993648925 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d6c0b5910 0x7f5d6c165a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:18.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6a800640 1 -- 172.21.15.100:0/3993648925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d6c0d0d90 msgr2=0x7f5d6c144b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:18.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.903+0000 7f5d6a800640 1 --2- 172.21.15.100:0/3993648925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d6c0d0d90 0x7f5d6c144b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:18.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.907+0000 7f5d6a800640 1 -- 172.21.15.100:0/3993648925 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6c15bc00 con 0x7f5d6c15b2f0 2024-04-23T14:34:18.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.907+0000 7f5d6a800640 1 --2- 172.21.15.100:0/3993648925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d6c15b2f0 0x7f5d6c06e4c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5d6400e880 tx=0x7f5d6400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:18.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.907+0000 7f5d62a00640 1 -- 172.21.15.100:0/3993648925 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d64017070 con 0x7f5d6c15b2f0 2024-04-23T14:34:18.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.907+0000 7f5d62a00640 1 -- 172.21.15.100:0/3993648925 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d6400c940 con 0x7f5d6c15b2f0 2024-04-23T14:34:18.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.907+0000 7f5d6b200640 1 -- 172.21.15.100:0/3993648925 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6c15bee0 con 0x7f5d6c15b2f0 2024-04-23T14:34:18.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.907+0000 7f5d62a00640 1 -- 172.21.15.100:0/3993648925 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6400cd10 con 0x7f5d6c15b2f0 2024-04-23T14:34:18.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.907+0000 7f5d6b200640 1 -- 172.21.15.100:0/3993648925 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d6c15c4a0 con 0x7f5d6c15b2f0 2024-04-23T14:34:18.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.907+0000 7f5d6b200640 1 -- 172.21.15.100:0/3993648925 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d6c068ad0 con 0x7f5d6c15b2f0 2024-04-23T14:34:18.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.907+0000 7f5d62a00640 1 -- 172.21.15.100:0/3993648925 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5d6401a070 con 0x7f5d6c15b2f0 2024-04-23T14:34:18.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.907+0000 7f5d62a00640 1 --2- 172.21.15.100:0/3993648925 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5d480683b0 0x7f5d4806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:18.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.907+0000 7f5d62a00640 1 -- 172.21.15.100:0/3993648925 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f5d64086b00 con 0x7f5d6c15b2f0 2024-04-23T14:34:18.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.907+0000 7f5d69e00640 1 --2- 172.21.15.100:0/3993648925 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5d480683b0 0x7f5d4806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:18.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.911+0000 7f5d69e00640 1 --2- 172.21.15.100:0/3993648925 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5d480683b0 0x7f5d4806a870 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0af000 tx=0x7f5d5c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:18.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:18.911+0000 7f5d62a00640 1 -- 172.21.15.100:0/3993648925 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5d64056bd0 con 0x7f5d6c15b2f0 2024-04-23T14:34:19.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.071+0000 7f5d6b200640 1 -- 172.21.15.100:0/3993648925 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5d6c1552f0 con 0x7f5d6c15b2f0 2024-04-23T14:34:19.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.075+0000 7f5d62a00640 1 -- 172.21.15.100:0/3993648925 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f5d6405aed0 con 0x7f5d6c15b2f0 2024-04-23T14:34:19.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.075+0000 7f5d6b200640 1 -- 172.21.15.100:0/3993648925 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5d480683b0 msgr2=0x7f5d4806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:19.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.075+0000 7f5d6b200640 1 --2- 172.21.15.100:0/3993648925 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5d480683b0 0x7f5d4806a870 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0af000 tx=0x7f5d5c0a7070 comp rx=0 tx=0).stop 2024-04-23T14:34:19.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.075+0000 7f5d6b200640 1 -- 172.21.15.100:0/3993648925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d6c15b2f0 msgr2=0x7f5d6c06e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:19.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.075+0000 7f5d6b200640 1 --2- 172.21.15.100:0/3993648925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d6c15b2f0 0x7f5d6c06e4c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5d6400e880 tx=0x7f5d6400ed40 comp rx=0 tx=0).stop 2024-04-23T14:34:19.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.075+0000 7f5d6b200640 1 -- 172.21.15.100:0/3993648925 shutdown_connections 2024-04-23T14:34:19.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.075+0000 7f5d6b200640 1 --2- 172.21.15.100:0/3993648925 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5d480683b0 0x7f5d4806a870 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.075+0000 7f5d6b200640 1 --2- 172.21.15.100:0/3993648925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d6c15b2f0 0x7f5d6c06e4c0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.075+0000 7f5d6b200640 1 --2- 172.21.15.100:0/3993648925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d6c0d0d90 0x7f5d6c144b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.075+0000 7f5d6b200640 1 --2- 172.21.15.100:0/3993648925 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d6c0b5910 0x7f5d6c165a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.075+0000 7f5d6b200640 1 -- 172.21.15.100:0/3993648925 >> 172.21.15.100:0/3993648925 conn(0x7f5d6c0d15f0 msgr2=0x7f5d6c0b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:19.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.075+0000 7f5d6b200640 1 -- 172.21.15.100:0/3993648925 shutdown_connections 2024-04-23T14:34:19.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.079+0000 7f5d6b200640 1 -- 172.21.15.100:0/3993648925 wait complete. 2024-04-23T14:34:19.102 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-23T14:34:19.103 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:34:19.185 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:34:19.254 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-23T14:34:19.255 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:34:19.255 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:19.263 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350140k,nr_inodes=4087535,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274960k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=820 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274956k,nr_inodes=818739,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274960k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:34:19.263 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:34:19.264 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:19.273 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3578 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:34:19.273 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:34:19.273 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:19.314 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350140k,nr_inodes=4087535,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274960k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=820 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274956k,nr_inodes=818739,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274960k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:34:19.314 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:34:19.314 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:19.328 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3578 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:34:19.328 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-23T14:34:19.329 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:34:19.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f280c200640 1 Processor -- start 2024-04-23T14:34:19.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f280c200640 1 -- start start 2024-04-23T14:34:19.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f280c200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f280415c7c0 0x7f28040cca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:19.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f280c200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f28040ccf90 0x7f28040d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:19.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f280c200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2804161150 con 0x7f280415c3e0 2024-04-23T14:34:19.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f280c200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f28041608e0 con 0x7f28040ccf90 2024-04-23T14:34:19.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f280c200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f2804160b90 con 0x7f280415c7c0 2024-04-23T14:34:19.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f2809800640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f280415c3e0 0x7f28041601d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48668/0 (socket says 172.21.15.100:48668) 2024-04-23T14:34:19.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f2809800640 1 -- 172.21.15.100:0/887059766 learned_addr learned my addr 172.21.15.100:0/887059766 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:19.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f2808e00640 1 --2- 172.21.15.100:0/887059766 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f280415c7c0 0x7f28040cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:19.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f280a200640 1 --2- 172.21.15.100:0/887059766 >> v2:172.21.15.151:3300/0 conn(0x7f28040ccf90 0x7f28040d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:19.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f2803e00640 1 -- 172.21.15.100:0/887059766 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 397254972 0 0) 0x7f2804161150 con 0x7f280415c3e0 2024-04-23T14:34:19.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f2803e00640 1 -- 172.21.15.100:0/887059766 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27ec003530 con 0x7f280415c3e0 2024-04-23T14:34:19.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f2803e00640 1 -- 172.21.15.100:0/887059766 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1196386838 0 0) 0x7f27ec003530 con 0x7f280415c3e0 2024-04-23T14:34:19.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f2803e00640 1 -- 172.21.15.100:0/887059766 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f280415c7c0 msgr2=0x7f28040cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:19.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f2803e00640 1 --2- 172.21.15.100:0/887059766 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f280415c7c0 0x7f28040cca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f2803e00640 1 -- 172.21.15.100:0/887059766 >> v2:172.21.15.151:3300/0 conn(0x7f28040ccf90 msgr2=0x7f28040d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:19.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f2803e00640 1 --2- 172.21.15.100:0/887059766 >> v2:172.21.15.151:3300/0 conn(0x7f28040ccf90 0x7f28040d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f2803e00640 1 -- 172.21.15.100:0/887059766 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2804160e70 con 0x7f280415c3e0 2024-04-23T14:34:19.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f2803e00640 1 -- 172.21.15.100:0/887059766 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f27f8001680 con 0x7f280415c3e0 2024-04-23T14:34:19.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f2803e00640 1 -- 172.21.15.100:0/887059766 >> v1:172.21.15.100:6789/0 conn(0x7f280415c3e0 legacy=0x7f28041601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:19.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f2803e00640 1 --2- 172.21.15.100:0/887059766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27ec0039c0 0x7f27ec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:19.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f2803e00640 1 --2- 172.21.15.100:0/887059766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27ec004810 0x7f27ec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:19.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f2803e00640 1 --2- 172.21.15.100:0/887059766 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f280415c7c0 0x7f27ec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:19.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f2803e00640 1 -- 172.21.15.100:0/887059766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f28041608e0 con 0x7f27ec004810 2024-04-23T14:34:19.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f2803e00640 1 -- 172.21.15.100:0/887059766 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2804160b90 con 0x7f27ec0039c0 2024-04-23T14:34:19.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.487+0000 7f2803e00640 1 -- 172.21.15.100:0/887059766 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f27ec00ba00 con 0x7f280415c7c0 2024-04-23T14:34:19.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.491+0000 7f2808e00640 1 --2- 172.21.15.100:0/887059766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27ec004810 0x7f27ec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:19.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.491+0000 7f2809800640 1 --2- 172.21.15.100:0/887059766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27ec0039c0 0x7f27ec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:19.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.491+0000 7f280a200640 1 --2- 172.21.15.100:0/887059766 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f280415c7c0 0x7f27ec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:19.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.491+0000 7f2808e00640 1 -- 172.21.15.100:0/887059766 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f280415c7c0 msgr2=0x7f27ec0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:19.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.491+0000 7f2808e00640 1 --2- 172.21.15.100:0/887059766 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f280415c7c0 0x7f27ec0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.491+0000 7f2808e00640 1 -- 172.21.15.100:0/887059766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27ec0039c0 msgr2=0x7f27ec003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:19.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.491+0000 7f2808e00640 1 --2- 172.21.15.100:0/887059766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27ec0039c0 0x7f27ec003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.491+0000 7f2808e00640 1 -- 172.21.15.100:0/887059766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ec003530 con 0x7f27ec004810 2024-04-23T14:34:19.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.491+0000 7f2808e00640 1 --2- 172.21.15.100:0/887059766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27ec004810 0x7f27ec004c90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f27f0009ae0 tx=0x7f27f001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2c34ab6fbcc1a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:19.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.491+0000 7f2803e00640 1 -- 172.21.15.100:0/887059766 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f00096b0 con 0x7f27ec004810 2024-04-23T14:34:19.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.491+0000 7f2803e00640 1 -- 172.21.15.100:0/887059766 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27f00a9470 con 0x7f27ec004810 2024-04-23T14:34:19.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.491+0000 7f2803e00640 1 -- 172.21.15.100:0/887059766 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f00a9860 con 0x7f27ec004810 2024-04-23T14:34:19.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280c200640 1 -- 172.21.15.100:0/887059766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27ec004810 msgr2=0x7f27ec004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:19.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280c200640 1 --2- 172.21.15.100:0/887059766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27ec004810 0x7f27ec004c90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f27f0009ae0 tx=0x7f27f001ca10 comp rx=0 tx=0).stop 2024-04-23T14:34:19.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280a200640 1 -- 172.21.15.100:0/887059766 reap_dead start 2024-04-23T14:34:19.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280c200640 1 -- 172.21.15.100:0/887059766 shutdown_connections 2024-04-23T14:34:19.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280c200640 1 -- 172.21.15.100:0/887059766 >> 172.21.15.100:0/887059766 conn(0x7f28040cee80 msgr2=0x7f28040cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:19.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280c200640 1 -- 172.21.15.100:0/887059766 shutdown_connections 2024-04-23T14:34:19.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280c200640 1 -- 172.21.15.100:0/887059766 wait complete. 2024-04-23T14:34:19.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280c200640 1 Processor -- start 2024-04-23T14:34:19.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280c200640 1 -- start start 2024-04-23T14:34:19.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280c200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27ec004810 0x7f2804146510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:19.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280c200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2804146a50 0x7f2804157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:19.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280c200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2804148000 0x7f2804158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:19.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280c200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f28040d1b70 con 0x7f2804148000 2024-04-23T14:34:19.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280c200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f280415cfd0 con 0x7f27ec004810 2024-04-23T14:34:19.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280c200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f28040d1cf0 con 0x7f2804146a50 2024-04-23T14:34:19.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280a200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2804148000 0x7f2804158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:19.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280a200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2804148000 0x7f2804158160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57044/0 (socket says 172.21.15.100:57044) 2024-04-23T14:34:19.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280a200640 1 -- 172.21.15.100:0/3108425502 learned_addr learned my addr 172.21.15.100:0/3108425502 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:19.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f2809800640 1 --2- 172.21.15.100:0/3108425502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27ec004810 0x7f2804146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:19.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f2808e00640 1 --2- 172.21.15.100:0/3108425502 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2804146a50 0x7f2804157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:19.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280a200640 1 -- 172.21.15.100:0/3108425502 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2804146a50 msgr2=0x7f2804157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:19.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280a200640 1 --2- 172.21.15.100:0/3108425502 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2804146a50 0x7f2804157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280a200640 1 -- 172.21.15.100:0/3108425502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27ec004810 msgr2=0x7f2804146510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:19.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280a200640 1 --2- 172.21.15.100:0/3108425502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27ec004810 0x7f2804146510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280a200640 1 -- 172.21.15.100:0/3108425502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2804146f60 con 0x7f2804148000 2024-04-23T14:34:19.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.495+0000 7f280a200640 1 --2- 172.21.15.100:0/3108425502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2804148000 0x7f2804158160 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2804075360 tx=0x7f27fc0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:19.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.499+0000 7f2801600640 1 -- 172.21.15.100:0/3108425502 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc0039a0 con 0x7f2804148000 2024-04-23T14:34:19.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.499+0000 7f2801600640 1 -- 172.21.15.100:0/3108425502 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27fc01b070 con 0x7f2804148000 2024-04-23T14:34:19.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.499+0000 7f280c200640 1 -- 172.21.15.100:0/3108425502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2804147240 con 0x7f2804148000 2024-04-23T14:34:19.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.499+0000 7f2801600640 1 -- 172.21.15.100:0/3108425502 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc003b80 con 0x7f2804148000 2024-04-23T14:34:19.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.499+0000 7f280c200640 1 -- 172.21.15.100:0/3108425502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2804147800 con 0x7f2804148000 2024-04-23T14:34:19.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.499+0000 7f280c200640 1 -- 172.21.15.100:0/3108425502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28040cc430 con 0x7f2804148000 2024-04-23T14:34:19.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.499+0000 7f2801600640 1 -- 172.21.15.100:0/3108425502 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f27fc004030 con 0x7f2804148000 2024-04-23T14:34:19.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.499+0000 7f2801600640 1 --2- 172.21.15.100:0/3108425502 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f27e40683b0 0x7f27e406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:19.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.499+0000 7f2801600640 1 -- 172.21.15.100:0/3108425502 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f27fc087290 con 0x7f2804148000 2024-04-23T14:34:19.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.499+0000 7f2809800640 1 --2- 172.21.15.100:0/3108425502 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f27e40683b0 0x7f27e406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:19.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.499+0000 7f2809800640 1 --2- 172.21.15.100:0/3108425502 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f27e40683b0 0x7f27e406a870 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f28040c70a0 tx=0x7f27f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:19.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.503+0000 7f2801600640 1 -- 172.21.15.100:0/3108425502 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f28040cc430 con 0x7f2804148000 2024-04-23T14:34:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.695+0000 7f280c200640 1 -- 172.21.15.100:0/3108425502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2804068ad0 con 0x7f2804148000 2024-04-23T14:34:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.695+0000 7f2801600640 1 -- 172.21.15.100:0/3108425502 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+3881 (secure 0 0 0) 0x7f27fc025020 con 0x7f2804148000 2024-04-23T14:34:19.701 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:19.701 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":21,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2601071793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2601071793},{"type":"v1","addr":"172.21.15.100:6837","nonce":2601071793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2930424296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2930424296},{"type":"v1","addr":"172.21.15.151:6833","nonce":2930424296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1406976216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1406976216},{"type":"v1","addr":"172.21.15.151:6835","nonce":1406976216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":6081,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/4059236033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4059236033},{"type":"v1","addr":"172.21.15.100:6836","nonce":4059236033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[]} 2024-04-23T14:34:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 21 2024-04-23T14:34:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.695+0000 7f280c200640 1 -- 172.21.15.100:0/3108425502 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f27e40683b0 msgr2=0x7f27e406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.695+0000 7f280c200640 1 --2- 172.21.15.100:0/3108425502 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f27e40683b0 0x7f27e406a870 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f28040c70a0 tx=0x7f27f8009040 comp rx=0 tx=0).stop 2024-04-23T14:34:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.695+0000 7f280c200640 1 -- 172.21.15.100:0/3108425502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2804148000 msgr2=0x7f2804158160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:19.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.695+0000 7f280c200640 1 --2- 172.21.15.100:0/3108425502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2804148000 0x7f2804158160 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2804075360 tx=0x7f27fc0115a0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.695+0000 7f280c200640 1 -- 172.21.15.100:0/3108425502 shutdown_connections 2024-04-23T14:34:19.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.695+0000 7f280c200640 1 --2- 172.21.15.100:0/3108425502 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f27e40683b0 0x7f27e406a870 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.695+0000 7f280c200640 1 --2- 172.21.15.100:0/3108425502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2804148000 0x7f2804158160 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.695+0000 7f280c200640 1 --2- 172.21.15.100:0/3108425502 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2804146a50 0x7f2804157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.695+0000 7f280c200640 1 --2- 172.21.15.100:0/3108425502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27ec004810 0x7f2804146510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.695+0000 7f280c200640 1 -- 172.21.15.100:0/3108425502 >> 172.21.15.100:0/3108425502 conn(0x7f28040cee80 msgr2=0x7f28040cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:19.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.695+0000 7f280c200640 1 -- 172.21.15.100:0/3108425502 shutdown_connections 2024-04-23T14:34:19.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.695+0000 7f280c200640 1 -- 172.21.15.100:0/3108425502 wait complete. 2024-04-23T14:34:19.721 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-23T14:34:19.721 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-23T14:34:19.721 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-23T14:34:19.722 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:34:19.722 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-23T14:34:19.722 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-23T14:34:19.722 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-23T14:34:19.722 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:34:19.722 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-23T14:34:19.722 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-23T14:34:19.723 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-23T14:34:19.723 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:34:19.723 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-23T14:34:19.723 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-23T14:34:19.723 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-23T14:34:19.723 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:34:19.735 INFO:tasks.ceph.mds.a:Stopped 2024-04-23T14:34:19.735 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-23T14:34:19.737 INFO:tasks.ceph.mds.c:Stopped 2024-04-23T14:34:19.737 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-23T14:34:19.738 INFO:tasks.ceph.mds.b:Stopped 2024-04-23T14:34:19.739 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-23T14:34:19.739 INFO:tasks.ceph.mds.d:Stopped 2024-04-23T14:34:19.739 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-23T14:34:19.785 INFO:tasks.ceph.mds.a:Started 2024-04-23T14:34:19.785 DEBUG:teuthology.parallel:result is None 2024-04-23T14:34:19.786 INFO:tasks.ceph.mds.b:Started 2024-04-23T14:34:19.786 DEBUG:teuthology.parallel:result is None 2024-04-23T14:34:19.787 INFO:tasks.ceph.mds.c:Started 2024-04-23T14:34:19.787 DEBUG:teuthology.parallel:result is None 2024-04-23T14:34:19.788 INFO:tasks.ceph.mds.d:Started 2024-04-23T14:34:19.789 DEBUG:teuthology.parallel:result is None 2024-04-23T14:34:19.789 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-23T14:34:19.871 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-04-23T14:34:19.872 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-04-23T14:34:19.875 INFO:tasks.ceph.mds.b.smithi151.stdout:starting mds.b at 2024-04-23T14:34:19.877 INFO:tasks.ceph.mds.d.smithi151.stdout:starting mds.d at 2024-04-23T14:34:19.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.927+0000 7f5a74c00640 1 Processor -- start 2024-04-23T14:34:19.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.927+0000 7f5a74c00640 1 -- start start 2024-04-23T14:34:19.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.927+0000 7f5a74c00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f5a700b32b0 0x7f5a701609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:19.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.927+0000 7f5a74c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5a7015d360 0x7f5a700d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:19.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.927+0000 7f5a74c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5a700d18c0 0x7f5a700b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:19.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.927+0000 7f5a74c00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f5a700b3690 con 0x7f5a700d18c0 2024-04-23T14:34:19.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.927+0000 7f5a74c00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f5a700d1cf0 con 0x7f5a700b32b0 2024-04-23T14:34:19.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.927+0000 7f5a74c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5a700ccbf0 con 0x7f5a7015d360 2024-04-23T14:34:19.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.927+0000 7f5a6fe00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5a700d18c0 0x7f5a700b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:19.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.927+0000 7f5a6fe00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5a700d18c0 0x7f5a700b1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57070/0 (socket says 172.21.15.100:57070) 2024-04-23T14:34:19.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.927+0000 7f5a6fe00640 1 -- 172.21.15.100:0/3894799128 learned_addr learned my addr 172.21.15.100:0/3894799128 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:19.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.927+0000 7f5a6ea00640 1 --2- 172.21.15.100:0/3894799128 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5a7015d360 0x7f5a700d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:19.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.927+0000 7f5a6f400640 1 --2- 172.21.15.100:0/3894799128 >> v2:172.21.15.151:3300/0 conn(0x7f5a700b32b0 0x7f5a701609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:19.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.927+0000 7f5a6fe00640 1 -- 172.21.15.100:0/3894799128 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5a7015d360 msgr2=0x7f5a700d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:19.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.927+0000 7f5a6fe00640 1 --2- 172.21.15.100:0/3894799128 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5a7015d360 0x7f5a700d1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.927+0000 7f5a6fe00640 1 -- 172.21.15.100:0/3894799128 >> v2:172.21.15.151:3300/0 conn(0x7f5a700b32b0 msgr2=0x7f5a701609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:19.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.927+0000 7f5a6fe00640 1 --2- 172.21.15.100:0/3894799128 >> v2:172.21.15.151:3300/0 conn(0x7f5a700b32b0 0x7f5a701609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.927+0000 7f5a6fe00640 1 -- 172.21.15.100:0/3894799128 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a700cced0 con 0x7f5a700d18c0 2024-04-23T14:34:19.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.927+0000 7f5a6f400640 1 --2- 172.21.15.100:0/3894799128 >> v2:172.21.15.151:3300/0 conn(0x7f5a700b32b0 0x7f5a701609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:19.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.931+0000 7f5a6fe00640 1 --2- 172.21.15.100:0/3894799128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a700d18c0 0x7f5a700b1bb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5a700cec20 tx=0x7f5a6801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1685e570071cfb21 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:19.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.931+0000 7f5a6e000640 1 -- 172.21.15.100:0/3894799128 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a680a5760 con 0x7f5a700d18c0 2024-04-23T14:34:19.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.931+0000 7f5a6e000640 1 -- 172.21.15.100:0/3894799128 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a680a5900 con 0x7f5a700d18c0 2024-04-23T14:34:19.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.931+0000 7f5a6e000640 1 -- 172.21.15.100:0/3894799128 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a680a5cd0 con 0x7f5a700d18c0 2024-04-23T14:34:19.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.931+0000 7f5a74c00640 1 -- 172.21.15.100:0/3894799128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a700d18c0 msgr2=0x7f5a700b1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:19.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.931+0000 7f5a74c00640 1 --2- 172.21.15.100:0/3894799128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a700d18c0 0x7f5a700b1bb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5a700cec20 tx=0x7f5a6801c9f0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.931+0000 7f5a74c00640 1 -- 172.21.15.100:0/3894799128 shutdown_connections 2024-04-23T14:34:19.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.931+0000 7f5a74c00640 1 --2- 172.21.15.100:0/3894799128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a700d18c0 0x7f5a700b1bb0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.931+0000 7f5a74c00640 1 --2- 172.21.15.100:0/3894799128 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5a7015d360 0x7f5a700d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.931+0000 7f5a74c00640 1 --2- 172.21.15.100:0/3894799128 >> v2:172.21.15.151:3300/0 conn(0x7f5a700b32b0 0x7f5a701609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.931+0000 7f5a74c00640 1 -- 172.21.15.100:0/3894799128 >> 172.21.15.100:0/3894799128 conn(0x7f5a700d2270 msgr2=0x7f5a700d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:19.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.931+0000 7f5a74c00640 1 -- 172.21.15.100:0/3894799128 shutdown_connections 2024-04-23T14:34:19.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.931+0000 7f5a74c00640 1 -- 172.21.15.100:0/3894799128 wait complete. 2024-04-23T14:34:19.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a74c00640 1 Processor -- start 2024-04-23T14:34:19.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a74c00640 1 -- start start 2024-04-23T14:34:19.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a74c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a700b32b0 0x7f5a7014ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:19.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a74c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a700d18c0 0x7f5a701f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:19.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a74c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5a7015d360 0x7f5a7014c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:19.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a74c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7015d770 con 0x7f5a700d18c0 2024-04-23T14:34:19.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a74c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a700cc8c0 con 0x7f5a700b32b0 2024-04-23T14:34:19.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a74c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5a7006d8f0 con 0x7f5a7015d360 2024-04-23T14:34:19.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a6ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a700d18c0 0x7f5a701f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:19.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a6ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a700d18c0 0x7f5a701f7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57080/0 (socket says 172.21.15.100:57080) 2024-04-23T14:34:19.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a6ea00640 1 -- 172.21.15.100:0/1080790378 learned_addr learned my addr 172.21.15.100:0/1080790378 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:19.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a6fe00640 1 --2- 172.21.15.100:0/1080790378 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5a7015d360 0x7f5a7014c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a6f400640 1 --2- 172.21.15.100:0/1080790378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a700b32b0 0x7f5a7014ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a6ea00640 1 -- 172.21.15.100:0/1080790378 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5a7015d360 msgr2=0x7f5a7014c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a6ea00640 1 --2- 172.21.15.100:0/1080790378 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5a7015d360 0x7f5a7014c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a6ea00640 1 -- 172.21.15.100:0/1080790378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a700b32b0 msgr2=0x7f5a7014ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a6ea00640 1 --2- 172.21.15.100:0/1080790378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a700b32b0 0x7f5a7014ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a6ea00640 1 -- 172.21.15.100:0/1080790378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a700cc140 con 0x7f5a700d18c0 2024-04-23T14:34:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a6fe00640 1 --2- 172.21.15.100:0/1080790378 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5a7015d360 0x7f5a7014c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:34:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a6ea00640 1 --2- 172.21.15.100:0/1080790378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a700d18c0 0x7f5a701f7550 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00e870 tx=0x7f5a5c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a67400640 1 -- 172.21.15.100:0/1080790378 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c017070 con 0x7f5a700d18c0 2024-04-23T14:34:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a67400640 1 -- 172.21.15.100:0/1080790378 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a5c00c940 con 0x7f5a700d18c0 2024-04-23T14:34:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a67400640 1 -- 172.21.15.100:0/1080790378 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c00ccf0 con 0x7f5a700d18c0 2024-04-23T14:34:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a74c00640 1 -- 172.21.15.100:0/1080790378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a700cbb80 con 0x7f5a700d18c0 2024-04-23T14:34:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a74c00640 1 -- 172.21.15.100:0/1080790378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a7006d580 con 0x7f5a700d18c0 2024-04-23T14:34:19.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.935+0000 7f5a74c00640 1 -- 172.21.15.100:0/1080790378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a70075c20 con 0x7f5a700d18c0 2024-04-23T14:34:19.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.939+0000 7f5a67400640 1 -- 172.21.15.100:0/1080790378 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5a5c01a070 con 0x7f5a700d18c0 2024-04-23T14:34:19.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.939+0000 7f5a67400640 1 --2- 172.21.15.100:0/1080790378 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5a50068380 0x7f5a5006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:19.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.939+0000 7f5a67400640 1 -- 172.21.15.100:0/1080790378 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f5a5c086af0 con 0x7f5a700d18c0 2024-04-23T14:34:19.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.939+0000 7f5a6f400640 1 --2- 172.21.15.100:0/1080790378 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5a50068380 0x7f5a5006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:19.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.939+0000 7f5a6f400640 1 --2- 172.21.15.100:0/1080790378 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5a50068380 0x7f5a5006a840 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5a600098d0 tx=0x7f5a60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:19.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:19.943+0000 7f5a67400640 1 -- 172.21.15.100:0/1080790378 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5a5c056c80 con 0x7f5a700d18c0 2024-04-23T14:34:20.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.111+0000 7f5a74c00640 1 -- 172.21.15.100:0/1080790378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f5a7015c420 con 0x7f5a700d18c0 2024-04-23T14:34:20.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.747+0000 7f5a67400640 1 -- 172.21.15.100:0/1080790378 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v47) ==== 97+0+0 (secure 0 0 0) 0x7f5a5c05af80 con 0x7f5a700d18c0 2024-04-23T14:34:20.752 INFO:teuthology.orchestra.run.smithi100.stderr: removed all blocklist entries 2024-04-23T14:34:20.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.747+0000 7f5a74c00640 1 -- 172.21.15.100:0/1080790378 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5a50068380 msgr2=0x7f5a5006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:20.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.747+0000 7f5a74c00640 1 --2- 172.21.15.100:0/1080790378 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5a50068380 0x7f5a5006a840 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5a600098d0 tx=0x7f5a60008040 comp rx=0 tx=0).stop 2024-04-23T14:34:20.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.747+0000 7f5a74c00640 1 -- 172.21.15.100:0/1080790378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a700d18c0 msgr2=0x7f5a701f7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:20.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.747+0000 7f5a74c00640 1 --2- 172.21.15.100:0/1080790378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a700d18c0 0x7f5a701f7550 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00e870 tx=0x7f5a5c00ed30 comp rx=0 tx=0).stop 2024-04-23T14:34:20.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.747+0000 7f5a74c00640 1 -- 172.21.15.100:0/1080790378 shutdown_connections 2024-04-23T14:34:20.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.747+0000 7f5a74c00640 1 --2- 172.21.15.100:0/1080790378 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5a50068380 0x7f5a5006a840 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:20.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.747+0000 7f5a74c00640 1 --2- 172.21.15.100:0/1080790378 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5a7015d360 0x7f5a7014c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:20.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.747+0000 7f5a74c00640 1 --2- 172.21.15.100:0/1080790378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a700d18c0 0x7f5a701f7550 secure :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00e870 tx=0x7f5a5c00ed30 comp rx=0 tx=0).stop 2024-04-23T14:34:20.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.747+0000 7f5a74c00640 1 --2- 172.21.15.100:0/1080790378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a700b32b0 0x7f5a7014ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:20.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.747+0000 7f5a74c00640 1 -- 172.21.15.100:0/1080790378 >> 172.21.15.100:0/1080790378 conn(0x7f5a700d2270 msgr2=0x7f5a700cffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:20.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.751+0000 7f5a74c00640 1 -- 172.21.15.100:0/1080790378 shutdown_connections 2024-04-23T14:34:20.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.751+0000 7f5a74c00640 1 -- 172.21.15.100:0/1080790378 wait complete. 2024-04-23T14:34:20.779 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-23T14:34:20.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.907+0000 7f1a8ce00640 1 Processor -- start 2024-04-23T14:34:20.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.907+0000 7f1a8ce00640 1 -- start start 2024-04-23T14:34:20.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.911+0000 7f1a8ce00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1a8815b640 0x7f1a880adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:20.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.911+0000 7f1a8ce00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a88073610 0x7f1a8806df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:20.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.911+0000 7f1a8ce00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a880af080 con 0x7f1a880b2a10 2024-04-23T14:34:20.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.911+0000 7f1a8ce00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f1a880afbc0 con 0x7f1a8815b640 2024-04-23T14:34:20.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.911+0000 7f1a8ce00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1a8815ba20 con 0x7f1a88073610 2024-04-23T14:34:20.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.911+0000 7f1a85e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1a8815b640 0x7f1a880adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:20.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.911+0000 7f1a85400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1a880b2a10 0x7f1a88072f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48690/0 (socket says 172.21.15.100:48690) 2024-04-23T14:34:20.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.911+0000 7f1a85400640 1 -- 172.21.15.100:0/2518827998 learned_addr learned my addr 172.21.15.100:0/2518827998 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:20.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.911+0000 7f1a86800640 1 --2- 172.21.15.100:0/2518827998 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a88073610 0x7f1a8806df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:20.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.911+0000 7f1a84a00640 1 -- 172.21.15.100:0/2518827998 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3710394426 0 0) 0x7f1a880af080 con 0x7f1a880b2a10 2024-04-23T14:34:20.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.911+0000 7f1a84a00640 1 -- 172.21.15.100:0/2518827998 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a6c003530 con 0x7f1a880b2a10 2024-04-23T14:34:20.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.911+0000 7f1a85e00640 1 -- 172.21.15.100:0/2518827998 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a88073610 msgr2=0x7f1a8806df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:20.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.911+0000 7f1a85e00640 1 --2- 172.21.15.100:0/2518827998 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a88073610 0x7f1a8806df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:20.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.911+0000 7f1a85e00640 1 -- 172.21.15.100:0/2518827998 >> v1:172.21.15.100:6789/0 conn(0x7f1a880b2a10 legacy=0x7f1a88072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:20.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.911+0000 7f1a85e00640 1 -- 172.21.15.100:0/2518827998 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a880af320 con 0x7f1a8815b640 2024-04-23T14:34:20.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.911+0000 7f1a86800640 1 --2- 172.21.15.100:0/2518827998 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a88073610 0x7f1a8806df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:20.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.915+0000 7f1a85e00640 1 --2- 172.21.15.100:0/2518827998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a8815b640 0x7f1a880adfc0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009d90 tx=0x7f1a7801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72964df209571bbc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:20.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.915+0000 7f1a84a00640 1 -- 172.21.15.100:0/2518827998 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a780b2070 con 0x7f1a8815b640 2024-04-23T14:34:20.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.915+0000 7f1a84a00640 1 -- 172.21.15.100:0/2518827998 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a780a5a00 con 0x7f1a8815b640 2024-04-23T14:34:20.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.915+0000 7f1a84a00640 1 -- 172.21.15.100:0/2518827998 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a780a5dd0 con 0x7f1a8815b640 2024-04-23T14:34:20.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.915+0000 7f1a8ce00640 1 -- 172.21.15.100:0/2518827998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a8815b640 msgr2=0x7f1a880adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:20.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.915+0000 7f1a8ce00640 1 --2- 172.21.15.100:0/2518827998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a8815b640 0x7f1a880adfc0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009d90 tx=0x7f1a7801c940 comp rx=0 tx=0).stop 2024-04-23T14:34:20.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.915+0000 7f1a8ce00640 1 -- 172.21.15.100:0/2518827998 shutdown_connections 2024-04-23T14:34:20.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.915+0000 7f1a8ce00640 1 --2- 172.21.15.100:0/2518827998 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a88073610 0x7f1a8806df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:20.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.915+0000 7f1a8ce00640 1 --2- 172.21.15.100:0/2518827998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a8815b640 0x7f1a880adfc0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:20.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.915+0000 7f1a8ce00640 1 -- 172.21.15.100:0/2518827998 >> 172.21.15.100:0/2518827998 conn(0x7f1a880b1e80 msgr2=0x7f1a880b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:20.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.915+0000 7f1a8ce00640 1 -- 172.21.15.100:0/2518827998 shutdown_connections 2024-04-23T14:34:20.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.915+0000 7f1a8ce00640 1 -- 172.21.15.100:0/2518827998 wait complete. 2024-04-23T14:34:20.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.915+0000 7f1a8ce00640 1 Processor -- start 2024-04-23T14:34:20.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.915+0000 7f1a8ce00640 1 -- start start 2024-04-23T14:34:20.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.915+0000 7f1a8ce00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a88073610 0x7f1a8814aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:20.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.915+0000 7f1a8ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a880b2a10 0x7f1a881487f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:20.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.915+0000 7f1a8ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a8815b640 0x7f1a881f79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:20.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.915+0000 7f1a8ce00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8806f0d0 con 0x7f1a8815b640 2024-04-23T14:34:20.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.915+0000 7f1a8ce00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8806ef50 con 0x7f1a880b2a10 2024-04-23T14:34:20.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.915+0000 7f1a8ce00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1a8806f250 con 0x7f1a88073610 2024-04-23T14:34:20.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.919+0000 7f1a86800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a8815b640 0x7f1a881f79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:20.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.919+0000 7f1a86800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a8815b640 0x7f1a881f79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57098/0 (socket says 172.21.15.100:57098) 2024-04-23T14:34:20.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.919+0000 7f1a86800640 1 -- 172.21.15.100:0/3588449401 learned_addr learned my addr 172.21.15.100:0/3588449401 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:20.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.919+0000 7f1a85e00640 1 --2- 172.21.15.100:0/3588449401 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a88073610 0x7f1a8814aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:20.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.919+0000 7f1a85400640 1 --2- 172.21.15.100:0/3588449401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a880b2a10 0x7f1a881487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:20.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.919+0000 7f1a86800640 1 -- 172.21.15.100:0/3588449401 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a88073610 msgr2=0x7f1a8814aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:20.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.919+0000 7f1a86800640 1 --2- 172.21.15.100:0/3588449401 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a88073610 0x7f1a8814aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:20.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.919+0000 7f1a86800640 1 -- 172.21.15.100:0/3588449401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a880b2a10 msgr2=0x7f1a881487f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:20.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.919+0000 7f1a86800640 1 --2- 172.21.15.100:0/3588449401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a880b2a10 0x7f1a881487f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:20.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.919+0000 7f1a86800640 1 -- 172.21.15.100:0/3588449401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8815e4e0 con 0x7f1a8815b640 2024-04-23T14:34:20.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.919+0000 7f1a85400640 1 --2- 172.21.15.100:0/3588449401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a880b2a10 0x7f1a881487f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:20.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.919+0000 7f1a86800640 1 --2- 172.21.15.100:0/3588449401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a8815b640 0x7f1a881f79a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1a8000e5b0 tx=0x7f1a8000ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:20.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.919+0000 7f1a7e000640 1 -- 172.21.15.100:0/3588449401 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80017070 con 0x7f1a8815b640 2024-04-23T14:34:20.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.919+0000 7f1a7e000640 1 -- 172.21.15.100:0/3588449401 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a80013610 con 0x7f1a8815b640 2024-04-23T14:34:20.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.919+0000 7f1a7e000640 1 -- 172.21.15.100:0/3588449401 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8000f040 con 0x7f1a8815b640 2024-04-23T14:34:20.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.919+0000 7f1a8ce00640 1 -- 172.21.15.100:0/3588449401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a8815ea60 con 0x7f1a8815b640 2024-04-23T14:34:20.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.919+0000 7f1a8ce00640 1 -- 172.21.15.100:0/3588449401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a8809db50 con 0x7f1a8815b640 2024-04-23T14:34:20.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.919+0000 7f1a8ce00640 1 -- 172.21.15.100:0/3588449401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a88068ad0 con 0x7f1a8815b640 2024-04-23T14:34:20.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.919+0000 7f1a7e000640 1 -- 172.21.15.100:0/3588449401 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1a8001a070 con 0x7f1a8815b640 2024-04-23T14:34:20.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.919+0000 7f1a7e000640 1 --2- 172.21.15.100:0/3588449401 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1a68068380 0x7f1a6806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:20.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.923+0000 7f1a7e000640 1 -- 172.21.15.100:0/3588449401 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5249+0+0 (secure 0 0 0) 0x7f1a80086320 con 0x7f1a8815b640 2024-04-23T14:34:20.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.923+0000 7f1a85e00640 1 --2- 172.21.15.100:0/3588449401 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1a68068380 0x7f1a6806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:20.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.923+0000 7f1a85e00640 1 --2- 172.21.15.100:0/3588449401 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1a68068380 0x7f1a6806a840 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1a780af000 tx=0x7f1a780a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:20.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:20.927+0000 7f1a7e000640 1 -- 172.21.15.100:0/3588449401 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1a800563b0 con 0x7f1a8815b640 2024-04-23T14:34:21.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.139+0000 7f1a8ce00640 1 -- 172.21.15.100:0/3588449401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1a88158440 con 0x7f1a8815b640 2024-04-23T14:34:21.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.139+0000 7f1a7e000640 1 -- 172.21.15.100:0/3588449401 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5298 (secure 0 0 0) 0x7f1a8005a6b0 con 0x7f1a8815b640 2024-04-23T14:34:21.147 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:21.147 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:34:21.147 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_dump": [ 2024-04-23T14:34:21.147 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:34:21.147 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.a", 2024-04-23T14:34:21.147 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtxidmcgr8GBAA944tuhsKzhnsmZrdxOKbAA==", 2024-04-23T14:34:21.147 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:34:21.147 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-04-23T14:34:21.147 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-04-23T14:34:21.147 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-04-23T14:34:21.147 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:34:21.147 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:21.148 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:21.148 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:34:21.148 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.b", 2024-04-23T14:34:21.148 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtxidmJcqnJBAA95zJpqTJW92I+2Nnh7SpXg==", 2024-04-23T14:34:21.148 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:34:21.148 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-04-23T14:34:21.148 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-04-23T14:34:21.148 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-04-23T14:34:21.148 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:34:21.148 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:21.148 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:21.148 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:34:21.148 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.c", 2024-04-23T14:34:21.148 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtxidmrYjvHxAABsf7ouy0ULqAFxwPrLCMsg==", 2024-04-23T14:34:21.148 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:34:21.149 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-04-23T14:34:21.149 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-04-23T14:34:21.149 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-04-23T14:34:21.149 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:34:21.149 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:21.149 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:21.149 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:34:21.149 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.d", 2024-04-23T14:34:21.149 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtxidmmN4JLBAANawA9RtKrB31GqCz/wuz3Q==", 2024-04-23T14:34:21.149 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:34:21.149 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-04-23T14:34:21.149 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-04-23T14:34:21.149 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-04-23T14:34:21.149 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:34:21.149 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:21.149 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:21.150 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:34:21.150 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.0", 2024-04-23T14:34:21.150 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAwxidmtJJBMBAAktb3nILtB6ywDU5lca43OQ==", 2024-04-23T14:34:21.150 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:34:21.150 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-04-23T14:34:21.150 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-04-23T14:34:21.150 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:34:21.150 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:21.150 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:21.150 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:34:21.150 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.1", 2024-04-23T14:34:21.150 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAzxidmUgz/ORAASX3vikkabjnWpzY1efthig==", 2024-04-23T14:34:21.150 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:34:21.150 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-04-23T14:34:21.151 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-04-23T14:34:21.151 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:34:21.151 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:21.151 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:21.151 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:34:21.151 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.2", 2024-04-23T14:34:21.151 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA3xidmUzrNBhAAI8ymZvKTD5dm+gLYh42WMA==", 2024-04-23T14:34:21.151 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:34:21.151 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-04-23T14:34:21.151 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-04-23T14:34:21.151 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:34:21.151 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:21.151 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:21.151 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:34:21.151 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.3", 2024-04-23T14:34:21.151 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA6xidmfEC8EBAAiv84aaTlpMGaG6ZFxKp2ng==", 2024-04-23T14:34:21.152 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:34:21.152 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-04-23T14:34:21.152 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-04-23T14:34:21.152 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:34:21.152 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:21.152 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:21.152 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:34:21.152 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.4", 2024-04-23T14:34:21.152 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBAxidmWoujCBAAclplSDUC/Hwhh4a4NWgH/Q==", 2024-04-23T14:34:21.152 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:34:21.152 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-04-23T14:34:21.152 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-04-23T14:34:21.152 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:34:21.152 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:21.152 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:21.152 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:34:21.153 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.5", 2024-04-23T14:34:21.153 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBDxidmRO+eEhAADfwa1qucvA1FGwuIY0A62w==", 2024-04-23T14:34:21.153 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:34:21.153 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-04-23T14:34:21.153 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-04-23T14:34:21.153 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:34:21.153 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:21.153 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:21.153 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:34:21.153 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.6", 2024-04-23T14:34:21.153 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBGxidmMujsHBAAKNjjM4nlyaUVesLu1CixhA==", 2024-04-23T14:34:21.153 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:34:21.153 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-04-23T14:34:21.153 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-04-23T14:34:21.154 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:34:21.154 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:21.154 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:21.154 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:34:21.154 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.7", 2024-04-23T14:34:21.154 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBJxidmHxlnJxAAk69hiCwS8GrIPMcLV+iVpg==", 2024-04-23T14:34:21.154 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:34:21.154 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-04-23T14:34:21.154 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-04-23T14:34:21.154 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:34:21.154 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:21.154 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:21.154 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:34:21.154 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.0", 2024-04-23T14:34:21.154 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtxidmyhImMBAAa49ApQEF2Lng6rY94F1HAQ==", 2024-04-23T14:34:21.155 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:34:21.155 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-04-23T14:34:21.155 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow r", 2024-04-23T14:34:21.155 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rw pool=cephfs_data" 2024-04-23T14:34:21.155 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:21.155 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:21.155 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:34:21.155 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.1", 2024-04-23T14:34:21.155 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtxidmpLhsORAAFLucAGu/GbEJTmkxy0LY8Q==", 2024-04-23T14:34:21.155 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:34:21.155 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-04-23T14:34:21.155 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-04-23T14:34:21.155 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-04-23T14:34:21.155 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-04-23T14:34:21.155 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:21.155 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:21.156 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:34:21.156 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.2", 2024-04-23T14:34:21.156 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtxidm34okNhAAkagR1bwlKj7wxsY/E/nRGg==", 2024-04-23T14:34:21.156 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:34:21.156 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-04-23T14:34:21.156 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-04-23T14:34:21.156 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-04-23T14:34:21.156 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-04-23T14:34:21.156 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:21.156 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:21.156 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:34:21.156 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.3", 2024-04-23T14:34:21.156 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAuxidmtJWrAxAAYBGSngF85+O1YO5on41OMw==", 2024-04-23T14:34:21.156 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:34:21.157 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-04-23T14:34:21.157 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-04-23T14:34:21.157 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-04-23T14:34:21.157 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-04-23T14:34:21.157 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:21.157 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:21.157 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:34:21.157 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.admin", 2024-04-23T14:34:21.157 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsxidmumdhOBAAa9JaVIetaPBZmY7B4sayLQ==", 2024-04-23T14:34:21.157 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:34:21.157 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-04-23T14:34:21.157 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-04-23T14:34:21.157 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-04-23T14:34:21.157 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:34:21.157 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:21.158 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:21.158 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:34:21.158 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.x", 2024-04-23T14:34:21.158 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtxidmKdCRFBAA4h8/t4aNmcR58fKgXGUB4Q==", 2024-04-23T14:34:21.158 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:34:21.158 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-04-23T14:34:21.158 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-04-23T14:34:21.158 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:34:21.158 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:21.158 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:21.158 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:34:21.158 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.y", 2024-04-23T14:34:21.158 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtxidmNTL4EBAAj88L1GzandAkaqg9AxEQqg==", 2024-04-23T14:34:21.158 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-04-23T14:34:21.158 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-04-23T14:34:21.158 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-04-23T14:34:21.159 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-04-23T14:34:21.159 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:21.159 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:21.159 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-04-23T14:34:21.159 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:34:21.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.143+0000 7f1a8ce00640 1 -- 172.21.15.100:0/3588449401 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1a68068380 msgr2=0x7f1a6806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:21.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.143+0000 7f1a8ce00640 1 --2- 172.21.15.100:0/3588449401 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1a68068380 0x7f1a6806a840 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1a780af000 tx=0x7f1a780a7070 comp rx=0 tx=0).stop 2024-04-23T14:34:21.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.143+0000 7f1a8ce00640 1 -- 172.21.15.100:0/3588449401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a8815b640 msgr2=0x7f1a881f79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:21.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.143+0000 7f1a8ce00640 1 --2- 172.21.15.100:0/3588449401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a8815b640 0x7f1a881f79a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1a8000e5b0 tx=0x7f1a8000ea70 comp rx=0 tx=0).stop 2024-04-23T14:34:21.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.143+0000 7f1a8ce00640 1 -- 172.21.15.100:0/3588449401 shutdown_connections 2024-04-23T14:34:21.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.143+0000 7f1a8ce00640 1 --2- 172.21.15.100:0/3588449401 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1a68068380 0x7f1a6806a840 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.143+0000 7f1a8ce00640 1 --2- 172.21.15.100:0/3588449401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a8815b640 0x7f1a881f79a0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.143+0000 7f1a8ce00640 1 --2- 172.21.15.100:0/3588449401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a880b2a10 0x7f1a881487f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.143+0000 7f1a8ce00640 1 --2- 172.21.15.100:0/3588449401 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a88073610 0x7f1a8814aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.143+0000 7f1a8ce00640 1 -- 172.21.15.100:0/3588449401 >> 172.21.15.100:0/3588449401 conn(0x7f1a880b1e80 msgr2=0x7f1a88158960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:21.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.143+0000 7f1a8ce00640 1 -- 172.21.15.100:0/3588449401 shutdown_connections 2024-04-23T14:34:21.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.143+0000 7f1a8ce00640 1 -- 172.21.15.100:0/3588449401 wait complete. 2024-04-23T14:34:21.167 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:21.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.295+0000 7f1912c00640 1 Processor -- start 2024-04-23T14:34:21.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.295+0000 7f1912c00640 1 -- start start 2024-04-23T14:34:21.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.299+0000 7f1912c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f190c072e70 0x7f190c0af390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:21.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.299+0000 7f1912c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f190c0af970 0x7f190c0adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:21.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.299+0000 7f1912c00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f190c06d130 con 0x7f190c15ad40 2024-04-23T14:34:21.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.299+0000 7f1912c00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f190c0ad720 con 0x7f190c0af970 2024-04-23T14:34:21.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.299+0000 7f1912c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f190c06d410 con 0x7f190c072e70 2024-04-23T14:34:21.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.299+0000 7f190be00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f190c15ad40 0x7f190c072760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48704/0 (socket says 172.21.15.100:48704) 2024-04-23T14:34:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.299+0000 7f190be00640 1 -- 172.21.15.100:0/3093366482 learned_addr learned my addr 172.21.15.100:0/3093366482 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.299+0000 7f1910c00640 1 --2- 172.21.15.100:0/3093366482 >> v2:172.21.15.100:3300/0 conn(0x7f190c0af970 0x7f190c0adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.299+0000 7f190b400640 1 --2- 172.21.15.100:0/3093366482 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f190c072e70 0x7f190c0af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.299+0000 7f190aa00640 1 -- 172.21.15.100:0/3093366482 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3768586404 0 0) 0x7f190c06d130 con 0x7f190c15ad40 2024-04-23T14:34:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.299+0000 7f190aa00640 1 -- 172.21.15.100:0/3093366482 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18f0005740 con 0x7f190c15ad40 2024-04-23T14:34:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.299+0000 7f1910c00640 1 -- 172.21.15.100:0/3093366482 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f190c072e70 msgr2=0x7f190c0af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.299+0000 7f1910c00640 1 --2- 172.21.15.100:0/3093366482 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f190c072e70 0x7f190c0af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.299+0000 7f1910c00640 1 -- 172.21.15.100:0/3093366482 >> v1:172.21.15.100:6789/0 conn(0x7f190c15ad40 legacy=0x7f190c072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:21.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.299+0000 7f1910c00640 1 -- 172.21.15.100:0/3093366482 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190c071e10 con 0x7f190c0af970 2024-04-23T14:34:21.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.303+0000 7f1910c00640 1 --2- 172.21.15.100:0/3093366482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c0af970 0x7f190c0adb90 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1904009d90 tx=0x7f190401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d23b962d432f7d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:21.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.303+0000 7f190aa00640 1 -- 172.21.15.100:0/3093366482 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19040b2070 con 0x7f190c0af970 2024-04-23T14:34:21.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.303+0000 7f190aa00640 1 -- 172.21.15.100:0/3093366482 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19040a5830 con 0x7f190c0af970 2024-04-23T14:34:21.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.303+0000 7f190aa00640 1 -- 172.21.15.100:0/3093366482 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19040a5c00 con 0x7f190c0af970 2024-04-23T14:34:21.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.303+0000 7f1912c00640 1 -- 172.21.15.100:0/3093366482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c0af970 msgr2=0x7f190c0adb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:21.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.303+0000 7f1912c00640 1 --2- 172.21.15.100:0/3093366482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c0af970 0x7f190c0adb90 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1904009d90 tx=0x7f190401c8e0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.303+0000 7f1912c00640 1 -- 172.21.15.100:0/3093366482 shutdown_connections 2024-04-23T14:34:21.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.303+0000 7f1912c00640 1 --2- 172.21.15.100:0/3093366482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c0af970 0x7f190c0adb90 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.303+0000 7f1912c00640 1 --2- 172.21.15.100:0/3093366482 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f190c072e70 0x7f190c0af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.303+0000 7f1912c00640 1 -- 172.21.15.100:0/3093366482 >> 172.21.15.100:0/3093366482 conn(0x7f190c0b1e80 msgr2=0x7f190c0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:21.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.303+0000 7f1912c00640 1 -- 172.21.15.100:0/3093366482 shutdown_connections 2024-04-23T14:34:21.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.303+0000 7f1912c00640 1 -- 172.21.15.100:0/3093366482 wait complete. 2024-04-23T14:34:21.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.303+0000 7f1912c00640 1 Processor -- start 2024-04-23T14:34:21.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.303+0000 7f1912c00640 1 -- start start 2024-04-23T14:34:21.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.303+0000 7f1912c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f190c072e70 0x7f190c146b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:21.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.303+0000 7f1912c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f190c0af970 0x7f190c14c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:21.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.303+0000 7f1912c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c15ad40 0x7f190c14ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:21.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.303+0000 7f1912c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f190c0ae460 con 0x7f190c15ad40 2024-04-23T14:34:21.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.303+0000 7f1912c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f190c0ae2e0 con 0x7f190c0af970 2024-04-23T14:34:21.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.303+0000 7f1912c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f190c0ae5e0 con 0x7f190c072e70 2024-04-23T14:34:21.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.303+0000 7f1910c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c15ad40 0x7f190c14ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:21.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.307+0000 7f1910c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c15ad40 0x7f190c14ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57128/0 (socket says 172.21.15.100:57128) 2024-04-23T14:34:21.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.307+0000 7f1910c00640 1 -- 172.21.15.100:0/20807392 learned_addr learned my addr 172.21.15.100:0/20807392 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:21.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.307+0000 7f190b400640 1 --2- 172.21.15.100:0/20807392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f190c0af970 0x7f190c14c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:21.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.307+0000 7f190be00640 1 --2- 172.21.15.100:0/20807392 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f190c072e70 0x7f190c146b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:21.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.307+0000 7f1910c00640 1 -- 172.21.15.100:0/20807392 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f190c072e70 msgr2=0x7f190c146b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:21.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.307+0000 7f1910c00640 1 --2- 172.21.15.100:0/20807392 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f190c072e70 0x7f190c146b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.307+0000 7f1910c00640 1 -- 172.21.15.100:0/20807392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f190c0af970 msgr2=0x7f190c14c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:21.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.307+0000 7f1910c00640 1 --2- 172.21.15.100:0/20807392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f190c0af970 0x7f190c14c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.307+0000 7f1910c00640 1 -- 172.21.15.100:0/20807392 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190c152280 con 0x7f190c15ad40 2024-04-23T14:34:21.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.307+0000 7f190b400640 1 --2- 172.21.15.100:0/20807392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f190c0af970 0x7f190c14c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:21.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.307+0000 7f190be00640 1 --2- 172.21.15.100:0/20807392 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f190c072e70 0x7f190c146b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:21.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.307+0000 7f1910c00640 1 --2- 172.21.15.100:0/20807392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c15ad40 0x7f190c14ed80 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f19040b0040 tx=0x7f1904019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:21.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.307+0000 7f1903e00640 1 -- 172.21.15.100:0/20807392 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19040b2050 con 0x7f190c15ad40 2024-04-23T14:34:21.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.307+0000 7f1903e00640 1 -- 172.21.15.100:0/20807392 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19040be070 con 0x7f190c15ad40 2024-04-23T14:34:21.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.307+0000 7f1903e00640 1 -- 172.21.15.100:0/20807392 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19040ad890 con 0x7f190c15ad40 2024-04-23T14:34:21.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.307+0000 7f1912c00640 1 -- 172.21.15.100:0/20807392 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f190c152560 con 0x7f190c15ad40 2024-04-23T14:34:21.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.307+0000 7f1912c00640 1 -- 172.21.15.100:0/20807392 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f190c152b20 con 0x7f190c15ad40 2024-04-23T14:34:21.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.307+0000 7f1912c00640 1 -- 172.21.15.100:0/20807392 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f190c068ad0 con 0x7f190c15ad40 2024-04-23T14:34:21.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.307+0000 7f1903e00640 1 -- 172.21.15.100:0/20807392 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f19040ada30 con 0x7f190c15ad40 2024-04-23T14:34:21.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.307+0000 7f1903e00640 1 --2- 172.21.15.100:0/20807392 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f18ec068380 0x7f18ec06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:21.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.307+0000 7f1903e00640 1 -- 172.21.15.100:0/20807392 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5249+0+0 (secure 0 0 0) 0x7f1904121790 con 0x7f190c15ad40 2024-04-23T14:34:21.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.307+0000 7f190be00640 1 --2- 172.21.15.100:0/20807392 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f18ec068380 0x7f18ec06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:21.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.311+0000 7f190be00640 1 --2- 172.21.15.100:0/20807392 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f18ec068380 0x7f18ec06a840 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f18fc005890 tx=0x7f18fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:21.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.311+0000 7f1903e00640 1 -- 172.21.15.100:0/20807392 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1904016560 con 0x7f190c15ad40 2024-04-23T14:34:21.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.471+0000 7f1912c00640 1 -- 172.21.15.100:0/20807392 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f190c0b2a30 con 0x7f190c15ad40 2024-04-23T14:34:21.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.471+0000 7f1903e00640 1 -- 172.21.15.100:0/20807392 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+13176 (secure 0 0 0) 0x7f19040f1f30 con 0x7f190c15ad40 2024-04-23T14:34:21.479 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:21.479 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":49,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:34:20.760184+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:21.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.471+0000 7f1912c00640 1 -- 172.21.15.100:0/20807392 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f18ec068380 msgr2=0x7f18ec06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:21.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.471+0000 7f1912c00640 1 --2- 172.21.15.100:0/20807392 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f18ec068380 0x7f18ec06a840 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f18fc005890 tx=0x7f18fc009040 comp rx=0 tx=0).stop 2024-04-23T14:34:21.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.471+0000 7f1912c00640 1 -- 172.21.15.100:0/20807392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c15ad40 msgr2=0x7f190c14ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:21.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.471+0000 7f1912c00640 1 --2- 172.21.15.100:0/20807392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c15ad40 0x7f190c14ed80 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f19040b0040 tx=0x7f1904019680 comp rx=0 tx=0).stop 2024-04-23T14:34:21.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.471+0000 7f1912c00640 1 -- 172.21.15.100:0/20807392 shutdown_connections 2024-04-23T14:34:21.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.471+0000 7f1912c00640 1 --2- 172.21.15.100:0/20807392 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f18ec068380 0x7f18ec06a840 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.471+0000 7f1912c00640 1 --2- 172.21.15.100:0/20807392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c15ad40 0x7f190c14ed80 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.471+0000 7f1912c00640 1 --2- 172.21.15.100:0/20807392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f190c0af970 0x7f190c14c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.471+0000 7f1912c00640 1 --2- 172.21.15.100:0/20807392 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f190c072e70 0x7f190c146b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.471+0000 7f1912c00640 1 -- 172.21.15.100:0/20807392 >> 172.21.15.100:0/20807392 conn(0x7f190c0b1e80 msgr2=0x7f190c06f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:21.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.471+0000 7f1912c00640 1 -- 172.21.15.100:0/20807392 shutdown_connections 2024-04-23T14:34:21.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.471+0000 7f1912c00640 1 -- 172.21.15.100:0/20807392 wait complete. 2024-04-23T14:34:21.498 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:34:21.499 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:34:21.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.631+0000 7f0543c00640 1 Processor -- start 2024-04-23T14:34:21.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.631+0000 7f0543c00640 1 -- start start 2024-04-23T14:34:21.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.635+0000 7f0543c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f053c0730b0 0x7f053c0ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:21.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.635+0000 7f0543c00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f053c06fe90 con 0x7f053c06d5b0 2024-04-23T14:34:21.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.635+0000 7f0543c00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f053c0703e0 con 0x7f053c06d990 2024-04-23T14:34:21.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.635+0000 7f0543c00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f053c06f610 con 0x7f053c0730b0 2024-04-23T14:34:21.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.635+0000 7f053be00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f053c0730b0 0x7f053c0ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:21.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.635+0000 7f053be00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f053c0730b0 0x7f053c0ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57142/0 (socket says 172.21.15.100:57142) 2024-04-23T14:34:21.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.635+0000 7f0541200640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f053c06d5b0 0x7f053c0cdf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48714/0 (socket says 172.21.15.100:48714) 2024-04-23T14:34:21.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.635+0000 7f053be00640 1 -- 172.21.15.100:0/3842727194 learned_addr learned my addr 172.21.15.100:0/3842727194 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:21.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.635+0000 7f053be00640 1 -- 172.21.15.100:0/3842727194 >> v1:172.21.15.151:6789/0 conn(0x7f053c06d990 legacy=0x7f053c0cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:21.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.635+0000 7f053be00640 1 -- 172.21.15.100:0/3842727194 >> v1:172.21.15.100:6789/0 conn(0x7f053c06d5b0 legacy=0x7f053c0cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:21.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.635+0000 7f053be00640 1 -- 172.21.15.100:0/3842727194 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053c0afff0 con 0x7f053c0730b0 2024-04-23T14:34:21.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.635+0000 7f053b400640 1 -- 172.21.15.100:0/3842727194 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 136914967 0 0) 0x7f053c06fe90 con 0x7f053c06d5b0 2024-04-23T14:34:21.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.635+0000 7f053be00640 1 --2- 172.21.15.100:0/3842727194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f053c0730b0 0x7f053c0ccca0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0528009870 tx=0x7f052801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c08c1bb95ab92c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:21.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f053b400640 1 -- 172.21.15.100:0/3842727194 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05280b0070 con 0x7f053c0730b0 2024-04-23T14:34:21.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f053b400640 1 -- 172.21.15.100:0/3842727194 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05280ab600 con 0x7f053c0730b0 2024-04-23T14:34:21.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f053b400640 1 -- 172.21.15.100:0/3842727194 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05280ab9d0 con 0x7f053c0730b0 2024-04-23T14:34:21.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f0543c00640 1 -- 172.21.15.100:0/3842727194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f053c0730b0 msgr2=0x7f053c0ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:21.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f0543c00640 1 --2- 172.21.15.100:0/3842727194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f053c0730b0 0x7f053c0ccca0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0528009870 tx=0x7f052801c520 comp rx=0 tx=0).stop 2024-04-23T14:34:21.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f0543c00640 1 -- 172.21.15.100:0/3842727194 shutdown_connections 2024-04-23T14:34:21.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f0543c00640 1 --2- 172.21.15.100:0/3842727194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f053c0730b0 0x7f053c0ccca0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f0543c00640 1 -- 172.21.15.100:0/3842727194 >> 172.21.15.100:0/3842727194 conn(0x7f053c072590 msgr2=0x7f053c0729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:21.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f0543c00640 1 -- 172.21.15.100:0/3842727194 shutdown_connections 2024-04-23T14:34:21.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f0543c00640 1 -- 172.21.15.100:0/3842727194 wait complete. 2024-04-23T14:34:21.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f0543c00640 1 Processor -- start 2024-04-23T14:34:21.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f0543c00640 1 -- start start 2024-04-23T14:34:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f0543c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f053c06d5b0 0x7f053c1565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f0543c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f053c06d990 0x7f053c147980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f0543c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f053c0730b0 0x7f053c1540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f0543c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f053c16a7d0 con 0x7f053c0730b0 2024-04-23T14:34:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f0543c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f053c16a650 con 0x7f053c06d990 2024-04-23T14:34:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f0543c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f053c16a950 con 0x7f053c06d5b0 2024-04-23T14:34:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f0541c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f053c0730b0 0x7f053c1540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f0541c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f053c0730b0 0x7f053c1540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57150/0 (socket says 172.21.15.100:57150) 2024-04-23T14:34:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f0541c00640 1 -- 172.21.15.100:0/1173870815 learned_addr learned my addr 172.21.15.100:0/1173870815 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f0541200640 1 --2- 172.21.15.100:0/1173870815 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f053c06d5b0 0x7f053c1565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.639+0000 7f053be00640 1 --2- 172.21.15.100:0/1173870815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f053c06d990 0x7f053c147980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.643+0000 7f0541c00640 1 -- 172.21.15.100:0/1173870815 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f053c06d5b0 msgr2=0x7f053c1565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.643+0000 7f0541c00640 1 --2- 172.21.15.100:0/1173870815 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f053c06d5b0 0x7f053c1565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.643+0000 7f0541c00640 1 -- 172.21.15.100:0/1173870815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f053c06d990 msgr2=0x7f053c147980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:21.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.643+0000 7f0541c00640 1 --2- 172.21.15.100:0/1173870815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f053c06d990 0x7f053c147980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.643+0000 7f0541c00640 1 -- 172.21.15.100:0/1173870815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053c1489b0 con 0x7f053c0730b0 2024-04-23T14:34:21.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.643+0000 7f0541c00640 1 --2- 172.21.15.100:0/1173870815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f053c0730b0 0x7f053c1540b0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f053400c3c0 tx=0x7f053400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:21.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.643+0000 7f0538c00640 1 -- 172.21.15.100:0/1173870815 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0534011790 con 0x7f053c0730b0 2024-04-23T14:34:21.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.643+0000 7f0538c00640 1 -- 172.21.15.100:0/1173870815 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0534017070 con 0x7f053c0730b0 2024-04-23T14:34:21.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.643+0000 7f0543c00640 1 -- 172.21.15.100:0/1173870815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f053c158bb0 con 0x7f053c0730b0 2024-04-23T14:34:21.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.643+0000 7f0538c00640 1 -- 172.21.15.100:0/1173870815 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0534011b70 con 0x7f053c0730b0 2024-04-23T14:34:21.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.643+0000 7f0543c00640 1 -- 172.21.15.100:0/1173870815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f053c159170 con 0x7f053c0730b0 2024-04-23T14:34:21.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.643+0000 7f0543c00640 1 -- 172.21.15.100:0/1173870815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f053c068ad0 con 0x7f053c0730b0 2024-04-23T14:34:21.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.643+0000 7f0538c00640 1 -- 172.21.15.100:0/1173870815 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0534011d10 con 0x7f053c0730b0 2024-04-23T14:34:21.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.643+0000 7f0538c00640 1 --2- 172.21.15.100:0/1173870815 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f051c0683b0 0x7f051c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:21.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.643+0000 7f0538c00640 1 -- 172.21.15.100:0/1173870815 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5249+0+0 (secure 0 0 0) 0x7f05340863e0 con 0x7f053c0730b0 2024-04-23T14:34:21.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.643+0000 7f0541200640 1 --2- 172.21.15.100:0/1173870815 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f051c0683b0 0x7f051c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:21.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.647+0000 7f0541200640 1 --2- 172.21.15.100:0/1173870815 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f051c0683b0 0x7f051c06a870 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f052c0012f0 tx=0x7f052c001690 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:21.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.647+0000 7f0538c00640 1 -- 172.21.15.100:0/1173870815 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f05340565a0 con 0x7f053c0730b0 2024-04-23T14:34:21.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.811+0000 7f0543c00640 1 -- 172.21.15.100:0/1173870815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f053c073840 con 0x7f053c0730b0 2024-04-23T14:34:21.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.811+0000 7f0538c00640 1 -- 172.21.15.100:0/1173870815 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) ==== 93+0+10 (secure 0 0 0) 0x7f053405a8a0 con 0x7f053c0730b0 2024-04-23T14:34:21.819 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:34:21.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.815+0000 7f0543c00640 1 -- 172.21.15.100:0/1173870815 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f051c0683b0 msgr2=0x7f051c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:21.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.815+0000 7f0543c00640 1 --2- 172.21.15.100:0/1173870815 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f051c0683b0 0x7f051c06a870 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f052c0012f0 tx=0x7f052c001690 comp rx=0 tx=0).stop 2024-04-23T14:34:21.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.815+0000 7f0543c00640 1 -- 172.21.15.100:0/1173870815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f053c0730b0 msgr2=0x7f053c1540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:21.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.815+0000 7f0543c00640 1 --2- 172.21.15.100:0/1173870815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f053c0730b0 0x7f053c1540b0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f053400c3c0 tx=0x7f053400c880 comp rx=0 tx=0).stop 2024-04-23T14:34:21.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.815+0000 7f0543c00640 1 -- 172.21.15.100:0/1173870815 shutdown_connections 2024-04-23T14:34:21.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.815+0000 7f0543c00640 1 --2- 172.21.15.100:0/1173870815 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f051c0683b0 0x7f051c06a870 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.815+0000 7f0543c00640 1 --2- 172.21.15.100:0/1173870815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f053c0730b0 0x7f053c1540b0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.815+0000 7f0543c00640 1 --2- 172.21.15.100:0/1173870815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f053c06d990 0x7f053c147980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.815+0000 7f0543c00640 1 --2- 172.21.15.100:0/1173870815 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f053c06d5b0 0x7f053c1565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.815+0000 7f0543c00640 1 -- 172.21.15.100:0/1173870815 >> 172.21.15.100:0/1173870815 conn(0x7f053c072590 msgr2=0x7f053c1528d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:21.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.815+0000 7f0543c00640 1 -- 172.21.15.100:0/1173870815 shutdown_connections 2024-04-23T14:34:21.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.815+0000 7f0543c00640 1 -- 172.21.15.100:0/1173870815 wait complete. 2024-04-23T14:34:21.840 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-23T14:34:21.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.967+0000 7f3e26a00640 1 Processor -- start 2024-04-23T14:34:21.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.971+0000 7f3e26a00640 1 -- start start 2024-04-23T14:34:21.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.971+0000 7f3e26a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3e2015c7e0 0x7f3e200d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:21.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.971+0000 7f3e26a00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e200d04a0 con 0x7f3e2015d600 2024-04-23T14:34:21.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.971+0000 7f3e26a00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e200d0780 con 0x7f3e200cca50 2024-04-23T14:34:21.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.971+0000 7f3e26a00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f3e2015db00 con 0x7f3e2015c7e0 2024-04-23T14:34:21.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.971+0000 7f3e1f400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3e2015c7e0 0x7f3e200d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:21.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.971+0000 7f3e24a00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f3e2015d600 0x7f3e200d17a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48716/0 (socket says 172.21.15.100:48716) 2024-04-23T14:34:21.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.971+0000 7f3e24a00640 1 -- 172.21.15.100:0/420592235 learned_addr learned my addr 172.21.15.100:0/420592235 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:21.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.971+0000 7f3e1f400640 1 -- 172.21.15.100:0/420592235 >> v1:172.21.15.151:6789/0 conn(0x7f3e200cca50 legacy=0x7f3e200cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:21.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.971+0000 7f3e1f400640 1 -- 172.21.15.100:0/420592235 >> v1:172.21.15.100:6789/0 conn(0x7f3e2015d600 legacy=0x7f3e200d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:21.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.971+0000 7f3e1f400640 1 -- 172.21.15.100:0/420592235 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e200d0a60 con 0x7f3e2015c7e0 2024-04-23T14:34:21.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.971+0000 7f3e1ea00640 1 -- 172.21.15.100:0/420592235 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 130738110 0 0) 0x7f3e200d04a0 con 0x7f3e2015d600 2024-04-23T14:34:21.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.975+0000 7f3e1f400640 1 --2- 172.21.15.100:0/420592235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e2015c7e0 0x7f3e200d1260 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0097c0 tx=0x7f3e0c01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52722bb5c1601fc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:21.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.975+0000 7f3e1ea00640 1 -- 172.21.15.100:0/420592235 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0b1070 con 0x7f3e2015c7e0 2024-04-23T14:34:21.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.975+0000 7f3e1ea00640 1 -- 172.21.15.100:0/420592235 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e0c0ac620 con 0x7f3e2015c7e0 2024-04-23T14:34:21.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.975+0000 7f3e1ea00640 1 -- 172.21.15.100:0/420592235 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0ac9f0 con 0x7f3e2015c7e0 2024-04-23T14:34:21.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.975+0000 7f3e26a00640 1 -- 172.21.15.100:0/420592235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e2015c7e0 msgr2=0x7f3e200d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:21.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.975+0000 7f3e26a00640 1 --2- 172.21.15.100:0/420592235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e2015c7e0 0x7f3e200d1260 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0097c0 tx=0x7f3e0c01c3a0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.975+0000 7f3e26a00640 1 -- 172.21.15.100:0/420592235 shutdown_connections 2024-04-23T14:34:21.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.975+0000 7f3e26a00640 1 --2- 172.21.15.100:0/420592235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e2015c7e0 0x7f3e200d1260 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.975+0000 7f3e26a00640 1 -- 172.21.15.100:0/420592235 >> 172.21.15.100:0/420592235 conn(0x7f3e200cee80 msgr2=0x7f3e200cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:21.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.975+0000 7f3e26a00640 1 -- 172.21.15.100:0/420592235 shutdown_connections 2024-04-23T14:34:21.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.975+0000 7f3e26a00640 1 -- 172.21.15.100:0/420592235 wait complete. 2024-04-23T14:34:21.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.975+0000 7f3e26a00640 1 Processor -- start 2024-04-23T14:34:21.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.975+0000 7f3e26a00640 1 -- start start 2024-04-23T14:34:21.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e26a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e200cca50 0x7f3e20154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:21.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e26a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e2015c7e0 0x7f3e20151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:21.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e26a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3e2015d600 0x7f3e20153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:21.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e26a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2016ac60 con 0x7f3e200cca50 2024-04-23T14:34:21.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e26a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2016aae0 con 0x7f3e2015c7e0 2024-04-23T14:34:21.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e26a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f3e2016ade0 con 0x7f3e2015d600 2024-04-23T14:34:21.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e1fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e200cca50 0x7f3e20154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:21.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e1fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e200cca50 0x7f3e20154f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57166/0 (socket says 172.21.15.100:57166) 2024-04-23T14:34:21.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e1fe00640 1 -- 172.21.15.100:0/1564857360 learned_addr learned my addr 172.21.15.100:0/1564857360 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:21.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e1f400640 1 --2- 172.21.15.100:0/1564857360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e2015c7e0 0x7f3e20151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:21.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e24a00640 1 --2- 172.21.15.100:0/1564857360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3e2015d600 0x7f3e20153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:21.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e1fe00640 1 -- 172.21.15.100:0/1564857360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3e2015d600 msgr2=0x7f3e20153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:21.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e1fe00640 1 --2- 172.21.15.100:0/1564857360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3e2015d600 0x7f3e20153c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e1fe00640 1 -- 172.21.15.100:0/1564857360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e2015c7e0 msgr2=0x7f3e20151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:21.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e1fe00640 1 --2- 172.21.15.100:0/1564857360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e2015c7e0 0x7f3e20151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:21.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e1fe00640 1 -- 172.21.15.100:0/1564857360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e20155610 con 0x7f3e200cca50 2024-04-23T14:34:21.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e24a00640 1 --2- 172.21.15.100:0/1564857360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3e2015d600 0x7f3e20153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:21.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e1fe00640 1 --2- 172.21.15.100:0/1564857360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e200cca50 0x7f3e20154f30 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000c450 tx=0x7f3e1000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:21.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e17e00640 1 -- 172.21.15.100:0/1564857360 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e10012830 con 0x7f3e200cca50 2024-04-23T14:34:21.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e17e00640 1 -- 172.21.15.100:0/1564857360 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e10017070 con 0x7f3e200cca50 2024-04-23T14:34:21.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e17e00640 1 -- 172.21.15.100:0/1564857360 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e10012c30 con 0x7f3e200cca50 2024-04-23T14:34:21.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e26a00640 1 -- 172.21.15.100:0/1564857360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e201558f0 con 0x7f3e200cca50 2024-04-23T14:34:21.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e26a00640 1 -- 172.21.15.100:0/1564857360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e20155bd0 con 0x7f3e200cca50 2024-04-23T14:34:21.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e26a00640 1 -- 172.21.15.100:0/1564857360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e20068ad0 con 0x7f3e200cca50 2024-04-23T14:34:21.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.979+0000 7f3e17e00640 1 -- 172.21.15.100:0/1564857360 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3e1001c460 con 0x7f3e200cca50 2024-04-23T14:34:21.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.983+0000 7f3e17e00640 1 --2- 172.21.15.100:0/1564857360 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3e000683b0 0x7f3e0006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:21.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.983+0000 7f3e17e00640 1 -- 172.21.15.100:0/1564857360 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5249+0+0 (secure 0 0 0) 0x7f3e100865d0 con 0x7f3e200cca50 2024-04-23T14:34:21.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.983+0000 7f3e1f400640 1 --2- 172.21.15.100:0/1564857360 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3e000683b0 0x7f3e0006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:21.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.983+0000 7f3e1f400640 1 --2- 172.21.15.100:0/1564857360 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3e000683b0 0x7f3e0006a870 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0ae000 tx=0x7f3e0c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:21.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:21.987+0000 7f3e17e00640 1 -- 172.21.15.100:0/1564857360 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3e10056d40 con 0x7f3e200cca50 2024-04-23T14:34:22.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.143+0000 7f3e26a00640 1 -- 172.21.15.100:0/1564857360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3e200b6bc0 con 0x7f3e200cca50 2024-04-23T14:34:22.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.143+0000 7f3e17e00640 1 -- 172.21.15.100:0/1564857360 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v49) ==== 84+0+65 (secure 0 0 0) 0x7f3e10056640 con 0x7f3e200cca50 2024-04-23T14:34:22.150 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:22.150 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-04-23T14:34:22.150 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-23T14:34:22.151 INFO:teuthology.orchestra.run.smithi100.stdout: "poolnum": 1, 2024-04-23T14:34:22.151 INFO:teuthology.orchestra.run.smithi100.stdout: "poolname": ".mgr" 2024-04-23T14:34:22.151 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:22.151 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-04-23T14:34:22.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.147+0000 7f3e26a00640 1 -- 172.21.15.100:0/1564857360 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3e000683b0 msgr2=0x7f3e0006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:22.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.147+0000 7f3e26a00640 1 --2- 172.21.15.100:0/1564857360 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3e000683b0 0x7f3e0006a870 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0ae000 tx=0x7f3e0c0a7000 comp rx=0 tx=0).stop 2024-04-23T14:34:22.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.147+0000 7f3e26a00640 1 -- 172.21.15.100:0/1564857360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e200cca50 msgr2=0x7f3e20154f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:22.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.147+0000 7f3e26a00640 1 --2- 172.21.15.100:0/1564857360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e200cca50 0x7f3e20154f30 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000c450 tx=0x7f3e1000c910 comp rx=0 tx=0).stop 2024-04-23T14:34:22.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.147+0000 7f3e26a00640 1 -- 172.21.15.100:0/1564857360 shutdown_connections 2024-04-23T14:34:22.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.147+0000 7f3e26a00640 1 --2- 172.21.15.100:0/1564857360 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3e000683b0 0x7f3e0006a870 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:22.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.147+0000 7f3e26a00640 1 --2- 172.21.15.100:0/1564857360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3e2015d600 0x7f3e20153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:22.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.147+0000 7f3e26a00640 1 --2- 172.21.15.100:0/1564857360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e2015c7e0 0x7f3e20151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:22.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.147+0000 7f3e26a00640 1 --2- 172.21.15.100:0/1564857360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e200cca50 0x7f3e20154f30 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:22.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.147+0000 7f3e26a00640 1 -- 172.21.15.100:0/1564857360 >> 172.21.15.100:0/1564857360 conn(0x7f3e200cee80 msgr2=0x7f3e2015a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:22.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.147+0000 7f3e26a00640 1 -- 172.21.15.100:0/1564857360 shutdown_connections 2024-04-23T14:34:22.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.147+0000 7f3e26a00640 1 -- 172.21.15.100:0/1564857360 wait complete. 2024-04-23T14:34:22.171 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-23T14:34:22.171 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-23T14:34:22.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.303+0000 7f9ec9200640 1 Processor -- start 2024-04-23T14:34:22.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.303+0000 7f9ec9200640 1 -- start start 2024-04-23T14:34:22.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.303+0000 7f9ec9200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9ec406c6c0 0x7f9ec40ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:22.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.303+0000 7f9ec9200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f9ec40ce620 0x7f9ec40add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:22.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.303+0000 7f9ec9200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ec406dd60 con 0x7f9ec406f8c0 2024-04-23T14:34:22.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.303+0000 7f9ec9200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f9ec40ad050 con 0x7f9ec406c6c0 2024-04-23T14:34:22.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.303+0000 7f9ec9200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f9ec40ad8f0 con 0x7f9ec40ce620 2024-04-23T14:34:22.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.307+0000 7f9ec1400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9ec406c6c0 0x7f9ec40ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:22.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.307+0000 7f9ec1400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9ec406c6c0 0x7f9ec40ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57180/0 (socket says 172.21.15.100:57180) 2024-04-23T14:34:22.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.307+0000 7f9ec1400640 1 -- 172.21.15.100:0/2430206274 learned_addr learned my addr 172.21.15.100:0/2430206274 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:22.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.307+0000 7f9ec1e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f9ec406f8c0 0x7f9ec40d0170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48730/0 (socket says 172.21.15.100:48730) 2024-04-23T14:34:22.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.307+0000 7f9ec2800640 1 --2- 172.21.15.100:0/2430206274 >> v2:172.21.15.151:3300/0 conn(0x7f9ec40ce620 0x7f9ec40add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:22.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.307+0000 7f9ec1400640 1 -- 172.21.15.100:0/2430206274 >> v2:172.21.15.151:3300/0 conn(0x7f9ec40ce620 msgr2=0x7f9ec40add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:22.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.307+0000 7f9ec1400640 1 --2- 172.21.15.100:0/2430206274 >> v2:172.21.15.151:3300/0 conn(0x7f9ec40ce620 0x7f9ec40add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:22.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.307+0000 7f9ec1400640 1 -- 172.21.15.100:0/2430206274 >> v1:172.21.15.100:6789/0 conn(0x7f9ec406f8c0 legacy=0x7f9ec40d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:22.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.307+0000 7f9ec1400640 1 -- 172.21.15.100:0/2430206274 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec40d09c0 con 0x7f9ec406c6c0 2024-04-23T14:34:22.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.307+0000 7f9ec0a00640 1 -- 172.21.15.100:0/2430206274 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3890492837 0 0) 0x7f9eb40014a0 con 0x7f9ec406f8c0 2024-04-23T14:34:22.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.307+0000 7f9ec1400640 1 --2- 172.21.15.100:0/2430206274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ec406c6c0 0x7f9ec40ce0e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40b3120 tx=0x7f9eb001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e302a747ff302 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:22.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.307+0000 7f9ec0a00640 1 -- 172.21.15.100:0/2430206274 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb00ab430 con 0x7f9ec406c6c0 2024-04-23T14:34:22.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.307+0000 7f9ec0a00640 1 -- 172.21.15.100:0/2430206274 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eb00b0070 con 0x7f9ec406c6c0 2024-04-23T14:34:22.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.307+0000 7f9ec0a00640 1 -- 172.21.15.100:0/2430206274 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb00ab7f0 con 0x7f9ec406c6c0 2024-04-23T14:34:22.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec9200640 1 -- 172.21.15.100:0/2430206274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ec406c6c0 msgr2=0x7f9ec40ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:22.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec9200640 1 --2- 172.21.15.100:0/2430206274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ec406c6c0 0x7f9ec40ce0e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40b3120 tx=0x7f9eb001c520 comp rx=0 tx=0).stop 2024-04-23T14:34:22.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec9200640 1 -- 172.21.15.100:0/2430206274 shutdown_connections 2024-04-23T14:34:22.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec9200640 1 --2- 172.21.15.100:0/2430206274 >> v2:172.21.15.151:3300/0 conn(0x7f9ec40ce620 0x7f9ec40add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:22.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec9200640 1 --2- 172.21.15.100:0/2430206274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ec406c6c0 0x7f9ec40ce0e0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:22.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec9200640 1 -- 172.21.15.100:0/2430206274 >> 172.21.15.100:0/2430206274 conn(0x7f9ec40b5200 msgr2=0x7f9ec40b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:22.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec9200640 1 -- 172.21.15.100:0/2430206274 shutdown_connections 2024-04-23T14:34:22.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec9200640 1 -- 172.21.15.100:0/2430206274 wait complete. 2024-04-23T14:34:22.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec9200640 1 Processor -- start 2024-04-23T14:34:22.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec9200640 1 -- start start 2024-04-23T14:34:22.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec9200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ec406c6c0 0x7f9ec41555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:22.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ec406f8c0 0x7f9ec40731f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:22.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec9200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ec40ce620 0x7f9ec4073730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:22.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec9200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec40aee60 con 0x7f9ec406f8c0 2024-04-23T14:34:22.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec9200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec40aece0 con 0x7f9ec406c6c0 2024-04-23T14:34:22.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec9200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9ec4168400 con 0x7f9ec40ce620 2024-04-23T14:34:22.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ec406f8c0 0x7f9ec40731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:22.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ec406f8c0 0x7f9ec40731f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57192/0 (socket says 172.21.15.100:57192) 2024-04-23T14:34:22.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec1400640 1 -- 172.21.15.100:0/2476164622 learned_addr learned my addr 172.21.15.100:0/2476164622 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:22.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec1e00640 1 --2- 172.21.15.100:0/2476164622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ec406c6c0 0x7f9ec41555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:22.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec2800640 1 --2- 172.21.15.100:0/2476164622 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ec40ce620 0x7f9ec4073730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:22.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec1400640 1 -- 172.21.15.100:0/2476164622 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ec40ce620 msgr2=0x7f9ec4073730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:22.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec1400640 1 --2- 172.21.15.100:0/2476164622 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ec40ce620 0x7f9ec4073730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:22.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec1400640 1 -- 172.21.15.100:0/2476164622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ec406c6c0 msgr2=0x7f9ec41555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:22.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec1400640 1 --2- 172.21.15.100:0/2476164622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ec406c6c0 0x7f9ec41555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:22.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec1400640 1 -- 172.21.15.100:0/2476164622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec40c7f60 con 0x7f9ec406f8c0 2024-04-23T14:34:22.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.311+0000 7f9ec2800640 1 --2- 172.21.15.100:0/2476164622 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ec40ce620 0x7f9ec4073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:22.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.315+0000 7f9ec1400640 1 --2- 172.21.15.100:0/2476164622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ec406f8c0 0x7f9ec40731f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00ad040 tx=0x7f9eb001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:22.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.315+0000 7f9eba000640 1 -- 172.21.15.100:0/2476164622 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb00be5f0 con 0x7f9ec406f8c0 2024-04-23T14:34:22.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.315+0000 7f9eba000640 1 -- 172.21.15.100:0/2476164622 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eb00be7b0 con 0x7f9ec406f8c0 2024-04-23T14:34:22.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.315+0000 7f9eba000640 1 -- 172.21.15.100:0/2476164622 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb00beba0 con 0x7f9ec406f8c0 2024-04-23T14:34:22.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.315+0000 7f9ec9200640 1 -- 172.21.15.100:0/2476164622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec40c8240 con 0x7f9ec406f8c0 2024-04-23T14:34:22.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.315+0000 7f9ec9200640 1 -- 172.21.15.100:0/2476164622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec40c8800 con 0x7f9ec406f8c0 2024-04-23T14:34:22.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.315+0000 7f9ec9200640 1 -- 172.21.15.100:0/2476164622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ec40c8a00 con 0x7f9ec406f8c0 2024-04-23T14:34:22.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.315+0000 7f9eba000640 1 -- 172.21.15.100:0/2476164622 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9eb00c7430 con 0x7f9ec406f8c0 2024-04-23T14:34:22.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.315+0000 7f9eba000640 1 --2- 172.21.15.100:0/2476164622 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9ea40683b0 0x7f9ea406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:22.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.315+0000 7f9eba000640 1 -- 172.21.15.100:0/2476164622 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5249+0+0 (secure 0 0 0) 0x7f9eb00a8030 con 0x7f9ec406f8c0 2024-04-23T14:34:22.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.315+0000 7f9ec1e00640 1 --2- 172.21.15.100:0/2476164622 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9ea40683b0 0x7f9ea406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:22.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.315+0000 7f9ec1e00640 1 --2- 172.21.15.100:0/2476164622 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9ea40683b0 0x7f9ea406a870 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4004fd0 tx=0x7f9eb4001260 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:22.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.319+0000 7f9eba000640 1 -- 172.21.15.100:0/2476164622 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9ec40c8a00 con 0x7f9ec406f8c0 2024-04-23T14:34:22.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.475+0000 7f9ec9200640 1 -- 172.21.15.100:0/2476164622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9ec4068ad0 con 0x7f9ec406f8c0 2024-04-23T14:34:22.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.759+0000 7f9eba000640 1 -- 172.21.15.100:0/2476164622 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v50) ==== 138+0+0 (secure 0 0 0) 0x7f9eb0016110 con 0x7f9ec406f8c0 2024-04-23T14:34:22.765 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' created 2024-04-23T14:34:22.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.759+0000 7f9ec9200640 1 -- 172.21.15.100:0/2476164622 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9ea40683b0 msgr2=0x7f9ea406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:22.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.759+0000 7f9ec9200640 1 --2- 172.21.15.100:0/2476164622 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9ea40683b0 0x7f9ea406a870 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4004fd0 tx=0x7f9eb4001260 comp rx=0 tx=0).stop 2024-04-23T14:34:22.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.759+0000 7f9ec9200640 1 -- 172.21.15.100:0/2476164622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ec406f8c0 msgr2=0x7f9ec40731f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:22.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.759+0000 7f9ec9200640 1 --2- 172.21.15.100:0/2476164622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ec406f8c0 0x7f9ec40731f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00ad040 tx=0x7f9eb001ad60 comp rx=0 tx=0).stop 2024-04-23T14:34:22.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.763+0000 7f9ec9200640 1 -- 172.21.15.100:0/2476164622 shutdown_connections 2024-04-23T14:34:22.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.763+0000 7f9ec9200640 1 --2- 172.21.15.100:0/2476164622 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9ea40683b0 0x7f9ea406a870 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:22.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.763+0000 7f9ec9200640 1 --2- 172.21.15.100:0/2476164622 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ec40ce620 0x7f9ec4073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:22.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.763+0000 7f9ec9200640 1 --2- 172.21.15.100:0/2476164622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ec406f8c0 0x7f9ec40731f0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:22.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.763+0000 7f9ec9200640 1 --2- 172.21.15.100:0/2476164622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ec406c6c0 0x7f9ec41555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:22.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.763+0000 7f9ec9200640 1 -- 172.21.15.100:0/2476164622 >> 172.21.15.100:0/2476164622 conn(0x7f9ec40b5200 msgr2=0x7f9ec40b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:22.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.763+0000 7f9ec9200640 1 -- 172.21.15.100:0/2476164622 shutdown_connections 2024-04-23T14:34:22.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.763+0000 7f9ec9200640 1 -- 172.21.15.100:0/2476164622 wait complete. 2024-04-23T14:34:22.792 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-23T14:34:22.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.927+0000 7fdb8e600640 1 Processor -- start 2024-04-23T14:34:22.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.931+0000 7fdb8e600640 1 -- start start 2024-04-23T14:34:22.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.931+0000 7fdb8e600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdb88158d60 0x7fdb8806d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:22.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.931+0000 7fdb8e600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb88160220 con 0x7fdb880d0650 2024-04-23T14:34:22.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.931+0000 7fdb8e600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb88160500 con 0x7fdb880b4130 2024-04-23T14:34:22.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.931+0000 7fdb8e600640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fdb8815ffd0 con 0x7fdb88158d60 2024-04-23T14:34:22.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.931+0000 7fdb8d200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdb88158d60 0x7fdb8806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:22.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.931+0000 7fdb8dc00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fdb880d0650 0x7fdb881612c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48732/0 (socket says 172.21.15.100:48732) 2024-04-23T14:34:22.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.931+0000 7fdb8dc00640 1 -- 172.21.15.100:0/3692016222 learned_addr learned my addr 172.21.15.100:0/3692016222 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:22.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.931+0000 7fdb8d200640 1 -- 172.21.15.100:0/3692016222 >> v1:172.21.15.151:6789/0 conn(0x7fdb880b4130 legacy=0x7fdb880cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:22.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.931+0000 7fdb8d200640 1 -- 172.21.15.100:0/3692016222 >> v1:172.21.15.100:6789/0 conn(0x7fdb880d0650 legacy=0x7fdb881612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:22.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.931+0000 7fdb87400640 1 -- 172.21.15.100:0/3692016222 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2368885085 0 0) 0x7fdb88160220 con 0x7fdb880d0650 2024-04-23T14:34:22.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.931+0000 7fdb8d200640 1 -- 172.21.15.100:0/3692016222 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb881607e0 con 0x7fdb88158d60 2024-04-23T14:34:22.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.935+0000 7fdb8d200640 1 --2- 172.21.15.100:0/3692016222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb88158d60 0x7fdb8806d450 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fdb88160150 tx=0x7fdb7801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e5353aa681f3b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:22.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.935+0000 7fdb87400640 1 -- 172.21.15.100:0/3692016222 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb780b2070 con 0x7fdb88158d60 2024-04-23T14:34:22.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.935+0000 7fdb87400640 1 -- 172.21.15.100:0/3692016222 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb780a58e0 con 0x7fdb88158d60 2024-04-23T14:34:22.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.935+0000 7fdb87400640 1 -- 172.21.15.100:0/3692016222 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb780a5cb0 con 0x7fdb88158d60 2024-04-23T14:34:22.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.935+0000 7fdb8e600640 1 -- 172.21.15.100:0/3692016222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb88158d60 msgr2=0x7fdb8806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:22.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.935+0000 7fdb8e600640 1 --2- 172.21.15.100:0/3692016222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb88158d60 0x7fdb8806d450 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fdb88160150 tx=0x7fdb7801c910 comp rx=0 tx=0).stop 2024-04-23T14:34:22.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.935+0000 7fdb8e600640 1 -- 172.21.15.100:0/3692016222 shutdown_connections 2024-04-23T14:34:22.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.935+0000 7fdb8e600640 1 --2- 172.21.15.100:0/3692016222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb88158d60 0x7fdb8806d450 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:22.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.935+0000 7fdb8e600640 1 -- 172.21.15.100:0/3692016222 >> 172.21.15.100:0/3692016222 conn(0x7fdb880cedf0 msgr2=0x7fdb880cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:22.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.935+0000 7fdb8e600640 1 -- 172.21.15.100:0/3692016222 shutdown_connections 2024-04-23T14:34:22.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.935+0000 7fdb8e600640 1 -- 172.21.15.100:0/3692016222 wait complete. 2024-04-23T14:34:22.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.935+0000 7fdb8e600640 1 Processor -- start 2024-04-23T14:34:22.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.935+0000 7fdb8e600640 1 -- start start 2024-04-23T14:34:22.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb8e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb880b4130 0x7fdb880ce140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:22.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb8e600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb880d0650 0x7fdb880c4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:22.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb8e600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb88158d60 0x7fdb88147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:22.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb8e600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdb880b4740 con 0x7fdb880b4130 2024-04-23T14:34:22.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb8e600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdb880b45c0 con 0x7fdb880d0650 2024-04-23T14:34:22.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb8e600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fdb8816cdb0 con 0x7fdb88158d60 2024-04-23T14:34:22.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb8d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb880b4130 0x7fdb880ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:22.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb8d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb880b4130 0x7fdb880ce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57204/0 (socket says 172.21.15.100:57204) 2024-04-23T14:34:22.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb8d200640 1 -- 172.21.15.100:0/3520841297 learned_addr learned my addr 172.21.15.100:0/3520841297 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:22.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb87e00640 1 --2- 172.21.15.100:0/3520841297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb880d0650 0x7fdb880c4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:22.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb8dc00640 1 --2- 172.21.15.100:0/3520841297 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb88158d60 0x7fdb88147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:22.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb8d200640 1 -- 172.21.15.100:0/3520841297 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb88158d60 msgr2=0x7fdb88147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:22.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb8d200640 1 --2- 172.21.15.100:0/3520841297 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb88158d60 0x7fdb88147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:22.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb8d200640 1 -- 172.21.15.100:0/3520841297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb880d0650 msgr2=0x7fdb880c4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:22.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb8d200640 1 --2- 172.21.15.100:0/3520841297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb880d0650 0x7fdb880c4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:22.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb8d200640 1 -- 172.21.15.100:0/3520841297 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb88162ae0 con 0x7fdb880b4130 2024-04-23T14:34:22.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb8dc00640 1 --2- 172.21.15.100:0/3520841297 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb88158d60 0x7fdb88147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:22.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb8d200640 1 --2- 172.21.15.100:0/3520841297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb880b4130 0x7fdb880ce140 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fdb780b0040 tx=0x7fdb780196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:22.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb84c00640 1 -- 172.21.15.100:0/3520841297 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb780bd020 con 0x7fdb880b4130 2024-04-23T14:34:22.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb84c00640 1 -- 172.21.15.100:0/3520841297 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb780b8750 con 0x7fdb880b4130 2024-04-23T14:34:22.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb8e600640 1 -- 172.21.15.100:0/3520841297 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb88155fa0 con 0x7fdb880b4130 2024-04-23T14:34:22.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb84c00640 1 -- 172.21.15.100:0/3520841297 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb780b2040 con 0x7fdb880b4130 2024-04-23T14:34:22.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb8e600640 1 -- 172.21.15.100:0/3520841297 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb88156240 con 0x7fdb880b4130 2024-04-23T14:34:22.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.939+0000 7fdb8e600640 1 -- 172.21.15.100:0/3520841297 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb88075c20 con 0x7fdb880b4130 2024-04-23T14:34:22.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.943+0000 7fdb84c00640 1 -- 172.21.15.100:0/3520841297 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdb780c0020 con 0x7fdb880b4130 2024-04-23T14:34:22.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.943+0000 7fdb84c00640 1 --2- 172.21.15.100:0/3520841297 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb68068380 0x7fdb6806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:22.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.943+0000 7fdb84c00640 1 -- 172.21.15.100:0/3520841297 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5619+0+0 (secure 0 0 0) 0x7fdb78121930 con 0x7fdb880b4130 2024-04-23T14:34:22.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.943+0000 7fdb87e00640 1 --2- 172.21.15.100:0/3520841297 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb68068380 0x7fdb6806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:22.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.943+0000 7fdb87e00640 1 --2- 172.21.15.100:0/3520841297 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb68068380 0x7fdb6806a840 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdb880d0300 tx=0x7fdb74009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:22.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:22.947+0000 7fdb84c00640 1 -- 172.21.15.100:0/3520841297 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdb780c7070 con 0x7fdb880b4130 2024-04-23T14:34:23.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.103+0000 7fdb8e600640 1 -- 172.21.15.100:0/3520841297 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fdb880d0430 con 0x7fdb880b4130 2024-04-23T14:34:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.763+0000 7fdb84c00640 1 -- 172.21.15.100:0/3520841297 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v51) ==== 170+0+0 (secure 0 0 0) 0x7fdb78016500 con 0x7fdb880b4130 2024-04-23T14:34:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' created 2024-04-23T14:34:23.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.767+0000 7fdb8e600640 1 -- 172.21.15.100:0/3520841297 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb68068380 msgr2=0x7fdb6806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:23.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.767+0000 7fdb8e600640 1 --2- 172.21.15.100:0/3520841297 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb68068380 0x7fdb6806a840 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdb880d0300 tx=0x7fdb74009040 comp rx=0 tx=0).stop 2024-04-23T14:34:23.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.767+0000 7fdb8e600640 1 -- 172.21.15.100:0/3520841297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb880b4130 msgr2=0x7fdb880ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:23.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.767+0000 7fdb8e600640 1 --2- 172.21.15.100:0/3520841297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb880b4130 0x7fdb880ce140 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fdb780b0040 tx=0x7fdb780196b0 comp rx=0 tx=0).stop 2024-04-23T14:34:23.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.767+0000 7fdb8e600640 1 -- 172.21.15.100:0/3520841297 shutdown_connections 2024-04-23T14:34:23.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.767+0000 7fdb8e600640 1 --2- 172.21.15.100:0/3520841297 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb68068380 0x7fdb6806a840 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:23.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.767+0000 7fdb8e600640 1 --2- 172.21.15.100:0/3520841297 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb88158d60 0x7fdb88147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:23.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.767+0000 7fdb8e600640 1 --2- 172.21.15.100:0/3520841297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb880d0650 0x7fdb880c4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:23.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.767+0000 7fdb8e600640 1 --2- 172.21.15.100:0/3520841297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb880b4130 0x7fdb880ce140 secure :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fdb780b0040 tx=0x7fdb780196b0 comp rx=0 tx=0).stop 2024-04-23T14:34:23.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.767+0000 7fdb8e600640 1 -- 172.21.15.100:0/3520841297 >> 172.21.15.100:0/3520841297 conn(0x7fdb880cedf0 msgr2=0x7fdb880cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:23.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.767+0000 7fdb8e600640 1 -- 172.21.15.100:0/3520841297 shutdown_connections 2024-04-23T14:34:23.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.771+0000 7fdb8e600640 1 -- 172.21.15.100:0/3520841297 wait complete. 2024-04-23T14:34:23.813 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-23T14:34:23.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.947+0000 7f9ddbc00640 1 Processor -- start 2024-04-23T14:34:23.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.947+0000 7f9ddbc00640 1 -- start start 2024-04-23T14:34:23.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.947+0000 7f9ddbc00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9dd406d450 0x7f9dd406d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:23.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.947+0000 7f9ddbc00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f9dd40cea60 0x7f9dd40d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:23.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.947+0000 7f9ddbc00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dd406dd70 con 0x7f9dd40cd460 2024-04-23T14:34:23.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.947+0000 7f9ddbc00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f9dd40b1f90 con 0x7f9dd406d450 2024-04-23T14:34:23.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.947+0000 7f9ddbc00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f9dd40b2270 con 0x7f9dd40cea60 2024-04-23T14:34:23.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.947+0000 7f9dd9200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9dd406d450 0x7f9dd406d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:23.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.947+0000 7f9dd9200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9dd406d450 0x7f9dd406d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57210/0 (socket says 172.21.15.100:57210) 2024-04-23T14:34:23.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.947+0000 7f9dd9200640 1 -- 172.21.15.100:0/2416416305 learned_addr learned my addr 172.21.15.100:0/2416416305 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:23.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.947+0000 7f9dcbe00640 1 --2- 172.21.15.100:0/2416416305 >> v2:172.21.15.151:3300/0 conn(0x7f9dd40cea60 0x7f9dd40d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:23.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.947+0000 7f9dd9200640 1 -- 172.21.15.100:0/2416416305 >> v2:172.21.15.151:3300/0 conn(0x7f9dd40cea60 msgr2=0x7f9dd40d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:23.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.947+0000 7f9dd9200640 1 --2- 172.21.15.100:0/2416416305 >> v2:172.21.15.151:3300/0 conn(0x7f9dd40cea60 0x7f9dd40d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:23.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.947+0000 7f9dd9200640 1 -- 172.21.15.100:0/2416416305 >> v1:172.21.15.151:6789/0 conn(0x7f9dd40cd460 legacy=0x7f9dd40b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:23.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.947+0000 7f9dd9200640 1 -- 172.21.15.100:0/2416416305 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd40b2550 con 0x7f9dd406d450 2024-04-23T14:34:23.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.947+0000 7f9dcbe00640 1 --2- 172.21.15.100:0/2416416305 >> v2:172.21.15.151:3300/0 conn(0x7f9dd40cea60 0x7f9dd40d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:23.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.951+0000 7f9dd9200640 1 --2- 172.21.15.100:0/2416416305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd406d450 0x7f9dd406d830 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9dd41622c0 tx=0x7f9dc401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb37a9e71a0b60a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:23.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.951+0000 7f9dd3e00640 1 -- 172.21.15.100:0/2416416305 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40b0070 con 0x7f9dd406d450 2024-04-23T14:34:23.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.951+0000 7f9dd3e00640 1 -- 172.21.15.100:0/2416416305 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc40ab600 con 0x7f9dd406d450 2024-04-23T14:34:23.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.951+0000 7f9dd3e00640 1 -- 172.21.15.100:0/2416416305 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40ab9d0 con 0x7f9dd406d450 2024-04-23T14:34:23.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.951+0000 7f9ddbc00640 1 -- 172.21.15.100:0/2416416305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd406d450 msgr2=0x7f9dd406d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:23.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.951+0000 7f9ddbc00640 1 --2- 172.21.15.100:0/2416416305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd406d450 0x7f9dd406d830 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9dd41622c0 tx=0x7f9dc401c450 comp rx=0 tx=0).stop 2024-04-23T14:34:23.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.951+0000 7f9ddbc00640 1 -- 172.21.15.100:0/2416416305 shutdown_connections 2024-04-23T14:34:23.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.951+0000 7f9ddbc00640 1 --2- 172.21.15.100:0/2416416305 >> v2:172.21.15.151:3300/0 conn(0x7f9dd40cea60 0x7f9dd40d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:23.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.951+0000 7f9ddbc00640 1 --2- 172.21.15.100:0/2416416305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd406d450 0x7f9dd406d830 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:23.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.951+0000 7f9ddbc00640 1 -- 172.21.15.100:0/2416416305 >> 172.21.15.100:0/2416416305 conn(0x7f9dd40cee80 msgr2=0x7f9dd40cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:23.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.951+0000 7f9ddbc00640 1 -- 172.21.15.100:0/2416416305 shutdown_connections 2024-04-23T14:34:23.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.951+0000 7f9ddbc00640 1 -- 172.21.15.100:0/2416416305 wait complete. 2024-04-23T14:34:23.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9ddbc00640 1 Processor -- start 2024-04-23T14:34:23.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9ddbc00640 1 -- start start 2024-04-23T14:34:23.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9ddbc00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9dd406d450 0x7f9dd4152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:23.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9ddbc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dd40cd460 0x7f9dd4158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:23.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9ddbc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd40cea60 0x7f9dd41586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:23.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9ddbc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd415fc70 con 0x7f9dd40cea60 2024-04-23T14:34:23.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9ddbc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd415faf0 con 0x7f9dd40cd460 2024-04-23T14:34:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9ddbc00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd416bbc0 con 0x7f9dd406d450 2024-04-23T14:34:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9dd9c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd40cea60 0x7f9dd41586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9dcbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dd40cd460 0x7f9dd4158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9dd9200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9dd406d450 0x7f9dd4152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9dcbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dd40cd460 0x7f9dd4158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:52104/0 (socket says 172.21.15.100:52104) 2024-04-23T14:34:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9dcbe00640 1 -- 172.21.15.100:0/3070989053 learned_addr learned my addr 172.21.15.100:0/3070989053 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9dd9c00640 1 -- 172.21.15.100:0/3070989053 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9dd406d450 msgr2=0x7f9dd4152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9dd9c00640 1 --2- 172.21.15.100:0/3070989053 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9dd406d450 0x7f9dd4152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9dd9c00640 1 -- 172.21.15.100:0/3070989053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dd40cd460 msgr2=0x7f9dd4158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9dd9c00640 1 --2- 172.21.15.100:0/3070989053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dd40cd460 0x7f9dd4158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9dd9c00640 1 -- 172.21.15.100:0/3070989053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd4156990 con 0x7f9dd40cea60 2024-04-23T14:34:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9dd9200640 1 --2- 172.21.15.100:0/3070989053 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9dd406d450 0x7f9dd4152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:34:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9dcbe00640 1 --2- 172.21.15.100:0/3070989053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dd40cd460 0x7f9dd4158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:34:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9dd9c00640 1 --2- 172.21.15.100:0/3070989053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd40cea60 0x7f9dd41586b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00be00 tx=0x7f9dcc00bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:23.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9dd1600640 1 -- 172.21.15.100:0/3070989053 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc003980 con 0x7f9dd40cea60 2024-04-23T14:34:23.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9dd1600640 1 -- 172.21.15.100:0/3070989053 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dcc003b20 con 0x7f9dd40cea60 2024-04-23T14:34:23.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9ddbc00640 1 -- 172.21.15.100:0/3070989053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd4156c70 con 0x7f9dd40cea60 2024-04-23T14:34:23.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9dd1600640 1 -- 172.21.15.100:0/3070989053 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc004020 con 0x7f9dd40cea60 2024-04-23T14:34:23.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.955+0000 7f9ddbc00640 1 -- 172.21.15.100:0/3070989053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd4157230 con 0x7f9dd40cea60 2024-04-23T14:34:23.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.959+0000 7f9ddbc00640 1 -- 172.21.15.100:0/3070989053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd4068ad0 con 0x7f9dd40cea60 2024-04-23T14:34:23.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.959+0000 7f9dd1600640 1 -- 172.21.15.100:0/3070989053 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9dcc006d10 con 0x7f9dd40cea60 2024-04-23T14:34:23.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.959+0000 7f9dd1600640 1 --2- 172.21.15.100:0/3070989053 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9db0068380 0x7f9db006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:23.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.959+0000 7f9dd1600640 1 -- 172.21.15.100:0/3070989053 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6001+0+0 (secure 0 0 0) 0x7f9dcc086fe0 con 0x7f9dd40cea60 2024-04-23T14:34:23.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.959+0000 7f9dd9200640 1 --2- 172.21.15.100:0/3070989053 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9db0068380 0x7f9db006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:23.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.959+0000 7f9dd9200640 1 --2- 172.21.15.100:0/3070989053 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9db0068380 0x7f9db006a840 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40ad000 tx=0x7f9dc40a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:23.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:23.963+0000 7f9dd1600640 1 -- 172.21.15.100:0/3070989053 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9dcc056ed0 con 0x7f9dd40cea60 2024-04-23T14:34:24.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.155+0000 7f9ddbc00640 1 -- 172.21.15.100:0/3070989053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9dd40b2df0 con 0x7f9dd40cea60 2024-04-23T14:34:24.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.775+0000 7f9dd1600640 1 -- 172.21.15.100:0/3070989053 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v24) ==== 172+0+0 (secure 0 0 0) 0x7f9dcc05b1d0 con 0x7f9dd40cea60 2024-04-23T14:34:24.779 INFO:teuthology.orchestra.run.smithi100.stderr:new fs with metadata pool 8 and data pool 9 2024-04-23T14:34:24.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.775+0000 7f9ddbc00640 1 -- 172.21.15.100:0/3070989053 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9db0068380 msgr2=0x7f9db006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:24.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.775+0000 7f9ddbc00640 1 --2- 172.21.15.100:0/3070989053 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9db0068380 0x7f9db006a840 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40ad000 tx=0x7f9dc40a6000 comp rx=0 tx=0).stop 2024-04-23T14:34:24.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.775+0000 7f9ddbc00640 1 -- 172.21.15.100:0/3070989053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd40cea60 msgr2=0x7f9dd41586b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:24.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.775+0000 7f9ddbc00640 1 --2- 172.21.15.100:0/3070989053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd40cea60 0x7f9dd41586b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00be00 tx=0x7f9dcc00bef0 comp rx=0 tx=0).stop 2024-04-23T14:34:24.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.779+0000 7f9ddbc00640 1 -- 172.21.15.100:0/3070989053 shutdown_connections 2024-04-23T14:34:24.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.779+0000 7f9ddbc00640 1 --2- 172.21.15.100:0/3070989053 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9db0068380 0x7f9db006a840 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:24.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.779+0000 7f9ddbc00640 1 --2- 172.21.15.100:0/3070989053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd40cea60 0x7f9dd41586b0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:24.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.779+0000 7f9ddbc00640 1 --2- 172.21.15.100:0/3070989053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dd40cd460 0x7f9dd4158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:24.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.779+0000 7f9ddbc00640 1 --2- 172.21.15.100:0/3070989053 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9dd406d450 0x7f9dd4152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:24.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.779+0000 7f9ddbc00640 1 -- 172.21.15.100:0/3070989053 >> 172.21.15.100:0/3070989053 conn(0x7f9dd40cee80 msgr2=0x7f9dd40d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:24.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.779+0000 7f9ddbc00640 1 -- 172.21.15.100:0/3070989053 shutdown_connections 2024-04-23T14:34:24.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.779+0000 7f9ddbc00640 1 -- 172.21.15.100:0/3070989053 wait complete. 2024-04-23T14:34:24.835 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:24.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.967+0000 7f0634e00640 1 Processor -- start 2024-04-23T14:34:24.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.967+0000 7f0634e00640 1 -- start start 2024-04-23T14:34:24.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.967+0000 7f0634e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f06300cd030 0x7f06300b2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:24.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.967+0000 7f0634e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f063015cb10 0x7f063015cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:24.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.967+0000 7f0634e00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f063015d690 con 0x7f0630071980 2024-04-23T14:34:24.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.967+0000 7f0634e00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f063015d970 con 0x7f063015cb10 2024-04-23T14:34:24.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.967+0000 7f0634e00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f06300b2000 con 0x7f06300cd030 2024-04-23T14:34:24.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.967+0000 7f0625400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f063015cb10 0x7f063015cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:24.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.967+0000 7f0625400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f063015cb10 0x7f063015cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57220/0 (socket says 172.21.15.100:57220) 2024-04-23T14:34:24.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.967+0000 7f0625400640 1 -- 172.21.15.100:0/4265270293 learned_addr learned my addr 172.21.15.100:0/4265270293 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:24.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.967+0000 7f062de00640 1 --2- 172.21.15.100:0/4265270293 >> v2:172.21.15.151:3300/0 conn(0x7f06300cd030 0x7f06300b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:24.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.967+0000 7f0625400640 1 -- 172.21.15.100:0/4265270293 >> v2:172.21.15.151:3300/0 conn(0x7f06300cd030 msgr2=0x7f06300b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:24.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.967+0000 7f0625400640 1 --2- 172.21.15.100:0/4265270293 >> v2:172.21.15.151:3300/0 conn(0x7f06300cd030 0x7f06300b2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:24.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.967+0000 7f0625400640 1 -- 172.21.15.100:0/4265270293 >> v1:172.21.15.151:6789/0 conn(0x7f0630071980 legacy=0x7f0630071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:24.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.967+0000 7f0625400640 1 -- 172.21.15.100:0/4265270293 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06300b22e0 con 0x7f063015cb10 2024-04-23T14:34:24.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.971+0000 7f0625400640 1 --2- 172.21.15.100:0/4265270293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f063015cb10 0x7f063015cfc0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f06300d35a0 tx=0x7f062001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6e397889490c03 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:24.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.971+0000 7f062d400640 1 -- 172.21.15.100:0/4265270293 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06200ab5f0 con 0x7f063015cb10 2024-04-23T14:34:24.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.971+0000 7f062d400640 1 -- 172.21.15.100:0/4265270293 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06200b0070 con 0x7f063015cb10 2024-04-23T14:34:24.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.971+0000 7f062d400640 1 -- 172.21.15.100:0/4265270293 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06200ab9b0 con 0x7f063015cb10 2024-04-23T14:34:24.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.971+0000 7f0634e00640 1 -- 172.21.15.100:0/4265270293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f063015cb10 msgr2=0x7f063015cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:24.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.971+0000 7f0634e00640 1 --2- 172.21.15.100:0/4265270293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f063015cb10 0x7f063015cfc0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f06300d35a0 tx=0x7f062001c4e0 comp rx=0 tx=0).stop 2024-04-23T14:34:24.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.971+0000 7f0634e00640 1 -- 172.21.15.100:0/4265270293 shutdown_connections 2024-04-23T14:34:24.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.971+0000 7f0634e00640 1 --2- 172.21.15.100:0/4265270293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f063015cb10 0x7f063015cfc0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:24.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.971+0000 7f0634e00640 1 --2- 172.21.15.100:0/4265270293 >> v2:172.21.15.151:3300/0 conn(0x7f06300cd030 0x7f06300b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:24.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.971+0000 7f0634e00640 1 -- 172.21.15.100:0/4265270293 >> 172.21.15.100:0/4265270293 conn(0x7f06300d24c0 msgr2=0x7f06300d28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:24.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.971+0000 7f0634e00640 1 -- 172.21.15.100:0/4265270293 shutdown_connections 2024-04-23T14:34:24.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.971+0000 7f0634e00640 1 -- 172.21.15.100:0/4265270293 wait complete. 2024-04-23T14:34:24.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0634e00640 1 Processor -- start 2024-04-23T14:34:24.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0634e00640 1 -- start start 2024-04-23T14:34:24.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0634e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0630071980 0x7f06301561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:24.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0634e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06300cd030 0x7f06300c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:24.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0634e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f063015cb10 0x7f0630159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:24.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0634e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f06300d0a20 con 0x7f06300cd030 2024-04-23T14:34:24.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0634e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f06300d08a0 con 0x7f0630071980 2024-04-23T14:34:24.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0634e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f06300d0ba0 con 0x7f063015cb10 2024-04-23T14:34:24.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0625400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06300cd030 0x7f06300c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:24.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0625400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06300cd030 0x7f06300c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57224/0 (socket says 172.21.15.100:57224) 2024-04-23T14:34:24.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0625400640 1 -- 172.21.15.100:0/3606720762 learned_addr learned my addr 172.21.15.100:0/3606720762 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:24.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f062de00640 1 --2- 172.21.15.100:0/3606720762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0630071980 0x7f06301561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:24.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f062e800640 1 --2- 172.21.15.100:0/3606720762 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f063015cb10 0x7f0630159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:24.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0625400640 1 -- 172.21.15.100:0/3606720762 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f063015cb10 msgr2=0x7f0630159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:24.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0625400640 1 --2- 172.21.15.100:0/3606720762 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f063015cb10 0x7f0630159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:24.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0625400640 1 -- 172.21.15.100:0/3606720762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0630071980 msgr2=0x7f06301561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:24.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0625400640 1 --2- 172.21.15.100:0/3606720762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0630071980 0x7f06301561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:24.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0625400640 1 -- 172.21.15.100:0/3606720762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06300c9ee0 con 0x7f06300cd030 2024-04-23T14:34:24.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0625400640 1 --2- 172.21.15.100:0/3606720762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06300cd030 0x7f06300c9590 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f062001a110 tx=0x7f062001ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:24.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0626a00640 1 -- 172.21.15.100:0/3606720762 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06200be5f0 con 0x7f06300cd030 2024-04-23T14:34:24.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0626a00640 1 -- 172.21.15.100:0/3606720762 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06200be790 con 0x7f06300cd030 2024-04-23T14:34:24.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0626a00640 1 -- 172.21.15.100:0/3606720762 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06200beb40 con 0x7f06300cd030 2024-04-23T14:34:24.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0634e00640 1 -- 172.21.15.100:0/3606720762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06300ca1c0 con 0x7f06300cd030 2024-04-23T14:34:24.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0634e00640 1 -- 172.21.15.100:0/3606720762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06300ca780 con 0x7f06300cd030 2024-04-23T14:34:24.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.975+0000 7f0634e00640 1 -- 172.21.15.100:0/3606720762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0630068ad0 con 0x7f06300cd030 2024-04-23T14:34:24.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.979+0000 7f0626a00640 1 -- 172.21.15.100:0/3606720762 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f06200b43f0 con 0x7f06300cd030 2024-04-23T14:34:24.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.979+0000 7f0626a00640 1 --2- 172.21.15.100:0/3606720762 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f060c068380 0x7f060c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:24.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.979+0000 7f0626a00640 1 -- 172.21.15.100:0/3606720762 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f06201211d0 con 0x7f06300cd030 2024-04-23T14:34:24.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.979+0000 7f062de00640 1 --2- 172.21.15.100:0/3606720762 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f060c068380 0x7f060c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:24.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.979+0000 7f062de00640 1 --2- 172.21.15.100:0/3606720762 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f060c068380 0x7f060c06a840 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f06301fa8d0 tx=0x7f0618008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:24.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:24.983+0000 7f0626a00640 1 -- 172.21.15.100:0/3606720762 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f06200160d0 con 0x7f06300cd030 2024-04-23T14:34:25.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.139+0000 7f0634e00640 1 -- 172.21.15.100:0/3606720762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06300b38d0 con 0x7f06300cd030 2024-04-23T14:34:25.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.139+0000 7f0626a00640 1 -- 172.21.15.100:0/3606720762 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16963 (secure 0 0 0) 0x7f06200f1030 con 0x7f06300cd030 2024-04-23T14:34:25.145 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:25.146 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":52,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:34:24.162572+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:34:22.491379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-23T14:34:23.120342+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:25.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.143+0000 7f0634e00640 1 -- 172.21.15.100:0/3606720762 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f060c068380 msgr2=0x7f060c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:25.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.143+0000 7f0634e00640 1 --2- 172.21.15.100:0/3606720762 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f060c068380 0x7f060c06a840 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f06301fa8d0 tx=0x7f0618008040 comp rx=0 tx=0).stop 2024-04-23T14:34:25.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.143+0000 7f0634e00640 1 -- 172.21.15.100:0/3606720762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06300cd030 msgr2=0x7f06300c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:25.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.143+0000 7f0634e00640 1 --2- 172.21.15.100:0/3606720762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06300cd030 0x7f06300c9590 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f062001a110 tx=0x7f062001ad20 comp rx=0 tx=0).stop 2024-04-23T14:34:25.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.143+0000 7f0634e00640 1 -- 172.21.15.100:0/3606720762 shutdown_connections 2024-04-23T14:34:25.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.143+0000 7f0634e00640 1 --2- 172.21.15.100:0/3606720762 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f060c068380 0x7f060c06a840 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:25.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.143+0000 7f0634e00640 1 --2- 172.21.15.100:0/3606720762 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f063015cb10 0x7f0630159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:25.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.143+0000 7f0634e00640 1 --2- 172.21.15.100:0/3606720762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06300cd030 0x7f06300c9590 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:25.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.143+0000 7f0634e00640 1 --2- 172.21.15.100:0/3606720762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0630071980 0x7f06301561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:25.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.143+0000 7f0634e00640 1 -- 172.21.15.100:0/3606720762 >> 172.21.15.100:0/3606720762 conn(0x7f06300d24c0 msgr2=0x7f06300b7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:25.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.143+0000 7f0634e00640 1 -- 172.21.15.100:0/3606720762 shutdown_connections 2024-04-23T14:34:25.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.143+0000 7f0634e00640 1 -- 172.21.15.100:0/3606720762 wait complete. 2024-04-23T14:34:25.166 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:25.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e46800640 1 Processor -- start 2024-04-23T14:34:25.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e46800640 1 -- start start 2024-04-23T14:34:25.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e46800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0e4015eb10 0x7f0e400d4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:25.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e46800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f0e400b39c0 0x7f0e400d4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:25.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e46800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e400cfc40 con 0x7f0e400d3e40 2024-04-23T14:34:25.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e46800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f0e400d4f90 con 0x7f0e4015eb10 2024-04-23T14:34:25.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e46800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f0e400cfee0 con 0x7f0e400b39c0 2024-04-23T14:34:25.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e3f400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0e4015eb10 0x7f0e400d4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:25.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e3ea00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f0e400d3e40 0x7f0e400d2240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48734/0 (socket says 172.21.15.100:48734) 2024-04-23T14:34:25.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e3ea00640 1 -- 172.21.15.100:0/3015644379 learned_addr learned my addr 172.21.15.100:0/3015644379 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:25.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e3fe00640 1 --2- 172.21.15.100:0/3015644379 >> v2:172.21.15.151:3300/0 conn(0x7f0e400b39c0 0x7f0e400d4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:25.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e3f400640 1 -- 172.21.15.100:0/3015644379 >> v2:172.21.15.151:3300/0 conn(0x7f0e400b39c0 msgr2=0x7f0e400d4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:25.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e3f400640 1 --2- 172.21.15.100:0/3015644379 >> v2:172.21.15.151:3300/0 conn(0x7f0e400b39c0 0x7f0e400d4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:25.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e3f400640 1 -- 172.21.15.100:0/3015644379 >> v1:172.21.15.100:6789/0 conn(0x7f0e400d3e40 legacy=0x7f0e400d2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:25.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e3f400640 1 -- 172.21.15.100:0/3015644379 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e400d01c0 con 0x7f0e4015eb10 2024-04-23T14:34:25.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e3e000640 1 -- 172.21.15.100:0/3015644379 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1845275827 0 0) 0x7f0e400cfc40 con 0x7f0e400d3e40 2024-04-23T14:34:25.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e3fe00640 1 --2- 172.21.15.100:0/3015644379 >> v2:172.21.15.151:3300/0 conn(0x7f0e400b39c0 0x7f0e400d4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:25.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e3f400640 1 --2- 172.21.15.100:0/3015644379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e4015eb10 0x7f0e400d4510 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0e4008e520 tx=0x7f0e3001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed647a4c09a4039c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:25.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e3e000640 1 -- 172.21.15.100:0/3015644379 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e300b2070 con 0x7f0e4015eb10 2024-04-23T14:34:25.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e3e000640 1 -- 172.21.15.100:0/3015644379 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e300a5820 con 0x7f0e4015eb10 2024-04-23T14:34:25.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e3e000640 1 -- 172.21.15.100:0/3015644379 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e300a5bf0 con 0x7f0e4015eb10 2024-04-23T14:34:25.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e46800640 1 -- 172.21.15.100:0/3015644379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e4015eb10 msgr2=0x7f0e400d4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:25.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e46800640 1 --2- 172.21.15.100:0/3015644379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e4015eb10 0x7f0e400d4510 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0e4008e520 tx=0x7f0e3001c930 comp rx=0 tx=0).stop 2024-04-23T14:34:25.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e46800640 1 -- 172.21.15.100:0/3015644379 shutdown_connections 2024-04-23T14:34:25.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e46800640 1 --2- 172.21.15.100:0/3015644379 >> v2:172.21.15.151:3300/0 conn(0x7f0e400b39c0 0x7f0e400d4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:25.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e46800640 1 --2- 172.21.15.100:0/3015644379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e4015eb10 0x7f0e400d4510 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:25.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e46800640 1 -- 172.21.15.100:0/3015644379 >> 172.21.15.100:0/3015644379 conn(0x7f0e400b9a30 msgr2=0x7f0e400b75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:25.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e46800640 1 -- 172.21.15.100:0/3015644379 shutdown_connections 2024-04-23T14:34:25.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.299+0000 7f0e46800640 1 -- 172.21.15.100:0/3015644379 wait complete. 2024-04-23T14:34:25.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e46800640 1 Processor -- start 2024-04-23T14:34:25.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e46800640 1 -- start start 2024-04-23T14:34:25.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e46800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e400b39c0 0x7f0e40154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:25.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e46800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0e4015eb10 0x7f0e40160480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:25.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e46800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e40155c40 0x7f0e401609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:25.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e46800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0e400d55a0 con 0x7f0e40155c40 2024-04-23T14:34:25.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e46800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0e400d5420 con 0x7f0e400b39c0 2024-04-23T14:34:25.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e46800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0e400b12c0 con 0x7f0e4015eb10 2024-04-23T14:34:25.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e3fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e40155c40 0x7f0e401609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:25.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e3fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e40155c40 0x7f0e401609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57240/0 (socket says 172.21.15.100:57240) 2024-04-23T14:34:25.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e3fe00640 1 -- 172.21.15.100:0/2284670646 learned_addr learned my addr 172.21.15.100:0/2284670646 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:25.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e3f400640 1 --2- 172.21.15.100:0/2284670646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e400b39c0 0x7f0e40154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:25.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e3ea00640 1 --2- 172.21.15.100:0/2284670646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0e4015eb10 0x7f0e40160480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:25.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e3fe00640 1 -- 172.21.15.100:0/2284670646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0e4015eb10 msgr2=0x7f0e40160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:25.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e3fe00640 1 --2- 172.21.15.100:0/2284670646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0e4015eb10 0x7f0e40160480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:25.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e3fe00640 1 -- 172.21.15.100:0/2284670646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e400b39c0 msgr2=0x7f0e40154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:25.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e3fe00640 1 --2- 172.21.15.100:0/2284670646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e400b39c0 0x7f0e40154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:25.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e3fe00640 1 -- 172.21.15.100:0/2284670646 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e40154a50 con 0x7f0e40155c40 2024-04-23T14:34:25.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e3fe00640 1 --2- 172.21.15.100:0/2284670646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e40155c40 0x7f0e401609c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800c7f0 tx=0x7f0e3800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:25.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e37400640 1 -- 172.21.15.100:0/2284670646 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e38015070 con 0x7f0e40155c40 2024-04-23T14:34:25.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e37400640 1 -- 172.21.15.100:0/2284670646 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e38007800 con 0x7f0e40155c40 2024-04-23T14:34:25.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e37400640 1 -- 172.21.15.100:0/2284670646 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e38007bf0 con 0x7f0e40155c40 2024-04-23T14:34:25.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e46800640 1 -- 172.21.15.100:0/2284670646 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e40154d30 con 0x7f0e40155c40 2024-04-23T14:34:25.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.303+0000 7f0e46800640 1 -- 172.21.15.100:0/2284670646 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e401552f0 con 0x7f0e40155c40 2024-04-23T14:34:25.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.307+0000 7f0e46800640 1 -- 172.21.15.100:0/2284670646 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e40068ad0 con 0x7f0e40155c40 2024-04-23T14:34:25.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.307+0000 7f0e37400640 1 -- 172.21.15.100:0/2284670646 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0e38018070 con 0x7f0e40155c40 2024-04-23T14:34:25.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.307+0000 7f0e37400640 1 --2- 172.21.15.100:0/2284670646 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0e200683b0 0x7f0e2006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:25.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.307+0000 7f0e37400640 1 -- 172.21.15.100:0/2284670646 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f0e38086c00 con 0x7f0e40155c40 2024-04-23T14:34:25.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.307+0000 7f0e3f400640 1 --2- 172.21.15.100:0/2284670646 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0e200683b0 0x7f0e2006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:25.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.307+0000 7f0e3f400640 1 --2- 172.21.15.100:0/2284670646 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0e200683b0 0x7f0e2006a870 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0e300af000 tx=0x7f0e300a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:25.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.311+0000 7f0e37400640 1 -- 172.21.15.100:0/2284670646 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0e38056940 con 0x7f0e40155c40 2024-04-23T14:34:25.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.471+0000 7f0e46800640 1 -- 172.21.15.100:0/2284670646 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e400cecc0 con 0x7f0e40155c40 2024-04-23T14:34:25.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.471+0000 7f0e37400640 1 -- 172.21.15.100:0/2284670646 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16963 (secure 0 0 0) 0x7f0e3805ac40 con 0x7f0e40155c40 2024-04-23T14:34:25.478 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:25.478 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":52,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:34:24.162572+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:34:22.491379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-23T14:34:23.120342+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:25.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.475+0000 7f0e46800640 1 -- 172.21.15.100:0/2284670646 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0e200683b0 msgr2=0x7f0e2006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:25.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.475+0000 7f0e46800640 1 --2- 172.21.15.100:0/2284670646 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0e200683b0 0x7f0e2006a870 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0e300af000 tx=0x7f0e300a7070 comp rx=0 tx=0).stop 2024-04-23T14:34:25.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.475+0000 7f0e46800640 1 -- 172.21.15.100:0/2284670646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e40155c40 msgr2=0x7f0e401609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:25.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.475+0000 7f0e46800640 1 --2- 172.21.15.100:0/2284670646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e40155c40 0x7f0e401609c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800c7f0 tx=0x7f0e3800ccb0 comp rx=0 tx=0).stop 2024-04-23T14:34:25.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.475+0000 7f0e46800640 1 -- 172.21.15.100:0/2284670646 shutdown_connections 2024-04-23T14:34:25.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.475+0000 7f0e46800640 1 --2- 172.21.15.100:0/2284670646 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0e200683b0 0x7f0e2006a870 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:25.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.475+0000 7f0e46800640 1 --2- 172.21.15.100:0/2284670646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e40155c40 0x7f0e401609c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:25.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.475+0000 7f0e46800640 1 --2- 172.21.15.100:0/2284670646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0e4015eb10 0x7f0e40160480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:25.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.475+0000 7f0e46800640 1 --2- 172.21.15.100:0/2284670646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e400b39c0 0x7f0e40154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:25.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.475+0000 7f0e46800640 1 -- 172.21.15.100:0/2284670646 >> 172.21.15.100:0/2284670646 conn(0x7f0e400b9a30 msgr2=0x7f0e400b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:25.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.475+0000 7f0e46800640 1 -- 172.21.15.100:0/2284670646 shutdown_connections 2024-04-23T14:34:25.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.475+0000 7f0e46800640 1 -- 172.21.15.100:0/2284670646 wait complete. 2024-04-23T14:34:25.499 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-23T14:34:25.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.635+0000 7fbfea400640 1 Processor -- start 2024-04-23T14:34:25.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.635+0000 7fbfea400640 1 -- start start 2024-04-23T14:34:25.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.635+0000 7fbfea400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fbfe415c7a0 0x7fbfe40af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:25.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.635+0000 7fbfea400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbfe406bb20 0x7fbfe40ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:25.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.635+0000 7fbfea400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfe406cbb0 con 0x7fbfe40cd130 2024-04-23T14:34:25.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.635+0000 7fbfea400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fbfe40cd5a0 con 0x7fbfe415c7a0 2024-04-23T14:34:25.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.635+0000 7fbfea400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fbfe406d450 con 0x7fbfe406bb20 2024-04-23T14:34:25.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe3e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fbfe40cd130 0x7fbfe406e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48744/0 (socket says 172.21.15.100:48744) 2024-04-23T14:34:25.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe3e00640 1 -- 172.21.15.100:0/4176549944 learned_addr learned my addr 172.21.15.100:0/4176549944 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:25.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfdaa00640 1 --2- 172.21.15.100:0/4176549944 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbfe406bb20 0x7fbfe40ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:25.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe3400640 1 --2- 172.21.15.100:0/4176549944 >> v2:172.21.15.151:3300/0 conn(0x7fbfe415c7a0 0x7fbfe40af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:25.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe2a00640 1 -- 172.21.15.100:0/4176549944 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2004295768 0 0) 0x7fbfe406cbb0 con 0x7fbfe40cd130 2024-04-23T14:34:25.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe2a00640 1 -- 172.21.15.100:0/4176549944 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfc8003530 con 0x7fbfe40cd130 2024-04-23T14:34:25.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe2a00640 1 -- 172.21.15.100:0/4176549944 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3650185878 0 0) 0x7fbfc8003530 con 0x7fbfe40cd130 2024-04-23T14:34:25.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe2a00640 1 -- 172.21.15.100:0/4176549944 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbfe406bb20 msgr2=0x7fbfe40ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:25.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe2a00640 1 --2- 172.21.15.100:0/4176549944 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbfe406bb20 0x7fbfe40ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:25.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe2a00640 1 -- 172.21.15.100:0/4176549944 >> v2:172.21.15.151:3300/0 conn(0x7fbfe415c7a0 msgr2=0x7fbfe40af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:25.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe2a00640 1 --2- 172.21.15.100:0/4176549944 >> v2:172.21.15.151:3300/0 conn(0x7fbfe415c7a0 0x7fbfe40af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:25.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe2a00640 1 -- 172.21.15.100:0/4176549944 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe406ce90 con 0x7fbfe40cd130 2024-04-23T14:34:25.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe2a00640 1 -- 172.21.15.100:0/4176549944 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7fbfdc001a00 con 0x7fbfe40cd130 2024-04-23T14:34:25.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe2a00640 1 -- 172.21.15.100:0/4176549944 >> v1:172.21.15.100:6789/0 conn(0x7fbfe40cd130 legacy=0x7fbfe406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:25.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe2a00640 1 --2- 172.21.15.100:0/4176549944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfc80039c0 0x7fbfc8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:25.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe2a00640 1 --2- 172.21.15.100:0/4176549944 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbfc8004810 0x7fbfc8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:25.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe2a00640 1 --2- 172.21.15.100:0/4176549944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfc80051d0 0x7fbfc800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:25.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe2a00640 1 -- 172.21.15.100:0/4176549944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe40cd5a0 con 0x7fbfc80051d0 2024-04-23T14:34:25.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe2a00640 1 -- 172.21.15.100:0/4176549944 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe406d450 con 0x7fbfc80039c0 2024-04-23T14:34:25.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe2a00640 1 -- 172.21.15.100:0/4176549944 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fbfc800bf60 con 0x7fbfc8004810 2024-04-23T14:34:25.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfdaa00640 1 --2- 172.21.15.100:0/4176549944 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbfe406bb20 0x7fbfe40ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:34:25.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe3e00640 1 --2- 172.21.15.100:0/4176549944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfc80051d0 0x7fbfc800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:25.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe3400640 1 --2- 172.21.15.100:0/4176549944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfc80039c0 0x7fbfc8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:25.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfdaa00640 1 --2- 172.21.15.100:0/4176549944 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbfc8004810 0x7fbfc8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:25.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe3e00640 1 -- 172.21.15.100:0/4176549944 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbfc8004810 msgr2=0x7fbfc8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:25.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe3e00640 1 --2- 172.21.15.100:0/4176549944 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbfc8004810 0x7fbfc8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:25.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe3e00640 1 -- 172.21.15.100:0/4176549944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfc80039c0 msgr2=0x7fbfc8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:25.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe3e00640 1 --2- 172.21.15.100:0/4176549944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfc80039c0 0x7fbfc8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:25.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.639+0000 7fbfe3e00640 1 -- 172.21.15.100:0/4176549944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc8003530 con 0x7fbfc80051d0 2024-04-23T14:34:25.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.643+0000 7fbfe3e00640 1 --2- 172.21.15.100:0/4176549944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfc80051d0 0x7fbfc800b9d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00a8d0 tx=0x7fbfdc01fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b0946c8b00e37 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:25.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.643+0000 7fbfe2a00640 1 -- 172.21.15.100:0/4176549944 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc00a4c0 con 0x7fbfc80051d0 2024-04-23T14:34:25.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.643+0000 7fbfe2a00640 1 -- 172.21.15.100:0/4176549944 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfdc0ad020 con 0x7fbfc80051d0 2024-04-23T14:34:25.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.643+0000 7fbfe2a00640 1 -- 172.21.15.100:0/4176549944 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc0a9740 con 0x7fbfc80051d0 2024-04-23T14:34:25.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.643+0000 7fbfea400640 1 -- 172.21.15.100:0/4176549944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfc80051d0 msgr2=0x7fbfc800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:25.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.643+0000 7fbfea400640 1 --2- 172.21.15.100:0/4176549944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfc80051d0 0x7fbfc800b9d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00a8d0 tx=0x7fbfdc01fb20 comp rx=0 tx=0).stop 2024-04-23T14:34:25.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.643+0000 7fbfe3e00640 1 -- 172.21.15.100:0/4176549944 reap_dead start 2024-04-23T14:34:25.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.643+0000 7fbfea400640 1 -- 172.21.15.100:0/4176549944 shutdown_connections 2024-04-23T14:34:25.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.643+0000 7fbfea400640 1 -- 172.21.15.100:0/4176549944 >> 172.21.15.100:0/4176549944 conn(0x7fbfe40b5200 msgr2=0x7fbfe40b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:25.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.643+0000 7fbfea400640 1 -- 172.21.15.100:0/4176549944 shutdown_connections 2024-04-23T14:34:25.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.643+0000 7fbfea400640 1 -- 172.21.15.100:0/4176549944 wait complete. 2024-04-23T14:34:25.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.643+0000 7fbfea400640 1 Processor -- start 2024-04-23T14:34:25.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.643+0000 7fbfea400640 1 -- start start 2024-04-23T14:34:25.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfea400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfc80051d0 0x7fbfe415a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:25.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfea400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfe41f3ff0 0x7fbfe40afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:25.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfea400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbfe40d0380 0x7fbfe40d0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:25.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfea400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe40af850 con 0x7fbfc80051d0 2024-04-23T14:34:25.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfea400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe40af6d0 con 0x7fbfe41f3ff0 2024-04-23T14:34:25.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfea400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fbfe40ae440 con 0x7fbfe40d0380 2024-04-23T14:34:25.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfe3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfc80051d0 0x7fbfe415a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:25.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfe3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfc80051d0 0x7fbfe415a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57248/0 (socket says 172.21.15.100:57248) 2024-04-23T14:34:25.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfe3400640 1 -- 172.21.15.100:0/21652716 learned_addr learned my addr 172.21.15.100:0/21652716 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:25.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfdaa00640 1 --2- 172.21.15.100:0/21652716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfe41f3ff0 0x7fbfe40afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:25.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfe3e00640 1 --2- 172.21.15.100:0/21652716 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbfe40d0380 0x7fbfe40d0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:25.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfe3400640 1 -- 172.21.15.100:0/21652716 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbfe40d0380 msgr2=0x7fbfe40d0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:25.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfe3400640 1 --2- 172.21.15.100:0/21652716 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbfe40d0380 0x7fbfe40d0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:25.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfe3400640 1 -- 172.21.15.100:0/21652716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfe41f3ff0 msgr2=0x7fbfe40afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:25.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfe3400640 1 --2- 172.21.15.100:0/21652716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfe41f3ff0 0x7fbfe40afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:25.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfe3400640 1 -- 172.21.15.100:0/21652716 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe40c83a0 con 0x7fbfc80051d0 2024-04-23T14:34:25.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfdaa00640 1 --2- 172.21.15.100:0/21652716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfe41f3ff0 0x7fbfe40afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:25.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfe3400640 1 --2- 172.21.15.100:0/21652716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfc80051d0 0x7fbfe415a710 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4011a30 tx=0x7fbfd4011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:25.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfdbe00640 1 -- 172.21.15.100:0/21652716 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd400fbb0 con 0x7fbfc80051d0 2024-04-23T14:34:25.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfdbe00640 1 -- 172.21.15.100:0/21652716 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfd400fd50 con 0x7fbfc80051d0 2024-04-23T14:34:25.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfdbe00640 1 -- 172.21.15.100:0/21652716 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4014660 con 0x7fbfc80051d0 2024-04-23T14:34:25.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfea400640 1 -- 172.21.15.100:0/21652716 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe40b0200 con 0x7fbfc80051d0 2024-04-23T14:34:25.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfea400640 1 -- 172.21.15.100:0/21652716 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe40b07b0 con 0x7fbfc80051d0 2024-04-23T14:34:25.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfea400640 1 -- 172.21.15.100:0/21652716 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfe4068ad0 con 0x7fbfc80051d0 2024-04-23T14:34:25.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.647+0000 7fbfdbe00640 1 -- 172.21.15.100:0/21652716 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbfd4012040 con 0x7fbfc80051d0 2024-04-23T14:34:25.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.651+0000 7fbfdbe00640 1 --2- 172.21.15.100:0/21652716 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fbfc00683b0 0x7fbfc006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:25.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.651+0000 7fbfdbe00640 1 -- 172.21.15.100:0/21652716 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fbfd4087170 con 0x7fbfc80051d0 2024-04-23T14:34:25.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.651+0000 7fbfdaa00640 1 --2- 172.21.15.100:0/21652716 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fbfc00683b0 0x7fbfc006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:25.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.651+0000 7fbfdaa00640 1 --2- 172.21.15.100:0/21652716 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fbfc00683b0 0x7fbfc006a870 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4158b80 tx=0x7fbfcc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:25.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.655+0000 7fbfdbe00640 1 -- 172.21.15.100:0/21652716 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbfd4019030 con 0x7fbfc80051d0 2024-04-23T14:34:25.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:25.851+0000 7fbfea400640 1 -- 172.21.15.100:0/21652716 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbfe41f43d0 con 0x7fbfc80051d0 2024-04-23T14:34:26.756 INFO:tasks.ceph.mds.a.smithi100.stderr:2024-04-23T14:34:26.755+0000 7f5de8400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:34:26.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.803+0000 7fbfdbe00640 1 -- 172.21.15.100:0/21652716 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v27) ==== 118+0+0 (secure 0 0 0) 0x7fbfd4057660 con 0x7fbfc80051d0 2024-04-23T14:34:26.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.807+0000 7fbfea400640 1 -- 172.21.15.100:0/21652716 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fbfc00683b0 msgr2=0x7fbfc006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:26.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.807+0000 7fbfea400640 1 --2- 172.21.15.100:0/21652716 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fbfc00683b0 0x7fbfc006a870 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4158b80 tx=0x7fbfcc008040 comp rx=0 tx=0).stop 2024-04-23T14:34:26.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.807+0000 7fbfea400640 1 -- 172.21.15.100:0/21652716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfc80051d0 msgr2=0x7fbfe415a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:26.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.807+0000 7fbfea400640 1 --2- 172.21.15.100:0/21652716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfc80051d0 0x7fbfe415a710 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4011a30 tx=0x7fbfd4011ef0 comp rx=0 tx=0).stop 2024-04-23T14:34:26.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.807+0000 7fbfea400640 1 -- 172.21.15.100:0/21652716 shutdown_connections 2024-04-23T14:34:26.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.807+0000 7fbfea400640 1 --2- 172.21.15.100:0/21652716 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fbfc00683b0 0x7fbfc006a870 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:26.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.807+0000 7fbfea400640 1 --2- 172.21.15.100:0/21652716 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbfe40d0380 0x7fbfe40d0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:26.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.807+0000 7fbfea400640 1 --2- 172.21.15.100:0/21652716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfe41f3ff0 0x7fbfe40afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:26.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.807+0000 7fbfea400640 1 --2- 172.21.15.100:0/21652716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfc80051d0 0x7fbfe415a710 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:26.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.807+0000 7fbfea400640 1 -- 172.21.15.100:0/21652716 >> 172.21.15.100:0/21652716 conn(0x7fbfe40b5200 msgr2=0x7fbfe4158f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:26.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.807+0000 7fbfea400640 1 -- 172.21.15.100:0/21652716 shutdown_connections 2024-04-23T14:34:26.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.807+0000 7fbfea400640 1 -- 172.21.15.100:0/21652716 wait complete. 2024-04-23T14:34:26.830 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-23T14:34:26.831 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-23T14:34:26.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.967+0000 7f5de7200640 1 Processor -- start 2024-04-23T14:34:26.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.967+0000 7f5de7200640 1 -- start start 2024-04-23T14:34:26.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de7200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5de815b370 0x7f5de815c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:26.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de7200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5de815efa0 con 0x7f5de80cc3a0 2024-04-23T14:34:26.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de7200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5de815f280 con 0x7f5de8073560 2024-04-23T14:34:26.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de7200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5de815b750 con 0x7f5de815b370 2024-04-23T14:34:26.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de5400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f5de80cc3a0 0x7f5de8072e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48748/0 (socket says 172.21.15.100:48748) 2024-04-23T14:34:26.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de5400640 1 -- 172.21.15.100:0/2545820186 learned_addr learned my addr 172.21.15.100:0/2545820186 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:26.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de5e00640 1 --2- 172.21.15.100:0/2545820186 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5de815b370 0x7f5de815c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:26.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de4a00640 1 -- 172.21.15.100:0/2545820186 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1122793337 0 0) 0x7f5de815efa0 con 0x7f5de80cc3a0 2024-04-23T14:34:26.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de4a00640 1 -- 172.21.15.100:0/2545820186 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5dcc003530 con 0x7f5de80cc3a0 2024-04-23T14:34:26.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de4a00640 1 -- 172.21.15.100:0/2545820186 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3782869718 0 0) 0x7f5dcc003530 con 0x7f5de80cc3a0 2024-04-23T14:34:26.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de4a00640 1 -- 172.21.15.100:0/2545820186 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5de815b370 msgr2=0x7f5de815c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:26.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de4a00640 1 --2- 172.21.15.100:0/2545820186 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5de815b370 0x7f5de815c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:26.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de4a00640 1 -- 172.21.15.100:0/2545820186 >> v1:172.21.15.151:6789/0 conn(0x7f5de8073560 legacy=0x7f5de80ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:26.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de4a00640 1 -- 172.21.15.100:0/2545820186 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de815f560 con 0x7f5de80cc3a0 2024-04-23T14:34:26.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de4a00640 1 -- 172.21.15.100:0/2545820186 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f5dd4001680 con 0x7f5de80cc3a0 2024-04-23T14:34:26.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de4a00640 1 -- 172.21.15.100:0/2545820186 >> v1:172.21.15.100:6789/0 conn(0x7f5de80cc3a0 legacy=0x7f5de8072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:26.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de4a00640 1 --2- 172.21.15.100:0/2545820186 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5de815b370 0x7f5dcc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:26.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de4a00640 1 --2- 172.21.15.100:0/2545820186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5dcc003f70 0x7f5dcc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:26.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de4a00640 1 --2- 172.21.15.100:0/2545820186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dcc004d50 0x7f5dcc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:26.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de4a00640 1 -- 172.21.15.100:0/2545820186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5de815b750 con 0x7f5dcc004d50 2024-04-23T14:34:26.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de4a00640 1 -- 172.21.15.100:0/2545820186 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc00bb60 con 0x7f5dcc003f70 2024-04-23T14:34:26.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de4a00640 1 -- 172.21.15.100:0/2545820186 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5dcc00df70 con 0x7f5de815b370 2024-04-23T14:34:26.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de6800640 1 --2- 172.21.15.100:0/2545820186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dcc004d50 0x7f5dcc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:26.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de5e00640 1 --2- 172.21.15.100:0/2545820186 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5de815b370 0x7f5dcc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:26.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de5400640 1 --2- 172.21.15.100:0/2545820186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5dcc003f70 0x7f5dcc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:26.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de6800640 1 -- 172.21.15.100:0/2545820186 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5de815b370 msgr2=0x7f5dcc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:26.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de6800640 1 --2- 172.21.15.100:0/2545820186 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5de815b370 0x7f5dcc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:26.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de6800640 1 -- 172.21.15.100:0/2545820186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5dcc003f70 msgr2=0x7f5dcc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:26.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de6800640 1 --2- 172.21.15.100:0/2545820186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5dcc003f70 0x7f5dcc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:26.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.971+0000 7f5de6800640 1 -- 172.21.15.100:0/2545820186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dcc003530 con 0x7f5dcc004d50 2024-04-23T14:34:26.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.975+0000 7f5de6800640 1 --2- 172.21.15.100:0/2545820186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dcc004d50 0x7f5dcc0051d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5de0001e40 tx=0x7f5de001c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603338f484f53f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:26.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.975+0000 7f5de4a00640 1 -- 172.21.15.100:0/2545820186 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0001a90 con 0x7f5dcc004d50 2024-04-23T14:34:26.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.975+0000 7f5de4a00640 1 -- 172.21.15.100:0/2545820186 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de0003b80 con 0x7f5dcc004d50 2024-04-23T14:34:26.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.975+0000 7f5de4a00640 1 -- 172.21.15.100:0/2545820186 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0005080 con 0x7f5dcc004d50 2024-04-23T14:34:26.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.975+0000 7f5de7200640 1 -- 172.21.15.100:0/2545820186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dcc004d50 msgr2=0x7f5dcc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:26.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.975+0000 7f5de7200640 1 --2- 172.21.15.100:0/2545820186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dcc004d50 0x7f5dcc0051d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5de0001e40 tx=0x7f5de001c760 comp rx=0 tx=0).stop 2024-04-23T14:34:26.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.975+0000 7f5de6800640 1 -- 172.21.15.100:0/2545820186 reap_dead start 2024-04-23T14:34:26.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.975+0000 7f5de7200640 1 -- 172.21.15.100:0/2545820186 shutdown_connections 2024-04-23T14:34:26.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.975+0000 7f5de7200640 1 -- 172.21.15.100:0/2545820186 >> 172.21.15.100:0/2545820186 conn(0x7f5de80b1d00 msgr2=0x7f5de80b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:26.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.975+0000 7f5de7200640 1 -- 172.21.15.100:0/2545820186 shutdown_connections 2024-04-23T14:34:26.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.975+0000 7f5de7200640 1 -- 172.21.15.100:0/2545820186 wait complete. 2024-04-23T14:34:26.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.975+0000 7f5de7200640 1 Processor -- start 2024-04-23T14:34:26.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.975+0000 7f5de7200640 1 -- start start 2024-04-23T14:34:26.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5de7200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5de80cd050 0x7f5de81f4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:26.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5de7200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5de80cd430 0x7f5de81501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:26.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5de7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de8150730 0x7f5de81590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:26.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5de7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5de815c300 con 0x7f5de8150730 2024-04-23T14:34:26.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5de7200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5de815fb20 con 0x7f5de80cd050 2024-04-23T14:34:26.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5de7200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5de815c480 con 0x7f5de80cd430 2024-04-23T14:34:26.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5de6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de8150730 0x7f5de81590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:26.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5de6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de8150730 0x7f5de81590d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57264/0 (socket says 172.21.15.100:57264) 2024-04-23T14:34:26.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5de6800640 1 -- 172.21.15.100:0/1296891500 learned_addr learned my addr 172.21.15.100:0/1296891500 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:26.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5de5e00640 1 --2- 172.21.15.100:0/1296891500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5de80cd050 0x7f5de81f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:26.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5de5400640 1 --2- 172.21.15.100:0/1296891500 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5de80cd430 0x7f5de81501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:26.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5de6800640 1 -- 172.21.15.100:0/1296891500 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5de80cd430 msgr2=0x7f5de81501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:26.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5de6800640 1 --2- 172.21.15.100:0/1296891500 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5de80cd430 0x7f5de81501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:26.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5de6800640 1 -- 172.21.15.100:0/1296891500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5de80cd050 msgr2=0x7f5de81f4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:26.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5de6800640 1 --2- 172.21.15.100:0/1296891500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5de80cd050 0x7f5de81f4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:26.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5de6800640 1 -- 172.21.15.100:0/1296891500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de8158050 con 0x7f5de8150730 2024-04-23T14:34:26.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5de5e00640 1 --2- 172.21.15.100:0/1296891500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5de80cd050 0x7f5de81f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:34:26.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5de6800640 1 --2- 172.21.15.100:0/1296891500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de8150730 0x7f5de81590d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5de0098060 tx=0x7f5dcc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:26.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5dde000640 1 -- 172.21.15.100:0/1296891500 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0001a90 con 0x7f5de8150730 2024-04-23T14:34:26.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5dde000640 1 -- 172.21.15.100:0/1296891500 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de00b8070 con 0x7f5de8150730 2024-04-23T14:34:26.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5dde000640 1 -- 172.21.15.100:0/1296891500 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0007dc0 con 0x7f5de8150730 2024-04-23T14:34:26.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5de7200640 1 -- 172.21.15.100:0/1296891500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de8150bb0 con 0x7f5de8150730 2024-04-23T14:34:26.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5de7200640 1 -- 172.21.15.100:0/1296891500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de81585f0 con 0x7f5de8150730 2024-04-23T14:34:26.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.979+0000 7f5de7200640 1 -- 172.21.15.100:0/1296891500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de8068ad0 con 0x7f5de8150730 2024-04-23T14:34:26.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.983+0000 7f5dde000640 1 -- 172.21.15.100:0/1296891500 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5de0006b60 con 0x7f5de8150730 2024-04-23T14:34:26.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.983+0000 7f5dde000640 1 --2- 172.21.15.100:0/1296891500 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5dc80683b0 0x7f5dc806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:26.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.983+0000 7f5dde000640 1 -- 172.21.15.100:0/1296891500 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f5de0120660 con 0x7f5de8150730 2024-04-23T14:34:26.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.983+0000 7f5de5e00640 1 --2- 172.21.15.100:0/1296891500 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5dc80683b0 0x7f5dc806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:26.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.983+0000 7f5de5e00640 1 --2- 172.21.15.100:0/1296891500 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5dc80683b0 0x7f5dc806a870 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009fd0 tx=0x7f5dd8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:26.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:26.987+0000 7f5dde000640 1 -- 172.21.15.100:0/1296891500 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5de00ad020 con 0x7f5de8150730 2024-04-23T14:34:27.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.175+0000 7f5de7200640 1 -- 172.21.15.100:0/1296891500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f5de80cfe60 con 0x7f5de8150730 2024-04-23T14:34:27.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.811+0000 7f5dde000640 1 -- 172.21.15.100:0/1296891500 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f5de0016470 con 0x7f5de8150730 2024-04-23T14:34:27.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.815+0000 7f5de7200640 1 -- 172.21.15.100:0/1296891500 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5dc80683b0 msgr2=0x7f5dc806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:27.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.815+0000 7f5de7200640 1 --2- 172.21.15.100:0/1296891500 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5dc80683b0 0x7f5dc806a870 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009fd0 tx=0x7f5dd8009190 comp rx=0 tx=0).stop 2024-04-23T14:34:27.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.815+0000 7f5de7200640 1 -- 172.21.15.100:0/1296891500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de8150730 msgr2=0x7f5de81590d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:27.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.815+0000 7f5de7200640 1 --2- 172.21.15.100:0/1296891500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de8150730 0x7f5de81590d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5de0098060 tx=0x7f5dcc005840 comp rx=0 tx=0).stop 2024-04-23T14:34:27.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.815+0000 7f5de7200640 1 -- 172.21.15.100:0/1296891500 shutdown_connections 2024-04-23T14:34:27.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.815+0000 7f5de7200640 1 --2- 172.21.15.100:0/1296891500 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5dc80683b0 0x7f5dc806a870 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:27.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.815+0000 7f5de7200640 1 --2- 172.21.15.100:0/1296891500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de8150730 0x7f5de81590d0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:27.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.815+0000 7f5de7200640 1 --2- 172.21.15.100:0/1296891500 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5de80cd430 0x7f5de81501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:27.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.815+0000 7f5de7200640 1 --2- 172.21.15.100:0/1296891500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5de80cd050 0x7f5de81f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:27.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.815+0000 7f5de7200640 1 -- 172.21.15.100:0/1296891500 >> 172.21.15.100:0/1296891500 conn(0x7f5de80b1d00 msgr2=0x7f5de806ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:27.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.815+0000 7f5de7200640 1 -- 172.21.15.100:0/1296891500 shutdown_connections 2024-04-23T14:34:27.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.815+0000 7f5de7200640 1 -- 172.21.15.100:0/1296891500 wait complete. 2024-04-23T14:34:27.836 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-23T14:34:27.837 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:34:27.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.971+0000 7f1639800640 1 Processor -- start 2024-04-23T14:34:27.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.971+0000 7f1639800640 1 -- start start 2024-04-23T14:34:27.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.971+0000 7f1639800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f163415b640 0x7f16340adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:27.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.971+0000 7f1639800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16340af080 con 0x7f1634073610 2024-04-23T14:34:27.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.971+0000 7f1639800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16340afbc0 con 0x7f16340b2a10 2024-04-23T14:34:27.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.971+0000 7f1639800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f163415ba20 con 0x7f163415b640 2024-04-23T14:34:27.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.975+0000 7f1633400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1634073610 0x7f163406df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48756/0 (socket says 172.21.15.100:48756) 2024-04-23T14:34:27.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.975+0000 7f1633400640 1 -- 172.21.15.100:0/644969578 learned_addr learned my addr 172.21.15.100:0/644969578 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:27.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.975+0000 7f1632a00640 1 --2- 172.21.15.100:0/644969578 >> v2:172.21.15.151:3300/0 conn(0x7f163415b640 0x7f16340adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:27.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.975+0000 7f1631600640 1 -- 172.21.15.100:0/644969578 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2531185554 0 0) 0x7f16340af080 con 0x7f1634073610 2024-04-23T14:34:27.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.975+0000 7f1631600640 1 -- 172.21.15.100:0/644969578 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1618003530 con 0x7f1634073610 2024-04-23T14:34:27.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.975+0000 7f1631600640 1 -- 172.21.15.100:0/644969578 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1114383577 0 0) 0x7f16340afbc0 con 0x7f16340b2a10 2024-04-23T14:34:27.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.975+0000 7f1631600640 1 -- 172.21.15.100:0/644969578 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16340af080 con 0x7f16340b2a10 2024-04-23T14:34:27.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.975+0000 7f1632a00640 1 -- 172.21.15.100:0/644969578 >> v1:172.21.15.151:6789/0 conn(0x7f16340b2a10 legacy=0x7f1634072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:27.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.975+0000 7f1632a00640 1 -- 172.21.15.100:0/644969578 >> v1:172.21.15.100:6789/0 conn(0x7f1634073610 legacy=0x7f163406df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:27.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.975+0000 7f1632a00640 1 -- 172.21.15.100:0/644969578 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16340af320 con 0x7f163415b640 2024-04-23T14:34:27.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.975+0000 7f1632a00640 1 --2- 172.21.15.100:0/644969578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f163415b640 0x7f16340adfc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f161c0097c0 tx=0x7f161c01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bc74bd1c3692152 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:27.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.975+0000 7f1631600640 1 -- 172.21.15.100:0/644969578 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c0b1070 con 0x7f163415b640 2024-04-23T14:34:27.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.979+0000 7f1631600640 1 -- 172.21.15.100:0/644969578 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f161c0a8050 con 0x7f163415b640 2024-04-23T14:34:27.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.979+0000 7f1631600640 1 -- 172.21.15.100:0/644969578 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c0ac840 con 0x7f163415b640 2024-04-23T14:34:27.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.979+0000 7f1639800640 1 -- 172.21.15.100:0/644969578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f163415b640 msgr2=0x7f16340adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:27.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.979+0000 7f1639800640 1 --2- 172.21.15.100:0/644969578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f163415b640 0x7f16340adfc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f161c0097c0 tx=0x7f161c01c3e0 comp rx=0 tx=0).stop 2024-04-23T14:34:27.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.979+0000 7f1639800640 1 -- 172.21.15.100:0/644969578 shutdown_connections 2024-04-23T14:34:27.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.979+0000 7f1639800640 1 --2- 172.21.15.100:0/644969578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f163415b640 0x7f16340adfc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:27.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.979+0000 7f1639800640 1 -- 172.21.15.100:0/644969578 >> 172.21.15.100:0/644969578 conn(0x7f16340b1e80 msgr2=0x7f16340b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:27.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.979+0000 7f1639800640 1 -- 172.21.15.100:0/644969578 shutdown_connections 2024-04-23T14:34:27.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.979+0000 7f1639800640 1 -- 172.21.15.100:0/644969578 wait complete. 2024-04-23T14:34:27.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.979+0000 7f1639800640 1 Processor -- start 2024-04-23T14:34:27.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.979+0000 7f1639800640 1 -- start start 2024-04-23T14:34:27.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.979+0000 7f1639800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1634073610 0x7f163415e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:27.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.979+0000 7f1639800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16340b2a10 0x7f1634157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:27.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.979+0000 7f1639800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f163415b640 0x7f16341faa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:27.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.979+0000 7f1639800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f163406c340 con 0x7f16340b2a10 2024-04-23T14:34:27.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.979+0000 7f1639800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f163406f160 con 0x7f163415b640 2024-04-23T14:34:27.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.979+0000 7f1639800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f163406c4c0 con 0x7f1634073610 2024-04-23T14:34:27.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.979+0000 7f1632000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16340b2a10 0x7f1634157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:27.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.979+0000 7f1632000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16340b2a10 0x7f1634157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57268/0 (socket says 172.21.15.100:57268) 2024-04-23T14:34:27.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.979+0000 7f1632000640 1 -- 172.21.15.100:0/600165787 learned_addr learned my addr 172.21.15.100:0/600165787 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:27.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.983+0000 7f1632000640 1 -- 172.21.15.100:0/600165787 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1634073610 msgr2=0x7f163415e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:27.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.983+0000 7f1632000640 1 --2- 172.21.15.100:0/600165787 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1634073610 0x7f163415e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:27.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.983+0000 7f1632000640 1 -- 172.21.15.100:0/600165787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f163415b640 msgr2=0x7f16341faa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:27.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.983+0000 7f1632000640 1 --2- 172.21.15.100:0/600165787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f163415b640 0x7f16341faa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:27.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.983+0000 7f1632000640 1 -- 172.21.15.100:0/600165787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163414af40 con 0x7f16340b2a10 2024-04-23T14:34:27.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.983+0000 7f1632000640 1 --2- 172.21.15.100:0/600165787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16340b2a10 0x7f1634157a70 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f162400c3e0 tx=0x7f162400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:27.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.983+0000 7f162aa00640 1 -- 172.21.15.100:0/600165787 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1624017070 con 0x7f16340b2a10 2024-04-23T14:34:27.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.983+0000 7f162aa00640 1 -- 172.21.15.100:0/600165787 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16240052d0 con 0x7f16340b2a10 2024-04-23T14:34:27.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.983+0000 7f1639800640 1 -- 172.21.15.100:0/600165787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f163414b220 con 0x7f16340b2a10 2024-04-23T14:34:27.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.983+0000 7f162aa00640 1 -- 172.21.15.100:0/600165787 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1624003840 con 0x7f16340b2a10 2024-04-23T14:34:27.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.983+0000 7f1639800640 1 -- 172.21.15.100:0/600165787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f163414bac0 con 0x7f16340b2a10 2024-04-23T14:34:27.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.983+0000 7f1639800640 1 -- 172.21.15.100:0/600165787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1634068ad0 con 0x7f16340b2a10 2024-04-23T14:34:27.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.983+0000 7f162aa00640 1 -- 172.21.15.100:0/600165787 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f162401a070 con 0x7f16340b2a10 2024-04-23T14:34:27.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.983+0000 7f162aa00640 1 --2- 172.21.15.100:0/600165787 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1610068380 0x7f161006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:27.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.983+0000 7f162aa00640 1 -- 172.21.15.100:0/600165787 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f1624085ef0 con 0x7f16340b2a10 2024-04-23T14:34:27.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.983+0000 7f1632a00640 1 --2- 172.21.15.100:0/600165787 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1610068380 0x7f161006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:27.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.987+0000 7f1632a00640 1 --2- 172.21.15.100:0/600165787 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1610068380 0x7f161006a840 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f161c0ae000 tx=0x7f161c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:27.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:27.987+0000 7f162aa00640 1 -- 172.21.15.100:0/600165787 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1624005480 con 0x7f16340b2a10 2024-04-23T14:34:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.179+0000 7f1639800640 1 -- 172.21.15.100:0/600165787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1634154c20 con 0x7f16340b2a10 2024-04-23T14:34:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.179+0000 7f162aa00640 1 -- 172.21.15.100:0/600165787 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5190 (secure 0 0 0) 0x7f16240563b0 con 0x7f16340b2a10 2024-04-23T14:34:28.180 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:28.180 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5528,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2774610498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2774610498},{"type":"v1","addr":"172.21.15.151:6833","nonce":2774610498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5929,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1347883282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1347883282},{"type":"v1","addr":"172.21.15.151:6835","nonce":1347883282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":6177,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3319827520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3319827520},{"type":"v1","addr":"172.21.15.100:6836","nonce":3319827520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:34:24.162539+0000","modified":"2024-04-23T14:34:27.179639+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6180":{"gid":6180,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/608349550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":608349550},{"type":"v1","addr":"172.21.15.100:6837","nonce":608349550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6180,"qdb_cluster":[6180]},"id":3}]} 2024-04-23T14:34:28.180 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 28 2024-04-23T14:34:28.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.179+0000 7f1639800640 1 -- 172.21.15.100:0/600165787 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1610068380 msgr2=0x7f161006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:28.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.179+0000 7f1639800640 1 --2- 172.21.15.100:0/600165787 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1610068380 0x7f161006a840 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f161c0ae000 tx=0x7f161c008040 comp rx=0 tx=0).stop 2024-04-23T14:34:28.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.179+0000 7f1639800640 1 -- 172.21.15.100:0/600165787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16340b2a10 msgr2=0x7f1634157a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:28.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.179+0000 7f1639800640 1 --2- 172.21.15.100:0/600165787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16340b2a10 0x7f1634157a70 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f162400c3e0 tx=0x7f162400c8a0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.179+0000 7f1639800640 1 -- 172.21.15.100:0/600165787 shutdown_connections 2024-04-23T14:34:28.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.179+0000 7f1639800640 1 --2- 172.21.15.100:0/600165787 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1610068380 0x7f161006a840 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.179+0000 7f1639800640 1 --2- 172.21.15.100:0/600165787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f163415b640 0x7f16341faa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.179+0000 7f1639800640 1 --2- 172.21.15.100:0/600165787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16340b2a10 0x7f1634157a70 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.179+0000 7f1639800640 1 --2- 172.21.15.100:0/600165787 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1634073610 0x7f163415e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.179+0000 7f1639800640 1 -- 172.21.15.100:0/600165787 >> 172.21.15.100:0/600165787 conn(0x7f16340b1e80 msgr2=0x7f16340b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:28.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.179+0000 7f1639800640 1 -- 172.21.15.100:0/600165787 shutdown_connections 2024-04-23T14:34:28.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.179+0000 7f1639800640 1 -- 172.21.15.100:0/600165787 wait complete. 2024-04-23T14:34:28.201 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:28.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.335+0000 7fdb38400640 1 Processor -- start 2024-04-23T14:34:28.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.335+0000 7fdb38400640 1 -- start start 2024-04-23T14:34:28.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.335+0000 7fdb38400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fdb3006d450 0x7fdb3006d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:28.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.335+0000 7fdb38400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdb300cea60 0x7fdb300d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:28.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.335+0000 7fdb38400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb3006dd70 con 0x7fdb300cd460 2024-04-23T14:34:28.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.335+0000 7fdb38400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fdb300b1f90 con 0x7fdb300cea60 2024-04-23T14:34:28.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.335+0000 7fdb38400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fdb300b2270 con 0x7fdb3006d450 2024-04-23T14:34:28.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.335+0000 7fdb35000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdb300cea60 0x7fdb300d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:28.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.335+0000 7fdb36400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fdb300cd460 0x7fdb300b2d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48758/0 (socket says 172.21.15.100:48758) 2024-04-23T14:34:28.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.335+0000 7fdb35000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdb300cea60 0x7fdb300d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57276/0 (socket says 172.21.15.100:57276) 2024-04-23T14:34:28.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.335+0000 7fdb36400640 1 -- 172.21.15.100:0/3622715483 learned_addr learned my addr 172.21.15.100:0/3622715483 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:28.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.335+0000 7fdb35a00640 1 --2- 172.21.15.100:0/3622715483 >> v2:172.21.15.151:3300/0 conn(0x7fdb3006d450 0x7fdb3006d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:28.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.335+0000 7fdb35000640 1 -- 172.21.15.100:0/3622715483 >> v2:172.21.15.151:3300/0 conn(0x7fdb3006d450 msgr2=0x7fdb3006d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:28.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.335+0000 7fdb35000640 1 --2- 172.21.15.100:0/3622715483 >> v2:172.21.15.151:3300/0 conn(0x7fdb3006d450 0x7fdb3006d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.335+0000 7fdb35000640 1 -- 172.21.15.100:0/3622715483 >> v1:172.21.15.100:6789/0 conn(0x7fdb300cd460 legacy=0x7fdb300b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:28.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.335+0000 7fdb35000640 1 -- 172.21.15.100:0/3622715483 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb300b2550 con 0x7fdb300cea60 2024-04-23T14:34:28.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.335+0000 7fdb35a00640 1 --2- 172.21.15.100:0/3622715483 >> v2:172.21.15.151:3300/0 conn(0x7fdb3006d450 0x7fdb3006d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:28.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.339+0000 7fdb35000640 1 --2- 172.21.15.100:0/3622715483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb300cea60 0x7fdb300d0300 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fdb30068600 tx=0x7fdb1c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72689af11987da43 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:28.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.339+0000 7fdb2fe00640 1 -- 172.21.15.100:0/3622715483 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c0b0070 con 0x7fdb300cea60 2024-04-23T14:34:28.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.339+0000 7fdb2fe00640 1 -- 172.21.15.100:0/3622715483 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb1c0ab600 con 0x7fdb300cea60 2024-04-23T14:34:28.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.339+0000 7fdb2fe00640 1 -- 172.21.15.100:0/3622715483 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c0ab9d0 con 0x7fdb300cea60 2024-04-23T14:34:28.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.339+0000 7fdb38400640 1 -- 172.21.15.100:0/3622715483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb300cea60 msgr2=0x7fdb300d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:28.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.339+0000 7fdb38400640 1 --2- 172.21.15.100:0/3622715483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb300cea60 0x7fdb300d0300 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fdb30068600 tx=0x7fdb1c01c3f0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.339+0000 7fdb38400640 1 -- 172.21.15.100:0/3622715483 shutdown_connections 2024-04-23T14:34:28.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.339+0000 7fdb38400640 1 --2- 172.21.15.100:0/3622715483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb300cea60 0x7fdb300d0300 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.339+0000 7fdb38400640 1 --2- 172.21.15.100:0/3622715483 >> v2:172.21.15.151:3300/0 conn(0x7fdb3006d450 0x7fdb3006d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.339+0000 7fdb38400640 1 -- 172.21.15.100:0/3622715483 >> 172.21.15.100:0/3622715483 conn(0x7fdb300cee80 msgr2=0x7fdb300cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:28.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.339+0000 7fdb38400640 1 -- 172.21.15.100:0/3622715483 shutdown_connections 2024-04-23T14:34:28.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.339+0000 7fdb38400640 1 -- 172.21.15.100:0/3622715483 wait complete. 2024-04-23T14:34:28.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb38400640 1 Processor -- start 2024-04-23T14:34:28.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb38400640 1 -- start start 2024-04-23T14:34:28.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb38400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb3006d450 0x7fdb300cb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:28.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb38400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb300cd460 0x7fdb301596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:28.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb38400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb300cea60 0x7fdb30158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:28.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb38400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3015fc70 con 0x7fdb300cd460 2024-04-23T14:34:28.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb38400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3015faf0 con 0x7fdb3006d450 2024-04-23T14:34:28.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb38400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fdb3016bbc0 con 0x7fdb300cea60 2024-04-23T14:34:28.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb35000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb300cd460 0x7fdb301596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:28.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb35000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb300cd460 0x7fdb301596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57282/0 (socket says 172.21.15.100:57282) 2024-04-23T14:34:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb35000640 1 -- 172.21.15.100:0/3572264463 learned_addr learned my addr 172.21.15.100:0/3572264463 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb35a00640 1 --2- 172.21.15.100:0/3572264463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb3006d450 0x7fdb300cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb36400640 1 --2- 172.21.15.100:0/3572264463 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb300cea60 0x7fdb30158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb35000640 1 -- 172.21.15.100:0/3572264463 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb300cea60 msgr2=0x7fdb30158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb35000640 1 --2- 172.21.15.100:0/3572264463 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb300cea60 0x7fdb30158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb35000640 1 -- 172.21.15.100:0/3572264463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb3006d450 msgr2=0x7fdb300cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb35000640 1 --2- 172.21.15.100:0/3572264463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb3006d450 0x7fdb300cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb35000640 1 -- 172.21.15.100:0/3572264463 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3015a1d0 con 0x7fdb300cd460 2024-04-23T14:34:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb35000640 1 --2- 172.21.15.100:0/3572264463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb300cd460 0x7fdb301596f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c0ae040 tx=0x7fdb1c019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb2d600640 1 -- 172.21.15.100:0/3572264463 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c0b8020 con 0x7fdb300cd460 2024-04-23T14:34:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb2d600640 1 -- 172.21.15.100:0/3572264463 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb1c0b4630 con 0x7fdb300cd460 2024-04-23T14:34:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb2d600640 1 -- 172.21.15.100:0/3572264463 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c0b0040 con 0x7fdb300cd460 2024-04-23T14:34:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb38400640 1 -- 172.21.15.100:0/3572264463 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb3015a4b0 con 0x7fdb300cd460 2024-04-23T14:34:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb38400640 1 -- 172.21.15.100:0/3572264463 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb300ad9a0 con 0x7fdb300cd460 2024-04-23T14:34:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.343+0000 7fdb38400640 1 -- 172.21.15.100:0/3572264463 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb30068ad0 con 0x7fdb300cd460 2024-04-23T14:34:28.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.347+0000 7fdb2d600640 1 -- 172.21.15.100:0/3572264463 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdb1c0bb020 con 0x7fdb300cd460 2024-04-23T14:34:28.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.347+0000 7fdb2d600640 1 --2- 172.21.15.100:0/3572264463 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb10068380 0x7fdb1006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:28.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.347+0000 7fdb2d600640 1 -- 172.21.15.100:0/3572264463 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fdb1c120950 con 0x7fdb300cd460 2024-04-23T14:34:28.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.347+0000 7fdb35a00640 1 --2- 172.21.15.100:0/3572264463 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb10068380 0x7fdb1006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.347+0000 7fdb35a00640 1 --2- 172.21.15.100:0/3572264463 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb10068380 0x7fdb1006a840 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdb24006fd0 tx=0x7fdb24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:28.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.351+0000 7fdb2d600640 1 -- 172.21.15.100:0/3572264463 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdb1c0c2030 con 0x7fdb300cd460 2024-04-23T14:34:28.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.507+0000 7fdb38400640 1 -- 172.21.15.100:0/3572264463 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb300cf720 con 0x7fdb300cd460 2024-04-23T14:34:28.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.507+0000 7fdb2d600640 1 -- 172.21.15.100:0/3572264463 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16950 (secure 0 0 0) 0x7fdb1c016070 con 0x7fdb300cd460 2024-04-23T14:34:28.510 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:28.511 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":53,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:34:25.795515+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:34:22.491379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-23T14:34:23.120342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.511+0000 7fdb38400640 1 -- 172.21.15.100:0/3572264463 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb10068380 msgr2=0x7fdb1006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.511+0000 7fdb38400640 1 --2- 172.21.15.100:0/3572264463 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb10068380 0x7fdb1006a840 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdb24006fd0 tx=0x7fdb24008040 comp rx=0 tx=0).stop 2024-04-23T14:34:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.511+0000 7fdb38400640 1 -- 172.21.15.100:0/3572264463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb300cd460 msgr2=0x7fdb301596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.511+0000 7fdb38400640 1 --2- 172.21.15.100:0/3572264463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb300cd460 0x7fdb301596f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c0ae040 tx=0x7fdb1c019190 comp rx=0 tx=0).stop 2024-04-23T14:34:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.511+0000 7fdb38400640 1 -- 172.21.15.100:0/3572264463 shutdown_connections 2024-04-23T14:34:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.511+0000 7fdb38400640 1 --2- 172.21.15.100:0/3572264463 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb10068380 0x7fdb1006a840 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.511+0000 7fdb38400640 1 --2- 172.21.15.100:0/3572264463 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb300cea60 0x7fdb30158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.511+0000 7fdb38400640 1 --2- 172.21.15.100:0/3572264463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb300cd460 0x7fdb301596f0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.511+0000 7fdb38400640 1 --2- 172.21.15.100:0/3572264463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb3006d450 0x7fdb300cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.511+0000 7fdb38400640 1 -- 172.21.15.100:0/3572264463 >> 172.21.15.100:0/3572264463 conn(0x7fdb300cee80 msgr2=0x7fdb3006f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:28.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.511+0000 7fdb38400640 1 -- 172.21.15.100:0/3572264463 shutdown_connections 2024-04-23T14:34:28.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.511+0000 7fdb38400640 1 -- 172.21.15.100:0/3572264463 wait complete. 2024-04-23T14:34:28.532 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-23T14:34:28.616 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:34:28.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.751+0000 7efeb2800640 1 Processor -- start 2024-04-23T14:34:28.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.751+0000 7efeb2800640 1 -- start start 2024-04-23T14:34:28.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.751+0000 7efeb2800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efeac15b640 0x7efeac0adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:28.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.751+0000 7efeb2800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7efeac0b2a10 0x7efeac072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:28.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.751+0000 7efeb2800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeac0af080 con 0x7efeac0cc750 2024-04-23T14:34:28.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.751+0000 7efeb2800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7efeac0afbc0 con 0x7efeac15b640 2024-04-23T14:34:28.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.751+0000 7efeb2800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7efeac15ba20 con 0x7efeac0b2a10 2024-04-23T14:34:28.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.751+0000 7efeab400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efeac15b640 0x7efeac0adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:28.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.751+0000 7efeab400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efeac15b640 0x7efeac0adfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48296/0 (socket says 172.21.15.100:48296) 2024-04-23T14:34:28.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.751+0000 7efeab400640 1 -- 172.21.15.100:0/3895432366 learned_addr learned my addr 172.21.15.100:0/3895432366 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:28.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.751+0000 7efeab400640 1 -- 172.21.15.100:0/3895432366 >> v2:172.21.15.151:3300/0 conn(0x7efeac0b2a10 msgr2=0x7efeac072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:28.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.751+0000 7efeaaa00640 1 --2- 172.21.15.100:0/3895432366 >> v2:172.21.15.151:3300/0 conn(0x7efeac0b2a10 0x7efeac072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:28.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.751+0000 7efeab400640 1 --2- 172.21.15.100:0/3895432366 >> v2:172.21.15.151:3300/0 conn(0x7efeac0b2a10 0x7efeac072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.751+0000 7efeab400640 1 -- 172.21.15.100:0/3895432366 >> v1:172.21.15.151:6789/0 conn(0x7efeac0cc750 legacy=0x7efeac071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:28.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.751+0000 7efeab400640 1 -- 172.21.15.100:0/3895432366 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeac0af320 con 0x7efeac15b640 2024-04-23T14:34:28.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.751+0000 7efeaaa00640 1 --2- 172.21.15.100:0/3895432366 >> v2:172.21.15.151:3300/0 conn(0x7efeac0b2a10 0x7efeac072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:28.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.755+0000 7efeab400640 1 --2- 172.21.15.100:0/3895432366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeac15b640 0x7efeac0adfc0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7efeac0ada50 tx=0x7efe9401eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e5b2f994fbbfb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:28.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.755+0000 7efeaa000640 1 -- 172.21.15.100:0/3895432366 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe940b4070 con 0x7efeac15b640 2024-04-23T14:34:28.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.755+0000 7efeaa000640 1 -- 172.21.15.100:0/3895432366 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe940a7bc0 con 0x7efeac15b640 2024-04-23T14:34:28.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.755+0000 7efeaa000640 1 -- 172.21.15.100:0/3895432366 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe940af440 con 0x7efeac15b640 2024-04-23T14:34:28.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.755+0000 7efeb2800640 1 -- 172.21.15.100:0/3895432366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeac15b640 msgr2=0x7efeac0adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:28.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.755+0000 7efeb2800640 1 --2- 172.21.15.100:0/3895432366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeac15b640 0x7efeac0adfc0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7efeac0ada50 tx=0x7efe9401eaf0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.755+0000 7efeb2800640 1 -- 172.21.15.100:0/3895432366 shutdown_connections 2024-04-23T14:34:28.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.755+0000 7efeb2800640 1 --2- 172.21.15.100:0/3895432366 >> v2:172.21.15.151:3300/0 conn(0x7efeac0b2a10 0x7efeac072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.755+0000 7efeb2800640 1 --2- 172.21.15.100:0/3895432366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeac15b640 0x7efeac0adfc0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.755+0000 7efeb2800640 1 -- 172.21.15.100:0/3895432366 >> 172.21.15.100:0/3895432366 conn(0x7efeac0b1e80 msgr2=0x7efeac0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:28.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.755+0000 7efeb2800640 1 -- 172.21.15.100:0/3895432366 shutdown_connections 2024-04-23T14:34:28.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.755+0000 7efeb2800640 1 -- 172.21.15.100:0/3895432366 wait complete. 2024-04-23T14:34:28.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efeb2800640 1 Processor -- start 2024-04-23T14:34:28.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efeb2800640 1 -- start start 2024-04-23T14:34:28.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efeb2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeac0b2a10 0x7efeac149440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:28.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efeb2800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efeac0cc750 0x7efeac06de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:28.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efeb2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeac15b640 0x7efeac06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:28.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efeb2800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efeac06c750 con 0x7efeac0b2a10 2024-04-23T14:34:28.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efeb2800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efeac06c5d0 con 0x7efeac15b640 2024-04-23T14:34:28.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efeb2800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7efeac06c8d0 con 0x7efeac0cc750 2024-04-23T14:34:28.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efeab400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeac0b2a10 0x7efeac149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:28.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efeab400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeac0b2a10 0x7efeac149440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48298/0 (socket says 172.21.15.100:48298) 2024-04-23T14:34:28.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efeab400640 1 -- 172.21.15.100:0/4247656752 learned_addr learned my addr 172.21.15.100:0/4247656752 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:28.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efeaaa00640 1 --2- 172.21.15.100:0/4247656752 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efeac0cc750 0x7efeac06de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:28.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efeabe00640 1 --2- 172.21.15.100:0/4247656752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeac15b640 0x7efeac06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:28.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efeab400640 1 -- 172.21.15.100:0/4247656752 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efeac0cc750 msgr2=0x7efeac06de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:28.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efeab400640 1 --2- 172.21.15.100:0/4247656752 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efeac0cc750 0x7efeac06de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efeab400640 1 -- 172.21.15.100:0/4247656752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeac15b640 msgr2=0x7efeac06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:28.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efeab400640 1 --2- 172.21.15.100:0/4247656752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeac15b640 0x7efeac06e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efeab400640 1 -- 172.21.15.100:0/4247656752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeac1fbbc0 con 0x7efeac0b2a10 2024-04-23T14:34:28.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efeab400640 1 --2- 172.21.15.100:0/4247656752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeac0b2a10 0x7efeac149440 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7efe9401c720 tx=0x7efe9401b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:28.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efea3400640 1 -- 172.21.15.100:0/4247656752 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe940b4030 con 0x7efeac0b2a10 2024-04-23T14:34:28.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efea3400640 1 -- 172.21.15.100:0/4247656752 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe940b6070 con 0x7efeac0b2a10 2024-04-23T14:34:28.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efeb2800640 1 -- 172.21.15.100:0/4247656752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeac1f7ba0 con 0x7efeac0b2a10 2024-04-23T14:34:28.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efea3400640 1 -- 172.21.15.100:0/4247656752 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe940ad7b0 con 0x7efeac0b2a10 2024-04-23T14:34:28.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efeb2800640 1 -- 172.21.15.100:0/4247656752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeac1f8160 con 0x7efeac0b2a10 2024-04-23T14:34:28.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.759+0000 7efeb2800640 1 -- 172.21.15.100:0/4247656752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeac068ad0 con 0x7efeac0b2a10 2024-04-23T14:34:28.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.763+0000 7efea3400640 1 -- 172.21.15.100:0/4247656752 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7efe940c1060 con 0x7efeac0b2a10 2024-04-23T14:34:28.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.763+0000 7efea3400640 1 --2- 172.21.15.100:0/4247656752 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7efe88068380 0x7efe8806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:28.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.763+0000 7efea3400640 1 -- 172.21.15.100:0/4247656752 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7efe94120ff0 con 0x7efeac0b2a10 2024-04-23T14:34:28.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.763+0000 7efeaaa00640 1 --2- 172.21.15.100:0/4247656752 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7efe88068380 0x7efe8806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:28.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.763+0000 7efeaaa00640 1 --2- 172.21.15.100:0/4247656752 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7efe88068380 0x7efe8806a840 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7efeac149de0 tx=0x7efe9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:28.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.767+0000 7efea3400640 1 -- 172.21.15.100:0/4247656752 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efe940187a0 con 0x7efeac0b2a10 2024-04-23T14:34:28.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.923+0000 7efeb2800640 1 -- 172.21.15.100:0/4247656752 --> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efeac1fbea0 con 0x7efe88068380 2024-04-23T14:34:28.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.931+0000 7efea3400640 1 -- 172.21.15.100:0/4247656752 <== mgr.4101 v2:172.21.15.100:6832/2859853957 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251953 (secure 0 0 0) 0x7efeac1fbea0 con 0x7efe88068380 2024-04-23T14:34:28.936 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:28.938 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-04-23T14:34:28.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.935+0000 7efeb2800640 1 -- 172.21.15.100:0/4247656752 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7efe88068380 msgr2=0x7efe8806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:28.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.935+0000 7efeb2800640 1 --2- 172.21.15.100:0/4247656752 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7efe88068380 0x7efe8806a840 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7efeac149de0 tx=0x7efe9c008040 comp rx=0 tx=0).stop 2024-04-23T14:34:28.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.935+0000 7efeb2800640 1 -- 172.21.15.100:0/4247656752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeac0b2a10 msgr2=0x7efeac149440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:28.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.935+0000 7efeb2800640 1 --2- 172.21.15.100:0/4247656752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeac0b2a10 0x7efeac149440 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7efe9401c720 tx=0x7efe9401b890 comp rx=0 tx=0).stop 2024-04-23T14:34:28.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.935+0000 7efeb2800640 1 -- 172.21.15.100:0/4247656752 shutdown_connections 2024-04-23T14:34:28.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.935+0000 7efeb2800640 1 --2- 172.21.15.100:0/4247656752 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7efe88068380 0x7efe8806a840 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.935+0000 7efeb2800640 1 --2- 172.21.15.100:0/4247656752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeac15b640 0x7efeac06e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.935+0000 7efeb2800640 1 --2- 172.21.15.100:0/4247656752 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efeac0cc750 0x7efeac06de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.935+0000 7efeb2800640 1 --2- 172.21.15.100:0/4247656752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efeac0b2a10 0x7efeac149440 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:28.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.935+0000 7efeb2800640 1 -- 172.21.15.100:0/4247656752 >> 172.21.15.100:0/4247656752 conn(0x7efeac0b1e80 msgr2=0x7efeac0b16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:28.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.935+0000 7efeb2800640 1 -- 172.21.15.100:0/4247656752 shutdown_connections 2024-04-23T14:34:28.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:28.935+0000 7efeb2800640 1 -- 172.21.15.100:0/4247656752 wait complete. 2024-04-23T14:34:28.956 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-04-23T14:34:28.763403+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226472,"kb_used_data":6136,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748248,"statfs":{"total":773094113280,"available":772862205952,"internally_reserved":0,"allocated":6283264,"data_stored":3446052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.995581"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421971+0000","last_change":"2024-04-23T14:34:24.816066+0000","last_active":"2024-04-23T14:34:26.421971+0000","last_peered":"2024-04-23T14:34:26.421971+0000","last_clean":"2024-04-23T14:34:26.421971+0000","last_became_active":"2024-04-23T14:34:24.815443+0000","last_became_peered":"2024-04-23T14:34:24.815443+0000","last_unstale":"2024-04-23T14:34:26.421971+0000","last_undegraded":"2024-04-23T14:34:26.421971+0000","last_fullsized":"2024-04-23T14:34:26.421971+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:11:34.330788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421519+0000","last_change":"2024-04-23T14:34:24.808825+0000","last_active":"2024-04-23T14:34:26.421519+0000","last_peered":"2024-04-23T14:34:26.421519+0000","last_clean":"2024-04-23T14:34:26.421519+0000","last_became_active":"2024-04-23T14:34:24.808517+0000","last_became_peered":"2024-04-23T14:34:24.808517+0000","last_unstale":"2024-04-23T14:34:26.421519+0000","last_undegraded":"2024-04-23T14:34:26.421519+0000","last_fullsized":"2024-04-23T14:34:26.421519+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:18:59.502027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.207364+0000","last_change":"2024-04-23T14:34:24.813082+0000","last_active":"2024-04-23T14:34:26.207364+0000","last_peered":"2024-04-23T14:34:26.207364+0000","last_clean":"2024-04-23T14:34:26.207364+0000","last_became_active":"2024-04-23T14:34:24.812788+0000","last_became_peered":"2024-04-23T14:34:24.812788+0000","last_unstale":"2024-04-23T14:34:26.207364+0000","last_undegraded":"2024-04-23T14:34:26.207364+0000","last_fullsized":"2024-04-23T14:34:26.207364+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:07:07.524078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805070+0000","last_change":"2024-04-23T14:34:24.798979+0000","last_active":"2024-04-23T14:34:25.805070+0000","last_peered":"2024-04-23T14:34:25.805070+0000","last_clean":"2024-04-23T14:34:25.805070+0000","last_became_active":"2024-04-23T14:34:24.798311+0000","last_became_peered":"2024-04-23T14:34:24.798311+0000","last_unstale":"2024-04-23T14:34:25.805070+0000","last_undegraded":"2024-04-23T14:34:25.805070+0000","last_fullsized":"2024-04-23T14:34:25.805070+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:42:57.268048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421292+0000","last_change":"2024-04-23T14:34:24.793063+0000","last_active":"2024-04-23T14:34:26.421292+0000","last_peered":"2024-04-23T14:34:26.421292+0000","last_clean":"2024-04-23T14:34:26.421292+0000","last_became_active":"2024-04-23T14:34:24.792787+0000","last_became_peered":"2024-04-23T14:34:24.792787+0000","last_unstale":"2024-04-23T14:34:26.421292+0000","last_undegraded":"2024-04-23T14:34:26.421292+0000","last_fullsized":"2024-04-23T14:34:26.421292+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:32:19.324839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329529+0000","last_change":"2024-04-23T14:34:24.810090+0000","last_active":"2024-04-23T14:34:26.329529+0000","last_peered":"2024-04-23T14:34:26.329529+0000","last_clean":"2024-04-23T14:34:26.329529+0000","last_became_active":"2024-04-23T14:34:24.809830+0000","last_became_peered":"2024-04-23T14:34:24.809830+0000","last_unstale":"2024-04-23T14:34:26.329529+0000","last_undegraded":"2024-04-23T14:34:26.329529+0000","last_fullsized":"2024-04-23T14:34:26.329529+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:07:07.524078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.804694+0000","last_change":"2024-04-23T14:34:24.794282+0000","last_active":"2024-04-23T14:34:25.804694+0000","last_peered":"2024-04-23T14:34:25.804694+0000","last_clean":"2024-04-23T14:34:25.804694+0000","last_became_active":"2024-04-23T14:34:24.794009+0000","last_became_peered":"2024-04-23T14:34:24.794009+0000","last_unstale":"2024-04-23T14:34:25.804694+0000","last_undegraded":"2024-04-23T14:34:25.804694+0000","last_fullsized":"2024-04-23T14:34:25.804694+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:43:43.259027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421784+0000","last_change":"2024-04-23T14:34:24.812554+0000","last_active":"2024-04-23T14:34:26.421784+0000","last_peered":"2024-04-23T14:34:26.421784+0000","last_clean":"2024-04-23T14:34:26.421784+0000","last_became_active":"2024-04-23T14:34:24.807887+0000","last_became_peered":"2024-04-23T14:34:24.807887+0000","last_unstale":"2024-04-23T14:34:26.421784+0000","last_undegraded":"2024-04-23T14:34:26.421784+0000","last_fullsized":"2024-04-23T14:34:26.421784+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:06:34.829966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.806067+0000","last_change":"2024-04-23T14:34:24.814581+0000","last_active":"2024-04-23T14:34:25.806067+0000","last_peered":"2024-04-23T14:34:25.806067+0000","last_clean":"2024-04-23T14:34:25.806067+0000","last_became_active":"2024-04-23T14:34:24.814306+0000","last_became_peered":"2024-04-23T14:34:24.814306+0000","last_unstale":"2024-04-23T14:34:25.806067+0000","last_undegraded":"2024-04-23T14:34:25.806067+0000","last_fullsized":"2024-04-23T14:34:25.806067+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:01:59.183007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.804126+0000","last_change":"2024-04-23T14:34:24.806753+0000","last_active":"2024-04-23T14:34:25.804126+0000","last_peered":"2024-04-23T14:34:25.804126+0000","last_clean":"2024-04-23T14:34:25.804126+0000","last_became_active":"2024-04-23T14:34:24.801081+0000","last_became_peered":"2024-04-23T14:34:24.801081+0000","last_unstale":"2024-04-23T14:34:25.804126+0000","last_undegraded":"2024-04-23T14:34:25.804126+0000","last_fullsized":"2024-04-23T14:34:25.804126+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:29:04.759935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.803905+0000","last_change":"2024-04-23T14:34:24.819028+0000","last_active":"2024-04-23T14:34:25.803905+0000","last_peered":"2024-04-23T14:34:25.803905+0000","last_clean":"2024-04-23T14:34:25.803905+0000","last_became_active":"2024-04-23T14:34:24.817326+0000","last_became_peered":"2024-04-23T14:34:24.817326+0000","last_unstale":"2024-04-23T14:34:25.803905+0000","last_undegraded":"2024-04-23T14:34:25.803905+0000","last_fullsized":"2024-04-23T14:34:25.803905+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:01:59.183007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805220+0000","last_change":"2024-04-23T14:34:24.814063+0000","last_active":"2024-04-23T14:34:25.805220+0000","last_peered":"2024-04-23T14:34:25.805220+0000","last_clean":"2024-04-23T14:34:25.805220+0000","last_became_active":"2024-04-23T14:34:24.813728+0000","last_became_peered":"2024-04-23T14:34:24.813728+0000","last_unstale":"2024-04-23T14:34:25.805220+0000","last_undegraded":"2024-04-23T14:34:25.805220+0000","last_fullsized":"2024-04-23T14:34:25.805220+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:22:43.967203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.422277+0000","last_change":"2024-04-23T14:34:24.808179+0000","last_active":"2024-04-23T14:34:26.422277+0000","last_peered":"2024-04-23T14:34:26.422277+0000","last_clean":"2024-04-23T14:34:26.422277+0000","last_became_active":"2024-04-23T14:34:24.805067+0000","last_became_peered":"2024-04-23T14:34:24.805067+0000","last_unstale":"2024-04-23T14:34:26.422277+0000","last_undegraded":"2024-04-23T14:34:26.422277+0000","last_fullsized":"2024-04-23T14:34:26.422277+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:02:42.230866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805997+0000","last_change":"2024-04-23T14:34:24.807039+0000","last_active":"2024-04-23T14:34:25.805997+0000","last_peered":"2024-04-23T14:34:25.805997+0000","last_clean":"2024-04-23T14:34:25.805997+0000","last_became_active":"2024-04-23T14:34:24.806760+0000","last_became_peered":"2024-04-23T14:34:24.806760+0000","last_unstale":"2024-04-23T14:34:25.805997+0000","last_undegraded":"2024-04-23T14:34:25.805997+0000","last_fullsized":"2024-04-23T14:34:25.805997+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:39:09.489633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805556+0000","last_change":"2024-04-23T14:34:24.807060+0000","last_active":"2024-04-23T14:34:25.805556+0000","last_peered":"2024-04-23T14:34:25.805556+0000","last_clean":"2024-04-23T14:34:25.805556+0000","last_became_active":"2024-04-23T14:34:24.806786+0000","last_became_peered":"2024-04-23T14:34:24.806786+0000","last_unstale":"2024-04-23T14:34:25.805556+0000","last_undegraded":"2024-04-23T14:34:25.805556+0000","last_fullsized":"2024-04-23T14:34:25.805556+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:24:07.696001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329487+0000","last_change":"2024-04-23T14:34:24.804474+0000","last_active":"2024-04-23T14:34:26.329487+0000","last_peered":"2024-04-23T14:34:26.329487+0000","last_clean":"2024-04-23T14:34:26.329487+0000","last_became_active":"2024-04-23T14:34:24.804212+0000","last_became_peered":"2024-04-23T14:34:24.804212+0000","last_unstale":"2024-04-23T14:34:26.329487+0000","last_undegraded":"2024-04-23T14:34:26.329487+0000","last_fullsized":"2024-04-23T14:34:26.329487+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:06.679211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.422144+0000","last_change":"2024-04-23T14:34:24.810400+0000","last_active":"2024-04-23T14:34:26.422144+0000","last_peered":"2024-04-23T14:34:26.422144+0000","last_clean":"2024-04-23T14:34:26.422144+0000","last_became_active":"2024-04-23T14:34:24.810125+0000","last_became_peered":"2024-04-23T14:34:24.810125+0000","last_unstale":"2024-04-23T14:34:26.422144+0000","last_undegraded":"2024-04-23T14:34:26.422144+0000","last_fullsized":"2024-04-23T14:34:26.422144+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:17:58.654366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.207588+0000","last_change":"2024-04-23T14:34:24.818240+0000","last_active":"2024-04-23T14:34:26.207588+0000","last_peered":"2024-04-23T14:34:26.207588+0000","last_clean":"2024-04-23T14:34:26.207588+0000","last_became_active":"2024-04-23T14:34:24.817890+0000","last_became_peered":"2024-04-23T14:34:24.817890+0000","last_unstale":"2024-04-23T14:34:26.207588+0000","last_undegraded":"2024-04-23T14:34:26.207588+0000","last_fullsized":"2024-04-23T14:34:26.207588+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:14:58.214732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421671+0000","last_change":"2024-04-23T14:34:24.817394+0000","last_active":"2024-04-23T14:34:26.421671+0000","last_peered":"2024-04-23T14:34:26.421671+0000","last_clean":"2024-04-23T14:34:26.421671+0000","last_became_active":"2024-04-23T14:34:24.817120+0000","last_became_peered":"2024-04-23T14:34:24.817120+0000","last_unstale":"2024-04-23T14:34:26.421671+0000","last_undegraded":"2024-04-23T14:34:26.421671+0000","last_fullsized":"2024-04-23T14:34:26.421671+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:46:10.550380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421832+0000","last_change":"2024-04-23T14:34:24.811477+0000","last_active":"2024-04-23T14:34:26.421832+0000","last_peered":"2024-04-23T14:34:26.421832+0000","last_clean":"2024-04-23T14:34:26.421832+0000","last_became_active":"2024-04-23T14:34:24.810943+0000","last_became_peered":"2024-04-23T14:34:24.810943+0000","last_unstale":"2024-04-23T14:34:26.421832+0000","last_undegraded":"2024-04-23T14:34:26.421832+0000","last_fullsized":"2024-04-23T14:34:26.421832+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:22:48.079754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.207016+0000","last_change":"2024-04-23T14:34:24.816877+0000","last_active":"2024-04-23T14:34:26.207016+0000","last_peered":"2024-04-23T14:34:26.207016+0000","last_clean":"2024-04-23T14:34:26.207016+0000","last_became_active":"2024-04-23T14:34:24.816179+0000","last_became_peered":"2024-04-23T14:34:24.816179+0000","last_unstale":"2024-04-23T14:34:26.207016+0000","last_undegraded":"2024-04-23T14:34:26.207016+0000","last_fullsized":"2024-04-23T14:34:26.207016+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:13:16.409968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421345+0000","last_change":"2024-04-23T14:34:24.814282+0000","last_active":"2024-04-23T14:34:26.421345+0000","last_peered":"2024-04-23T14:34:26.421345+0000","last_clean":"2024-04-23T14:34:26.421345+0000","last_became_active":"2024-04-23T14:34:24.813821+0000","last_became_peered":"2024-04-23T14:34:24.813821+0000","last_unstale":"2024-04-23T14:34:26.421345+0000","last_undegraded":"2024-04-23T14:34:26.421345+0000","last_fullsized":"2024-04-23T14:34:26.421345+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:03.630387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.206871+0000","last_change":"2024-04-23T14:34:24.813158+0000","last_active":"2024-04-23T14:34:26.206871+0000","last_peered":"2024-04-23T14:34:26.206871+0000","last_clean":"2024-04-23T14:34:26.206871+0000","last_became_active":"2024-04-23T14:34:24.810614+0000","last_became_peered":"2024-04-23T14:34:24.810614+0000","last_unstale":"2024-04-23T14:34:26.206871+0000","last_undegraded":"2024-04-23T14:34:26.206871+0000","last_fullsized":"2024-04-23T14:34:26.206871+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:03:02.301543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.422017+0000","last_change":"2024-04-23T14:34:24.807779+0000","last_active":"2024-04-23T14:34:26.422017+0000","last_peered":"2024-04-23T14:34:26.422017+0000","last_clean":"2024-04-23T14:34:26.422017+0000","last_became_active":"2024-04-23T14:34:24.807332+0000","last_became_peered":"2024-04-23T14:34:24.807332+0000","last_unstale":"2024-04-23T14:34:26.422017+0000","last_undegraded":"2024-04-23T14:34:26.422017+0000","last_fullsized":"2024-04-23T14:34:26.422017+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:24:31.326993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.422075+0000","last_change":"2024-04-23T14:34:24.816308+0000","last_active":"2024-04-23T14:34:26.422075+0000","last_peered":"2024-04-23T14:34:26.422075+0000","last_clean":"2024-04-23T14:34:26.422075+0000","last_became_active":"2024-04-23T14:34:24.815807+0000","last_became_peered":"2024-04-23T14:34:24.815807+0000","last_unstale":"2024-04-23T14:34:26.422075+0000","last_undegraded":"2024-04-23T14:34:26.422075+0000","last_fullsized":"2024-04-23T14:34:26.422075+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:03.539500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.806258+0000","last_change":"2024-04-23T14:34:24.800542+0000","last_active":"2024-04-23T14:34:25.806258+0000","last_peered":"2024-04-23T14:34:25.806258+0000","last_clean":"2024-04-23T14:34:25.806258+0000","last_became_active":"2024-04-23T14:34:24.800292+0000","last_became_peered":"2024-04-23T14:34:24.800292+0000","last_unstale":"2024-04-23T14:34:25.806258+0000","last_undegraded":"2024-04-23T14:34:25.806258+0000","last_fullsized":"2024-04-23T14:34:25.806258+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:33:23.882396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329723+0000","last_change":"2024-04-23T14:34:24.808739+0000","last_active":"2024-04-23T14:34:26.329723+0000","last_peered":"2024-04-23T14:34:26.329723+0000","last_clean":"2024-04-23T14:34:26.329723+0000","last_became_active":"2024-04-23T14:34:24.808468+0000","last_became_peered":"2024-04-23T14:34:24.808468+0000","last_unstale":"2024-04-23T14:34:26.329723+0000","last_undegraded":"2024-04-23T14:34:26.329723+0000","last_fullsized":"2024-04-23T14:34:26.329723+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:42:57.268048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421464+0000","last_change":"2024-04-23T14:34:24.791972+0000","last_active":"2024-04-23T14:34:26.421464+0000","last_peered":"2024-04-23T14:34:26.421464+0000","last_clean":"2024-04-23T14:34:26.421464+0000","last_became_active":"2024-04-23T14:34:24.791684+0000","last_became_peered":"2024-04-23T14:34:24.791684+0000","last_unstale":"2024-04-23T14:34:26.421464+0000","last_undegraded":"2024-04-23T14:34:26.421464+0000","last_fullsized":"2024-04-23T14:34:26.421464+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:45:55.677313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421769+0000","last_change":"2024-04-23T14:34:24.812289+0000","last_active":"2024-04-23T14:34:26.421769+0000","last_peered":"2024-04-23T14:34:26.421769+0000","last_clean":"2024-04-23T14:34:26.421769+0000","last_became_active":"2024-04-23T14:34:24.811957+0000","last_became_peered":"2024-04-23T14:34:24.811957+0000","last_unstale":"2024-04-23T14:34:26.421769+0000","last_undegraded":"2024-04-23T14:34:26.421769+0000","last_fullsized":"2024-04-23T14:34:26.421769+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:56.109504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.206883+0000","last_change":"2024-04-23T14:34:24.812721+0000","last_active":"2024-04-23T14:34:26.206883+0000","last_peered":"2024-04-23T14:34:26.206883+0000","last_clean":"2024-04-23T14:34:26.206883+0000","last_became_active":"2024-04-23T14:34:24.812406+0000","last_became_peered":"2024-04-23T14:34:24.812406+0000","last_unstale":"2024-04-23T14:34:26.206883+0000","last_undegraded":"2024-04-23T14:34:26.206883+0000","last_fullsized":"2024-04-23T14:34:26.206883+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:57.460216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421063+0000","last_change":"2024-04-23T14:34:24.797417+0000","last_active":"2024-04-23T14:34:26.421063+0000","last_peered":"2024-04-23T14:34:26.421063+0000","last_clean":"2024-04-23T14:34:26.421063+0000","last_became_active":"2024-04-23T14:34:24.797135+0000","last_became_peered":"2024-04-23T14:34:24.797135+0000","last_unstale":"2024-04-23T14:34:26.421063+0000","last_undegraded":"2024-04-23T14:34:26.421063+0000","last_fullsized":"2024-04-23T14:34:26.421063+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:08:06.351163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329594+0000","last_change":"2024-04-23T14:34:24.821730+0000","last_active":"2024-04-23T14:34:26.329594+0000","last_peered":"2024-04-23T14:34:26.329594+0000","last_clean":"2024-04-23T14:34:26.329594+0000","last_became_active":"2024-04-23T14:34:24.821223+0000","last_became_peered":"2024-04-23T14:34:24.821223+0000","last_unstale":"2024-04-23T14:34:26.329594+0000","last_undegraded":"2024-04-23T14:34:26.329594+0000","last_fullsized":"2024-04-23T14:34:26.329594+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:14:58.214732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329886+0000","last_change":"2024-04-23T14:34:24.820085+0000","last_active":"2024-04-23T14:34:26.329886+0000","last_peered":"2024-04-23T14:34:26.329886+0000","last_clean":"2024-04-23T14:34:26.329886+0000","last_became_active":"2024-04-23T14:34:24.818343+0000","last_became_peered":"2024-04-23T14:34:24.818343+0000","last_unstale":"2024-04-23T14:34:26.329886+0000","last_undegraded":"2024-04-23T14:34:26.329886+0000","last_fullsized":"2024-04-23T14:34:26.329886+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:57:41.597696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"52'4","reported_seq":18,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329884+0000","last_change":"2024-04-23T14:34:23.783227+0000","last_active":"2024-04-23T14:34:26.329884+0000","last_peered":"2024-04-23T14:34:26.329884+0000","last_clean":"2024-04-23T14:34:26.329884+0000","last_became_active":"2024-04-23T14:34:23.782921+0000","last_became_peered":"2024-04-23T14:34:23.782921+0000","last_unstale":"2024-04-23T14:34:26.329884+0000","last_undegraded":"2024-04-23T14:34:26.329884+0000","last_fullsized":"2024-04-23T14:34:26.329884+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:27:55.173851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.804251+0000","last_change":"2024-04-23T14:34:24.814940+0000","last_active":"2024-04-23T14:34:25.804251+0000","last_peered":"2024-04-23T14:34:25.804251+0000","last_clean":"2024-04-23T14:34:25.804251+0000","last_became_active":"2024-04-23T14:34:24.814678+0000","last_became_peered":"2024-04-23T14:34:24.814678+0000","last_unstale":"2024-04-23T14:34:25.804251+0000","last_undegraded":"2024-04-23T14:34:25.804251+0000","last_fullsized":"2024-04-23T14:34:25.804251+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:54:40.578942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421909+0000","last_change":"2024-04-23T14:34:23.794894+0000","last_active":"2024-04-23T14:34:26.421909+0000","last_peered":"2024-04-23T14:34:26.421909+0000","last_clean":"2024-04-23T14:34:26.421909+0000","last_became_active":"2024-04-23T14:34:23.794654+0000","last_became_peered":"2024-04-23T14:34:23.794654+0000","last_unstale":"2024-04-23T14:34:26.421909+0000","last_undegraded":"2024-04-23T14:34:26.421909+0000","last_fullsized":"2024-04-23T14:34:26.421909+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:56.458224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329734+0000","last_change":"2024-04-23T14:34:24.811820+0000","last_active":"2024-04-23T14:34:26.329734+0000","last_peered":"2024-04-23T14:34:26.329734+0000","last_clean":"2024-04-23T14:34:26.329734+0000","last_became_active":"2024-04-23T14:34:24.811485+0000","last_became_peered":"2024-04-23T14:34:24.811485+0000","last_unstale":"2024-04-23T14:34:26.329734+0000","last_undegraded":"2024-04-23T14:34:26.329734+0000","last_fullsized":"2024-04-23T14:34:26.329734+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:22:43.967203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329957+0000","last_change":"2024-04-23T14:34:23.789955+0000","last_active":"2024-04-23T14:34:26.329957+0000","last_peered":"2024-04-23T14:34:26.329957+0000","last_clean":"2024-04-23T14:34:26.329957+0000","last_became_active":"2024-04-23T14:34:23.788171+0000","last_became_peered":"2024-04-23T14:34:23.788171+0000","last_unstale":"2024-04-23T14:34:26.329957+0000","last_undegraded":"2024-04-23T14:34:26.329957+0000","last_fullsized":"2024-04-23T14:34:26.329957+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:54:39.576949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329865+0000","last_change":"2024-04-23T14:34:24.801791+0000","last_active":"2024-04-23T14:34:26.329865+0000","last_peered":"2024-04-23T14:34:26.329865+0000","last_clean":"2024-04-23T14:34:26.329865+0000","last_became_active":"2024-04-23T14:34:24.801530+0000","last_became_peered":"2024-04-23T14:34:24.801530+0000","last_unstale":"2024-04-23T14:34:26.329865+0000","last_undegraded":"2024-04-23T14:34:26.329865+0000","last_fullsized":"2024-04-23T14:34:26.329865+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:33:04.830057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805238+0000","last_change":"2024-04-23T14:34:23.795809+0000","last_active":"2024-04-23T14:34:25.805238+0000","last_peered":"2024-04-23T14:34:25.805238+0000","last_clean":"2024-04-23T14:34:25.805238+0000","last_became_active":"2024-04-23T14:34:23.795529+0000","last_became_peered":"2024-04-23T14:34:23.795529+0000","last_unstale":"2024-04-23T14:34:25.805238+0000","last_undegraded":"2024-04-23T14:34:25.805238+0000","last_fullsized":"2024-04-23T14:34:25.805238+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:25:50.733726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421350+0000","last_change":"2024-04-23T14:34:24.814076+0000","last_active":"2024-04-23T14:34:26.421350+0000","last_peered":"2024-04-23T14:34:26.421350+0000","last_clean":"2024-04-23T14:34:26.421350+0000","last_became_active":"2024-04-23T14:34:24.813436+0000","last_became_peered":"2024-04-23T14:34:24.813436+0000","last_unstale":"2024-04-23T14:34:26.421350+0000","last_undegraded":"2024-04-23T14:34:26.421350+0000","last_fullsized":"2024-04-23T14:34:26.421350+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:36:46.948840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805958+0000","last_change":"2024-04-23T14:34:23.796231+0000","last_active":"2024-04-23T14:34:25.805958+0000","last_peered":"2024-04-23T14:34:25.805958+0000","last_clean":"2024-04-23T14:34:25.805958+0000","last_became_active":"2024-04-23T14:34:23.795952+0000","last_became_peered":"2024-04-23T14:34:23.795952+0000","last_unstale":"2024-04-23T14:34:25.805958+0000","last_undegraded":"2024-04-23T14:34:25.805958+0000","last_fullsized":"2024-04-23T14:34:25.805958+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:27:23.226059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421549+0000","last_change":"2024-04-23T14:34:24.813571+0000","last_active":"2024-04-23T14:34:26.421549+0000","last_peered":"2024-04-23T14:34:26.421549+0000","last_clean":"2024-04-23T14:34:26.421549+0000","last_became_active":"2024-04-23T14:34:24.812515+0000","last_became_peered":"2024-04-23T14:34:24.812515+0000","last_unstale":"2024-04-23T14:34:26.421549+0000","last_undegraded":"2024-04-23T14:34:26.421549+0000","last_fullsized":"2024-04-23T14:34:26.421549+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:15:47.496695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329518+0000","last_change":"2024-04-23T14:34:23.785763+0000","last_active":"2024-04-23T14:34:26.329518+0000","last_peered":"2024-04-23T14:34:26.329518+0000","last_clean":"2024-04-23T14:34:26.329518+0000","last_became_active":"2024-04-23T14:34:23.785518+0000","last_became_peered":"2024-04-23T14:34:23.785518+0000","last_unstale":"2024-04-23T14:34:26.329518+0000","last_undegraded":"2024-04-23T14:34:26.329518+0000","last_fullsized":"2024-04-23T14:34:26.329518+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:39:08.487640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.422104+0000","last_change":"2024-04-23T14:34:24.807190+0000","last_active":"2024-04-23T14:34:26.422104+0000","last_peered":"2024-04-23T14:34:26.422104+0000","last_clean":"2024-04-23T14:34:26.422104+0000","last_became_active":"2024-04-23T14:34:24.804960+0000","last_became_peered":"2024-04-23T14:34:24.804960+0000","last_unstale":"2024-04-23T14:34:26.422104+0000","last_undegraded":"2024-04-23T14:34:26.422104+0000","last_fullsized":"2024-04-23T14:34:26.422104+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:09:21.395477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.804956+0000","last_change":"2024-04-23T14:34:23.791773+0000","last_active":"2024-04-23T14:34:25.804956+0000","last_peered":"2024-04-23T14:34:25.804956+0000","last_clean":"2024-04-23T14:34:25.804956+0000","last_became_active":"2024-04-23T14:34:23.790851+0000","last_became_peered":"2024-04-23T14:34:23.790851+0000","last_unstale":"2024-04-23T14:34:25.804956+0000","last_undegraded":"2024-04-23T14:34:25.804956+0000","last_fullsized":"2024-04-23T14:34:25.804956+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:38:03.046355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421329+0000","last_change":"2024-04-23T14:34:24.801784+0000","last_active":"2024-04-23T14:34:26.421329+0000","last_peered":"2024-04-23T14:34:26.421329+0000","last_clean":"2024-04-23T14:34:26.421329+0000","last_became_active":"2024-04-23T14:34:24.801513+0000","last_became_peered":"2024-04-23T14:34:24.801513+0000","last_unstale":"2024-04-23T14:34:26.421329+0000","last_undegraded":"2024-04-23T14:34:26.421329+0000","last_fullsized":"2024-04-23T14:34:26.421329+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:27.091148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805453+0000","last_change":"2024-04-23T14:34:23.796931+0000","last_active":"2024-04-23T14:34:25.805453+0000","last_peered":"2024-04-23T14:34:25.805453+0000","last_clean":"2024-04-23T14:34:25.805453+0000","last_became_active":"2024-04-23T14:34:23.796602+0000","last_became_peered":"2024-04-23T14:34:23.796602+0000","last_unstale":"2024-04-23T14:34:25.805453+0000","last_undegraded":"2024-04-23T14:34:25.805453+0000","last_fullsized":"2024-04-23T14:34:25.805453+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:47:52.275367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.806270+0000","last_change":"2024-04-23T14:34:24.801735+0000","last_active":"2024-04-23T14:34:25.806270+0000","last_peered":"2024-04-23T14:34:25.806270+0000","last_clean":"2024-04-23T14:34:25.806270+0000","last_became_active":"2024-04-23T14:34:24.801461+0000","last_became_peered":"2024-04-23T14:34:24.801461+0000","last_unstale":"2024-04-23T14:34:25.806270+0000","last_undegraded":"2024-04-23T14:34:25.806270+0000","last_fullsized":"2024-04-23T14:34:25.806270+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:29:04.759935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.207553+0000","last_change":"2024-04-23T14:34:23.798356+0000","last_active":"2024-04-23T14:34:26.207553+0000","last_peered":"2024-04-23T14:34:26.207553+0000","last_clean":"2024-04-23T14:34:26.207553+0000","last_became_active":"2024-04-23T14:34:23.798021+0000","last_became_peered":"2024-04-23T14:34:23.798021+0000","last_unstale":"2024-04-23T14:34:26.207553+0000","last_undegraded":"2024-04-23T14:34:26.207553+0000","last_fullsized":"2024-04-23T14:34:26.207553+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:43:42.257035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.207642+0000","last_change":"2024-04-23T14:34:24.825621+0000","last_active":"2024-04-23T14:34:26.207642+0000","last_peered":"2024-04-23T14:34:26.207642+0000","last_clean":"2024-04-23T14:34:26.207642+0000","last_became_active":"2024-04-23T14:34:24.825408+0000","last_became_peered":"2024-04-23T14:34:24.825408+0000","last_unstale":"2024-04-23T14:34:26.207642+0000","last_undegraded":"2024-04-23T14:34:26.207642+0000","last_fullsized":"2024-04-23T14:34:26.207642+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:22:43.967203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.422067+0000","last_change":"2024-04-23T14:34:23.788488+0000","last_active":"2024-04-23T14:34:26.422067+0000","last_peered":"2024-04-23T14:34:26.422067+0000","last_clean":"2024-04-23T14:34:26.422067+0000","last_became_active":"2024-04-23T14:34:23.788219+0000","last_became_peered":"2024-04-23T14:34:23.788219+0000","last_unstale":"2024-04-23T14:34:26.422067+0000","last_undegraded":"2024-04-23T14:34:26.422067+0000","last_fullsized":"2024-04-23T14:34:26.422067+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:13:15.407976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.803737+0000","last_change":"2024-04-23T14:34:24.814634+0000","last_active":"2024-04-23T14:34:25.803737+0000","last_peered":"2024-04-23T14:34:25.803737+0000","last_clean":"2024-04-23T14:34:25.803737+0000","last_became_active":"2024-04-23T14:34:24.811479+0000","last_became_peered":"2024-04-23T14:34:24.811479+0000","last_unstale":"2024-04-23T14:34:25.803737+0000","last_undegraded":"2024-04-23T14:34:25.803737+0000","last_fullsized":"2024-04-23T14:34:25.803737+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:24:07.696001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"52'3","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.207400+0000","last_change":"2024-04-23T14:34:23.796614+0000","last_active":"2024-04-23T14:34:26.207400+0000","last_peered":"2024-04-23T14:34:26.207400+0000","last_clean":"2024-04-23T14:34:26.207400+0000","last_became_active":"2024-04-23T14:34:23.796331+0000","last_became_peered":"2024-04-23T14:34:23.796331+0000","last_unstale":"2024-04-23T14:34:26.207400+0000","last_undegraded":"2024-04-23T14:34:26.207400+0000","last_fullsized":"2024-04-23T14:34:26.207400+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:42:20.098074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.207086+0000","last_change":"2024-04-23T14:34:24.824592+0000","last_active":"2024-04-23T14:34:26.207086+0000","last_peered":"2024-04-23T14:34:26.207086+0000","last_clean":"2024-04-23T14:34:26.207086+0000","last_became_active":"2024-04-23T14:34:24.824282+0000","last_became_peered":"2024-04-23T14:34:24.824282+0000","last_unstale":"2024-04-23T14:34:26.207086+0000","last_undegraded":"2024-04-23T14:34:26.207086+0000","last_fullsized":"2024-04-23T14:34:26.207086+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:58.861642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"52'6","reported_seq":20,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.806443+0000","last_change":"2024-04-23T14:34:23.793905+0000","last_active":"2024-04-23T14:34:25.806443+0000","last_peered":"2024-04-23T14:34:25.806443+0000","last_clean":"2024-04-23T14:34:25.806443+0000","last_became_active":"2024-04-23T14:34:23.793652+0000","last_became_peered":"2024-04-23T14:34:23.793652+0000","last_unstale":"2024-04-23T14:34:25.806443+0000","last_undegraded":"2024-04-23T14:34:25.806443+0000","last_fullsized":"2024-04-23T14:34:25.806443+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:14.298493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.804743+0000","last_change":"2024-04-23T14:34:24.802096+0000","last_active":"2024-04-23T14:34:25.804743+0000","last_peered":"2024-04-23T14:34:25.804743+0000","last_clean":"2024-04-23T14:34:25.804743+0000","last_became_active":"2024-04-23T14:34:24.801608+0000","last_became_peered":"2024-04-23T14:34:24.801608+0000","last_unstale":"2024-04-23T14:34:25.804743+0000","last_undegraded":"2024-04-23T14:34:25.804743+0000","last_fullsized":"2024-04-23T14:34:25.804743+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:07:07.524078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421663+0000","last_change":"2024-04-23T14:34:23.797317+0000","last_active":"2024-04-23T14:34:26.421663+0000","last_peered":"2024-04-23T14:34:26.421663+0000","last_clean":"2024-04-23T14:34:26.421663+0000","last_became_active":"2024-04-23T14:34:23.796717+0000","last_became_peered":"2024-04-23T14:34:23.796717+0000","last_unstale":"2024-04-23T14:34:26.421663+0000","last_undegraded":"2024-04-23T14:34:26.421663+0000","last_fullsized":"2024-04-23T14:34:26.421663+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:05.677218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.207107+0000","last_change":"2024-04-23T14:34:24.816642+0000","last_active":"2024-04-23T14:34:26.207107+0000","last_peered":"2024-04-23T14:34:26.207107+0000","last_clean":"2024-04-23T14:34:26.207107+0000","last_became_active":"2024-04-23T14:34:24.813340+0000","last_became_peered":"2024-04-23T14:34:24.813340+0000","last_unstale":"2024-04-23T14:34:26.207107+0000","last_undegraded":"2024-04-23T14:34:26.207107+0000","last_fullsized":"2024-04-23T14:34:26.207107+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:14:07.060990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.422074+0000","last_change":"2024-04-23T14:34:23.795146+0000","last_active":"2024-04-23T14:34:26.422074+0000","last_peered":"2024-04-23T14:34:26.422074+0000","last_clean":"2024-04-23T14:34:26.422074+0000","last_became_active":"2024-04-23T14:34:23.794811+0000","last_became_peered":"2024-04-23T14:34:23.794811+0000","last_unstale":"2024-04-23T14:34:26.422074+0000","last_undegraded":"2024-04-23T14:34:26.422074+0000","last_fullsized":"2024-04-23T14:34:26.422074+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:27:23.226059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805440+0000","last_change":"2024-04-23T14:34:24.811947+0000","last_active":"2024-04-23T14:34:25.805440+0000","last_peered":"2024-04-23T14:34:25.805440+0000","last_clean":"2024-04-23T14:34:25.805440+0000","last_became_active":"2024-04-23T14:34:24.811632+0000","last_became_peered":"2024-04-23T14:34:24.811632+0000","last_unstale":"2024-04-23T14:34:25.805440+0000","last_undegraded":"2024-04-23T14:34:25.805440+0000","last_fullsized":"2024-04-23T14:34:25.805440+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:04:53.451687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805082+0000","last_change":"2024-04-23T14:34:23.793321+0000","last_active":"2024-04-23T14:34:25.805082+0000","last_peered":"2024-04-23T14:34:25.805082+0000","last_clean":"2024-04-23T14:34:25.805082+0000","last_became_active":"2024-04-23T14:34:23.790416+0000","last_became_peered":"2024-04-23T14:34:23.790416+0000","last_unstale":"2024-04-23T14:34:25.805082+0000","last_undegraded":"2024-04-23T14:34:25.805082+0000","last_fullsized":"2024-04-23T14:34:25.805082+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:49:34.958705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.206887+0000","last_change":"2024-04-23T14:34:24.813008+0000","last_active":"2024-04-23T14:34:26.206887+0000","last_peered":"2024-04-23T14:34:26.206887+0000","last_clean":"2024-04-23T14:34:26.206887+0000","last_became_active":"2024-04-23T14:34:24.812052+0000","last_became_peered":"2024-04-23T14:34:24.812052+0000","last_unstale":"2024-04-23T14:34:26.206887+0000","last_undegraded":"2024-04-23T14:34:26.206887+0000","last_fullsized":"2024-04-23T14:34:26.206887+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:06:15.735233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"52'8","reported_seq":22,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.206772+0000","last_change":"2024-04-23T14:34:23.795623+0000","last_active":"2024-04-23T14:34:26.206772+0000","last_peered":"2024-04-23T14:34:26.206772+0000","last_clean":"2024-04-23T14:34:26.206772+0000","last_became_active":"2024-04-23T14:34:23.795321+0000","last_became_peered":"2024-04-23T14:34:23.795321+0000","last_unstale":"2024-04-23T14:34:26.206772+0000","last_undegraded":"2024-04-23T14:34:26.206772+0000","last_fullsized":"2024-04-23T14:34:26.206772+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:02:41.228874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.207618+0000","last_change":"2024-04-23T14:34:24.813927+0000","last_active":"2024-04-23T14:34:26.207618+0000","last_peered":"2024-04-23T14:34:26.207618+0000","last_clean":"2024-04-23T14:34:26.207618+0000","last_became_active":"2024-04-23T14:34:24.813605+0000","last_became_peered":"2024-04-23T14:34:24.813605+0000","last_unstale":"2024-04-23T14:34:26.207618+0000","last_undegraded":"2024-04-23T14:34:26.207618+0000","last_fullsized":"2024-04-23T14:34:26.207618+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:42:57.268048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.422214+0000","last_change":"2024-04-23T14:34:23.788663+0000","last_active":"2024-04-23T14:34:26.422214+0000","last_peered":"2024-04-23T14:34:26.422214+0000","last_clean":"2024-04-23T14:34:26.422214+0000","last_became_active":"2024-04-23T14:34:23.788338+0000","last_became_peered":"2024-04-23T14:34:23.788338+0000","last_unstale":"2024-04-23T14:34:26.422214+0000","last_undegraded":"2024-04-23T14:34:26.422214+0000","last_fullsized":"2024-04-23T14:34:26.422214+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:14:06.058997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805862+0000","last_change":"2024-04-23T14:34:24.804264+0000","last_active":"2024-04-23T14:34:25.805862+0000","last_peered":"2024-04-23T14:34:25.805862+0000","last_clean":"2024-04-23T14:34:25.805862+0000","last_became_active":"2024-04-23T14:34:24.803910+0000","last_became_peered":"2024-04-23T14:34:24.803910+0000","last_unstale":"2024-04-23T14:34:25.805862+0000","last_undegraded":"2024-04-23T14:34:25.805862+0000","last_fullsized":"2024-04-23T14:34:25.805862+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:33:04.830057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329810+0000","last_change":"2024-04-23T14:34:23.784914+0000","last_active":"2024-04-23T14:34:26.329810+0000","last_peered":"2024-04-23T14:34:26.329810+0000","last_clean":"2024-04-23T14:34:26.329810+0000","last_became_active":"2024-04-23T14:34:23.784284+0000","last_became_peered":"2024-04-23T14:34:23.784284+0000","last_unstale":"2024-04-23T14:34:26.329810+0000","last_undegraded":"2024-04-23T14:34:26.329810+0000","last_fullsized":"2024-04-23T14:34:26.329810+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:01:58.181015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421971+0000","last_change":"2024-04-23T14:34:24.815082+0000","last_active":"2024-04-23T14:34:26.421971+0000","last_peered":"2024-04-23T14:34:26.421971+0000","last_clean":"2024-04-23T14:34:26.421971+0000","last_became_active":"2024-04-23T14:34:24.809746+0000","last_became_peered":"2024-04-23T14:34:24.809746+0000","last_unstale":"2024-04-23T14:34:26.421971+0000","last_undegraded":"2024-04-23T14:34:26.421971+0000","last_fullsized":"2024-04-23T14:34:26.421971+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:41:08.525219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805494+0000","last_change":"2024-04-23T14:34:23.790346+0000","last_active":"2024-04-23T14:34:25.805494+0000","last_peered":"2024-04-23T14:34:25.805494+0000","last_clean":"2024-04-23T14:34:25.805494+0000","last_became_active":"2024-04-23T14:34:23.789259+0000","last_became_peered":"2024-04-23T14:34:23.789259+0000","last_unstale":"2024-04-23T14:34:25.805494+0000","last_undegraded":"2024-04-23T14:34:25.805494+0000","last_fullsized":"2024-04-23T14:34:25.805494+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:13:23.310847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.207275+0000","last_change":"2024-04-23T14:34:24.818308+0000","last_active":"2024-04-23T14:34:26.207275+0000","last_peered":"2024-04-23T14:34:26.207275+0000","last_clean":"2024-04-23T14:34:26.207275+0000","last_became_active":"2024-04-23T14:34:24.818052+0000","last_became_peered":"2024-04-23T14:34:24.818052+0000","last_unstale":"2024-04-23T14:34:26.207275+0000","last_undegraded":"2024-04-23T14:34:26.207275+0000","last_fullsized":"2024-04-23T14:34:26.207275+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:29:27.755632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329787+0000","last_change":"2024-04-23T14:34:23.792656+0000","last_active":"2024-04-23T14:34:26.329787+0000","last_peered":"2024-04-23T14:34:26.329787+0000","last_clean":"2024-04-23T14:34:26.329787+0000","last_became_active":"2024-04-23T14:34:23.792407+0000","last_became_peered":"2024-04-23T14:34:23.792407+0000","last_unstale":"2024-04-23T14:34:26.329787+0000","last_undegraded":"2024-04-23T14:34:26.329787+0000","last_fullsized":"2024-04-23T14:34:26.329787+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:29:03.757942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.804953+0000","last_change":"2024-04-23T14:34:24.801888+0000","last_active":"2024-04-23T14:34:25.804953+0000","last_peered":"2024-04-23T14:34:25.804953+0000","last_clean":"2024-04-23T14:34:25.804953+0000","last_became_active":"2024-04-23T14:34:24.801271+0000","last_became_peered":"2024-04-23T14:34:24.801271+0000","last_unstale":"2024-04-23T14:34:25.804953+0000","last_undegraded":"2024-04-23T14:34:25.804953+0000","last_fullsized":"2024-04-23T14:34:25.804953+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:14:58.214732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421773+0000","last_change":"2024-04-23T14:34:23.791100+0000","last_active":"2024-04-23T14:34:26.421773+0000","last_peered":"2024-04-23T14:34:26.421773+0000","last_clean":"2024-04-23T14:34:26.421773+0000","last_became_active":"2024-04-23T14:34:23.790887+0000","last_became_peered":"2024-04-23T14:34:23.790887+0000","last_unstale":"2024-04-23T14:34:26.421773+0000","last_undegraded":"2024-04-23T14:34:26.421773+0000","last_fullsized":"2024-04-23T14:34:26.421773+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:29:26.753640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329782+0000","last_change":"2024-04-23T14:34:24.809276+0000","last_active":"2024-04-23T14:34:26.329782+0000","last_peered":"2024-04-23T14:34:26.329782+0000","last_clean":"2024-04-23T14:34:26.329782+0000","last_became_active":"2024-04-23T14:34:24.808961+0000","last_became_peered":"2024-04-23T14:34:24.808961+0000","last_unstale":"2024-04-23T14:34:26.329782+0000","last_undegraded":"2024-04-23T14:34:26.329782+0000","last_fullsized":"2024-04-23T14:34:26.329782+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:05:05.416514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.803763+0000","last_change":"2024-04-23T14:34:23.783936+0000","last_active":"2024-04-23T14:34:25.803763+0000","last_peered":"2024-04-23T14:34:25.803763+0000","last_clean":"2024-04-23T14:34:25.803763+0000","last_became_active":"2024-04-23T14:34:23.783599+0000","last_became_peered":"2024-04-23T14:34:23.783599+0000","last_unstale":"2024-04-23T14:34:25.803763+0000","last_undegraded":"2024-04-23T14:34:25.803763+0000","last_fullsized":"2024-04-23T14:34:25.803763+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:27:23.226059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329525+0000","last_change":"2024-04-23T14:34:24.818601+0000","last_active":"2024-04-23T14:34:26.329525+0000","last_peered":"2024-04-23T14:34:26.329525+0000","last_clean":"2024-04-23T14:34:26.329525+0000","last_became_active":"2024-04-23T14:34:24.818315+0000","last_became_peered":"2024-04-23T14:34:24.818315+0000","last_unstale":"2024-04-23T14:34:26.329525+0000","last_undegraded":"2024-04-23T14:34:26.329525+0000","last_fullsized":"2024-04-23T14:34:26.329525+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:29:27.755632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.804995+0000","last_change":"2024-04-23T14:34:23.792653+0000","last_active":"2024-04-23T14:34:25.804995+0000","last_peered":"2024-04-23T14:34:25.804995+0000","last_clean":"2024-04-23T14:34:25.804995+0000","last_became_active":"2024-04-23T14:34:23.791851+0000","last_became_peered":"2024-04-23T14:34:23.791851+0000","last_unstale":"2024-04-23T14:34:25.804995+0000","last_undegraded":"2024-04-23T14:34:25.804995+0000","last_fullsized":"2024-04-23T14:34:25.804995+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:27:55.173851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805565+0000","last_change":"2024-04-23T14:34:24.798776+0000","last_active":"2024-04-23T14:34:25.805565+0000","last_peered":"2024-04-23T14:34:25.805565+0000","last_clean":"2024-04-23T14:34:25.805565+0000","last_became_active":"2024-04-23T14:34:24.798487+0000","last_became_peered":"2024-04-23T14:34:24.798487+0000","last_unstale":"2024-04-23T14:34:25.805565+0000","last_undegraded":"2024-04-23T14:34:25.805565+0000","last_fullsized":"2024-04-23T14:34:25.805565+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:05:05.416514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.804230+0000","last_change":"2024-04-23T14:34:23.797511+0000","last_active":"2024-04-23T14:34:25.804230+0000","last_peered":"2024-04-23T14:34:25.804230+0000","last_clean":"2024-04-23T14:34:25.804230+0000","last_became_active":"2024-04-23T14:34:23.796153+0000","last_became_peered":"2024-04-23T14:34:23.796153+0000","last_unstale":"2024-04-23T14:34:25.804230+0000","last_undegraded":"2024-04-23T14:34:25.804230+0000","last_fullsized":"2024-04-23T14:34:25.804230+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:49:59.206097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805971+0000","last_change":"2024-04-23T14:34:24.799868+0000","last_active":"2024-04-23T14:34:25.805971+0000","last_peered":"2024-04-23T14:34:25.805971+0000","last_clean":"2024-04-23T14:34:25.805971+0000","last_became_active":"2024-04-23T14:34:24.799575+0000","last_became_peered":"2024-04-23T14:34:24.799575+0000","last_unstale":"2024-04-23T14:34:25.805971+0000","last_undegraded":"2024-04-23T14:34:25.805971+0000","last_fullsized":"2024-04-23T14:34:25.805971+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:57:41.597696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329923+0000","last_change":"2024-04-23T14:34:23.785148+0000","last_active":"2024-04-23T14:34:26.329923+0000","last_peered":"2024-04-23T14:34:26.329923+0000","last_clean":"2024-04-23T14:34:26.329923+0000","last_became_active":"2024-04-23T14:34:23.784649+0000","last_became_peered":"2024-04-23T14:34:23.784649+0000","last_unstale":"2024-04-23T14:34:26.329923+0000","last_undegraded":"2024-04-23T14:34:26.329923+0000","last_fullsized":"2024-04-23T14:34:26.329923+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:39:40.724648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.803894+0000","last_change":"2024-04-23T14:34:24.811573+0000","last_active":"2024-04-23T14:34:25.803894+0000","last_peered":"2024-04-23T14:34:25.803894+0000","last_clean":"2024-04-23T14:34:25.803894+0000","last_became_active":"2024-04-23T14:34:24.811156+0000","last_became_peered":"2024-04-23T14:34:24.811156+0000","last_unstale":"2024-04-23T14:34:25.803894+0000","last_undegraded":"2024-04-23T14:34:25.803894+0000","last_fullsized":"2024-04-23T14:34:25.803894+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:39:09.489633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.422252+0000","last_change":"2024-04-23T14:34:23.789714+0000","last_active":"2024-04-23T14:34:26.422252+0000","last_peered":"2024-04-23T14:34:26.422252+0000","last_clean":"2024-04-23T14:34:26.422252+0000","last_became_active":"2024-04-23T14:34:23.789435+0000","last_became_peered":"2024-04-23T14:34:23.789435+0000","last_unstale":"2024-04-23T14:34:26.422252+0000","last_undegraded":"2024-04-23T14:34:26.422252+0000","last_fullsized":"2024-04-23T14:34:26.422252+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:31:51.656961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329935+0000","last_change":"2024-04-23T14:34:24.802004+0000","last_active":"2024-04-23T14:34:26.329935+0000","last_peered":"2024-04-23T14:34:26.329935+0000","last_clean":"2024-04-23T14:34:26.329935+0000","last_became_active":"2024-04-23T14:34:24.799728+0000","last_became_peered":"2024-04-23T14:34:24.799728+0000","last_unstale":"2024-04-23T14:34:26.329935+0000","last_undegraded":"2024-04-23T14:34:26.329935+0000","last_fullsized":"2024-04-23T14:34:26.329935+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:33:23.882396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805511+0000","last_change":"2024-04-23T14:34:23.787899+0000","last_active":"2024-04-23T14:34:25.805511+0000","last_peered":"2024-04-23T14:34:25.805511+0000","last_clean":"2024-04-23T14:34:25.805511+0000","last_became_active":"2024-04-23T14:34:23.787614+0000","last_became_peered":"2024-04-23T14:34:23.787614+0000","last_unstale":"2024-04-23T14:34:25.805511+0000","last_undegraded":"2024-04-23T14:34:25.805511+0000","last_fullsized":"2024-04-23T14:34:25.805511+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:10.096243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421524+0000","last_change":"2024-04-23T14:34:24.807497+0000","last_active":"2024-04-23T14:34:26.421524+0000","last_peered":"2024-04-23T14:34:26.421524+0000","last_clean":"2024-04-23T14:34:26.421524+0000","last_became_active":"2024-04-23T14:34:24.807240+0000","last_became_peered":"2024-04-23T14:34:24.807240+0000","last_unstale":"2024-04-23T14:34:26.421524+0000","last_undegraded":"2024-04-23T14:34:26.421524+0000","last_fullsized":"2024-04-23T14:34:26.421524+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:35:43.913771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.803587+0000","last_change":"2024-04-23T14:34:23.785163+0000","last_active":"2024-04-23T14:34:25.803587+0000","last_peered":"2024-04-23T14:34:25.803587+0000","last_clean":"2024-04-23T14:34:25.803587+0000","last_became_active":"2024-04-23T14:34:23.784848+0000","last_became_peered":"2024-04-23T14:34:23.784848+0000","last_unstale":"2024-04-23T14:34:25.803587+0000","last_undegraded":"2024-04-23T14:34:25.803587+0000","last_fullsized":"2024-04-23T14:34:25.803587+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:14.298493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.207176+0000","last_change":"2024-04-23T14:34:24.818757+0000","last_active":"2024-04-23T14:34:26.207176+0000","last_peered":"2024-04-23T14:34:26.207176+0000","last_clean":"2024-04-23T14:34:26.207176+0000","last_became_active":"2024-04-23T14:34:24.818011+0000","last_became_peered":"2024-04-23T14:34:24.818011+0000","last_unstale":"2024-04-23T14:34:26.207176+0000","last_undegraded":"2024-04-23T14:34:26.207176+0000","last_fullsized":"2024-04-23T14:34:26.207176+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:06.679211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.806105+0000","last_change":"2024-04-23T14:34:23.787146+0000","last_active":"2024-04-23T14:34:25.806105+0000","last_peered":"2024-04-23T14:34:25.806105+0000","last_clean":"2024-04-23T14:34:25.806105+0000","last_became_active":"2024-04-23T14:34:23.786842+0000","last_became_peered":"2024-04-23T14:34:23.786842+0000","last_unstale":"2024-04-23T14:34:25.806105+0000","last_undegraded":"2024-04-23T14:34:25.806105+0000","last_fullsized":"2024-04-23T14:34:25.806105+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:31:51.656961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.803582+0000","last_change":"2024-04-23T14:34:24.806226+0000","last_active":"2024-04-23T14:34:25.803582+0000","last_peered":"2024-04-23T14:34:25.803582+0000","last_clean":"2024-04-23T14:34:25.803582+0000","last_became_active":"2024-04-23T14:34:24.801068+0000","last_became_peered":"2024-04-23T14:34:24.801068+0000","last_unstale":"2024-04-23T14:34:25.803582+0000","last_undegraded":"2024-04-23T14:34:25.803582+0000","last_fullsized":"2024-04-23T14:34:25.803582+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:04:53.451687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.803961+0000","last_change":"2024-04-23T14:34:23.787530+0000","last_active":"2024-04-23T14:34:25.803961+0000","last_peered":"2024-04-23T14:34:25.803961+0000","last_clean":"2024-04-23T14:34:25.803961+0000","last_became_active":"2024-04-23T14:34:23.787239+0000","last_became_peered":"2024-04-23T14:34:23.787239+0000","last_unstale":"2024-04-23T14:34:25.803961+0000","last_undegraded":"2024-04-23T14:34:25.803961+0000","last_fullsized":"2024-04-23T14:34:25.803961+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:47:52.275367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421570+0000","last_change":"2024-04-23T14:34:24.800571+0000","last_active":"2024-04-23T14:34:26.421570+0000","last_peered":"2024-04-23T14:34:26.421570+0000","last_clean":"2024-04-23T14:34:26.421570+0000","last_became_active":"2024-04-23T14:34:24.800278+0000","last_became_peered":"2024-04-23T14:34:24.800278+0000","last_unstale":"2024-04-23T14:34:26.421570+0000","last_undegraded":"2024-04-23T14:34:26.421570+0000","last_fullsized":"2024-04-23T14:34:26.421570+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:10:27.472762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421905+0000","last_change":"2024-04-23T14:34:23.794762+0000","last_active":"2024-04-23T14:34:26.421905+0000","last_peered":"2024-04-23T14:34:26.421905+0000","last_clean":"2024-04-23T14:34:26.421905+0000","last_became_active":"2024-04-23T14:34:23.794402+0000","last_became_peered":"2024-04-23T14:34:23.794402+0000","last_unstale":"2024-04-23T14:34:26.421905+0000","last_undegraded":"2024-04-23T14:34:26.421905+0000","last_fullsized":"2024-04-23T14:34:26.421905+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:57.859650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421804+0000","last_change":"2024-04-23T14:34:24.822585+0000","last_active":"2024-04-23T14:34:26.421804+0000","last_peered":"2024-04-23T14:34:26.421804+0000","last_clean":"2024-04-23T14:34:26.421804+0000","last_became_active":"2024-04-23T14:34:24.822200+0000","last_became_peered":"2024-04-23T14:34:24.822200+0000","last_unstale":"2024-04-23T14:34:26.421804+0000","last_undegraded":"2024-04-23T14:34:26.421804+0000","last_fullsized":"2024-04-23T14:34:26.421804+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:39:06.009341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.422071+0000","last_change":"2024-04-23T14:34:23.789329+0000","last_active":"2024-04-23T14:34:26.422071+0000","last_peered":"2024-04-23T14:34:26.422071+0000","last_clean":"2024-04-23T14:34:26.422071+0000","last_became_active":"2024-04-23T14:34:23.789054+0000","last_became_peered":"2024-04-23T14:34:23.789054+0000","last_unstale":"2024-04-23T14:34:26.422071+0000","last_undegraded":"2024-04-23T14:34:26.422071+0000","last_fullsized":"2024-04-23T14:34:26.422071+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:17:13.032214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":231,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.206850+0000","last_change":"2024-04-23T14:31:58.314235+0000","last_active":"2024-04-23T14:34:26.206850+0000","last_peered":"2024-04-23T14:34:26.206850+0000","last_clean":"2024-04-23T14:34:26.206850+0000","last_became_active":"2024-04-23T14:31:58.313165+0000","last_became_peered":"2024-04-23T14:31:58.313165+0000","last_unstale":"2024-04-23T14:34:26.206850+0000","last_undegraded":"2024-04-23T14:34:26.206850+0000","last_fullsized":"2024-04-23T14:34:26.206850+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_clean_scrub_stamp":"2024-04-23T14:31:57.140712+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:44.170436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1224704,"data_stored":869744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":279216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":643072,"data_stored":286560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":634880,"data_stored":280828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":643072,"data_stored":285964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":279272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":638976,"data_stored":285998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1245184,"data_stored":878470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:34:28.961 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:34:29.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.135+0000 7f2904200640 1 Processor -- start 2024-04-23T14:34:29.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.135+0000 7f2904200640 1 -- start start 2024-04-23T14:34:29.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.135+0000 7f2904200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f28fc15e820 0x7f28fc0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.135+0000 7f2904200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28fc0b5820 con 0x7f28fc0d0200 2024-04-23T14:34:29.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.135+0000 7f2904200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28fc0b5b00 con 0x7f28fc0b52c0 2024-04-23T14:34:29.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.135+0000 7f2904200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f28fc15ec00 con 0x7f28fc15e820 2024-04-23T14:34:29.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.135+0000 7f2901800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f28fc15e820 0x7f28fc0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.135+0000 7f2901800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f28fc15e820 0x7f28fc0b4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48312/0 (socket says 172.21.15.100:48312) 2024-04-23T14:34:29.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.135+0000 7f2901800640 1 -- 172.21.15.100:0/1794628119 learned_addr learned my addr 172.21.15.100:0/1794628119 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:29.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.135+0000 7f2901800640 1 -- 172.21.15.100:0/1794628119 >> v1:172.21.15.151:6789/0 conn(0x7f28fc0b52c0 legacy=0x7f28fc0cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:29.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.135+0000 7f2901800640 1 -- 172.21.15.100:0/1794628119 >> v1:172.21.15.100:6789/0 conn(0x7f28fc0d0200 legacy=0x7f28fc06cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.135+0000 7f2901800640 1 -- 172.21.15.100:0/1794628119 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28fc0b5de0 con 0x7f28fc15e820 2024-04-23T14:34:29.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.139+0000 7f2901800640 1 --2- 172.21.15.100:0/1794628119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28fc15e820 0x7f28fc0b4d80 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f28f0009870 tx=0x7f28f001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9174ba21c6ff9493 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:29.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.139+0000 7f28fbe00640 1 -- 172.21.15.100:0/1794628119 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f00aa650 con 0x7f28fc15e820 2024-04-23T14:34:29.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.139+0000 7f28fbe00640 1 -- 172.21.15.100:0/1794628119 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28f00af070 con 0x7f28fc15e820 2024-04-23T14:34:29.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.139+0000 7f28fbe00640 1 -- 172.21.15.100:0/1794628119 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f00aaa10 con 0x7f28fc15e820 2024-04-23T14:34:29.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.139+0000 7f2904200640 1 -- 172.21.15.100:0/1794628119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28fc15e820 msgr2=0x7f28fc0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.139+0000 7f2904200640 1 --2- 172.21.15.100:0/1794628119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28fc15e820 0x7f28fc0b4d80 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f28f0009870 tx=0x7f28f001c4f0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.139+0000 7f2904200640 1 -- 172.21.15.100:0/1794628119 shutdown_connections 2024-04-23T14:34:29.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.139+0000 7f2904200640 1 --2- 172.21.15.100:0/1794628119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28fc15e820 0x7f28fc0b4d80 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.139+0000 7f2904200640 1 -- 172.21.15.100:0/1794628119 >> 172.21.15.100:0/1794628119 conn(0x7f28fc070690 msgr2=0x7f28fc070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:29.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.139+0000 7f2904200640 1 -- 172.21.15.100:0/1794628119 shutdown_connections 2024-04-23T14:34:29.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.139+0000 7f2904200640 1 -- 172.21.15.100:0/1794628119 wait complete. 2024-04-23T14:34:29.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2904200640 1 Processor -- start 2024-04-23T14:34:29.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2904200640 1 -- start start 2024-04-23T14:34:29.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2904200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28fc0b52c0 0x7f28fc1fb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2904200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28fc0d0200 0x7f28fc150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2904200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28fc15e820 0x7f28fc151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2904200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc06dbe0 con 0x7f28fc0d0200 2024-04-23T14:34:29.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2904200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc06da60 con 0x7f28fc15e820 2024-04-23T14:34:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2904200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f28fc06dd60 con 0x7f28fc0b52c0 2024-04-23T14:34:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2900e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28fc0d0200 0x7f28fc150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2900e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28fc0d0200 0x7f28fc150b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48320/0 (socket says 172.21.15.100:48320) 2024-04-23T14:34:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2900e00640 1 -- 172.21.15.100:0/1595970340 learned_addr learned my addr 172.21.15.100:0/1595970340 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2901800640 1 --2- 172.21.15.100:0/1595970340 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28fc0b52c0 0x7f28fc1fb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2902200640 1 --2- 172.21.15.100:0/1595970340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28fc15e820 0x7f28fc151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2900e00640 1 -- 172.21.15.100:0/1595970340 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28fc0b52c0 msgr2=0x7f28fc1fb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2900e00640 1 --2- 172.21.15.100:0/1595970340 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28fc0b52c0 0x7f28fc1fb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2900e00640 1 -- 172.21.15.100:0/1595970340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28fc15e820 msgr2=0x7f28fc151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2900e00640 1 --2- 172.21.15.100:0/1595970340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28fc15e820 0x7f28fc151090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2900e00640 1 -- 172.21.15.100:0/1595970340 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28fc1517a0 con 0x7f28fc0d0200 2024-04-23T14:34:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2901800640 1 --2- 172.21.15.100:0/1595970340 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28fc0b52c0 0x7f28fc1fb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2902200640 1 --2- 172.21.15.100:0/1595970340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28fc15e820 0x7f28fc151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2900e00640 1 --2- 172.21.15.100:0/1595970340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28fc0d0200 0x7f28fc150b50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f28e800b410 tx=0x7f28e800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f28f9600640 1 -- 172.21.15.100:0/1595970340 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8015030 con 0x7f28fc0d0200 2024-04-23T14:34:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f28f9600640 1 -- 172.21.15.100:0/1595970340 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e8004020 con 0x7f28fc0d0200 2024-04-23T14:34:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2904200640 1 -- 172.21.15.100:0/1595970340 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28fc151a80 con 0x7f28fc0d0200 2024-04-23T14:34:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f28f9600640 1 -- 172.21.15.100:0/1595970340 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8004450 con 0x7f28fc0d0200 2024-04-23T14:34:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.143+0000 7f2904200640 1 -- 172.21.15.100:0/1595970340 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28fc152040 con 0x7f28fc0d0200 2024-04-23T14:34:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.147+0000 7f2904200640 1 -- 172.21.15.100:0/1595970340 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28fc068ad0 con 0x7f28fc0d0200 2024-04-23T14:34:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.147+0000 7f28f9600640 1 -- 172.21.15.100:0/1595970340 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f28e8018070 con 0x7f28fc0d0200 2024-04-23T14:34:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.147+0000 7f28f9600640 1 --2- 172.21.15.100:0/1595970340 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f28dc068380 0x7f28dc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.147+0000 7f28f9600640 1 -- 172.21.15.100:0/1595970340 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f28e8086670 con 0x7f28fc0d0200 2024-04-23T14:34:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.147+0000 7f2901800640 1 --2- 172.21.15.100:0/1595970340 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f28dc068380 0x7f28dc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.147+0000 7f2901800640 1 --2- 172.21.15.100:0/1595970340 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f28dc068380 0x7f28dc06a840 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f28f00ae000 tx=0x7f28f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:29.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.151+0000 7f28f9600640 1 -- 172.21.15.100:0/1595970340 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f28e8056b30 con 0x7f28fc0d0200 2024-04-23T14:34:29.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.303+0000 7f2904200640 1 -- 172.21.15.100:0/1595970340 --> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f28fc06fb40 con 0x7f28dc068380 2024-04-23T14:34:29.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.311+0000 7f28f9600640 1 -- 172.21.15.100:0/1595970340 <== mgr.4101 v2:172.21.15.100:6832/2859853957 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251953 (secure 0 0 0) 0x7f28fc06fb40 con 0x7f28dc068380 2024-04-23T14:34:29.316 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:29.317 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-04-23T14:34:29.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.315+0000 7f2904200640 1 -- 172.21.15.100:0/1595970340 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f28dc068380 msgr2=0x7f28dc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.315+0000 7f2904200640 1 --2- 172.21.15.100:0/1595970340 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f28dc068380 0x7f28dc06a840 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f28f00ae000 tx=0x7f28f0008040 comp rx=0 tx=0).stop 2024-04-23T14:34:29.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.315+0000 7f2904200640 1 -- 172.21.15.100:0/1595970340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28fc0d0200 msgr2=0x7f28fc150b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.315+0000 7f2904200640 1 --2- 172.21.15.100:0/1595970340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28fc0d0200 0x7f28fc150b50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f28e800b410 tx=0x7f28e800b8d0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.315+0000 7f2904200640 1 -- 172.21.15.100:0/1595970340 shutdown_connections 2024-04-23T14:34:29.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.315+0000 7f2904200640 1 --2- 172.21.15.100:0/1595970340 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f28dc068380 0x7f28dc06a840 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.315+0000 7f2904200640 1 --2- 172.21.15.100:0/1595970340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28fc15e820 0x7f28fc151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.315+0000 7f2904200640 1 --2- 172.21.15.100:0/1595970340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28fc0d0200 0x7f28fc150b50 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.315+0000 7f2904200640 1 --2- 172.21.15.100:0/1595970340 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28fc0b52c0 0x7f28fc1fb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.315+0000 7f2904200640 1 -- 172.21.15.100:0/1595970340 >> 172.21.15.100:0/1595970340 conn(0x7f28fc070690 msgr2=0x7f28fc0b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:29.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.315+0000 7f2904200640 1 -- 172.21.15.100:0/1595970340 shutdown_connections 2024-04-23T14:34:29.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.315+0000 7f2904200640 1 -- 172.21.15.100:0/1595970340 wait complete. 2024-04-23T14:34:29.335 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-04-23T14:34:28.763403+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226472,"kb_used_data":6136,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748248,"statfs":{"total":773094113280,"available":772862205952,"internally_reserved":0,"allocated":6283264,"data_stored":3446052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.995581"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421971+0000","last_change":"2024-04-23T14:34:24.816066+0000","last_active":"2024-04-23T14:34:26.421971+0000","last_peered":"2024-04-23T14:34:26.421971+0000","last_clean":"2024-04-23T14:34:26.421971+0000","last_became_active":"2024-04-23T14:34:24.815443+0000","last_became_peered":"2024-04-23T14:34:24.815443+0000","last_unstale":"2024-04-23T14:34:26.421971+0000","last_undegraded":"2024-04-23T14:34:26.421971+0000","last_fullsized":"2024-04-23T14:34:26.421971+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:11:34.330788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421519+0000","last_change":"2024-04-23T14:34:24.808825+0000","last_active":"2024-04-23T14:34:26.421519+0000","last_peered":"2024-04-23T14:34:26.421519+0000","last_clean":"2024-04-23T14:34:26.421519+0000","last_became_active":"2024-04-23T14:34:24.808517+0000","last_became_peered":"2024-04-23T14:34:24.808517+0000","last_unstale":"2024-04-23T14:34:26.421519+0000","last_undegraded":"2024-04-23T14:34:26.421519+0000","last_fullsized":"2024-04-23T14:34:26.421519+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:18:59.502027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.207364+0000","last_change":"2024-04-23T14:34:24.813082+0000","last_active":"2024-04-23T14:34:26.207364+0000","last_peered":"2024-04-23T14:34:26.207364+0000","last_clean":"2024-04-23T14:34:26.207364+0000","last_became_active":"2024-04-23T14:34:24.812788+0000","last_became_peered":"2024-04-23T14:34:24.812788+0000","last_unstale":"2024-04-23T14:34:26.207364+0000","last_undegraded":"2024-04-23T14:34:26.207364+0000","last_fullsized":"2024-04-23T14:34:26.207364+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:07:07.524078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805070+0000","last_change":"2024-04-23T14:34:24.798979+0000","last_active":"2024-04-23T14:34:25.805070+0000","last_peered":"2024-04-23T14:34:25.805070+0000","last_clean":"2024-04-23T14:34:25.805070+0000","last_became_active":"2024-04-23T14:34:24.798311+0000","last_became_peered":"2024-04-23T14:34:24.798311+0000","last_unstale":"2024-04-23T14:34:25.805070+0000","last_undegraded":"2024-04-23T14:34:25.805070+0000","last_fullsized":"2024-04-23T14:34:25.805070+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:42:57.268048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421292+0000","last_change":"2024-04-23T14:34:24.793063+0000","last_active":"2024-04-23T14:34:26.421292+0000","last_peered":"2024-04-23T14:34:26.421292+0000","last_clean":"2024-04-23T14:34:26.421292+0000","last_became_active":"2024-04-23T14:34:24.792787+0000","last_became_peered":"2024-04-23T14:34:24.792787+0000","last_unstale":"2024-04-23T14:34:26.421292+0000","last_undegraded":"2024-04-23T14:34:26.421292+0000","last_fullsized":"2024-04-23T14:34:26.421292+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:32:19.324839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329529+0000","last_change":"2024-04-23T14:34:24.810090+0000","last_active":"2024-04-23T14:34:26.329529+0000","last_peered":"2024-04-23T14:34:26.329529+0000","last_clean":"2024-04-23T14:34:26.329529+0000","last_became_active":"2024-04-23T14:34:24.809830+0000","last_became_peered":"2024-04-23T14:34:24.809830+0000","last_unstale":"2024-04-23T14:34:26.329529+0000","last_undegraded":"2024-04-23T14:34:26.329529+0000","last_fullsized":"2024-04-23T14:34:26.329529+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:07:07.524078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.804694+0000","last_change":"2024-04-23T14:34:24.794282+0000","last_active":"2024-04-23T14:34:25.804694+0000","last_peered":"2024-04-23T14:34:25.804694+0000","last_clean":"2024-04-23T14:34:25.804694+0000","last_became_active":"2024-04-23T14:34:24.794009+0000","last_became_peered":"2024-04-23T14:34:24.794009+0000","last_unstale":"2024-04-23T14:34:25.804694+0000","last_undegraded":"2024-04-23T14:34:25.804694+0000","last_fullsized":"2024-04-23T14:34:25.804694+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:43:43.259027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421784+0000","last_change":"2024-04-23T14:34:24.812554+0000","last_active":"2024-04-23T14:34:26.421784+0000","last_peered":"2024-04-23T14:34:26.421784+0000","last_clean":"2024-04-23T14:34:26.421784+0000","last_became_active":"2024-04-23T14:34:24.807887+0000","last_became_peered":"2024-04-23T14:34:24.807887+0000","last_unstale":"2024-04-23T14:34:26.421784+0000","last_undegraded":"2024-04-23T14:34:26.421784+0000","last_fullsized":"2024-04-23T14:34:26.421784+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:06:34.829966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.806067+0000","last_change":"2024-04-23T14:34:24.814581+0000","last_active":"2024-04-23T14:34:25.806067+0000","last_peered":"2024-04-23T14:34:25.806067+0000","last_clean":"2024-04-23T14:34:25.806067+0000","last_became_active":"2024-04-23T14:34:24.814306+0000","last_became_peered":"2024-04-23T14:34:24.814306+0000","last_unstale":"2024-04-23T14:34:25.806067+0000","last_undegraded":"2024-04-23T14:34:25.806067+0000","last_fullsized":"2024-04-23T14:34:25.806067+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:01:59.183007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.804126+0000","last_change":"2024-04-23T14:34:24.806753+0000","last_active":"2024-04-23T14:34:25.804126+0000","last_peered":"2024-04-23T14:34:25.804126+0000","last_clean":"2024-04-23T14:34:25.804126+0000","last_became_active":"2024-04-23T14:34:24.801081+0000","last_became_peered":"2024-04-23T14:34:24.801081+0000","last_unstale":"2024-04-23T14:34:25.804126+0000","last_undegraded":"2024-04-23T14:34:25.804126+0000","last_fullsized":"2024-04-23T14:34:25.804126+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:29:04.759935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.803905+0000","last_change":"2024-04-23T14:34:24.819028+0000","last_active":"2024-04-23T14:34:25.803905+0000","last_peered":"2024-04-23T14:34:25.803905+0000","last_clean":"2024-04-23T14:34:25.803905+0000","last_became_active":"2024-04-23T14:34:24.817326+0000","last_became_peered":"2024-04-23T14:34:24.817326+0000","last_unstale":"2024-04-23T14:34:25.803905+0000","last_undegraded":"2024-04-23T14:34:25.803905+0000","last_fullsized":"2024-04-23T14:34:25.803905+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:01:59.183007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805220+0000","last_change":"2024-04-23T14:34:24.814063+0000","last_active":"2024-04-23T14:34:25.805220+0000","last_peered":"2024-04-23T14:34:25.805220+0000","last_clean":"2024-04-23T14:34:25.805220+0000","last_became_active":"2024-04-23T14:34:24.813728+0000","last_became_peered":"2024-04-23T14:34:24.813728+0000","last_unstale":"2024-04-23T14:34:25.805220+0000","last_undegraded":"2024-04-23T14:34:25.805220+0000","last_fullsized":"2024-04-23T14:34:25.805220+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:22:43.967203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.422277+0000","last_change":"2024-04-23T14:34:24.808179+0000","last_active":"2024-04-23T14:34:26.422277+0000","last_peered":"2024-04-23T14:34:26.422277+0000","last_clean":"2024-04-23T14:34:26.422277+0000","last_became_active":"2024-04-23T14:34:24.805067+0000","last_became_peered":"2024-04-23T14:34:24.805067+0000","last_unstale":"2024-04-23T14:34:26.422277+0000","last_undegraded":"2024-04-23T14:34:26.422277+0000","last_fullsized":"2024-04-23T14:34:26.422277+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:02:42.230866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805997+0000","last_change":"2024-04-23T14:34:24.807039+0000","last_active":"2024-04-23T14:34:25.805997+0000","last_peered":"2024-04-23T14:34:25.805997+0000","last_clean":"2024-04-23T14:34:25.805997+0000","last_became_active":"2024-04-23T14:34:24.806760+0000","last_became_peered":"2024-04-23T14:34:24.806760+0000","last_unstale":"2024-04-23T14:34:25.805997+0000","last_undegraded":"2024-04-23T14:34:25.805997+0000","last_fullsized":"2024-04-23T14:34:25.805997+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:39:09.489633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805556+0000","last_change":"2024-04-23T14:34:24.807060+0000","last_active":"2024-04-23T14:34:25.805556+0000","last_peered":"2024-04-23T14:34:25.805556+0000","last_clean":"2024-04-23T14:34:25.805556+0000","last_became_active":"2024-04-23T14:34:24.806786+0000","last_became_peered":"2024-04-23T14:34:24.806786+0000","last_unstale":"2024-04-23T14:34:25.805556+0000","last_undegraded":"2024-04-23T14:34:25.805556+0000","last_fullsized":"2024-04-23T14:34:25.805556+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:24:07.696001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329487+0000","last_change":"2024-04-23T14:34:24.804474+0000","last_active":"2024-04-23T14:34:26.329487+0000","last_peered":"2024-04-23T14:34:26.329487+0000","last_clean":"2024-04-23T14:34:26.329487+0000","last_became_active":"2024-04-23T14:34:24.804212+0000","last_became_peered":"2024-04-23T14:34:24.804212+0000","last_unstale":"2024-04-23T14:34:26.329487+0000","last_undegraded":"2024-04-23T14:34:26.329487+0000","last_fullsized":"2024-04-23T14:34:26.329487+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:06.679211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.422144+0000","last_change":"2024-04-23T14:34:24.810400+0000","last_active":"2024-04-23T14:34:26.422144+0000","last_peered":"2024-04-23T14:34:26.422144+0000","last_clean":"2024-04-23T14:34:26.422144+0000","last_became_active":"2024-04-23T14:34:24.810125+0000","last_became_peered":"2024-04-23T14:34:24.810125+0000","last_unstale":"2024-04-23T14:34:26.422144+0000","last_undegraded":"2024-04-23T14:34:26.422144+0000","last_fullsized":"2024-04-23T14:34:26.422144+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:17:58.654366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.207588+0000","last_change":"2024-04-23T14:34:24.818240+0000","last_active":"2024-04-23T14:34:26.207588+0000","last_peered":"2024-04-23T14:34:26.207588+0000","last_clean":"2024-04-23T14:34:26.207588+0000","last_became_active":"2024-04-23T14:34:24.817890+0000","last_became_peered":"2024-04-23T14:34:24.817890+0000","last_unstale":"2024-04-23T14:34:26.207588+0000","last_undegraded":"2024-04-23T14:34:26.207588+0000","last_fullsized":"2024-04-23T14:34:26.207588+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:14:58.214732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421671+0000","last_change":"2024-04-23T14:34:24.817394+0000","last_active":"2024-04-23T14:34:26.421671+0000","last_peered":"2024-04-23T14:34:26.421671+0000","last_clean":"2024-04-23T14:34:26.421671+0000","last_became_active":"2024-04-23T14:34:24.817120+0000","last_became_peered":"2024-04-23T14:34:24.817120+0000","last_unstale":"2024-04-23T14:34:26.421671+0000","last_undegraded":"2024-04-23T14:34:26.421671+0000","last_fullsized":"2024-04-23T14:34:26.421671+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:46:10.550380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421832+0000","last_change":"2024-04-23T14:34:24.811477+0000","last_active":"2024-04-23T14:34:26.421832+0000","last_peered":"2024-04-23T14:34:26.421832+0000","last_clean":"2024-04-23T14:34:26.421832+0000","last_became_active":"2024-04-23T14:34:24.810943+0000","last_became_peered":"2024-04-23T14:34:24.810943+0000","last_unstale":"2024-04-23T14:34:26.421832+0000","last_undegraded":"2024-04-23T14:34:26.421832+0000","last_fullsized":"2024-04-23T14:34:26.421832+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:22:48.079754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.207016+0000","last_change":"2024-04-23T14:34:24.816877+0000","last_active":"2024-04-23T14:34:26.207016+0000","last_peered":"2024-04-23T14:34:26.207016+0000","last_clean":"2024-04-23T14:34:26.207016+0000","last_became_active":"2024-04-23T14:34:24.816179+0000","last_became_peered":"2024-04-23T14:34:24.816179+0000","last_unstale":"2024-04-23T14:34:26.207016+0000","last_undegraded":"2024-04-23T14:34:26.207016+0000","last_fullsized":"2024-04-23T14:34:26.207016+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:13:16.409968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421345+0000","last_change":"2024-04-23T14:34:24.814282+0000","last_active":"2024-04-23T14:34:26.421345+0000","last_peered":"2024-04-23T14:34:26.421345+0000","last_clean":"2024-04-23T14:34:26.421345+0000","last_became_active":"2024-04-23T14:34:24.813821+0000","last_became_peered":"2024-04-23T14:34:24.813821+0000","last_unstale":"2024-04-23T14:34:26.421345+0000","last_undegraded":"2024-04-23T14:34:26.421345+0000","last_fullsized":"2024-04-23T14:34:26.421345+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:32:03.630387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.206871+0000","last_change":"2024-04-23T14:34:24.813158+0000","last_active":"2024-04-23T14:34:26.206871+0000","last_peered":"2024-04-23T14:34:26.206871+0000","last_clean":"2024-04-23T14:34:26.206871+0000","last_became_active":"2024-04-23T14:34:24.810614+0000","last_became_peered":"2024-04-23T14:34:24.810614+0000","last_unstale":"2024-04-23T14:34:26.206871+0000","last_undegraded":"2024-04-23T14:34:26.206871+0000","last_fullsized":"2024-04-23T14:34:26.206871+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:03:02.301543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.422017+0000","last_change":"2024-04-23T14:34:24.807779+0000","last_active":"2024-04-23T14:34:26.422017+0000","last_peered":"2024-04-23T14:34:26.422017+0000","last_clean":"2024-04-23T14:34:26.422017+0000","last_became_active":"2024-04-23T14:34:24.807332+0000","last_became_peered":"2024-04-23T14:34:24.807332+0000","last_unstale":"2024-04-23T14:34:26.422017+0000","last_undegraded":"2024-04-23T14:34:26.422017+0000","last_fullsized":"2024-04-23T14:34:26.422017+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:24:31.326993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.422075+0000","last_change":"2024-04-23T14:34:24.816308+0000","last_active":"2024-04-23T14:34:26.422075+0000","last_peered":"2024-04-23T14:34:26.422075+0000","last_clean":"2024-04-23T14:34:26.422075+0000","last_became_active":"2024-04-23T14:34:24.815807+0000","last_became_peered":"2024-04-23T14:34:24.815807+0000","last_unstale":"2024-04-23T14:34:26.422075+0000","last_undegraded":"2024-04-23T14:34:26.422075+0000","last_fullsized":"2024-04-23T14:34:26.422075+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:03.539500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.806258+0000","last_change":"2024-04-23T14:34:24.800542+0000","last_active":"2024-04-23T14:34:25.806258+0000","last_peered":"2024-04-23T14:34:25.806258+0000","last_clean":"2024-04-23T14:34:25.806258+0000","last_became_active":"2024-04-23T14:34:24.800292+0000","last_became_peered":"2024-04-23T14:34:24.800292+0000","last_unstale":"2024-04-23T14:34:25.806258+0000","last_undegraded":"2024-04-23T14:34:25.806258+0000","last_fullsized":"2024-04-23T14:34:25.806258+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:33:23.882396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329723+0000","last_change":"2024-04-23T14:34:24.808739+0000","last_active":"2024-04-23T14:34:26.329723+0000","last_peered":"2024-04-23T14:34:26.329723+0000","last_clean":"2024-04-23T14:34:26.329723+0000","last_became_active":"2024-04-23T14:34:24.808468+0000","last_became_peered":"2024-04-23T14:34:24.808468+0000","last_unstale":"2024-04-23T14:34:26.329723+0000","last_undegraded":"2024-04-23T14:34:26.329723+0000","last_fullsized":"2024-04-23T14:34:26.329723+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:42:57.268048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421464+0000","last_change":"2024-04-23T14:34:24.791972+0000","last_active":"2024-04-23T14:34:26.421464+0000","last_peered":"2024-04-23T14:34:26.421464+0000","last_clean":"2024-04-23T14:34:26.421464+0000","last_became_active":"2024-04-23T14:34:24.791684+0000","last_became_peered":"2024-04-23T14:34:24.791684+0000","last_unstale":"2024-04-23T14:34:26.421464+0000","last_undegraded":"2024-04-23T14:34:26.421464+0000","last_fullsized":"2024-04-23T14:34:26.421464+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:45:55.677313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421769+0000","last_change":"2024-04-23T14:34:24.812289+0000","last_active":"2024-04-23T14:34:26.421769+0000","last_peered":"2024-04-23T14:34:26.421769+0000","last_clean":"2024-04-23T14:34:26.421769+0000","last_became_active":"2024-04-23T14:34:24.811957+0000","last_became_peered":"2024-04-23T14:34:24.811957+0000","last_unstale":"2024-04-23T14:34:26.421769+0000","last_undegraded":"2024-04-23T14:34:26.421769+0000","last_fullsized":"2024-04-23T14:34:26.421769+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:56.109504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.206883+0000","last_change":"2024-04-23T14:34:24.812721+0000","last_active":"2024-04-23T14:34:26.206883+0000","last_peered":"2024-04-23T14:34:26.206883+0000","last_clean":"2024-04-23T14:34:26.206883+0000","last_became_active":"2024-04-23T14:34:24.812406+0000","last_became_peered":"2024-04-23T14:34:24.812406+0000","last_unstale":"2024-04-23T14:34:26.206883+0000","last_undegraded":"2024-04-23T14:34:26.206883+0000","last_fullsized":"2024-04-23T14:34:26.206883+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:57.460216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421063+0000","last_change":"2024-04-23T14:34:24.797417+0000","last_active":"2024-04-23T14:34:26.421063+0000","last_peered":"2024-04-23T14:34:26.421063+0000","last_clean":"2024-04-23T14:34:26.421063+0000","last_became_active":"2024-04-23T14:34:24.797135+0000","last_became_peered":"2024-04-23T14:34:24.797135+0000","last_unstale":"2024-04-23T14:34:26.421063+0000","last_undegraded":"2024-04-23T14:34:26.421063+0000","last_fullsized":"2024-04-23T14:34:26.421063+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:08:06.351163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329594+0000","last_change":"2024-04-23T14:34:24.821730+0000","last_active":"2024-04-23T14:34:26.329594+0000","last_peered":"2024-04-23T14:34:26.329594+0000","last_clean":"2024-04-23T14:34:26.329594+0000","last_became_active":"2024-04-23T14:34:24.821223+0000","last_became_peered":"2024-04-23T14:34:24.821223+0000","last_unstale":"2024-04-23T14:34:26.329594+0000","last_undegraded":"2024-04-23T14:34:26.329594+0000","last_fullsized":"2024-04-23T14:34:26.329594+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:14:58.214732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329886+0000","last_change":"2024-04-23T14:34:24.820085+0000","last_active":"2024-04-23T14:34:26.329886+0000","last_peered":"2024-04-23T14:34:26.329886+0000","last_clean":"2024-04-23T14:34:26.329886+0000","last_became_active":"2024-04-23T14:34:24.818343+0000","last_became_peered":"2024-04-23T14:34:24.818343+0000","last_unstale":"2024-04-23T14:34:26.329886+0000","last_undegraded":"2024-04-23T14:34:26.329886+0000","last_fullsized":"2024-04-23T14:34:26.329886+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:57:41.597696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"52'4","reported_seq":18,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329884+0000","last_change":"2024-04-23T14:34:23.783227+0000","last_active":"2024-04-23T14:34:26.329884+0000","last_peered":"2024-04-23T14:34:26.329884+0000","last_clean":"2024-04-23T14:34:26.329884+0000","last_became_active":"2024-04-23T14:34:23.782921+0000","last_became_peered":"2024-04-23T14:34:23.782921+0000","last_unstale":"2024-04-23T14:34:26.329884+0000","last_undegraded":"2024-04-23T14:34:26.329884+0000","last_fullsized":"2024-04-23T14:34:26.329884+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:27:55.173851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.804251+0000","last_change":"2024-04-23T14:34:24.814940+0000","last_active":"2024-04-23T14:34:25.804251+0000","last_peered":"2024-04-23T14:34:25.804251+0000","last_clean":"2024-04-23T14:34:25.804251+0000","last_became_active":"2024-04-23T14:34:24.814678+0000","last_became_peered":"2024-04-23T14:34:24.814678+0000","last_unstale":"2024-04-23T14:34:25.804251+0000","last_undegraded":"2024-04-23T14:34:25.804251+0000","last_fullsized":"2024-04-23T14:34:25.804251+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:54:40.578942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421909+0000","last_change":"2024-04-23T14:34:23.794894+0000","last_active":"2024-04-23T14:34:26.421909+0000","last_peered":"2024-04-23T14:34:26.421909+0000","last_clean":"2024-04-23T14:34:26.421909+0000","last_became_active":"2024-04-23T14:34:23.794654+0000","last_became_peered":"2024-04-23T14:34:23.794654+0000","last_unstale":"2024-04-23T14:34:26.421909+0000","last_undegraded":"2024-04-23T14:34:26.421909+0000","last_fullsized":"2024-04-23T14:34:26.421909+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:56.458224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329734+0000","last_change":"2024-04-23T14:34:24.811820+0000","last_active":"2024-04-23T14:34:26.329734+0000","last_peered":"2024-04-23T14:34:26.329734+0000","last_clean":"2024-04-23T14:34:26.329734+0000","last_became_active":"2024-04-23T14:34:24.811485+0000","last_became_peered":"2024-04-23T14:34:24.811485+0000","last_unstale":"2024-04-23T14:34:26.329734+0000","last_undegraded":"2024-04-23T14:34:26.329734+0000","last_fullsized":"2024-04-23T14:34:26.329734+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:22:43.967203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329957+0000","last_change":"2024-04-23T14:34:23.789955+0000","last_active":"2024-04-23T14:34:26.329957+0000","last_peered":"2024-04-23T14:34:26.329957+0000","last_clean":"2024-04-23T14:34:26.329957+0000","last_became_active":"2024-04-23T14:34:23.788171+0000","last_became_peered":"2024-04-23T14:34:23.788171+0000","last_unstale":"2024-04-23T14:34:26.329957+0000","last_undegraded":"2024-04-23T14:34:26.329957+0000","last_fullsized":"2024-04-23T14:34:26.329957+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:54:39.576949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329865+0000","last_change":"2024-04-23T14:34:24.801791+0000","last_active":"2024-04-23T14:34:26.329865+0000","last_peered":"2024-04-23T14:34:26.329865+0000","last_clean":"2024-04-23T14:34:26.329865+0000","last_became_active":"2024-04-23T14:34:24.801530+0000","last_became_peered":"2024-04-23T14:34:24.801530+0000","last_unstale":"2024-04-23T14:34:26.329865+0000","last_undegraded":"2024-04-23T14:34:26.329865+0000","last_fullsized":"2024-04-23T14:34:26.329865+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:33:04.830057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805238+0000","last_change":"2024-04-23T14:34:23.795809+0000","last_active":"2024-04-23T14:34:25.805238+0000","last_peered":"2024-04-23T14:34:25.805238+0000","last_clean":"2024-04-23T14:34:25.805238+0000","last_became_active":"2024-04-23T14:34:23.795529+0000","last_became_peered":"2024-04-23T14:34:23.795529+0000","last_unstale":"2024-04-23T14:34:25.805238+0000","last_undegraded":"2024-04-23T14:34:25.805238+0000","last_fullsized":"2024-04-23T14:34:25.805238+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:25:50.733726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421350+0000","last_change":"2024-04-23T14:34:24.814076+0000","last_active":"2024-04-23T14:34:26.421350+0000","last_peered":"2024-04-23T14:34:26.421350+0000","last_clean":"2024-04-23T14:34:26.421350+0000","last_became_active":"2024-04-23T14:34:24.813436+0000","last_became_peered":"2024-04-23T14:34:24.813436+0000","last_unstale":"2024-04-23T14:34:26.421350+0000","last_undegraded":"2024-04-23T14:34:26.421350+0000","last_fullsized":"2024-04-23T14:34:26.421350+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:36:46.948840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805958+0000","last_change":"2024-04-23T14:34:23.796231+0000","last_active":"2024-04-23T14:34:25.805958+0000","last_peered":"2024-04-23T14:34:25.805958+0000","last_clean":"2024-04-23T14:34:25.805958+0000","last_became_active":"2024-04-23T14:34:23.795952+0000","last_became_peered":"2024-04-23T14:34:23.795952+0000","last_unstale":"2024-04-23T14:34:25.805958+0000","last_undegraded":"2024-04-23T14:34:25.805958+0000","last_fullsized":"2024-04-23T14:34:25.805958+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:27:23.226059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421549+0000","last_change":"2024-04-23T14:34:24.813571+0000","last_active":"2024-04-23T14:34:26.421549+0000","last_peered":"2024-04-23T14:34:26.421549+0000","last_clean":"2024-04-23T14:34:26.421549+0000","last_became_active":"2024-04-23T14:34:24.812515+0000","last_became_peered":"2024-04-23T14:34:24.812515+0000","last_unstale":"2024-04-23T14:34:26.421549+0000","last_undegraded":"2024-04-23T14:34:26.421549+0000","last_fullsized":"2024-04-23T14:34:26.421549+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:15:47.496695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329518+0000","last_change":"2024-04-23T14:34:23.785763+0000","last_active":"2024-04-23T14:34:26.329518+0000","last_peered":"2024-04-23T14:34:26.329518+0000","last_clean":"2024-04-23T14:34:26.329518+0000","last_became_active":"2024-04-23T14:34:23.785518+0000","last_became_peered":"2024-04-23T14:34:23.785518+0000","last_unstale":"2024-04-23T14:34:26.329518+0000","last_undegraded":"2024-04-23T14:34:26.329518+0000","last_fullsized":"2024-04-23T14:34:26.329518+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:39:08.487640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.422104+0000","last_change":"2024-04-23T14:34:24.807190+0000","last_active":"2024-04-23T14:34:26.422104+0000","last_peered":"2024-04-23T14:34:26.422104+0000","last_clean":"2024-04-23T14:34:26.422104+0000","last_became_active":"2024-04-23T14:34:24.804960+0000","last_became_peered":"2024-04-23T14:34:24.804960+0000","last_unstale":"2024-04-23T14:34:26.422104+0000","last_undegraded":"2024-04-23T14:34:26.422104+0000","last_fullsized":"2024-04-23T14:34:26.422104+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:09:21.395477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.804956+0000","last_change":"2024-04-23T14:34:23.791773+0000","last_active":"2024-04-23T14:34:25.804956+0000","last_peered":"2024-04-23T14:34:25.804956+0000","last_clean":"2024-04-23T14:34:25.804956+0000","last_became_active":"2024-04-23T14:34:23.790851+0000","last_became_peered":"2024-04-23T14:34:23.790851+0000","last_unstale":"2024-04-23T14:34:25.804956+0000","last_undegraded":"2024-04-23T14:34:25.804956+0000","last_fullsized":"2024-04-23T14:34:25.804956+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:38:03.046355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421329+0000","last_change":"2024-04-23T14:34:24.801784+0000","last_active":"2024-04-23T14:34:26.421329+0000","last_peered":"2024-04-23T14:34:26.421329+0000","last_clean":"2024-04-23T14:34:26.421329+0000","last_became_active":"2024-04-23T14:34:24.801513+0000","last_became_peered":"2024-04-23T14:34:24.801513+0000","last_unstale":"2024-04-23T14:34:26.421329+0000","last_undegraded":"2024-04-23T14:34:26.421329+0000","last_fullsized":"2024-04-23T14:34:26.421329+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:27.091148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805453+0000","last_change":"2024-04-23T14:34:23.796931+0000","last_active":"2024-04-23T14:34:25.805453+0000","last_peered":"2024-04-23T14:34:25.805453+0000","last_clean":"2024-04-23T14:34:25.805453+0000","last_became_active":"2024-04-23T14:34:23.796602+0000","last_became_peered":"2024-04-23T14:34:23.796602+0000","last_unstale":"2024-04-23T14:34:25.805453+0000","last_undegraded":"2024-04-23T14:34:25.805453+0000","last_fullsized":"2024-04-23T14:34:25.805453+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:47:52.275367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.806270+0000","last_change":"2024-04-23T14:34:24.801735+0000","last_active":"2024-04-23T14:34:25.806270+0000","last_peered":"2024-04-23T14:34:25.806270+0000","last_clean":"2024-04-23T14:34:25.806270+0000","last_became_active":"2024-04-23T14:34:24.801461+0000","last_became_peered":"2024-04-23T14:34:24.801461+0000","last_unstale":"2024-04-23T14:34:25.806270+0000","last_undegraded":"2024-04-23T14:34:25.806270+0000","last_fullsized":"2024-04-23T14:34:25.806270+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:29:04.759935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.207553+0000","last_change":"2024-04-23T14:34:23.798356+0000","last_active":"2024-04-23T14:34:26.207553+0000","last_peered":"2024-04-23T14:34:26.207553+0000","last_clean":"2024-04-23T14:34:26.207553+0000","last_became_active":"2024-04-23T14:34:23.798021+0000","last_became_peered":"2024-04-23T14:34:23.798021+0000","last_unstale":"2024-04-23T14:34:26.207553+0000","last_undegraded":"2024-04-23T14:34:26.207553+0000","last_fullsized":"2024-04-23T14:34:26.207553+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:43:42.257035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.207642+0000","last_change":"2024-04-23T14:34:24.825621+0000","last_active":"2024-04-23T14:34:26.207642+0000","last_peered":"2024-04-23T14:34:26.207642+0000","last_clean":"2024-04-23T14:34:26.207642+0000","last_became_active":"2024-04-23T14:34:24.825408+0000","last_became_peered":"2024-04-23T14:34:24.825408+0000","last_unstale":"2024-04-23T14:34:26.207642+0000","last_undegraded":"2024-04-23T14:34:26.207642+0000","last_fullsized":"2024-04-23T14:34:26.207642+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:22:43.967203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.422067+0000","last_change":"2024-04-23T14:34:23.788488+0000","last_active":"2024-04-23T14:34:26.422067+0000","last_peered":"2024-04-23T14:34:26.422067+0000","last_clean":"2024-04-23T14:34:26.422067+0000","last_became_active":"2024-04-23T14:34:23.788219+0000","last_became_peered":"2024-04-23T14:34:23.788219+0000","last_unstale":"2024-04-23T14:34:26.422067+0000","last_undegraded":"2024-04-23T14:34:26.422067+0000","last_fullsized":"2024-04-23T14:34:26.422067+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:13:15.407976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.803737+0000","last_change":"2024-04-23T14:34:24.814634+0000","last_active":"2024-04-23T14:34:25.803737+0000","last_peered":"2024-04-23T14:34:25.803737+0000","last_clean":"2024-04-23T14:34:25.803737+0000","last_became_active":"2024-04-23T14:34:24.811479+0000","last_became_peered":"2024-04-23T14:34:24.811479+0000","last_unstale":"2024-04-23T14:34:25.803737+0000","last_undegraded":"2024-04-23T14:34:25.803737+0000","last_fullsized":"2024-04-23T14:34:25.803737+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:24:07.696001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"52'3","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.207400+0000","last_change":"2024-04-23T14:34:23.796614+0000","last_active":"2024-04-23T14:34:26.207400+0000","last_peered":"2024-04-23T14:34:26.207400+0000","last_clean":"2024-04-23T14:34:26.207400+0000","last_became_active":"2024-04-23T14:34:23.796331+0000","last_became_peered":"2024-04-23T14:34:23.796331+0000","last_unstale":"2024-04-23T14:34:26.207400+0000","last_undegraded":"2024-04-23T14:34:26.207400+0000","last_fullsized":"2024-04-23T14:34:26.207400+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:42:20.098074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.207086+0000","last_change":"2024-04-23T14:34:24.824592+0000","last_active":"2024-04-23T14:34:26.207086+0000","last_peered":"2024-04-23T14:34:26.207086+0000","last_clean":"2024-04-23T14:34:26.207086+0000","last_became_active":"2024-04-23T14:34:24.824282+0000","last_became_peered":"2024-04-23T14:34:24.824282+0000","last_unstale":"2024-04-23T14:34:26.207086+0000","last_undegraded":"2024-04-23T14:34:26.207086+0000","last_fullsized":"2024-04-23T14:34:26.207086+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:58.861642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"52'6","reported_seq":20,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.806443+0000","last_change":"2024-04-23T14:34:23.793905+0000","last_active":"2024-04-23T14:34:25.806443+0000","last_peered":"2024-04-23T14:34:25.806443+0000","last_clean":"2024-04-23T14:34:25.806443+0000","last_became_active":"2024-04-23T14:34:23.793652+0000","last_became_peered":"2024-04-23T14:34:23.793652+0000","last_unstale":"2024-04-23T14:34:25.806443+0000","last_undegraded":"2024-04-23T14:34:25.806443+0000","last_fullsized":"2024-04-23T14:34:25.806443+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:14.298493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.804743+0000","last_change":"2024-04-23T14:34:24.802096+0000","last_active":"2024-04-23T14:34:25.804743+0000","last_peered":"2024-04-23T14:34:25.804743+0000","last_clean":"2024-04-23T14:34:25.804743+0000","last_became_active":"2024-04-23T14:34:24.801608+0000","last_became_peered":"2024-04-23T14:34:24.801608+0000","last_unstale":"2024-04-23T14:34:25.804743+0000","last_undegraded":"2024-04-23T14:34:25.804743+0000","last_fullsized":"2024-04-23T14:34:25.804743+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:07:07.524078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421663+0000","last_change":"2024-04-23T14:34:23.797317+0000","last_active":"2024-04-23T14:34:26.421663+0000","last_peered":"2024-04-23T14:34:26.421663+0000","last_clean":"2024-04-23T14:34:26.421663+0000","last_became_active":"2024-04-23T14:34:23.796717+0000","last_became_peered":"2024-04-23T14:34:23.796717+0000","last_unstale":"2024-04-23T14:34:26.421663+0000","last_undegraded":"2024-04-23T14:34:26.421663+0000","last_fullsized":"2024-04-23T14:34:26.421663+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:05.677218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.207107+0000","last_change":"2024-04-23T14:34:24.816642+0000","last_active":"2024-04-23T14:34:26.207107+0000","last_peered":"2024-04-23T14:34:26.207107+0000","last_clean":"2024-04-23T14:34:26.207107+0000","last_became_active":"2024-04-23T14:34:24.813340+0000","last_became_peered":"2024-04-23T14:34:24.813340+0000","last_unstale":"2024-04-23T14:34:26.207107+0000","last_undegraded":"2024-04-23T14:34:26.207107+0000","last_fullsized":"2024-04-23T14:34:26.207107+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:14:07.060990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.422074+0000","last_change":"2024-04-23T14:34:23.795146+0000","last_active":"2024-04-23T14:34:26.422074+0000","last_peered":"2024-04-23T14:34:26.422074+0000","last_clean":"2024-04-23T14:34:26.422074+0000","last_became_active":"2024-04-23T14:34:23.794811+0000","last_became_peered":"2024-04-23T14:34:23.794811+0000","last_unstale":"2024-04-23T14:34:26.422074+0000","last_undegraded":"2024-04-23T14:34:26.422074+0000","last_fullsized":"2024-04-23T14:34:26.422074+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:27:23.226059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805440+0000","last_change":"2024-04-23T14:34:24.811947+0000","last_active":"2024-04-23T14:34:25.805440+0000","last_peered":"2024-04-23T14:34:25.805440+0000","last_clean":"2024-04-23T14:34:25.805440+0000","last_became_active":"2024-04-23T14:34:24.811632+0000","last_became_peered":"2024-04-23T14:34:24.811632+0000","last_unstale":"2024-04-23T14:34:25.805440+0000","last_undegraded":"2024-04-23T14:34:25.805440+0000","last_fullsized":"2024-04-23T14:34:25.805440+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:04:53.451687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805082+0000","last_change":"2024-04-23T14:34:23.793321+0000","last_active":"2024-04-23T14:34:25.805082+0000","last_peered":"2024-04-23T14:34:25.805082+0000","last_clean":"2024-04-23T14:34:25.805082+0000","last_became_active":"2024-04-23T14:34:23.790416+0000","last_became_peered":"2024-04-23T14:34:23.790416+0000","last_unstale":"2024-04-23T14:34:25.805082+0000","last_undegraded":"2024-04-23T14:34:25.805082+0000","last_fullsized":"2024-04-23T14:34:25.805082+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:49:34.958705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.206887+0000","last_change":"2024-04-23T14:34:24.813008+0000","last_active":"2024-04-23T14:34:26.206887+0000","last_peered":"2024-04-23T14:34:26.206887+0000","last_clean":"2024-04-23T14:34:26.206887+0000","last_became_active":"2024-04-23T14:34:24.812052+0000","last_became_peered":"2024-04-23T14:34:24.812052+0000","last_unstale":"2024-04-23T14:34:26.206887+0000","last_undegraded":"2024-04-23T14:34:26.206887+0000","last_fullsized":"2024-04-23T14:34:26.206887+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:06:15.735233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"52'8","reported_seq":22,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.206772+0000","last_change":"2024-04-23T14:34:23.795623+0000","last_active":"2024-04-23T14:34:26.206772+0000","last_peered":"2024-04-23T14:34:26.206772+0000","last_clean":"2024-04-23T14:34:26.206772+0000","last_became_active":"2024-04-23T14:34:23.795321+0000","last_became_peered":"2024-04-23T14:34:23.795321+0000","last_unstale":"2024-04-23T14:34:26.206772+0000","last_undegraded":"2024-04-23T14:34:26.206772+0000","last_fullsized":"2024-04-23T14:34:26.206772+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:02:41.228874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.207618+0000","last_change":"2024-04-23T14:34:24.813927+0000","last_active":"2024-04-23T14:34:26.207618+0000","last_peered":"2024-04-23T14:34:26.207618+0000","last_clean":"2024-04-23T14:34:26.207618+0000","last_became_active":"2024-04-23T14:34:24.813605+0000","last_became_peered":"2024-04-23T14:34:24.813605+0000","last_unstale":"2024-04-23T14:34:26.207618+0000","last_undegraded":"2024-04-23T14:34:26.207618+0000","last_fullsized":"2024-04-23T14:34:26.207618+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:42:57.268048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.422214+0000","last_change":"2024-04-23T14:34:23.788663+0000","last_active":"2024-04-23T14:34:26.422214+0000","last_peered":"2024-04-23T14:34:26.422214+0000","last_clean":"2024-04-23T14:34:26.422214+0000","last_became_active":"2024-04-23T14:34:23.788338+0000","last_became_peered":"2024-04-23T14:34:23.788338+0000","last_unstale":"2024-04-23T14:34:26.422214+0000","last_undegraded":"2024-04-23T14:34:26.422214+0000","last_fullsized":"2024-04-23T14:34:26.422214+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:14:06.058997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805862+0000","last_change":"2024-04-23T14:34:24.804264+0000","last_active":"2024-04-23T14:34:25.805862+0000","last_peered":"2024-04-23T14:34:25.805862+0000","last_clean":"2024-04-23T14:34:25.805862+0000","last_became_active":"2024-04-23T14:34:24.803910+0000","last_became_peered":"2024-04-23T14:34:24.803910+0000","last_unstale":"2024-04-23T14:34:25.805862+0000","last_undegraded":"2024-04-23T14:34:25.805862+0000","last_fullsized":"2024-04-23T14:34:25.805862+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:33:04.830057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329810+0000","last_change":"2024-04-23T14:34:23.784914+0000","last_active":"2024-04-23T14:34:26.329810+0000","last_peered":"2024-04-23T14:34:26.329810+0000","last_clean":"2024-04-23T14:34:26.329810+0000","last_became_active":"2024-04-23T14:34:23.784284+0000","last_became_peered":"2024-04-23T14:34:23.784284+0000","last_unstale":"2024-04-23T14:34:26.329810+0000","last_undegraded":"2024-04-23T14:34:26.329810+0000","last_fullsized":"2024-04-23T14:34:26.329810+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:01:58.181015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421971+0000","last_change":"2024-04-23T14:34:24.815082+0000","last_active":"2024-04-23T14:34:26.421971+0000","last_peered":"2024-04-23T14:34:26.421971+0000","last_clean":"2024-04-23T14:34:26.421971+0000","last_became_active":"2024-04-23T14:34:24.809746+0000","last_became_peered":"2024-04-23T14:34:24.809746+0000","last_unstale":"2024-04-23T14:34:26.421971+0000","last_undegraded":"2024-04-23T14:34:26.421971+0000","last_fullsized":"2024-04-23T14:34:26.421971+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:41:08.525219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805494+0000","last_change":"2024-04-23T14:34:23.790346+0000","last_active":"2024-04-23T14:34:25.805494+0000","last_peered":"2024-04-23T14:34:25.805494+0000","last_clean":"2024-04-23T14:34:25.805494+0000","last_became_active":"2024-04-23T14:34:23.789259+0000","last_became_peered":"2024-04-23T14:34:23.789259+0000","last_unstale":"2024-04-23T14:34:25.805494+0000","last_undegraded":"2024-04-23T14:34:25.805494+0000","last_fullsized":"2024-04-23T14:34:25.805494+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:13:23.310847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.207275+0000","last_change":"2024-04-23T14:34:24.818308+0000","last_active":"2024-04-23T14:34:26.207275+0000","last_peered":"2024-04-23T14:34:26.207275+0000","last_clean":"2024-04-23T14:34:26.207275+0000","last_became_active":"2024-04-23T14:34:24.818052+0000","last_became_peered":"2024-04-23T14:34:24.818052+0000","last_unstale":"2024-04-23T14:34:26.207275+0000","last_undegraded":"2024-04-23T14:34:26.207275+0000","last_fullsized":"2024-04-23T14:34:26.207275+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:29:27.755632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329787+0000","last_change":"2024-04-23T14:34:23.792656+0000","last_active":"2024-04-23T14:34:26.329787+0000","last_peered":"2024-04-23T14:34:26.329787+0000","last_clean":"2024-04-23T14:34:26.329787+0000","last_became_active":"2024-04-23T14:34:23.792407+0000","last_became_peered":"2024-04-23T14:34:23.792407+0000","last_unstale":"2024-04-23T14:34:26.329787+0000","last_undegraded":"2024-04-23T14:34:26.329787+0000","last_fullsized":"2024-04-23T14:34:26.329787+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:29:03.757942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.804953+0000","last_change":"2024-04-23T14:34:24.801888+0000","last_active":"2024-04-23T14:34:25.804953+0000","last_peered":"2024-04-23T14:34:25.804953+0000","last_clean":"2024-04-23T14:34:25.804953+0000","last_became_active":"2024-04-23T14:34:24.801271+0000","last_became_peered":"2024-04-23T14:34:24.801271+0000","last_unstale":"2024-04-23T14:34:25.804953+0000","last_undegraded":"2024-04-23T14:34:25.804953+0000","last_fullsized":"2024-04-23T14:34:25.804953+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:14:58.214732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421773+0000","last_change":"2024-04-23T14:34:23.791100+0000","last_active":"2024-04-23T14:34:26.421773+0000","last_peered":"2024-04-23T14:34:26.421773+0000","last_clean":"2024-04-23T14:34:26.421773+0000","last_became_active":"2024-04-23T14:34:23.790887+0000","last_became_peered":"2024-04-23T14:34:23.790887+0000","last_unstale":"2024-04-23T14:34:26.421773+0000","last_undegraded":"2024-04-23T14:34:26.421773+0000","last_fullsized":"2024-04-23T14:34:26.421773+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:29:26.753640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329782+0000","last_change":"2024-04-23T14:34:24.809276+0000","last_active":"2024-04-23T14:34:26.329782+0000","last_peered":"2024-04-23T14:34:26.329782+0000","last_clean":"2024-04-23T14:34:26.329782+0000","last_became_active":"2024-04-23T14:34:24.808961+0000","last_became_peered":"2024-04-23T14:34:24.808961+0000","last_unstale":"2024-04-23T14:34:26.329782+0000","last_undegraded":"2024-04-23T14:34:26.329782+0000","last_fullsized":"2024-04-23T14:34:26.329782+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:05:05.416514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.803763+0000","last_change":"2024-04-23T14:34:23.783936+0000","last_active":"2024-04-23T14:34:25.803763+0000","last_peered":"2024-04-23T14:34:25.803763+0000","last_clean":"2024-04-23T14:34:25.803763+0000","last_became_active":"2024-04-23T14:34:23.783599+0000","last_became_peered":"2024-04-23T14:34:23.783599+0000","last_unstale":"2024-04-23T14:34:25.803763+0000","last_undegraded":"2024-04-23T14:34:25.803763+0000","last_fullsized":"2024-04-23T14:34:25.803763+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:27:23.226059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329525+0000","last_change":"2024-04-23T14:34:24.818601+0000","last_active":"2024-04-23T14:34:26.329525+0000","last_peered":"2024-04-23T14:34:26.329525+0000","last_clean":"2024-04-23T14:34:26.329525+0000","last_became_active":"2024-04-23T14:34:24.818315+0000","last_became_peered":"2024-04-23T14:34:24.818315+0000","last_unstale":"2024-04-23T14:34:26.329525+0000","last_undegraded":"2024-04-23T14:34:26.329525+0000","last_fullsized":"2024-04-23T14:34:26.329525+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:29:27.755632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.804995+0000","last_change":"2024-04-23T14:34:23.792653+0000","last_active":"2024-04-23T14:34:25.804995+0000","last_peered":"2024-04-23T14:34:25.804995+0000","last_clean":"2024-04-23T14:34:25.804995+0000","last_became_active":"2024-04-23T14:34:23.791851+0000","last_became_peered":"2024-04-23T14:34:23.791851+0000","last_unstale":"2024-04-23T14:34:25.804995+0000","last_undegraded":"2024-04-23T14:34:25.804995+0000","last_fullsized":"2024-04-23T14:34:25.804995+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:27:55.173851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805565+0000","last_change":"2024-04-23T14:34:24.798776+0000","last_active":"2024-04-23T14:34:25.805565+0000","last_peered":"2024-04-23T14:34:25.805565+0000","last_clean":"2024-04-23T14:34:25.805565+0000","last_became_active":"2024-04-23T14:34:24.798487+0000","last_became_peered":"2024-04-23T14:34:24.798487+0000","last_unstale":"2024-04-23T14:34:25.805565+0000","last_undegraded":"2024-04-23T14:34:25.805565+0000","last_fullsized":"2024-04-23T14:34:25.805565+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:05:05.416514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.804230+0000","last_change":"2024-04-23T14:34:23.797511+0000","last_active":"2024-04-23T14:34:25.804230+0000","last_peered":"2024-04-23T14:34:25.804230+0000","last_clean":"2024-04-23T14:34:25.804230+0000","last_became_active":"2024-04-23T14:34:23.796153+0000","last_became_peered":"2024-04-23T14:34:23.796153+0000","last_unstale":"2024-04-23T14:34:25.804230+0000","last_undegraded":"2024-04-23T14:34:25.804230+0000","last_fullsized":"2024-04-23T14:34:25.804230+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:49:59.206097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805971+0000","last_change":"2024-04-23T14:34:24.799868+0000","last_active":"2024-04-23T14:34:25.805971+0000","last_peered":"2024-04-23T14:34:25.805971+0000","last_clean":"2024-04-23T14:34:25.805971+0000","last_became_active":"2024-04-23T14:34:24.799575+0000","last_became_peered":"2024-04-23T14:34:24.799575+0000","last_unstale":"2024-04-23T14:34:25.805971+0000","last_undegraded":"2024-04-23T14:34:25.805971+0000","last_fullsized":"2024-04-23T14:34:25.805971+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:57:41.597696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329923+0000","last_change":"2024-04-23T14:34:23.785148+0000","last_active":"2024-04-23T14:34:26.329923+0000","last_peered":"2024-04-23T14:34:26.329923+0000","last_clean":"2024-04-23T14:34:26.329923+0000","last_became_active":"2024-04-23T14:34:23.784649+0000","last_became_peered":"2024-04-23T14:34:23.784649+0000","last_unstale":"2024-04-23T14:34:26.329923+0000","last_undegraded":"2024-04-23T14:34:26.329923+0000","last_fullsized":"2024-04-23T14:34:26.329923+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:39:40.724648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.803894+0000","last_change":"2024-04-23T14:34:24.811573+0000","last_active":"2024-04-23T14:34:25.803894+0000","last_peered":"2024-04-23T14:34:25.803894+0000","last_clean":"2024-04-23T14:34:25.803894+0000","last_became_active":"2024-04-23T14:34:24.811156+0000","last_became_peered":"2024-04-23T14:34:24.811156+0000","last_unstale":"2024-04-23T14:34:25.803894+0000","last_undegraded":"2024-04-23T14:34:25.803894+0000","last_fullsized":"2024-04-23T14:34:25.803894+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:39:09.489633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.422252+0000","last_change":"2024-04-23T14:34:23.789714+0000","last_active":"2024-04-23T14:34:26.422252+0000","last_peered":"2024-04-23T14:34:26.422252+0000","last_clean":"2024-04-23T14:34:26.422252+0000","last_became_active":"2024-04-23T14:34:23.789435+0000","last_became_peered":"2024-04-23T14:34:23.789435+0000","last_unstale":"2024-04-23T14:34:26.422252+0000","last_undegraded":"2024-04-23T14:34:26.422252+0000","last_fullsized":"2024-04-23T14:34:26.422252+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:31:51.656961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.329935+0000","last_change":"2024-04-23T14:34:24.802004+0000","last_active":"2024-04-23T14:34:26.329935+0000","last_peered":"2024-04-23T14:34:26.329935+0000","last_clean":"2024-04-23T14:34:26.329935+0000","last_became_active":"2024-04-23T14:34:24.799728+0000","last_became_peered":"2024-04-23T14:34:24.799728+0000","last_unstale":"2024-04-23T14:34:26.329935+0000","last_undegraded":"2024-04-23T14:34:26.329935+0000","last_fullsized":"2024-04-23T14:34:26.329935+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:33:23.882396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.805511+0000","last_change":"2024-04-23T14:34:23.787899+0000","last_active":"2024-04-23T14:34:25.805511+0000","last_peered":"2024-04-23T14:34:25.805511+0000","last_clean":"2024-04-23T14:34:25.805511+0000","last_became_active":"2024-04-23T14:34:23.787614+0000","last_became_peered":"2024-04-23T14:34:23.787614+0000","last_unstale":"2024-04-23T14:34:25.805511+0000","last_undegraded":"2024-04-23T14:34:25.805511+0000","last_fullsized":"2024-04-23T14:34:25.805511+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:10.096243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421524+0000","last_change":"2024-04-23T14:34:24.807497+0000","last_active":"2024-04-23T14:34:26.421524+0000","last_peered":"2024-04-23T14:34:26.421524+0000","last_clean":"2024-04-23T14:34:26.421524+0000","last_became_active":"2024-04-23T14:34:24.807240+0000","last_became_peered":"2024-04-23T14:34:24.807240+0000","last_unstale":"2024-04-23T14:34:26.421524+0000","last_undegraded":"2024-04-23T14:34:26.421524+0000","last_fullsized":"2024-04-23T14:34:26.421524+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:35:43.913771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.803587+0000","last_change":"2024-04-23T14:34:23.785163+0000","last_active":"2024-04-23T14:34:25.803587+0000","last_peered":"2024-04-23T14:34:25.803587+0000","last_clean":"2024-04-23T14:34:25.803587+0000","last_became_active":"2024-04-23T14:34:23.784848+0000","last_became_peered":"2024-04-23T14:34:23.784848+0000","last_unstale":"2024-04-23T14:34:25.803587+0000","last_undegraded":"2024-04-23T14:34:25.803587+0000","last_fullsized":"2024-04-23T14:34:25.803587+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:14.298493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.207176+0000","last_change":"2024-04-23T14:34:24.818757+0000","last_active":"2024-04-23T14:34:26.207176+0000","last_peered":"2024-04-23T14:34:26.207176+0000","last_clean":"2024-04-23T14:34:26.207176+0000","last_became_active":"2024-04-23T14:34:24.818011+0000","last_became_peered":"2024-04-23T14:34:24.818011+0000","last_unstale":"2024-04-23T14:34:26.207176+0000","last_undegraded":"2024-04-23T14:34:26.207176+0000","last_fullsized":"2024-04-23T14:34:26.207176+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:06.679211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.806105+0000","last_change":"2024-04-23T14:34:23.787146+0000","last_active":"2024-04-23T14:34:25.806105+0000","last_peered":"2024-04-23T14:34:25.806105+0000","last_clean":"2024-04-23T14:34:25.806105+0000","last_became_active":"2024-04-23T14:34:23.786842+0000","last_became_peered":"2024-04-23T14:34:23.786842+0000","last_unstale":"2024-04-23T14:34:25.806105+0000","last_undegraded":"2024-04-23T14:34:25.806105+0000","last_fullsized":"2024-04-23T14:34:25.806105+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:31:51.656961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.803582+0000","last_change":"2024-04-23T14:34:24.806226+0000","last_active":"2024-04-23T14:34:25.803582+0000","last_peered":"2024-04-23T14:34:25.803582+0000","last_clean":"2024-04-23T14:34:25.803582+0000","last_became_active":"2024-04-23T14:34:24.801068+0000","last_became_peered":"2024-04-23T14:34:24.801068+0000","last_unstale":"2024-04-23T14:34:25.803582+0000","last_undegraded":"2024-04-23T14:34:25.803582+0000","last_fullsized":"2024-04-23T14:34:25.803582+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:04:53.451687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:25.803961+0000","last_change":"2024-04-23T14:34:23.787530+0000","last_active":"2024-04-23T14:34:25.803961+0000","last_peered":"2024-04-23T14:34:25.803961+0000","last_clean":"2024-04-23T14:34:25.803961+0000","last_became_active":"2024-04-23T14:34:23.787239+0000","last_became_peered":"2024-04-23T14:34:23.787239+0000","last_unstale":"2024-04-23T14:34:25.803961+0000","last_undegraded":"2024-04-23T14:34:25.803961+0000","last_fullsized":"2024-04-23T14:34:25.803961+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:47:52.275367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421570+0000","last_change":"2024-04-23T14:34:24.800571+0000","last_active":"2024-04-23T14:34:26.421570+0000","last_peered":"2024-04-23T14:34:26.421570+0000","last_clean":"2024-04-23T14:34:26.421570+0000","last_became_active":"2024-04-23T14:34:24.800278+0000","last_became_peered":"2024-04-23T14:34:24.800278+0000","last_unstale":"2024-04-23T14:34:26.421570+0000","last_undegraded":"2024-04-23T14:34:26.421570+0000","last_fullsized":"2024-04-23T14:34:26.421570+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:10:27.472762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421905+0000","last_change":"2024-04-23T14:34:23.794762+0000","last_active":"2024-04-23T14:34:26.421905+0000","last_peered":"2024-04-23T14:34:26.421905+0000","last_clean":"2024-04-23T14:34:26.421905+0000","last_became_active":"2024-04-23T14:34:23.794402+0000","last_became_peered":"2024-04-23T14:34:23.794402+0000","last_unstale":"2024-04-23T14:34:26.421905+0000","last_undegraded":"2024-04-23T14:34:26.421905+0000","last_fullsized":"2024-04-23T14:34:26.421905+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:57.859650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.421804+0000","last_change":"2024-04-23T14:34:24.822585+0000","last_active":"2024-04-23T14:34:26.421804+0000","last_peered":"2024-04-23T14:34:26.421804+0000","last_clean":"2024-04-23T14:34:26.421804+0000","last_became_active":"2024-04-23T14:34:24.822200+0000","last_became_peered":"2024-04-23T14:34:24.822200+0000","last_unstale":"2024-04-23T14:34:26.421804+0000","last_undegraded":"2024-04-23T14:34:26.421804+0000","last_fullsized":"2024-04-23T14:34:26.421804+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:23.764964+0000","last_clean_scrub_stamp":"2024-04-23T14:34:23.764964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:39:06.009341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.422071+0000","last_change":"2024-04-23T14:34:23.789329+0000","last_active":"2024-04-23T14:34:26.422071+0000","last_peered":"2024-04-23T14:34:26.422071+0000","last_clean":"2024-04-23T14:34:26.422071+0000","last_became_active":"2024-04-23T14:34:23.789054+0000","last_became_peered":"2024-04-23T14:34:23.789054+0000","last_unstale":"2024-04-23T14:34:26.422071+0000","last_undegraded":"2024-04-23T14:34:26.422071+0000","last_fullsized":"2024-04-23T14:34:26.422071+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:22.762971+0000","last_clean_scrub_stamp":"2024-04-23T14:34:22.762971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:17:13.032214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":231,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-23T14:34:26.206850+0000","last_change":"2024-04-23T14:31:58.314235+0000","last_active":"2024-04-23T14:34:26.206850+0000","last_peered":"2024-04-23T14:34:26.206850+0000","last_clean":"2024-04-23T14:34:26.206850+0000","last_became_active":"2024-04-23T14:31:58.313165+0000","last_became_peered":"2024-04-23T14:31:58.313165+0000","last_unstale":"2024-04-23T14:34:26.206850+0000","last_undegraded":"2024-04-23T14:34:26.206850+0000","last_fullsized":"2024-04-23T14:34:26.206850+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_clean_scrub_stamp":"2024-04-23T14:31:57.140712+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:44.170436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1224704,"data_stored":869744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":279216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":643072,"data_stored":286560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":634880,"data_stored":280828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":643072,"data_stored":285964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":279272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":638976,"data_stored":285998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1245184,"data_stored":878470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:34:29.339 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-23T14:34:29.339 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-23T14:34:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.519+0000 7f578e000640 1 Processor -- start 2024-04-23T14:34:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.519+0000 7f578e000640 1 -- start start 2024-04-23T14:34:29.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.519+0000 7f578e000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5788160700 0x7f57880d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.519+0000 7f578e000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f57880ce730 0x7f57880d0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.519+0000 7f578e000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f578815cd50 con 0x7f5788156b60 2024-04-23T14:34:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.519+0000 7f578e000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f5788156f40 con 0x7f57880ce730 2024-04-23T14:34:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.519+0000 7f578e000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f578815d030 con 0x7f5788160700 2024-04-23T14:34:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.519+0000 7f578d600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f57880ce730 0x7f57880d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.519+0000 7f578cc00640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f5788156b60 0x7f578815fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:59408/0 (socket says 172.21.15.100:59408) 2024-04-23T14:34:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.519+0000 7f578cc00640 1 -- 172.21.15.100:0/1066103150 learned_addr learned my addr 172.21.15.100:0/1066103150 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.519+0000 7f5787e00640 1 --2- 172.21.15.100:0/1066103150 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5788160700 0x7f57880d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.519+0000 7f578d600640 1 -- 172.21.15.100:0/1066103150 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5788160700 msgr2=0x7f57880d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:29.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.519+0000 7f5787400640 1 -- 172.21.15.100:0/1066103150 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1824986349 0 0) 0x7f578815cd50 con 0x7f5788156b60 2024-04-23T14:34:29.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.519+0000 7f578d600640 1 --2- 172.21.15.100:0/1066103150 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5788160700 0x7f57880d1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.519+0000 7f578d600640 1 -- 172.21.15.100:0/1066103150 >> v1:172.21.15.151:6789/0 conn(0x7f5788156b60 legacy=0x7f578815fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.519+0000 7f578d600640 1 -- 172.21.15.100:0/1066103150 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578815d310 con 0x7f57880ce730 2024-04-23T14:34:29.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.523+0000 7f578d600640 1 --2- 172.21.15.100:0/1066103150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57880ce730 0x7f57880d0270 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5780009d90 tx=0x7f578001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87144b08ee45a1b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:29.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.523+0000 7f5787400640 1 -- 172.21.15.100:0/1066103150 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57800b2070 con 0x7f57880ce730 2024-04-23T14:34:29.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.523+0000 7f5787400640 1 -- 172.21.15.100:0/1066103150 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57800a5a10 con 0x7f57880ce730 2024-04-23T14:34:29.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.523+0000 7f5787400640 1 -- 172.21.15.100:0/1066103150 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57800a5de0 con 0x7f57880ce730 2024-04-23T14:34:29.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.523+0000 7f578e000640 1 -- 172.21.15.100:0/1066103150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57880ce730 msgr2=0x7f57880d0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.523+0000 7f578e000640 1 --2- 172.21.15.100:0/1066103150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57880ce730 0x7f57880d0270 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5780009d90 tx=0x7f578001c970 comp rx=0 tx=0).stop 2024-04-23T14:34:29.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.523+0000 7f578e000640 1 -- 172.21.15.100:0/1066103150 shutdown_connections 2024-04-23T14:34:29.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.523+0000 7f578e000640 1 --2- 172.21.15.100:0/1066103150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57880ce730 0x7f57880d0270 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.523+0000 7f578e000640 1 --2- 172.21.15.100:0/1066103150 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5788160700 0x7f57880d1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.523+0000 7f578e000640 1 -- 172.21.15.100:0/1066103150 >> 172.21.15.100:0/1066103150 conn(0x7f57880cedf0 msgr2=0x7f57880cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:29.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.523+0000 7f578e000640 1 -- 172.21.15.100:0/1066103150 shutdown_connections 2024-04-23T14:34:29.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.523+0000 7f578e000640 1 -- 172.21.15.100:0/1066103150 wait complete. 2024-04-23T14:34:29.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.523+0000 7f578e000640 1 Processor -- start 2024-04-23T14:34:29.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f578e000640 1 -- start start 2024-04-23T14:34:29.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f578e000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f57880ce730 0x7f5788151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f578e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5788156b60 0x7f57880c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f578e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5788160700 0x7f578814c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f578e000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f57880d0fc0 con 0x7f5788156b60 2024-04-23T14:34:29.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f578e000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f578815ca00 con 0x7f5788160700 2024-04-23T14:34:29.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f578e000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f57880d1140 con 0x7f57880ce730 2024-04-23T14:34:29.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f5787e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5788156b60 0x7f57880c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f5787e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5788156b60 0x7f57880c7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48344/0 (socket says 172.21.15.100:48344) 2024-04-23T14:34:29.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f5787e00640 1 -- 172.21.15.100:0/3897274824 learned_addr learned my addr 172.21.15.100:0/3897274824 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:29.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f578cc00640 1 --2- 172.21.15.100:0/3897274824 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f57880ce730 0x7f5788151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f578d600640 1 --2- 172.21.15.100:0/3897274824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5788160700 0x7f578814c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f5787e00640 1 -- 172.21.15.100:0/3897274824 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f57880ce730 msgr2=0x7f5788151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f5787e00640 1 --2- 172.21.15.100:0/3897274824 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f57880ce730 0x7f5788151450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f5787e00640 1 -- 172.21.15.100:0/3897274824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5788160700 msgr2=0x7f578814c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f5787e00640 1 --2- 172.21.15.100:0/3897274824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5788160700 0x7f578814c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f5787e00640 1 -- 172.21.15.100:0/3897274824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578814cf90 con 0x7f5788156b60 2024-04-23T14:34:29.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f5787e00640 1 --2- 172.21.15.100:0/3897274824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5788156b60 0x7f57880c7d10 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f577400e8a0 tx=0x7f577400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:29.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f5784c00640 1 -- 172.21.15.100:0/3897274824 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774017070 con 0x7f5788156b60 2024-04-23T14:34:29.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f5784c00640 1 -- 172.21.15.100:0/3897274824 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f577400c8e0 con 0x7f5788156b60 2024-04-23T14:34:29.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f5784c00640 1 -- 172.21.15.100:0/3897274824 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577400cc90 con 0x7f5788156b60 2024-04-23T14:34:29.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f578e000640 1 -- 172.21.15.100:0/3897274824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f578814d270 con 0x7f5788156b60 2024-04-23T14:34:29.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f578e000640 1 -- 172.21.15.100:0/3897274824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57880c7510 con 0x7f5788156b60 2024-04-23T14:34:29.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.527+0000 7f578e000640 1 -- 172.21.15.100:0/3897274824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5788075c20 con 0x7f5788156b60 2024-04-23T14:34:29.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.531+0000 7f5784c00640 1 -- 172.21.15.100:0/3897274824 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f577401a070 con 0x7f5788156b60 2024-04-23T14:34:29.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.531+0000 7f5784c00640 1 --2- 172.21.15.100:0/3897274824 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5768068380 0x7f576806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.531+0000 7f5784c00640 1 -- 172.21.15.100:0/3897274824 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f5774086d60 con 0x7f5788156b60 2024-04-23T14:34:29.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.531+0000 7f578cc00640 1 --2- 172.21.15.100:0/3897274824 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5768068380 0x7f576806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.531+0000 7f578cc00640 1 --2- 172.21.15.100:0/3897274824 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5768068380 0x7f576806a840 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f577c000f70 tx=0x7f577c007240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:29.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.535+0000 7f5784c00640 1 -- 172.21.15.100:0/3897274824 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5774056aa0 con 0x7f5788156b60 2024-04-23T14:34:29.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.747+0000 7f578e000640 1 -- 172.21.15.100:0/3897274824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f57880b46a0 con 0x7f5788156b60 2024-04-23T14:34:29.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.747+0000 7f5784c00640 1 -- 172.21.15.100:0/3897274824 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f577405ada0 con 0x7f5788156b60 2024-04-23T14:34:29.751 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-04-23T14:34:29.751 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAtxidmyhImMBAAa49ApQEF2Lng6rY94F1HAQ== 2024-04-23T14:34:29.751 INFO:teuthology.orchestra.run.smithi100.stdout: caps mds = "allow" 2024-04-23T14:34:29.751 INFO:teuthology.orchestra.run.smithi100.stdout: caps mon = "allow r" 2024-04-23T14:34:29.751 INFO:teuthology.orchestra.run.smithi100.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-23T14:34:29.751 INFO:teuthology.orchestra.run.smithi100.stderr:updated caps for client.0 2024-04-23T14:34:29.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.751+0000 7f578e000640 1 -- 172.21.15.100:0/3897274824 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5768068380 msgr2=0x7f576806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.751+0000 7f578e000640 1 --2- 172.21.15.100:0/3897274824 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5768068380 0x7f576806a840 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f577c000f70 tx=0x7f577c007240 comp rx=0 tx=0).stop 2024-04-23T14:34:29.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.751+0000 7f578e000640 1 -- 172.21.15.100:0/3897274824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5788156b60 msgr2=0x7f57880c7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.751+0000 7f578e000640 1 --2- 172.21.15.100:0/3897274824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5788156b60 0x7f57880c7d10 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f577400e8a0 tx=0x7f577400eca0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.751+0000 7f578e000640 1 -- 172.21.15.100:0/3897274824 shutdown_connections 2024-04-23T14:34:29.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.751+0000 7f578e000640 1 --2- 172.21.15.100:0/3897274824 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5768068380 0x7f576806a840 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.751+0000 7f578e000640 1 --2- 172.21.15.100:0/3897274824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5788160700 0x7f578814c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.751+0000 7f578e000640 1 --2- 172.21.15.100:0/3897274824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5788156b60 0x7f57880c7d10 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.751+0000 7f578e000640 1 --2- 172.21.15.100:0/3897274824 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f57880ce730 0x7f5788151450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.751+0000 7f578e000640 1 -- 172.21.15.100:0/3897274824 >> 172.21.15.100:0/3897274824 conn(0x7f57880cedf0 msgr2=0x7f57880b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:29.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.751+0000 7f578e000640 1 -- 172.21.15.100:0/3897274824 shutdown_connections 2024-04-23T14:34:29.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.751+0000 7f578e000640 1 -- 172.21.15.100:0/3897274824 wait complete. 2024-04-23T14:34:29.772 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:34:29.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dd3a00640 1 Processor -- start 2024-04-23T14:34:29.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dd3a00640 1 -- start start 2024-04-23T14:34:29.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dd3a00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f0dcc0cea60 0x7f0dcc0d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dd3a00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dcc0b1f90 con 0x7f0dcc06d450 2024-04-23T14:34:29.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dd3a00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dcc0b2270 con 0x7f0dcc0cd460 2024-04-23T14:34:29.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dd3a00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f0dcc0b2550 con 0x7f0dcc0cea60 2024-04-23T14:34:29.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dd1000640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f0dcc06d450 0x7f0dcc06d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:39830/0 (socket says 172.21.15.100:39830) 2024-04-23T14:34:29.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dd1000640 1 -- 172.21.15.100:0/2514516255 learned_addr learned my addr 172.21.15.100:0/2514516255 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:29.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dcbe00640 1 --2- 172.21.15.100:0/2514516255 >> v2:172.21.15.151:3300/0 conn(0x7f0dcc0cea60 0x7f0dcc0d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dcb400640 1 -- 172.21.15.100:0/2514516255 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 851311791 0 0) 0x7f0dcc0b1f90 con 0x7f0dcc06d450 2024-04-23T14:34:29.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dcb400640 1 -- 172.21.15.100:0/2514516255 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0db0003530 con 0x7f0dcc06d450 2024-04-23T14:34:29.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dcb400640 1 -- 172.21.15.100:0/2514516255 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 258897678 0 0) 0x7f0dcc0b2270 con 0x7f0dcc0cd460 2024-04-23T14:34:29.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dcb400640 1 -- 172.21.15.100:0/2514516255 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0dcc0b1f90 con 0x7f0dcc0cd460 2024-04-23T14:34:29.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dcb400640 1 -- 172.21.15.100:0/2514516255 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1612354201 0 0) 0x7f0db0003530 con 0x7f0dcc06d450 2024-04-23T14:34:29.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dcb400640 1 -- 172.21.15.100:0/2514516255 >> v2:172.21.15.151:3300/0 conn(0x7f0dcc0cea60 msgr2=0x7f0dcc0d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:29.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dcb400640 1 --2- 172.21.15.100:0/2514516255 >> v2:172.21.15.151:3300/0 conn(0x7f0dcc0cea60 0x7f0dcc0d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dcb400640 1 -- 172.21.15.100:0/2514516255 >> v1:172.21.15.151:6789/0 conn(0x7f0dcc0cd460 legacy=0x7f0dcc0b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dcb400640 1 -- 172.21.15.100:0/2514516255 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dcc0b2830 con 0x7f0dcc06d450 2024-04-23T14:34:29.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dcb400640 1 -- 172.21.15.100:0/2514516255 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f0dbc001a00 con 0x7f0dcc06d450 2024-04-23T14:34:29.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dcb400640 1 -- 172.21.15.100:0/2514516255 >> v1:172.21.15.100:6789/0 conn(0x7f0dcc06d450 legacy=0x7f0dcc06d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dcb400640 1 --2- 172.21.15.100:0/2514516255 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0db0003b50 0x7f0db0003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dcb400640 1 --2- 172.21.15.100:0/2514516255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0db0004810 0x7f0db0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dcb400640 1 --2- 172.21.15.100:0/2514516255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0db00051d0 0x7f0db000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dcb400640 1 -- 172.21.15.100:0/2514516255 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc0b2550 con 0x7f0db0004810 2024-04-23T14:34:29.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dcb400640 1 -- 172.21.15.100:0/2514516255 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0db000c100 con 0x7f0db00051d0 2024-04-23T14:34:29.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dcb400640 1 -- 172.21.15.100:0/2514516255 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0db000c300 con 0x7f0db0003b50 2024-04-23T14:34:29.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dcbe00640 1 --2- 172.21.15.100:0/2514516255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0db0004810 0x7f0db0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dd1000640 1 --2- 172.21.15.100:0/2514516255 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0db0003b50 0x7f0db0003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.907+0000 7f0dd1a00640 1 --2- 172.21.15.100:0/2514516255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0db00051d0 0x7f0db000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.911+0000 7f0dcbe00640 1 -- 172.21.15.100:0/2514516255 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0db0003b50 msgr2=0x7f0db0003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:29.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.911+0000 7f0dcbe00640 1 --2- 172.21.15.100:0/2514516255 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0db0003b50 0x7f0db0003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.911+0000 7f0dcbe00640 1 -- 172.21.15.100:0/2514516255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0db00051d0 msgr2=0x7f0db000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:29.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.911+0000 7f0dcbe00640 1 --2- 172.21.15.100:0/2514516255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0db00051d0 0x7f0db000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.911+0000 7f0dcbe00640 1 -- 172.21.15.100:0/2514516255 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db0003530 con 0x7f0db0004810 2024-04-23T14:34:29.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.911+0000 7f0dd1000640 1 --2- 172.21.15.100:0/2514516255 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0db0003b50 0x7f0db0003f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:29.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.911+0000 7f0dd1a00640 1 --2- 172.21.15.100:0/2514516255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0db00051d0 0x7f0db000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:29.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.911+0000 7f0dd1a00640 1 -- 172.21.15.100:0/2514516255 reap_dead start 2024-04-23T14:34:29.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.911+0000 7f0dcbe00640 1 --2- 172.21.15.100:0/2514516255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0db0004810 0x7f0db0004c90 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009ae0 tx=0x7f0db401c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c716a69625714c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:29.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.911+0000 7f0dcb400640 1 -- 172.21.15.100:0/2514516255 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db40b1070 con 0x7f0db0004810 2024-04-23T14:34:29.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.911+0000 7f0dcb400640 1 -- 172.21.15.100:0/2514516255 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db40a4cb0 con 0x7f0db0004810 2024-04-23T14:34:29.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.911+0000 7f0dcb400640 1 -- 172.21.15.100:0/2514516255 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db40096b0 con 0x7f0db0004810 2024-04-23T14:34:29.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.911+0000 7f0dd3a00640 1 -- 172.21.15.100:0/2514516255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0db0004810 msgr2=0x7f0db0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.911+0000 7f0dd3a00640 1 --2- 172.21.15.100:0/2514516255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0db0004810 0x7f0db0004c90 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009ae0 tx=0x7f0db401c6f0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.911+0000 7f0dd3a00640 1 -- 172.21.15.100:0/2514516255 shutdown_connections 2024-04-23T14:34:29.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.911+0000 7f0dd3a00640 1 --2- 172.21.15.100:0/2514516255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0db0004810 0x7f0db0004c90 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.911+0000 7f0dd3a00640 1 -- 172.21.15.100:0/2514516255 >> 172.21.15.100:0/2514516255 conn(0x7f0dcc0cee80 msgr2=0x7f0dcc0cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:29.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.911+0000 7f0dd3a00640 1 -- 172.21.15.100:0/2514516255 shutdown_connections 2024-04-23T14:34:29.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.911+0000 7f0dd3a00640 1 -- 172.21.15.100:0/2514516255 wait complete. 2024-04-23T14:34:29.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dd3a00640 1 Processor -- start 2024-04-23T14:34:29.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dd3a00640 1 -- start start 2024-04-23T14:34:29.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dd3a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0db0004810 0x7f0dcc0ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dd3a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dcc0c8590 0x7f0dcc14c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dd3a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dcc14cea0 0x7f0dcc1ec000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dd3a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc0b38f0 con 0x7f0dcc0c8590 2024-04-23T14:34:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dd3a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc0b3770 con 0x7f0dcc14cea0 2024-04-23T14:34:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dd3a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0dcc15eae0 con 0x7f0db0004810 2024-04-23T14:34:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dcbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dcc0c8590 0x7f0dcc14c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dcbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dcc0c8590 0x7f0dcc14c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48362/0 (socket says 172.21.15.100:48362) 2024-04-23T14:34:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dcbe00640 1 -- 172.21.15.100:0/734881234 learned_addr learned my addr 172.21.15.100:0/734881234 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dd1000640 1 --2- 172.21.15.100:0/734881234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0db0004810 0x7f0dcc0ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dd1a00640 1 --2- 172.21.15.100:0/734881234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dcc14cea0 0x7f0dcc1ec000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dcbe00640 1 -- 172.21.15.100:0/734881234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0db0004810 msgr2=0x7f0dcc0ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dcbe00640 1 --2- 172.21.15.100:0/734881234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0db0004810 0x7f0dcc0ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dcbe00640 1 -- 172.21.15.100:0/734881234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dcc14cea0 msgr2=0x7f0dcc1ec000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dcbe00640 1 --2- 172.21.15.100:0/734881234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dcc14cea0 0x7f0dcc1ec000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dcbe00640 1 -- 172.21.15.100:0/734881234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dcc0caf50 con 0x7f0dcc0c8590 2024-04-23T14:34:29.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dcbe00640 1 --2- 172.21.15.100:0/734881234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dcc0c8590 0x7f0dcc14c960 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0db40af040 tx=0x7f0db4019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:29.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dc8c00640 1 -- 172.21.15.100:0/734881234 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db40b1040 con 0x7f0dcc0c8590 2024-04-23T14:34:29.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dc8c00640 1 -- 172.21.15.100:0/734881234 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db4004020 con 0x7f0dcc0c8590 2024-04-23T14:34:29.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dc8c00640 1 -- 172.21.15.100:0/734881234 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db40a4e60 con 0x7f0dcc0c8590 2024-04-23T14:34:29.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dd3a00640 1 -- 172.21.15.100:0/734881234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dcc14d500 con 0x7f0dcc0c8590 2024-04-23T14:34:29.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.915+0000 7f0dd3a00640 1 -- 172.21.15.100:0/734881234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dcc14d7e0 con 0x7f0dcc0c8590 2024-04-23T14:34:29.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.919+0000 7f0dd3a00640 1 -- 172.21.15.100:0/734881234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dcc14d280 con 0x7f0dcc0c8590 2024-04-23T14:34:29.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.919+0000 7f0dc8c00640 1 -- 172.21.15.100:0/734881234 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0db40be060 con 0x7f0dcc0c8590 2024-04-23T14:34:29.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.919+0000 7f0dc8c00640 1 --2- 172.21.15.100:0/734881234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0da8068380 0x7f0da806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.919+0000 7f0dc8c00640 1 -- 172.21.15.100:0/734881234 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f0db41200d0 con 0x7f0dcc0c8590 2024-04-23T14:34:29.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.919+0000 7f0dd1000640 1 --2- 172.21.15.100:0/734881234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0da8068380 0x7f0da806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.919+0000 7f0dd1000640 1 --2- 172.21.15.100:0/734881234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0da8068380 0x7f0da806a840 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc000f90 tx=0x7f0dbc001310 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:29.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:29.923+0000 7f0dc8c00640 1 -- 172.21.15.100:0/734881234 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0dcc14d280 con 0x7f0dcc0c8590 2024-04-23T14:34:30.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.111+0000 7f0dd3a00640 1 -- 172.21.15.100:0/734881234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0dcc068ad0 con 0x7f0dcc0c8590 2024-04-23T14:34:30.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.111+0000 7f0dc8c00640 1 -- 172.21.15.100:0/734881234 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5190 (secure 0 0 0) 0x7f0db4016780 con 0x7f0dcc0c8590 2024-04-23T14:34:30.114 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:30.114 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5528,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2774610498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2774610498},{"type":"v1","addr":"172.21.15.151:6833","nonce":2774610498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5929,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1347883282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1347883282},{"type":"v1","addr":"172.21.15.151:6835","nonce":1347883282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":6177,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3319827520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3319827520},{"type":"v1","addr":"172.21.15.100:6836","nonce":3319827520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:34:24.162539+0000","modified":"2024-04-23T14:34:27.179639+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6180":{"gid":6180,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/608349550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":608349550},{"type":"v1","addr":"172.21.15.100:6837","nonce":608349550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6180,"qdb_cluster":[6180]},"id":3}]} 2024-04-23T14:34:30.114 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 28 2024-04-23T14:34:30.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.115+0000 7f0dd3a00640 1 -- 172.21.15.100:0/734881234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0da8068380 msgr2=0x7f0da806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.115+0000 7f0dd3a00640 1 --2- 172.21.15.100:0/734881234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0da8068380 0x7f0da806a840 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc000f90 tx=0x7f0dbc001310 comp rx=0 tx=0).stop 2024-04-23T14:34:30.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.115+0000 7f0dd3a00640 1 -- 172.21.15.100:0/734881234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dcc0c8590 msgr2=0x7f0dcc14c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.115+0000 7f0dd3a00640 1 --2- 172.21.15.100:0/734881234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dcc0c8590 0x7f0dcc14c960 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0db40af040 tx=0x7f0db4019810 comp rx=0 tx=0).stop 2024-04-23T14:34:30.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.115+0000 7f0dd3a00640 1 -- 172.21.15.100:0/734881234 shutdown_connections 2024-04-23T14:34:30.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.115+0000 7f0dd3a00640 1 --2- 172.21.15.100:0/734881234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0da8068380 0x7f0da806a840 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.115+0000 7f0dd3a00640 1 --2- 172.21.15.100:0/734881234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dcc14cea0 0x7f0dcc1ec000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.115+0000 7f0dd3a00640 1 --2- 172.21.15.100:0/734881234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dcc0c8590 0x7f0dcc14c960 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.115+0000 7f0dd3a00640 1 --2- 172.21.15.100:0/734881234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0db0004810 0x7f0dcc0ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.115+0000 7f0dd3a00640 1 -- 172.21.15.100:0/734881234 >> 172.21.15.100:0/734881234 conn(0x7f0dcc0cee80 msgr2=0x7f0dcc15dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:30.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.115+0000 7f0dd3a00640 1 -- 172.21.15.100:0/734881234 shutdown_connections 2024-04-23T14:34:30.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.115+0000 7f0dd3a00640 1 -- 172.21.15.100:0/734881234 wait complete. 2024-04-23T14:34:30.538 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:34:24.162539+0000', 'modified': '2024-04-23T14:34:27.179639+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6180}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6180': {'gid': 6180, 'name': 'a', 'rank': 0, 'incarnation': 24, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6837/608349550', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6835', 'nonce': 608349550}, {'type': 'v1', 'addr': '172.21.15.100:6837', 'nonce': 608349550}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6180, 'qdb_cluster': [6180]} 2024-04-23T14:34:30.538 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-23T14:34:30.539 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-23T14:34:30.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.671+0000 7ff03ae00640 1 Processor -- start 2024-04-23T14:34:30.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.671+0000 7ff03ae00640 1 -- start start 2024-04-23T14:34:30.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.675+0000 7ff03ae00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff03415e750 0x7ff0340cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.675+0000 7ff03ae00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff03415eb30 0x7ff034162f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.675+0000 7ff03ae00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff034163480 con 0x7ff0340c6310 2024-04-23T14:34:30.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.675+0000 7ff03ae00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff034099e80 con 0x7ff03415eb30 2024-04-23T14:34:30.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.675+0000 7ff03ae00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff0340cd120 con 0x7ff03415e750 2024-04-23T14:34:30.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.675+0000 7ff033400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff03415eb30 0x7ff034162f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.675+0000 7ff033400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff03415eb30 0x7ff034162f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48364/0 (socket says 172.21.15.100:48364) 2024-04-23T14:34:30.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.675+0000 7ff033400640 1 -- 172.21.15.100:0/1805105682 learned_addr learned my addr 172.21.15.100:0/1805105682 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:30.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.675+0000 7ff033e00640 1 --2- 172.21.15.100:0/1805105682 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff03415e750 0x7ff0340cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.675+0000 7ff033400640 1 -- 172.21.15.100:0/1805105682 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff03415e750 msgr2=0x7ff0340cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:30.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.675+0000 7ff033400640 1 --2- 172.21.15.100:0/1805105682 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff03415e750 0x7ff0340cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.675+0000 7ff033400640 1 -- 172.21.15.100:0/1805105682 >> v1:172.21.15.151:6789/0 conn(0x7ff0340c6310 legacy=0x7ff03406c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.675+0000 7ff033400640 1 -- 172.21.15.100:0/1805105682 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff034099ba0 con 0x7ff03415eb30 2024-04-23T14:34:30.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.679+0000 7ff033400640 1 --2- 172.21.15.100:0/1805105682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff03415eb30 0x7ff034162f40 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff020009870 tx=0x7ff02001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ea8e83e6bb79c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:30.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.679+0000 7ff032a00640 1 -- 172.21.15.100:0/1805105682 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0200ab610 con 0x7ff03415eb30 2024-04-23T14:34:30.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.679+0000 7ff032a00640 1 -- 172.21.15.100:0/1805105682 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0200b0070 con 0x7ff03415eb30 2024-04-23T14:34:30.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.679+0000 7ff032a00640 1 -- 172.21.15.100:0/1805105682 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0200ab9d0 con 0x7ff03415eb30 2024-04-23T14:34:30.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.679+0000 7ff03ae00640 1 -- 172.21.15.100:0/1805105682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff03415eb30 msgr2=0x7ff034162f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.679+0000 7ff03ae00640 1 --2- 172.21.15.100:0/1805105682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff03415eb30 0x7ff034162f40 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff020009870 tx=0x7ff02001c410 comp rx=0 tx=0).stop 2024-04-23T14:34:30.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.679+0000 7ff03ae00640 1 -- 172.21.15.100:0/1805105682 shutdown_connections 2024-04-23T14:34:30.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.679+0000 7ff03ae00640 1 --2- 172.21.15.100:0/1805105682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff03415eb30 0x7ff034162f40 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.679+0000 7ff03ae00640 1 --2- 172.21.15.100:0/1805105682 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff03415e750 0x7ff0340cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.679+0000 7ff03ae00640 1 -- 172.21.15.100:0/1805105682 >> 172.21.15.100:0/1805105682 conn(0x7ff0340b5310 msgr2=0x7ff0340b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:30.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.679+0000 7ff03ae00640 1 -- 172.21.15.100:0/1805105682 shutdown_connections 2024-04-23T14:34:30.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.679+0000 7ff03ae00640 1 -- 172.21.15.100:0/1805105682 wait complete. 2024-04-23T14:34:30.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.679+0000 7ff03ae00640 1 Processor -- start 2024-04-23T14:34:30.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.679+0000 7ff03ae00640 1 -- start start 2024-04-23T14:34:30.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff03ae00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0340c6310 0x7ff034161b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff03ae00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff03415e750 0x7ff03414c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff03ae00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff03415eb30 0x7ff0340af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff03ae00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff0340c6b30 con 0x7ff0340c6310 2024-04-23T14:34:30.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff03ae00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0340997c0 con 0x7ff03415e750 2024-04-23T14:34:30.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff03ae00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff03414cf30 con 0x7ff03415eb30 2024-04-23T14:34:30.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff033e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0340c6310 0x7ff034161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff033e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0340c6310 0x7ff034161b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48380/0 (socket says 172.21.15.100:48380) 2024-04-23T14:34:30.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff033e00640 1 -- 172.21.15.100:0/1922307949 learned_addr learned my addr 172.21.15.100:0/1922307949 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:30.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff033400640 1 --2- 172.21.15.100:0/1922307949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff03415e750 0x7ff03414c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff038e00640 1 --2- 172.21.15.100:0/1922307949 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff03415eb30 0x7ff0340af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff033e00640 1 -- 172.21.15.100:0/1922307949 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff03415eb30 msgr2=0x7ff0340af8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff033e00640 1 --2- 172.21.15.100:0/1922307949 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff03415eb30 0x7ff0340af8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff033e00640 1 -- 172.21.15.100:0/1922307949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff03415e750 msgr2=0x7ff03414c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff033e00640 1 --2- 172.21.15.100:0/1922307949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff03415e750 0x7ff03414c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff033e00640 1 -- 172.21.15.100:0/1922307949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0340afe30 con 0x7ff0340c6310 2024-04-23T14:34:30.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff038e00640 1 --2- 172.21.15.100:0/1922307949 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff03415eb30 0x7ff0340af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:30.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff033e00640 1 --2- 172.21.15.100:0/1922307949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0340c6310 0x7ff034161b60 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff02400d800 tx=0x7ff02400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:30.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff02be00640 1 -- 172.21.15.100:0/1922307949 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02400b6c0 con 0x7ff0340c6310 2024-04-23T14:34:30.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff02be00640 1 -- 172.21.15.100:0/1922307949 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff024019070 con 0x7ff0340c6310 2024-04-23T14:34:30.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff03ae00640 1 -- 172.21.15.100:0/1922307949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03414d160 con 0x7ff0340c6310 2024-04-23T14:34:30.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff02be00640 1 -- 172.21.15.100:0/1922307949 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02400bae0 con 0x7ff0340c6310 2024-04-23T14:34:30.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff03ae00640 1 -- 172.21.15.100:0/1922307949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff03406ec90 con 0x7ff0340c6310 2024-04-23T14:34:30.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.683+0000 7ff02be00640 1 -- 172.21.15.100:0/1922307949 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff02400bc80 con 0x7ff0340c6310 2024-04-23T14:34:30.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff02be00640 1 --2- 172.21.15.100:0/1922307949 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff014068380 0x7ff01406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff02be00640 1 -- 172.21.15.100:0/1922307949 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7ff024087070 con 0x7ff0340c6310 2024-04-23T14:34:30.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff033400640 1 --2- 172.21.15.100:0/1922307949 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff014068380 0x7ff01406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff03e39d000 1 Processor -- start 2024-04-23T14:34:30.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff03e39d000 1 -- start start 2024-04-23T14:34:30.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff033400640 1 --2- 172.21.15.100:0/1922307949 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff014068380 0x7ff01406a840 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff0200af000 tx=0x7ff0200a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:30.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff03e39d000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd6b65b0 0x5624dd6b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff03e39d000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd6b6ed0 0x5624dd669c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff03e39d000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5624dd66a140 0x5624dd66a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff03e39d000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5624dd5536f0 con 0x5624dd6b65b0 2024-04-23T14:34:30.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff03e39d000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5624dd4ba4d0 con 0x5624dd6b6ed0 2024-04-23T14:34:30.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff03e39d000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x5624dd454bf0 con 0x5624dd66a140 2024-04-23T14:34:30.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff033e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd6b65b0 0x5624dd6b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff033e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd6b65b0 0x5624dd6b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48382/0 (socket says 172.21.15.100:48382) 2024-04-23T14:34:30.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff033e00640 1 -- 172.21.15.100:0/432874325 learned_addr learned my addr 172.21.15.100:0/432874325 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff033400640 1 --2- 172.21.15.100:0/432874325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd6b6ed0 0x5624dd669c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff033e00640 1 -- 172.21.15.100:0/432874325 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5624dd66a140 msgr2=0x5624dd66a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff038e00640 1 --2- 172.21.15.100:0/432874325 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5624dd66a140 0x5624dd66a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff033e00640 1 --2- 172.21.15.100:0/432874325 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5624dd66a140 0x5624dd66a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff033e00640 1 -- 172.21.15.100:0/432874325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd6b6ed0 msgr2=0x5624dd669c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff033e00640 1 --2- 172.21.15.100:0/432874325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd6b6ed0 0x5624dd669c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff033e00640 1 -- 172.21.15.100:0/432874325 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624dd454e40 con 0x5624dd6b65b0 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff033400640 1 --2- 172.21.15.100:0/432874325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd6b6ed0 0x5624dd669c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff038e00640 1 --2- 172.21.15.100:0/432874325 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5624dd66a140 0x5624dd66a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff033e00640 1 --2- 172.21.15.100:0/432874325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd6b65b0 0x5624dd6b6990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff024053160 tx=0x7ff0240574c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d55c6f87f9d5f22 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff028c00640 1 -- 172.21.15.100:0/432874325 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024083070 con 0x5624dd6b65b0 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff028c00640 1 -- 172.21.15.100:0/432874325 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff02405f8e0 con 0x5624dd6b65b0 2024-04-23T14:34:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff028c00640 1 -- 172.21.15.100:0/432874325 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02406f070 con 0x5624dd6b65b0 2024-04-23T14:34:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff03e39d000 1 -- 172.21.15.100:0/432874325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd6b65b0 msgr2=0x5624dd6b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff03e39d000 1 --2- 172.21.15.100:0/432874325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd6b65b0 0x5624dd6b6990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff024053160 tx=0x7ff0240574c0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff03e39d000 1 -- 172.21.15.100:0/432874325 shutdown_connections 2024-04-23T14:34:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff03e39d000 1 --2- 172.21.15.100:0/432874325 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5624dd66a140 0x5624dd66a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff03e39d000 1 --2- 172.21.15.100:0/432874325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd6b6ed0 0x5624dd669c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff03e39d000 1 --2- 172.21.15.100:0/432874325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd6b65b0 0x5624dd6b6990 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff03e39d000 1 -- 172.21.15.100:0/432874325 >> 172.21.15.100:0/432874325 conn(0x5624dd697350 msgr2=0x5624dd699740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff03e39d000 1 -- 172.21.15.100:0/432874325 shutdown_connections 2024-04-23T14:34:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.687+0000 7ff03e39d000 1 -- 172.21.15.100:0/432874325 wait complete. 2024-04-23T14:34:30.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff03e39d000 1 -- start start 2024-04-23T14:34:30.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff03e39d000 1 Processor -- start 2024-04-23T14:34:30.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff03e39d000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5624dd66a140 0x5624dd670280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff03e39d000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd6b65b0 0x5624dd6707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff03e39d000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd6b6ed0 0x5624dd670d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff03e39d000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5624dd576e50 con 0x5624dd6b65b0 2024-04-23T14:34:30.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff03e39d000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5624dd477a60 con 0x5624dd6b6ed0 2024-04-23T14:34:30.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff03e39d000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x5624dd475cf0 con 0x5624dd66a140 2024-04-23T14:34:30.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff03e39d000 10 client.0 _ll_register_callbacks cb 0x5624dd657a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:34:30.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff033400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd6b65b0 0x5624dd6707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff033400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd6b65b0 0x5624dd6707c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48386/0 (socket says 172.21.15.100:48386) 2024-04-23T14:34:30.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff033400640 1 -- 172.21.15.100:0/500595654 learned_addr learned my addr 172.21.15.100:0/500595654 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:30.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff038e00640 1 --2- 172.21.15.100:0/500595654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd6b6ed0 0x5624dd670d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff033e00640 1 --2- 172.21.15.100:0/500595654 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5624dd66a140 0x5624dd670280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff033400640 1 -- 172.21.15.100:0/500595654 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5624dd66a140 msgr2=0x5624dd670280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:30.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff033400640 1 --2- 172.21.15.100:0/500595654 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5624dd66a140 0x5624dd670280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff033400640 1 -- 172.21.15.100:0/500595654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd6b6ed0 msgr2=0x5624dd670d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:30.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff033400640 1 --2- 172.21.15.100:0/500595654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd6b6ed0 0x5624dd670d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff033400640 1 -- 172.21.15.100:0/500595654 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624dd485e00 con 0x5624dd6b65b0 2024-04-23T14:34:30.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff038e00640 1 --2- 172.21.15.100:0/500595654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd6b6ed0 0x5624dd670d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:30.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff033400640 1 --2- 172.21.15.100:0/500595654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd6b65b0 0x5624dd6707c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ff02001a040 tx=0x7ff02001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1aa52158342d3a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:30.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff01d600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-04-23T14:34:30.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff01d600640 1 -- 172.21.15.100:0/500595654 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0200cd060 con 0x5624dd6b65b0 2024-04-23T14:34:30.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff01d600640 1 -- 172.21.15.100:0/500595654 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7ff0200c0ce0 con 0x5624dd6b65b0 2024-04-23T14:34:30.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff03e39d000 1 -- 172.21.15.100:0/500595654 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff02405f8e0 con 0x5624dd6b65b0 2024-04-23T14:34:30.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff01d600640 1 -- 172.21.15.100:0/500595654 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff0200be420 con 0x5624dd6b65b0 2024-04-23T14:34:30.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff03e39d000 10 client.6333 fetch_fsmap learned FSMap version 28 2024-04-23T14:34:30.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff03e39d000 1 -- 172.21.15.100:0/500595654 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x5624dd53c670 con 0x5624dd6b65b0 2024-04-23T14:34:30.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff01d600640 1 -- 172.21.15.100:0/500595654 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff02405f8e0 con 0x5624dd6b65b0 2024-04-23T14:34:30.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff01d600640 1 -- 172.21.15.100:0/500595654 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7ff0200c7030 con 0x5624dd6b65b0 2024-04-23T14:34:30.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff03e39d000 10 client.6333 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T14:34:30.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff03e39d000 10 client.6333 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-23T14:34:30.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff03e39d000 20 client.6333 populate_metadata read hostname 'smithi100' 2024-04-23T14:34:30.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff03e39d000 1 --2- 172.21.15.100:0/500595654 >> [v2:172.21.15.100:6835/608349550,v1:172.21.15.100:6837/608349550] conn(0x5624dd671570 0x5624dd6c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff03e39d000 4 client.6333 mds_command: new command op to 6180 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:34:30.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff03e39d000 1 -- 172.21.15.100:0/500595654 --> [v2:172.21.15.100:6835/608349550,v1:172.21.15.100:6837/608349550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624dd453bc0 con 0x5624dd671570 2024-04-23T14:34:30.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.691+0000 7ff033e00640 1 --2- 172.21.15.100:0/500595654 >> [v2:172.21.15.100:6835/608349550,v1:172.21.15.100:6837/608349550] conn(0x5624dd671570 0x5624dd6c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.695+0000 7ff033e00640 1 --2- 172.21.15.100:0/500595654 >> [v2:172.21.15.100:6835/608349550,v1:172.21.15.100:6837/608349550] conn(0x5624dd671570 0x5624dd6c7d30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=343a1af354971dc2 server_cookie=88781854e72cdf19 in_seq=0 out_seq=0 2024-04-23T14:34:30.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.695+0000 7ff01d600640 10 client.6333 ms_handle_connect on v2:172.21.15.100:6835/608349550 2024-04-23T14:34:30.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.695+0000 7ff01d600640 1 -- 172.21.15.100:0/500595654 <== mds.0 v2:172.21.15.100:6835/608349550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5624dd453bc0 con 0x5624dd671570 2024-04-23T14:34:30.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.695+0000 7ff01d600640 10 client.6333 handle_command_reply: tid=0 2024-04-23T14:34:30.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.695+0000 7ff03e39d000 1 client.6333 shutdown 2024-04-23T14:34:30.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.695+0000 7ff03e39d000 1 -- 172.21.15.100:0/500595654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd6b65b0 msgr2=0x5624dd6707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.695+0000 7ff03e39d000 1 --2- 172.21.15.100:0/500595654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd6b65b0 0x5624dd6707c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ff02001a040 tx=0x7ff02001ac50 comp rx=0 tx=0).stop 2024-04-23T14:34:30.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.695+0000 7ff03e39d000 1 -- 172.21.15.100:0/500595654 shutdown_connections 2024-04-23T14:34:30.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.695+0000 7ff03e39d000 1 --2- 172.21.15.100:0/500595654 >> [v2:172.21.15.100:6835/608349550,v1:172.21.15.100:6837/608349550] conn(0x5624dd671570 0x5624dd6c7d30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.695+0000 7ff03e39d000 1 --2- 172.21.15.100:0/500595654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd6b6ed0 0x5624dd670d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.695+0000 7ff03e39d000 1 --2- 172.21.15.100:0/500595654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd6b65b0 0x5624dd6707c0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.695+0000 7ff01d600640 0 client.6333 ms_handle_reset on v2:172.21.15.100:6835/608349550 2024-04-23T14:34:30.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.695+0000 7ff03e39d000 1 --2- 172.21.15.100:0/500595654 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5624dd66a140 0x5624dd670280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.695+0000 7ff03e39d000 1 -- 172.21.15.100:0/500595654 >> 172.21.15.100:0/500595654 conn(0x5624dd697350 msgr2=0x5624dd699740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:30.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.695+0000 7ff03e39d000 1 -- 172.21.15.100:0/500595654 shutdown_connections 2024-04-23T14:34:30.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.695+0000 7ff03e39d000 1 -- 172.21.15.100:0/500595654 wait complete. 2024-04-23T14:34:30.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.695+0000 7ff03e39d000 20 client.6333 trim_cache size 0 max 16384 2024-04-23T14:34:30.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.711+0000 7ff03e39d000 1 Processor -- start 2024-04-23T14:34:30.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.711+0000 7ff03e39d000 1 -- start start 2024-04-23T14:34:30.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff03e39d000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd697350 0x5624dd66f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff03e39d000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5624dd66a140 0x5624dd66f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff03e39d000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd6b65b0 0x5624dd6b4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff03e39d000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff034203800 con 0x5624dd6b65b0 2024-04-23T14:34:30.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff03e39d000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0340aef20 con 0x5624dd697350 2024-04-23T14:34:30.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff03e39d000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff03406fe50 con 0x5624dd66a140 2024-04-23T14:34:30.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff038e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd6b65b0 0x5624dd6b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff038e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd6b65b0 0x5624dd6b4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48392/0 (socket says 172.21.15.100:48392) 2024-04-23T14:34:30.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff038e00640 1 -- 172.21.15.100:0/4120790764 learned_addr learned my addr 172.21.15.100:0/4120790764 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:30.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff033e00640 1 --2- 172.21.15.100:0/4120790764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd697350 0x5624dd66f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff033400640 1 --2- 172.21.15.100:0/4120790764 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5624dd66a140 0x5624dd66f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff038e00640 1 -- 172.21.15.100:0/4120790764 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5624dd66a140 msgr2=0x5624dd66f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:30.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff038e00640 1 --2- 172.21.15.100:0/4120790764 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5624dd66a140 0x5624dd66f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff038e00640 1 -- 172.21.15.100:0/4120790764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd697350 msgr2=0x5624dd66f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:30.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff038e00640 1 --2- 172.21.15.100:0/4120790764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd697350 0x5624dd66f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff038e00640 1 -- 172.21.15.100:0/4120790764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624dd513340 con 0x5624dd6b65b0 2024-04-23T14:34:30.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff033e00640 1 --2- 172.21.15.100:0/4120790764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd697350 0x5624dd66f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:30.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff038e00640 1 --2- 172.21.15.100:0/4120790764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd6b65b0 0x5624dd6b4660 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0053a0 tx=0x7ff02c005860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b030dbfb6eb40b1e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:30.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff01fe00640 1 -- 172.21.15.100:0/4120790764 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c001e40 con 0x5624dd6b65b0 2024-04-23T14:34:30.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff01fe00640 1 -- 172.21.15.100:0/4120790764 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff02c01a070 con 0x5624dd6b65b0 2024-04-23T14:34:30.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff01fe00640 1 -- 172.21.15.100:0/4120790764 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c015470 con 0x5624dd6b65b0 2024-04-23T14:34:30.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff03e39d000 1 -- 172.21.15.100:0/4120790764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd6b65b0 msgr2=0x5624dd6b4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff03e39d000 1 --2- 172.21.15.100:0/4120790764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd6b65b0 0x5624dd6b4660 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0053a0 tx=0x7ff02c005860 comp rx=0 tx=0).stop 2024-04-23T14:34:30.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff03e39d000 1 -- 172.21.15.100:0/4120790764 shutdown_connections 2024-04-23T14:34:30.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff03e39d000 1 --2- 172.21.15.100:0/4120790764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd6b65b0 0x5624dd6b4660 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff03e39d000 1 --2- 172.21.15.100:0/4120790764 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5624dd66a140 0x5624dd66f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff03e39d000 1 --2- 172.21.15.100:0/4120790764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd697350 0x5624dd66f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff03e39d000 1 -- 172.21.15.100:0/4120790764 >> 172.21.15.100:0/4120790764 conn(0x5624dd670cc0 msgr2=0x5624dd66ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:30.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff03e39d000 1 -- 172.21.15.100:0/4120790764 shutdown_connections 2024-04-23T14:34:30.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff03e39d000 1 -- 172.21.15.100:0/4120790764 wait complete. 2024-04-23T14:34:30.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff03e39d000 1 -- start start 2024-04-23T14:34:30.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff03e39d000 1 Processor -- start 2024-04-23T14:34:30.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff03e39d000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd66a140 0x5624dd6c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.715+0000 7ff03e39d000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5624dd697350 0x5624dd6c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd6b65b0 0x5624dd6c9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5624dd4beb00 con 0x5624dd66a140 2024-04-23T14:34:30.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5624dd4b97e0 con 0x5624dd6b65b0 2024-04-23T14:34:30.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x5624dd4c76e0 con 0x5624dd697350 2024-04-23T14:34:30.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 10 client.0 _ll_register_callbacks cb 0x7ff0200094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:34:30.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff033e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd66a140 0x5624dd6c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff033e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd66a140 0x5624dd6c8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48402/0 (socket says 172.21.15.100:48402) 2024-04-23T14:34:30.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff033e00640 1 -- 172.21.15.100:0/1203646386 learned_addr learned my addr 172.21.15.100:0/1203646386 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:30.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff033400640 1 --2- 172.21.15.100:0/1203646386 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5624dd697350 0x5624dd6c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff038e00640 1 --2- 172.21.15.100:0/1203646386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd6b65b0 0x5624dd6c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff033e00640 1 -- 172.21.15.100:0/1203646386 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5624dd697350 msgr2=0x5624dd6c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:30.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff033e00640 1 --2- 172.21.15.100:0/1203646386 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5624dd697350 0x5624dd6c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff033e00640 1 -- 172.21.15.100:0/1203646386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd6b65b0 msgr2=0x5624dd6c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:30.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff033e00640 1 --2- 172.21.15.100:0/1203646386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd6b65b0 0x5624dd6c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff033e00640 1 -- 172.21.15.100:0/1203646386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624dd44c7c0 con 0x5624dd66a140 2024-04-23T14:34:30.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff033e00640 1 --2- 172.21.15.100:0/1203646386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd66a140 0x5624dd6c8d80 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff0240569b0 tx=0x7ff024020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3383c5e5d25c9ea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:30.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff01d600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-04-23T14:34:30.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff01d600640 1 -- 172.21.15.100:0/1203646386 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024083040 con 0x5624dd66a140 2024-04-23T14:34:30.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 1 -- 172.21.15.100:0/1203646386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff02c01a070 con 0x5624dd66a140 2024-04-23T14:34:30.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff01d600640 1 -- 172.21.15.100:0/1203646386 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7ff024055000 con 0x5624dd66a140 2024-04-23T14:34:30.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff01d600640 1 -- 172.21.15.100:0/1203646386 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff02407d050 con 0x5624dd66a140 2024-04-23T14:34:30.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 10 client.6339 fetch_fsmap learned FSMap version 28 2024-04-23T14:34:30.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 1 -- 172.21.15.100:0/1203646386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x5624dd45f650 con 0x5624dd66a140 2024-04-23T14:34:30.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff01d600640 1 -- 172.21.15.100:0/1203646386 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff02c01a070 con 0x5624dd66a140 2024-04-23T14:34:30.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff01d600640 1 -- 172.21.15.100:0/1203646386 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) ==== 3494+0+0 (secure 0 0 0) 0x7ff024013070 con 0x5624dd66a140 2024-04-23T14:34:30.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 10 client.6339 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T14:34:30.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 10 client.6339 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-23T14:34:30.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 20 client.6339 populate_metadata read hostname 'smithi100' 2024-04-23T14:34:30.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 1 --2- 172.21.15.100:0/1203646386 >> [v2:172.21.15.100:6835/608349550,v1:172.21.15.100:6837/608349550] conn(0x5624dd6b6ed0 0x5624dd6ca190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 4 client.6339 mds_command: new command op to 6180 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:34:30.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 1 -- 172.21.15.100:0/1203646386 --> [v2:172.21.15.100:6835/608349550,v1:172.21.15.100:6837/608349550] -- command(tid 0: {"prefix": "status"}) -- 0x5624dd48f4e0 con 0x5624dd6b6ed0 2024-04-23T14:34:30.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff033400640 1 --2- 172.21.15.100:0/1203646386 >> [v2:172.21.15.100:6835/608349550,v1:172.21.15.100:6837/608349550] conn(0x5624dd6b6ed0 0x5624dd6ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff033400640 1 --2- 172.21.15.100:0/1203646386 >> [v2:172.21.15.100:6835/608349550,v1:172.21.15.100:6837/608349550] conn(0x5624dd6b6ed0 0x5624dd6ca190 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76aa62e67dde4c18 server_cookie=9c897f3e1bde2fae in_seq=0 out_seq=0 2024-04-23T14:34:30.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff01d600640 10 client.6339 ms_handle_connect on v2:172.21.15.100:6835/608349550 2024-04-23T14:34:30.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff01d600640 1 -- 172.21.15.100:0/1203646386 <== mds.0 v2:172.21.15.100:6835/608349550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5624dd48f4e0 con 0x5624dd6b6ed0 2024-04-23T14:34:30.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff01d600640 10 client.6339 handle_command_reply: tid=0 2024-04-23T14:34:30.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 1 client.6339 shutdown 2024-04-23T14:34:30.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 1 -- 172.21.15.100:0/1203646386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd66a140 msgr2=0x5624dd6c8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 1 --2- 172.21.15.100:0/1203646386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd66a140 0x5624dd6c8d80 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff0240569b0 tx=0x7ff024020070 comp rx=0 tx=0).stop 2024-04-23T14:34:30.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 1 -- 172.21.15.100:0/1203646386 shutdown_connections 2024-04-23T14:34:30.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 1 --2- 172.21.15.100:0/1203646386 >> [v2:172.21.15.100:6835/608349550,v1:172.21.15.100:6837/608349550] conn(0x5624dd6b6ed0 0x5624dd6ca190 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 1 --2- 172.21.15.100:0/1203646386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624dd6b65b0 0x5624dd6c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 1 --2- 172.21.15.100:0/1203646386 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5624dd697350 0x5624dd6c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 1 --2- 172.21.15.100:0/1203646386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624dd66a140 0x5624dd6c8d80 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff01d600640 0 client.6339 ms_handle_reset on v2:172.21.15.100:6835/608349550 2024-04-23T14:34:30.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 1 -- 172.21.15.100:0/1203646386 >> 172.21.15.100:0/1203646386 conn(0x5624dd670cc0 msgr2=0x5624dd66f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:30.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 1 -- 172.21.15.100:0/1203646386 shutdown_connections 2024-04-23T14:34:30.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 1 -- 172.21.15.100:0/1203646386 wait complete. 2024-04-23T14:34:30.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03e39d000 20 client.6339 trim_cache size 0 max 16384 2024-04-23T14:34:30.726 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:34:30.727 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "167d2af1-01c4-4f87-8bce-0de22791f823", 2024-04-23T14:34:30.727 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-04-23T14:34:30.727 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 6180, 2024-04-23T14:34:30.727 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-04-23T14:34:30.727 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-04-23T14:34:30.727 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-04-23T14:34:30.727 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 5.9416761710000001, 2024-04-23T14:34:30.727 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 28, 2024-04-23T14:34:30.727 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 53, 2024-04-23T14:34:30.727 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 53, 2024-04-23T14:34:30.727 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 10.846130235 2024-04-23T14:34:30.727 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:34:30.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03ae00640 1 -- 172.21.15.100:0/1922307949 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff014068380 msgr2=0x7ff01406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03ae00640 1 --2- 172.21.15.100:0/1922307949 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff014068380 0x7ff01406a840 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff0200af000 tx=0x7ff0200a7000 comp rx=0 tx=0).stop 2024-04-23T14:34:30.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03ae00640 1 -- 172.21.15.100:0/1922307949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0340c6310 msgr2=0x7ff034161b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.719+0000 7ff03ae00640 1 --2- 172.21.15.100:0/1922307949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0340c6310 0x7ff034161b60 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff02400d800 tx=0x7ff02400dcc0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.723+0000 7ff03ae00640 1 -- 172.21.15.100:0/1922307949 shutdown_connections 2024-04-23T14:34:30.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.723+0000 7ff03ae00640 1 --2- 172.21.15.100:0/1922307949 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff014068380 0x7ff01406a840 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.723+0000 7ff03ae00640 1 --2- 172.21.15.100:0/1922307949 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff03415eb30 0x7ff0340af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.723+0000 7ff03ae00640 1 --2- 172.21.15.100:0/1922307949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff03415e750 0x7ff03414c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.723+0000 7ff03ae00640 1 --2- 172.21.15.100:0/1922307949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0340c6310 0x7ff034161b60 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.723+0000 7ff03ae00640 1 -- 172.21.15.100:0/1922307949 >> 172.21.15.100:0/1922307949 conn(0x7ff0340b5310 msgr2=0x7ff03415d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:30.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.723+0000 7ff03ae00640 1 -- 172.21.15.100:0/1922307949 shutdown_connections 2024-04-23T14:34:30.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.723+0000 7ff03ae00640 1 -- 172.21.15.100:0/1922307949 wait complete. 2024-04-23T14:34:30.741 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:34:30.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.875+0000 7f9357a00640 1 Processor -- start 2024-04-23T14:34:30.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.875+0000 7f9357a00640 1 -- start start 2024-04-23T14:34:30.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.879+0000 7f9357a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f935015c470 0x7f93500b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.879+0000 7f9357a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f93500ad870 0x7f935006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.879+0000 7f9357a00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9350160320 con 0x7f9350160de0 2024-04-23T14:34:30.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.879+0000 7f9357a00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f93500cc7e0 con 0x7f93500ad870 2024-04-23T14:34:30.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.879+0000 7f9357a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f93501605e0 con 0x7f935015c470 2024-04-23T14:34:30.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.879+0000 7f9355a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f93500ad870 0x7f935006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.879+0000 7f934fe00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f9350160de0 0x7f93500ad160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:39834/0 (socket says 172.21.15.100:39834) 2024-04-23T14:34:30.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.879+0000 7f934fe00640 1 -- 172.21.15.100:0/589723046 learned_addr learned my addr 172.21.15.100:0/589723046 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:30.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.879+0000 7f9355a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f93500ad870 0x7f935006bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48410/0 (socket says 172.21.15.100:48410) 2024-04-23T14:34:30.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.879+0000 7f9355000640 1 --2- 172.21.15.100:0/589723046 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f935015c470 0x7f93500b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.879+0000 7f9355a00640 1 -- 172.21.15.100:0/589723046 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f935015c470 msgr2=0x7f93500b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:30.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.879+0000 7f9355a00640 1 --2- 172.21.15.100:0/589723046 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f935015c470 0x7f93500b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.879+0000 7f9355a00640 1 -- 172.21.15.100:0/589723046 >> v1:172.21.15.100:6789/0 conn(0x7f9350160de0 legacy=0x7f93500ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.879+0000 7f9355a00640 1 -- 172.21.15.100:0/589723046 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93501608c0 con 0x7f93500ad870 2024-04-23T14:34:30.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.879+0000 7f9355000640 1 --2- 172.21.15.100:0/589723046 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f935015c470 0x7f93500b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:30.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.879+0000 7f9355a00640 1 --2- 172.21.15.100:0/589723046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93500ad870 0x7f935006bab0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9348009d90 tx=0x7f934801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a21b3728ca52f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:30.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f934f400640 1 -- 172.21.15.100:0/589723046 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93480b2070 con 0x7f93500ad870 2024-04-23T14:34:30.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f934f400640 1 -- 172.21.15.100:0/589723046 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93480a58b0 con 0x7f93500ad870 2024-04-23T14:34:30.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f934f400640 1 -- 172.21.15.100:0/589723046 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93480a5c80 con 0x7f93500ad870 2024-04-23T14:34:30.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f9357a00640 1 -- 172.21.15.100:0/589723046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93500ad870 msgr2=0x7f935006bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f9357a00640 1 --2- 172.21.15.100:0/589723046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93500ad870 0x7f935006bab0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9348009d90 tx=0x7f934801c8e0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f9357a00640 1 -- 172.21.15.100:0/589723046 shutdown_connections 2024-04-23T14:34:30.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f9357a00640 1 --2- 172.21.15.100:0/589723046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93500ad870 0x7f935006bab0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f9357a00640 1 --2- 172.21.15.100:0/589723046 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f935015c470 0x7f93500b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f9357a00640 1 -- 172.21.15.100:0/589723046 >> 172.21.15.100:0/589723046 conn(0x7f93500b1e80 msgr2=0x7f93500b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:30.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f9357a00640 1 -- 172.21.15.100:0/589723046 shutdown_connections 2024-04-23T14:34:30.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f9357a00640 1 -- 172.21.15.100:0/589723046 wait complete. 2024-04-23T14:34:30.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f9357a00640 1 Processor -- start 2024-04-23T14:34:30.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f9357a00640 1 -- start start 2024-04-23T14:34:30.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f9357a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f93500ad870 0x7f935015b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f9357a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935015c470 0x7f9350099190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f9357a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9350160de0 0x7f935015cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f9357a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f935015ca60 con 0x7f935015c470 2024-04-23T14:34:30.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f9357a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f93500cc580 con 0x7f9350160de0 2024-04-23T14:34:30.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f9357a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9350169540 con 0x7f93500ad870 2024-04-23T14:34:30.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f934fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935015c470 0x7f9350099190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f934fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935015c470 0x7f9350099190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48422/0 (socket says 172.21.15.100:48422) 2024-04-23T14:34:30.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f934fe00640 1 -- 172.21.15.100:0/1385200986 learned_addr learned my addr 172.21.15.100:0/1385200986 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:30.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f934fe00640 1 -- 172.21.15.100:0/1385200986 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f93500ad870 msgr2=0x7f935015b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f934fe00640 1 --2- 172.21.15.100:0/1385200986 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f93500ad870 0x7f935015b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f934fe00640 1 -- 172.21.15.100:0/1385200986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9350160de0 msgr2=0x7f935015cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f934fe00640 1 --2- 172.21.15.100:0/1385200986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9350160de0 0x7f935015cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.883+0000 7f934fe00640 1 -- 172.21.15.100:0/1385200986 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935015ee60 con 0x7f935015c470 2024-04-23T14:34:30.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.887+0000 7f934fe00640 1 --2- 172.21.15.100:0/1385200986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935015c470 0x7f9350099190 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f933c00c370 tx=0x7f933c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:30.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.887+0000 7f934cc00640 1 -- 172.21.15.100:0/1385200986 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c017070 con 0x7f935015c470 2024-04-23T14:34:30.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.887+0000 7f934cc00640 1 -- 172.21.15.100:0/1385200986 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f933c005cf0 con 0x7f935015c470 2024-04-23T14:34:30.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.887+0000 7f934cc00640 1 -- 172.21.15.100:0/1385200986 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c0072f0 con 0x7f935015c470 2024-04-23T14:34:30.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.887+0000 7f9357a00640 1 -- 172.21.15.100:0/1385200986 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f935015f140 con 0x7f935015c470 2024-04-23T14:34:30.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.887+0000 7f9357a00640 1 -- 172.21.15.100:0/1385200986 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f935015f700 con 0x7f935015c470 2024-04-23T14:34:30.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.887+0000 7f9357a00640 1 -- 172.21.15.100:0/1385200986 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9350068ad0 con 0x7f935015c470 2024-04-23T14:34:30.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.887+0000 7f934cc00640 1 -- 172.21.15.100:0/1385200986 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f933c01a070 con 0x7f935015c470 2024-04-23T14:34:30.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.887+0000 7f934cc00640 1 --2- 172.21.15.100:0/1385200986 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9330068380 0x7f933006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.887+0000 7f934cc00640 1 -- 172.21.15.100:0/1385200986 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f933c085e90 con 0x7f935015c470 2024-04-23T14:34:30.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.887+0000 7f9355000640 1 --2- 172.21.15.100:0/1385200986 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9330068380 0x7f933006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.891+0000 7f9355000640 1 --2- 172.21.15.100:0/1385200986 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9330068380 0x7f933006a840 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f934400a8d0 tx=0x7f9344008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:30.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:30.891+0000 7f934cc00640 1 -- 172.21.15.100:0/1385200986 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f933c056220 con 0x7f935015c470 2024-04-23T14:34:31.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:31.083+0000 7f9357a00640 1 -- 172.21.15.100:0/1385200986 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f93500b5460 con 0x7f935015c470 2024-04-23T14:34:31.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:31.083+0000 7f934cc00640 1 -- 172.21.15.100:0/1385200986 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+73 (secure 0 0 0) 0x7f933c087070 con 0x7f935015c470 2024-04-23T14:34:31.083 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:34:31.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:31.083+0000 7f9357a00640 1 -- 172.21.15.100:0/1385200986 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9330068380 msgr2=0x7f933006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:31.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:31.083+0000 7f9357a00640 1 --2- 172.21.15.100:0/1385200986 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9330068380 0x7f933006a840 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f934400a8d0 tx=0x7f9344008040 comp rx=0 tx=0).stop 2024-04-23T14:34:31.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:31.083+0000 7f9357a00640 1 -- 172.21.15.100:0/1385200986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935015c470 msgr2=0x7f9350099190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:31.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:31.083+0000 7f9357a00640 1 --2- 172.21.15.100:0/1385200986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935015c470 0x7f9350099190 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f933c00c370 tx=0x7f933c00c830 comp rx=0 tx=0).stop 2024-04-23T14:34:31.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:31.083+0000 7f9357a00640 1 -- 172.21.15.100:0/1385200986 shutdown_connections 2024-04-23T14:34:31.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:31.083+0000 7f9357a00640 1 --2- 172.21.15.100:0/1385200986 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9330068380 0x7f933006a840 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:31.083+0000 7f9357a00640 1 --2- 172.21.15.100:0/1385200986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9350160de0 0x7f935015cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:31.083+0000 7f9357a00640 1 --2- 172.21.15.100:0/1385200986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935015c470 0x7f9350099190 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:31.083+0000 7f9357a00640 1 --2- 172.21.15.100:0/1385200986 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f93500ad870 0x7f935015b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:31.083+0000 7f9357a00640 1 -- 172.21.15.100:0/1385200986 >> 172.21.15.100:0/1385200986 conn(0x7f93500b1e80 msgr2=0x7f935006dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:31.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:31.083+0000 7f9357a00640 1 -- 172.21.15.100:0/1385200986 shutdown_connections 2024-04-23T14:34:31.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:31.083+0000 7f9357a00640 1 -- 172.21.15.100:0/1385200986 wait complete. 2024-04-23T14:34:31.107 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:34:31.108 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:34:31.108 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-04-23T14:34:31.108 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:34:31.108 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:31.108 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:34:31.108 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:34:31.108 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:34:31.108 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:34:31.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:31.108 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-04-23T14:34:31.117 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:34:31.118 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:34:31.118 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:34:31.118 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:31.118 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-23T14:34:31.118 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:31.118 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:34:31.118 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:31.118 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-04-23T14:34:31.118 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:34:31.118 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:31.118 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-04-23T14:34:31.118 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:34:31.118 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:31.119 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:34:31.119 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:31.119 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-23T14:34:31.119 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42313sec preferred_lft 42313sec 2024-04-23T14:34:31.119 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-04-23T14:34:31.119 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:31.119 INFO:teuthology.orchestra.run.smithi100.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-23T14:34:31.119 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 02:42:ff:fc:d1:cc brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:31.119 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-23T14:34:31.119 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:31.119 INFO:teuthology.orchestra.run.smithi100.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:34:31.119 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:31.119 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:34:31.119 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:31.120 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-23T14:34:31.120 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:31.120 INFO:teuthology.orchestra.run.smithi100.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:34:31.120 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:34:31.120 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-23T14:34:31.120 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:31.120 INFO:teuthology.orchestra.run.smithi100.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:34:31.120 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:34:31.120 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-23T14:34:31.120 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:31.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:31.120 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:34:31.176 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-04-23T14:34:31.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:31.178 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-04-23T14:34:31.229 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-04-23T14:34:31.229 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:34:31.229 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-23T14:34:31.229 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-23T14:34:31.229 INFO:teuthology.orchestra.run.smithi100.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-23T14:34:31.229 INFO:teuthology.orchestra.run.smithi100.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-23T14:34:31.229 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-23T14:34:31.230 INFO:teuthology.orchestra.run.smithi100.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:34:31.230 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:34:31.230 INFO:teuthology.orchestra.run.smithi100.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:34:31.230 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:34:31.230 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:34:31.230 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-23T14:34:31.230 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-23T14:34:31.230 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-23T14:34:31.230 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-23T14:34:31.230 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-23T14:34:31.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:34:31.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:34:31.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:31.363 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-23T14:34:31.372 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:34:31.372 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:34:31.373 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:34:31.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:31.373 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-04-23T14:34:31.433 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:34:31.434 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:34:31.434 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:31.482 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:34:31.482 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:34:31.482 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:31.530 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-23T14:34:31.591 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:34:31.643 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-23T14:34:31.644 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-23T14:34:31.644 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-23T14:34:31.644 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-23T14:34:31.644 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-23T14:34:31.644 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-23T14:34:31.644 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-23T14:34:31.644 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-23T14:34:31.644 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-23T14:34:31.644 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-23T14:34:31.644 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-23T14:34:31.644 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-23T14:34:31.644 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:34:31.644 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-23T14:34:31.644 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-23T14:34:31.645 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-23T14:34:31.645 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-23T14:34:31.645 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-23T14:34:31.645 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-23T14:34:31.645 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-23T14:34:31.645 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:34:31.645 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-23T14:34:31.646 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-23T14:34:31.646 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-23T14:34:31.646 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:34:31.646 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:34:31.646 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:34:31.646 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:34:31.646 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-23T14:34:31.646 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:34:31.646 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:34:31.646 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:34:31.646 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:34:31.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:31.647 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-23T14:34:31.698 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:34:31.698 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:34:31.741 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-23T14:34:31.761 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:34:31.812 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-23T14:34:31.812 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-23T14:34:31.812 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-23T14:34:31.812 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-23T14:34:31.812 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-23T14:34:31.813 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-23T14:34:31.813 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-23T14:34:31.813 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-23T14:34:31.813 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-23T14:34:31.813 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-23T14:34:31.813 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-23T14:34:31.813 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-23T14:34:31.813 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:34:31.813 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-23T14:34:31.813 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-23T14:34:31.813 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-23T14:34:31.813 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-23T14:34:31.813 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-23T14:34:31.813 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-23T14:34:31.813 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-23T14:34:31.813 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:34:31.814 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-23T14:34:31.814 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-23T14:34:31.814 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-23T14:34:31.814 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:34:31.814 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:34:31.814 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:34:31.814 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:34:31.814 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-23T14:34:31.814 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:34:31.814 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:34:31.814 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:34:31.814 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:34:31.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:31.815 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-23T14:34:31.837 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:2024-04-23T14:34:31.835+0000 7f6bc4e89580 -1 init, newargv = 0x55c513ec0950 newargc=15 2024-04-23T14:34:31.837 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[41994]: starting ceph client 2024-04-23T14:34:31.845 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[41994]: starting fuse 2024-04-23T14:34:32.947 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-23T14:34:32.970 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:34:33.020 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-23T14:34:33.020 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-23T14:34:33.020 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-23T14:34:33.020 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-23T14:34:33.020 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-23T14:34:33.020 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-23T14:34:33.021 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-23T14:34:33.021 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-23T14:34:33.021 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-23T14:34:33.021 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-23T14:34:33.021 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-23T14:34:33.021 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-23T14:34:33.021 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:34:33.021 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-23T14:34:33.021 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-23T14:34:33.021 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-23T14:34:33.021 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-23T14:34:33.021 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-23T14:34:33.021 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-23T14:34:33.022 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-23T14:34:33.022 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:34:33.022 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-23T14:34:33.022 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-23T14:34:33.022 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-23T14:34:33.022 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:34:33.022 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:34:33.022 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:34:33.022 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:34:33.022 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-23T14:34:33.022 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:34:33.022 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:34:33.022 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:34:33.023 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:34:33.023 INFO:teuthology.orchestra.run.smithi100.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:33.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:33.024 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-23T14:34:33.074 INFO:teuthology.orchestra.run.smithi100.stdout:44 2024-04-23T14:34:33.075 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-23T14:34:33.075 DEBUG:teuthology.orchestra.run.smithi100:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:34:33.075 DEBUG:teuthology.orchestra.run.smithi100:> import glob 2024-04-23T14:34:33.075 DEBUG:teuthology.orchestra.run.smithi100:> import re 2024-04-23T14:34:33.075 DEBUG:teuthology.orchestra.run.smithi100:> import os 2024-04-23T14:34:33.075 DEBUG:teuthology.orchestra.run.smithi100:> import subprocess 2024-04-23T14:34:33.075 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:34:33.075 DEBUG:teuthology.orchestra.run.smithi100:> def _find_admin_socket(client_name): 2024-04-23T14:34:33.075 DEBUG:teuthology.orchestra.run.smithi100:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:34:33.075 DEBUG:teuthology.orchestra.run.smithi100:> files = glob.glob(asok_path) 2024-04-23T14:34:33.075 DEBUG:teuthology.orchestra.run.smithi100:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:34:33.075 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:34:33.075 DEBUG:teuthology.orchestra.run.smithi100:> # Given a non-glob path, it better be there 2024-04-23T14:34:33.076 DEBUG:teuthology.orchestra.run.smithi100:> if "*" not in asok_path: 2024-04-23T14:34:33.076 DEBUG:teuthology.orchestra.run.smithi100:> assert(len(files) == 1) 2024-04-23T14:34:33.076 DEBUG:teuthology.orchestra.run.smithi100:> return files[0] 2024-04-23T14:34:33.076 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:34:33.076 DEBUG:teuthology.orchestra.run.smithi100:> for f in files: 2024-04-23T14:34:33.076 DEBUG:teuthology.orchestra.run.smithi100:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:34:33.076 DEBUG:teuthology.orchestra.run.smithi100:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:34:33.076 DEBUG:teuthology.orchestra.run.smithi100:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:34:33.076 DEBUG:teuthology.orchestra.run.smithi100:> contents = proc_f.read() 2024-04-23T14:34:33.076 DEBUG:teuthology.orchestra.run.smithi100:> if mountpoint in contents: 2024-04-23T14:34:33.076 DEBUG:teuthology.orchestra.run.smithi100:> return f 2024-04-23T14:34:33.076 DEBUG:teuthology.orchestra.run.smithi100:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:34:33.076 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:34:33.076 DEBUG:teuthology.orchestra.run.smithi100:> print(_find_admin_socket("client.0")) 2024-04-23T14:34:33.076 DEBUG:teuthology.orchestra.run.smithi100:> ' 2024-04-23T14:34:33.208 INFO:teuthology.orchestra.run.smithi100.stdout:/var/run/ceph/ceph-client.0.41994.asok 2024-04-23T14:34:33.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:34:33.221 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41994.asok 2024-04-23T14:34:33.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:33.221 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41994.asok status 2024-04-23T14:34:33.389 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:34:33.390 INFO:teuthology.orchestra.run.smithi100.stdout: "metadata": { 2024-04-23T14:34:33.390 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:34:33.390 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:34:33.390 INFO:teuthology.orchestra.run.smithi100.stdout: "entity_id": "0", 2024-04-23T14:34:33.390 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": "smithi100", 2024-04-23T14:34:33.390 INFO:teuthology.orchestra.run.smithi100.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:34:33.390 INFO:teuthology.orchestra.run.smithi100.stdout: "pid": "41994", 2024-04-23T14:34:33.390 INFO:teuthology.orchestra.run.smithi100.stdout: "root": "/" 2024-04-23T14:34:33.390 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:33.390 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_count": 0, 2024-04-23T14:34:33.390 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_pinned_count": 0, 2024-04-23T14:34:33.390 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 6354, 2024-04-23T14:34:33.391 INFO:teuthology.orchestra.run.smithi100.stdout: "inst": { 2024-04-23T14:34:33.391 INFO:teuthology.orchestra.run.smithi100.stdout: "name": { 2024-04-23T14:34:33.391 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "client", 2024-04-23T14:34:33.391 INFO:teuthology.orchestra.run.smithi100.stdout: "num": 6354 2024-04-23T14:34:33.391 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:33.391 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-23T14:34:33.391 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-23T14:34:33.391 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:34:33.391 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 119076797 2024-04-23T14:34:33.391 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:33.391 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:33.391 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-23T14:34:33.391 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-23T14:34:33.391 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:34:33.391 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 119076797 2024-04-23T14:34:33.392 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:33.392 INFO:teuthology.orchestra.run.smithi100.stdout: "inst_str": "client.6354 192.168.0.1:0/119076797", 2024-04-23T14:34:33.392 INFO:teuthology.orchestra.run.smithi100.stdout: "addr_str": "192.168.0.1:0/119076797", 2024-04-23T14:34:33.392 INFO:teuthology.orchestra.run.smithi100.stdout: "inode_count": 1, 2024-04-23T14:34:33.392 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_epoch": 28, 2024-04-23T14:34:33.392 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch": 53, 2024-04-23T14:34:33.392 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:34:33.392 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklisted": false, 2024-04-23T14:34:33.392 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs" 2024-04-23T14:34:33.392 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:34:33.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:33.404 DEBUG:teuthology.orchestra.run.smithi100:> stat --file-system '--printf=%T 2024-04-23T14:34:33.405 DEBUG:teuthology.orchestra.run.smithi100:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:33.413 INFO:teuthology.orchestra.run.smithi100.stdout:fuseblk 2024-04-23T14:34:33.413 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:33.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:33.414 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:33.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:33.473 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch test_file) 2024-04-23T14:34:33.529 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow r pool=cephfs_data' 2024-04-23T14:34:33.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.711+0000 7fdb58000640 1 Processor -- start 2024-04-23T14:34:33.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.711+0000 7fdb58000640 1 -- start start 2024-04-23T14:34:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.711+0000 7fdb58000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdb5006d450 0x7fdb5006d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.711+0000 7fdb58000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb5006dd70 con 0x7fdb500cea60 2024-04-23T14:34:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.711+0000 7fdb58000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb500b1f90 con 0x7fdb500cd460 2024-04-23T14:34:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.711+0000 7fdb58000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fdb500b2270 con 0x7fdb5006d450 2024-04-23T14:34:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.711+0000 7fdb55600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdb5006d450 0x7fdb5006d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.711+0000 7fdb54c00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fdb500cea60 0x7fdb500d0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:39848/0 (socket says 172.21.15.100:39848) 2024-04-23T14:34:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.711+0000 7fdb54c00640 1 -- 172.21.15.100:0/287863888 learned_addr learned my addr 172.21.15.100:0/287863888 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:33.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.711+0000 7fdb55600640 1 -- 172.21.15.100:0/287863888 >> v1:172.21.15.151:6789/0 conn(0x7fdb500cd460 legacy=0x7fdb500b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:33.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.711+0000 7fdb55600640 1 -- 172.21.15.100:0/287863888 >> v1:172.21.15.100:6789/0 conn(0x7fdb500cea60 legacy=0x7fdb500d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:33.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.711+0000 7fdb55600640 1 -- 172.21.15.100:0/287863888 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb500b2550 con 0x7fdb5006d450 2024-04-23T14:34:33.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.711+0000 7fdb4fe00640 1 -- 172.21.15.100:0/287863888 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3050868511 0 0) 0x7fdb5006dd70 con 0x7fdb500cea60 2024-04-23T14:34:33.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.715+0000 7fdb55600640 1 --2- 172.21.15.100:0/287863888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb5006d450 0x7fdb5006d830 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fdb440097c0 tx=0x7fdb4401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da50f1ec873fe7e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:33.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.715+0000 7fdb4fe00640 1 -- 172.21.15.100:0/287863888 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb440b0070 con 0x7fdb5006d450 2024-04-23T14:34:33.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.715+0000 7fdb4fe00640 1 -- 172.21.15.100:0/287863888 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb440ab600 con 0x7fdb5006d450 2024-04-23T14:34:33.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.715+0000 7fdb4fe00640 1 -- 172.21.15.100:0/287863888 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb440ab9d0 con 0x7fdb5006d450 2024-04-23T14:34:33.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.715+0000 7fdb58000640 1 -- 172.21.15.100:0/287863888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb5006d450 msgr2=0x7fdb5006d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:33.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.715+0000 7fdb58000640 1 --2- 172.21.15.100:0/287863888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb5006d450 0x7fdb5006d830 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fdb440097c0 tx=0x7fdb4401c410 comp rx=0 tx=0).stop 2024-04-23T14:34:33.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.715+0000 7fdb58000640 1 -- 172.21.15.100:0/287863888 shutdown_connections 2024-04-23T14:34:33.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.715+0000 7fdb58000640 1 --2- 172.21.15.100:0/287863888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb5006d450 0x7fdb5006d830 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:33.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.715+0000 7fdb58000640 1 -- 172.21.15.100:0/287863888 >> 172.21.15.100:0/287863888 conn(0x7fdb500cee80 msgr2=0x7fdb500cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:33.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.715+0000 7fdb58000640 1 -- 172.21.15.100:0/287863888 shutdown_connections 2024-04-23T14:34:33.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.715+0000 7fdb58000640 1 -- 172.21.15.100:0/287863888 wait complete. 2024-04-23T14:34:33.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb58000640 1 Processor -- start 2024-04-23T14:34:33.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb58000640 1 -- start start 2024-04-23T14:34:33.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb58000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb5006d450 0x7fdb500c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:33.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb58000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb500cd460 0x7fdb5015d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:33.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb58000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb500cea60 0x7fdb50158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:33.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb58000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5016cd50 con 0x7fdb500cd460 2024-04-23T14:34:33.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb58000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5016cbd0 con 0x7fdb5006d450 2024-04-23T14:34:33.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb58000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fdb5016ced0 con 0x7fdb500cea60 2024-04-23T14:34:33.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb54c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb500cd460 0x7fdb5015d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:33.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb54c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb500cd460 0x7fdb5015d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48442/0 (socket says 172.21.15.100:48442) 2024-04-23T14:34:33.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb54c00640 1 -- 172.21.15.100:0/3924996360 learned_addr learned my addr 172.21.15.100:0/3924996360 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:33.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb55600640 1 --2- 172.21.15.100:0/3924996360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb5006d450 0x7fdb500c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:33.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb56000640 1 --2- 172.21.15.100:0/3924996360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb500cea60 0x7fdb50158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:33.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb54c00640 1 -- 172.21.15.100:0/3924996360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb500cea60 msgr2=0x7fdb50158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:33.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb54c00640 1 --2- 172.21.15.100:0/3924996360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb500cea60 0x7fdb50158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:33.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb54c00640 1 -- 172.21.15.100:0/3924996360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb5006d450 msgr2=0x7fdb500c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:33.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb54c00640 1 --2- 172.21.15.100:0/3924996360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb5006d450 0x7fdb500c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:33.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb54c00640 1 -- 172.21.15.100:0/3924996360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb50157580 con 0x7fdb500cd460 2024-04-23T14:34:33.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb56000640 1 --2- 172.21.15.100:0/3924996360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb500cea60 0x7fdb50158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:33.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb54c00640 1 --2- 172.21.15.100:0/3924996360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb500cd460 0x7fdb5015d6d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00c480 tx=0x7fdb3c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:33.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb4d600640 1 -- 172.21.15.100:0/3924996360 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c018070 con 0x7fdb500cd460 2024-04-23T14:34:33.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb4d600640 1 -- 172.21.15.100:0/3924996360 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb3c0074e0 con 0x7fdb500cd460 2024-04-23T14:34:33.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb4d600640 1 -- 172.21.15.100:0/3924996360 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c004a30 con 0x7fdb500cd460 2024-04-23T14:34:33.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb58000640 1 -- 172.21.15.100:0/3924996360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb50157860 con 0x7fdb500cd460 2024-04-23T14:34:33.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb58000640 1 -- 172.21.15.100:0/3924996360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb50157e20 con 0x7fdb500cd460 2024-04-23T14:34:33.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.719+0000 7fdb58000640 1 -- 172.21.15.100:0/3924996360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb50068ad0 con 0x7fdb500cd460 2024-04-23T14:34:33.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.723+0000 7fdb4d600640 1 -- 172.21.15.100:0/3924996360 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdb3c004bd0 con 0x7fdb500cd460 2024-04-23T14:34:33.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.723+0000 7fdb4d600640 1 --2- 172.21.15.100:0/3924996360 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb30068380 0x7fdb3006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:33.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.723+0000 7fdb4d600640 1 -- 172.21.15.100:0/3924996360 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fdb3c085f60 con 0x7fdb500cd460 2024-04-23T14:34:33.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.723+0000 7fdb55600640 1 --2- 172.21.15.100:0/3924996360 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb30068380 0x7fdb3006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:33.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.723+0000 7fdb55600640 1 --2- 172.21.15.100:0/3924996360 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb30068380 0x7fdb3006a840 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdb440ad000 tx=0x7fdb44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:33.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.727+0000 7fdb4d600640 1 -- 172.21.15.100:0/3924996360 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdb3c056380 con 0x7fdb500cd460 2024-04-23T14:34:33.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.943+0000 7fdb58000640 1 -- 172.21.15.100:0/3924996360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]} v 0) -- 0x7fdb500b2df0 con 0x7fdb500cd460 2024-04-23T14:34:33.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.943+0000 7fdb4d600640 1 -- 172.21.15.100:0/3924996360 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]}]=0 updated caps for client.0 v16) ==== 183+0+140 (secure 0 0 0) 0x7fdb3c055c80 con 0x7fdb500cd460 2024-04-23T14:34:33.946 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-04-23T14:34:33.946 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAtxidmyhImMBAAa49ApQEF2Lng6rY94F1HAQ== 2024-04-23T14:34:33.946 INFO:teuthology.orchestra.run.smithi100.stdout: caps mds = "allow" 2024-04-23T14:34:33.946 INFO:teuthology.orchestra.run.smithi100.stdout: caps mon = "allow r" 2024-04-23T14:34:33.946 INFO:teuthology.orchestra.run.smithi100.stdout: caps osd = "allow r pool=cephfs_data" 2024-04-23T14:34:33.946 INFO:teuthology.orchestra.run.smithi100.stderr:updated caps for client.0 2024-04-23T14:34:33.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.947+0000 7fdb58000640 1 -- 172.21.15.100:0/3924996360 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb30068380 msgr2=0x7fdb3006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:33.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.947+0000 7fdb58000640 1 --2- 172.21.15.100:0/3924996360 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb30068380 0x7fdb3006a840 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdb440ad000 tx=0x7fdb44008040 comp rx=0 tx=0).stop 2024-04-23T14:34:33.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.947+0000 7fdb58000640 1 -- 172.21.15.100:0/3924996360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb500cd460 msgr2=0x7fdb5015d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:33.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.947+0000 7fdb58000640 1 --2- 172.21.15.100:0/3924996360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb500cd460 0x7fdb5015d6d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00c480 tx=0x7fdb3c00c940 comp rx=0 tx=0).stop 2024-04-23T14:34:33.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.947+0000 7fdb58000640 1 -- 172.21.15.100:0/3924996360 shutdown_connections 2024-04-23T14:34:33.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.947+0000 7fdb58000640 1 --2- 172.21.15.100:0/3924996360 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdb30068380 0x7fdb3006a840 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:33.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.947+0000 7fdb58000640 1 --2- 172.21.15.100:0/3924996360 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb500cea60 0x7fdb50158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:33.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.947+0000 7fdb58000640 1 --2- 172.21.15.100:0/3924996360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb500cd460 0x7fdb5015d6d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:33.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.947+0000 7fdb58000640 1 --2- 172.21.15.100:0/3924996360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb5006d450 0x7fdb500c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:33.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.947+0000 7fdb58000640 1 -- 172.21.15.100:0/3924996360 >> 172.21.15.100:0/3924996360 conn(0x7fdb500cee80 msgr2=0x7fdb50161630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:33.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.947+0000 7fdb58000640 1 -- 172.21.15.100:0/3924996360 shutdown_connections 2024-04-23T14:34:33.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:33.947+0000 7fdb58000640 1 -- 172.21.15.100:0/3924996360 wait complete. 2024-04-23T14:34:33.968 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:34:33.968 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:33.977 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:34.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.151+0000 7fcb59a00640 1 Processor -- start 2024-04-23T14:34:34.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.151+0000 7fcb59a00640 1 -- start start 2024-04-23T14:34:34.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.151+0000 7fcb59a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcb5415c470 0x7fcb540b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.151+0000 7fcb59a00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fcb540ad870 0x7fcb5406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.151+0000 7fcb59a00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb54160320 con 0x7fcb54160de0 2024-04-23T14:34:34.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.151+0000 7fcb59a00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fcb540cc7e0 con 0x7fcb5415c470 2024-04-23T14:34:34.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.151+0000 7fcb59a00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fcb541605e0 con 0x7fcb540ad870 2024-04-23T14:34:34.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.151+0000 7fcb4a000640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fcb54160de0 0x7fcb540ad160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:39860/0 (socket says 172.21.15.100:39860) 2024-04-23T14:34:34.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.151+0000 7fcb4a000640 1 -- 172.21.15.100:0/2512825691 learned_addr learned my addr 172.21.15.100:0/2512825691 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:34.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.151+0000 7fcb52a00640 1 --2- 172.21.15.100:0/2512825691 >> v2:172.21.15.100:3300/0 conn(0x7fcb5415c470 0x7fcb540b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.151+0000 7fcb53400640 1 --2- 172.21.15.100:0/2512825691 >> v2:172.21.15.151:3300/0 conn(0x7fcb540ad870 0x7fcb5406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.151+0000 7fcb52a00640 1 -- 172.21.15.100:0/2512825691 >> v2:172.21.15.151:3300/0 conn(0x7fcb540ad870 msgr2=0x7fcb5406bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:34.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.151+0000 7fcb52a00640 1 --2- 172.21.15.100:0/2512825691 >> v2:172.21.15.151:3300/0 conn(0x7fcb540ad870 0x7fcb5406bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.151+0000 7fcb52a00640 1 -- 172.21.15.100:0/2512825691 >> v1:172.21.15.100:6789/0 conn(0x7fcb54160de0 legacy=0x7fcb540ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.151+0000 7fcb52a00640 1 -- 172.21.15.100:0/2512825691 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb541608c0 con 0x7fcb5415c470 2024-04-23T14:34:34.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.151+0000 7fcb52000640 1 -- 172.21.15.100:0/2512825691 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3213406633 0 0) 0x7fcb54160320 con 0x7fcb54160de0 2024-04-23T14:34:34.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.155+0000 7fcb52a00640 1 --2- 172.21.15.100:0/2512825691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb5415c470 0x7fcb540b29a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fcb5406d5c0 tx=0x7fcb4401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d63c72a0dc3b82e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:34.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.155+0000 7fcb52000640 1 -- 172.21.15.100:0/2512825691 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb440b0070 con 0x7fcb5415c470 2024-04-23T14:34:34.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.155+0000 7fcb52000640 1 -- 172.21.15.100:0/2512825691 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb440ab600 con 0x7fcb5415c470 2024-04-23T14:34:34.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.155+0000 7fcb52000640 1 -- 172.21.15.100:0/2512825691 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb440ab9d0 con 0x7fcb5415c470 2024-04-23T14:34:34.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.155+0000 7fcb59a00640 1 -- 172.21.15.100:0/2512825691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb5415c470 msgr2=0x7fcb540b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.155+0000 7fcb59a00640 1 --2- 172.21.15.100:0/2512825691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb5415c470 0x7fcb540b29a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fcb5406d5c0 tx=0x7fcb4401c3f0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.155+0000 7fcb59a00640 1 -- 172.21.15.100:0/2512825691 shutdown_connections 2024-04-23T14:34:34.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.155+0000 7fcb59a00640 1 --2- 172.21.15.100:0/2512825691 >> v2:172.21.15.151:3300/0 conn(0x7fcb540ad870 0x7fcb5406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.155+0000 7fcb59a00640 1 --2- 172.21.15.100:0/2512825691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb5415c470 0x7fcb540b29a0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.155+0000 7fcb59a00640 1 -- 172.21.15.100:0/2512825691 >> 172.21.15.100:0/2512825691 conn(0x7fcb540b1e80 msgr2=0x7fcb540b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:34.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.155+0000 7fcb59a00640 1 -- 172.21.15.100:0/2512825691 shutdown_connections 2024-04-23T14:34:34.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.155+0000 7fcb59a00640 1 -- 172.21.15.100:0/2512825691 wait complete. 2024-04-23T14:34:34.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb59a00640 1 Processor -- start 2024-04-23T14:34:34.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb59a00640 1 -- start start 2024-04-23T14:34:34.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb59a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb540ad870 0x7fcb5415da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb59a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcb5415c470 0x7fcb5415f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb59a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb54160de0 0x7fcb540b2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb59a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5415ca60 con 0x7fcb54160de0 2024-04-23T14:34:34.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb59a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcb540cc580 con 0x7fcb540ad870 2024-04-23T14:34:34.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb59a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fcb54169540 con 0x7fcb5415c470 2024-04-23T14:34:34.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb53400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb54160de0 0x7fcb540b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb53400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb54160de0 0x7fcb540b2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48454/0 (socket says 172.21.15.100:48454) 2024-04-23T14:34:34.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb53400640 1 -- 172.21.15.100:0/3294094273 learned_addr learned my addr 172.21.15.100:0/3294094273 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:34.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb52a00640 1 --2- 172.21.15.100:0/3294094273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb540ad870 0x7fcb5415da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb4a000640 1 --2- 172.21.15.100:0/3294094273 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcb5415c470 0x7fcb5415f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb53400640 1 -- 172.21.15.100:0/3294094273 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcb5415c470 msgr2=0x7fcb5415f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb53400640 1 --2- 172.21.15.100:0/3294094273 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcb5415c470 0x7fcb5415f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb53400640 1 -- 172.21.15.100:0/3294094273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb540ad870 msgr2=0x7fcb5415da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb53400640 1 --2- 172.21.15.100:0/3294094273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb540ad870 0x7fcb5415da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb53400640 1 -- 172.21.15.100:0/3294094273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb54098ed0 con 0x7fcb54160de0 2024-04-23T14:34:34.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb4a000640 1 --2- 172.21.15.100:0/3294094273 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcb5415c470 0x7fcb5415f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:34.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb53400640 1 --2- 172.21.15.100:0/3294094273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb54160de0 0x7fcb540b2260 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c007dc0 tx=0x7fcb4c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:34.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb4b400640 1 -- 172.21.15.100:0/3294094273 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c014420 con 0x7fcb54160de0 2024-04-23T14:34:34.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb4b400640 1 -- 172.21.15.100:0/3294094273 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb4c019070 con 0x7fcb54160de0 2024-04-23T14:34:34.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb59a00640 1 -- 172.21.15.100:0/3294094273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb5415e1c0 con 0x7fcb54160de0 2024-04-23T14:34:34.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb4b400640 1 -- 172.21.15.100:0/3294094273 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c014820 con 0x7fcb54160de0 2024-04-23T14:34:34.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb59a00640 1 -- 172.21.15.100:0/3294094273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb5415e780 con 0x7fcb54160de0 2024-04-23T14:34:34.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.159+0000 7fcb59a00640 1 -- 172.21.15.100:0/3294094273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb54068ad0 con 0x7fcb54160de0 2024-04-23T14:34:34.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.163+0000 7fcb4b400640 1 -- 172.21.15.100:0/3294094273 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcb4c0149c0 con 0x7fcb54160de0 2024-04-23T14:34:34.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.163+0000 7fcb4b400640 1 --2- 172.21.15.100:0/3294094273 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcb30068380 0x7fcb3006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.163+0000 7fcb4b400640 1 -- 172.21.15.100:0/3294094273 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fcb4c0873a0 con 0x7fcb54160de0 2024-04-23T14:34:34.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.163+0000 7fcb52a00640 1 --2- 172.21.15.100:0/3294094273 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcb30068380 0x7fcb3006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.163+0000 7fcb52a00640 1 --2- 172.21.15.100:0/3294094273 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcb30068380 0x7fcb3006a840 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcb440ad000 tx=0x7fcb440a4f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:34.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.167+0000 7fcb4b400640 1 -- 172.21.15.100:0/3294094273 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcb4c057730 con 0x7fcb54160de0 2024-04-23T14:34:34.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.323+0000 7fcb59a00640 1 -- 172.21.15.100:0/3294094273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb5406fea0 con 0x7fcb54160de0 2024-04-23T14:34:34.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.327+0000 7fcb4b400640 1 -- 172.21.15.100:0/3294094273 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16950 (secure 0 0 0) 0x7fcb4c057030 con 0x7fcb54160de0 2024-04-23T14:34:34.327 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:34.327 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":53,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:34:25.795515+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:34:22.491379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-23T14:34:23.120342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.327+0000 7fcb59a00640 1 -- 172.21.15.100:0/3294094273 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcb30068380 msgr2=0x7fcb3006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.327+0000 7fcb59a00640 1 --2- 172.21.15.100:0/3294094273 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcb30068380 0x7fcb3006a840 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcb440ad000 tx=0x7fcb440a4f90 comp rx=0 tx=0).stop 2024-04-23T14:34:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.327+0000 7fcb59a00640 1 -- 172.21.15.100:0/3294094273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb54160de0 msgr2=0x7fcb540b2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.327+0000 7fcb59a00640 1 --2- 172.21.15.100:0/3294094273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb54160de0 0x7fcb540b2260 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c007dc0 tx=0x7fcb4c00e4b0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.327+0000 7fcb59a00640 1 -- 172.21.15.100:0/3294094273 shutdown_connections 2024-04-23T14:34:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.327+0000 7fcb59a00640 1 --2- 172.21.15.100:0/3294094273 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcb30068380 0x7fcb3006a840 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.327+0000 7fcb59a00640 1 --2- 172.21.15.100:0/3294094273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb54160de0 0x7fcb540b2260 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.327+0000 7fcb59a00640 1 --2- 172.21.15.100:0/3294094273 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcb5415c470 0x7fcb5415f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.327+0000 7fcb59a00640 1 --2- 172.21.15.100:0/3294094273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb540ad870 0x7fcb5415da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.327+0000 7fcb59a00640 1 -- 172.21.15.100:0/3294094273 >> 172.21.15.100:0/3294094273 conn(0x7fcb540b1e80 msgr2=0x7fcb540990e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:34.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.327+0000 7fcb59a00640 1 -- 172.21.15.100:0/3294094273 shutdown_connections 2024-04-23T14:34:34.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.327+0000 7fcb59a00640 1 -- 172.21.15.100:0/3294094273 wait complete. 2024-04-23T14:34:34.348 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:34:22.491379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:34:23.120342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:34:34.348 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:34:34.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.483+0000 7fc658600640 1 Processor -- start 2024-04-23T14:34:34.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.483+0000 7fc658600640 1 -- start start 2024-04-23T14:34:34.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.487+0000 7fc658600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc65006f8c0 0x7fc65006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.487+0000 7fc658600640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc6500d0170 0x7fc6500ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.487+0000 7fc658600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc65006dd60 con 0x7fc6500ce620 2024-04-23T14:34:34.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.487+0000 7fc658600640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fc6500ad050 con 0x7fc6500d0170 2024-04-23T14:34:34.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.487+0000 7fc658600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc6500ad8f0 con 0x7fc65006f8c0 2024-04-23T14:34:34.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.487+0000 7fc655c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc65006f8c0 0x7fc65006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.487+0000 7fc655c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc65006f8c0 0x7fc65006c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:57724/0 (socket says 172.21.15.100:57724) 2024-04-23T14:34:34.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.487+0000 7fc655200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc6500d0170 0x7fc6500ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.487+0000 7fc655c00640 1 -- 172.21.15.100:0/1164110837 learned_addr learned my addr 172.21.15.100:0/1164110837 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:34.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.487+0000 7fc655c00640 1 -- 172.21.15.100:0/1164110837 >> v2:172.21.15.151:3300/0 conn(0x7fc6500d0170 msgr2=0x7fc6500ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:34.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.487+0000 7fc655c00640 1 --2- 172.21.15.100:0/1164110837 >> v2:172.21.15.151:3300/0 conn(0x7fc6500d0170 0x7fc6500ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.487+0000 7fc655c00640 1 -- 172.21.15.100:0/1164110837 >> v1:172.21.15.151:6789/0 conn(0x7fc6500ce620 legacy=0x7fc65015f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.487+0000 7fc655c00640 1 -- 172.21.15.100:0/1164110837 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6500d09c0 con 0x7fc65006f8c0 2024-04-23T14:34:34.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.491+0000 7fc655c00640 1 --2- 172.21.15.100:0/1164110837 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc65006f8c0 0x7fc65006c6c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc644009870 tx=0x7fc64401e770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ed4baa7c5045b07 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:34.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.491+0000 7fc64fe00640 1 -- 172.21.15.100:0/1164110837 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6440ab670 con 0x7fc65006f8c0 2024-04-23T14:34:34.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.491+0000 7fc64fe00640 1 -- 172.21.15.100:0/1164110837 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6440ab810 con 0x7fc65006f8c0 2024-04-23T14:34:34.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.491+0000 7fc64fe00640 1 -- 172.21.15.100:0/1164110837 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6440abbe0 con 0x7fc65006f8c0 2024-04-23T14:34:34.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.491+0000 7fc658600640 1 -- 172.21.15.100:0/1164110837 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc65006f8c0 msgr2=0x7fc65006c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.491+0000 7fc658600640 1 --2- 172.21.15.100:0/1164110837 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc65006f8c0 0x7fc65006c6c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc644009870 tx=0x7fc64401e770 comp rx=0 tx=0).stop 2024-04-23T14:34:34.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.491+0000 7fc658600640 1 -- 172.21.15.100:0/1164110837 shutdown_connections 2024-04-23T14:34:34.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.491+0000 7fc658600640 1 --2- 172.21.15.100:0/1164110837 >> v2:172.21.15.151:3300/0 conn(0x7fc6500d0170 0x7fc6500ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.491+0000 7fc658600640 1 --2- 172.21.15.100:0/1164110837 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc65006f8c0 0x7fc65006c6c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.491+0000 7fc658600640 1 -- 172.21.15.100:0/1164110837 >> 172.21.15.100:0/1164110837 conn(0x7fc6500b5200 msgr2=0x7fc6500b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:34.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.491+0000 7fc658600640 1 -- 172.21.15.100:0/1164110837 shutdown_connections 2024-04-23T14:34:34.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.491+0000 7fc658600640 1 -- 172.21.15.100:0/1164110837 wait complete. 2024-04-23T14:34:34.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.491+0000 7fc658600640 1 Processor -- start 2024-04-23T14:34:34.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.491+0000 7fc658600640 1 -- start start 2024-04-23T14:34:34.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc658600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc65006f8c0 0x7fc6500c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc658600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6500ce620 0x7fc6500c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc658600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6500d0170 0x7fc65009d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc658600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc6500d06d0 con 0x7fc6500d0170 2024-04-23T14:34:34.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc658600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc6500d0550 con 0x7fc6500ce620 2024-04-23T14:34:34.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc658600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc6500ccc60 con 0x7fc65006f8c0 2024-04-23T14:34:34.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc655c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc65006f8c0 0x7fc6500c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc656600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6500d0170 0x7fc65009d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc655c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc65006f8c0 0x7fc6500c9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:57726/0 (socket says 172.21.15.100:57726) 2024-04-23T14:34:34.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc655c00640 1 -- 172.21.15.100:0/3604657540 learned_addr learned my addr 172.21.15.100:0/3604657540 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:34.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc655200640 1 --2- 172.21.15.100:0/3604657540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6500ce620 0x7fc6500c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc655c00640 1 -- 172.21.15.100:0/3604657540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6500ce620 msgr2=0x7fc6500c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc655c00640 1 --2- 172.21.15.100:0/3604657540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6500ce620 0x7fc6500c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc655c00640 1 -- 172.21.15.100:0/3604657540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6500d0170 msgr2=0x7fc65009d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc655c00640 1 --2- 172.21.15.100:0/3604657540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6500d0170 0x7fc65009d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc655c00640 1 -- 172.21.15.100:0/3604657540 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65015ddb0 con 0x7fc65006f8c0 2024-04-23T14:34:34.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc655c00640 1 --2- 172.21.15.100:0/3604657540 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc65006f8c0 0x7fc6500c9f00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc6440ad040 tx=0x7fc64401cfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:34.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc64d600640 1 -- 172.21.15.100:0/3604657540 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6440bd5f0 con 0x7fc65006f8c0 2024-04-23T14:34:34.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc64d600640 1 -- 172.21.15.100:0/3604657540 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6440bd790 con 0x7fc65006f8c0 2024-04-23T14:34:34.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc658600640 1 -- 172.21.15.100:0/3604657540 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65015e090 con 0x7fc65006f8c0 2024-04-23T14:34:34.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc64d600640 1 -- 172.21.15.100:0/3604657540 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6440bdb80 con 0x7fc65006f8c0 2024-04-23T14:34:34.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc658600640 1 -- 172.21.15.100:0/3604657540 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc65015e650 con 0x7fc65006f8c0 2024-04-23T14:34:34.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc658600640 1 -- 172.21.15.100:0/3604657540 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc650068ad0 con 0x7fc65006f8c0 2024-04-23T14:34:34.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.495+0000 7fc64d600640 1 -- 172.21.15.100:0/3604657540 <== mon.2 v2:172.21.15.151:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc6440b3600 con 0x7fc65006f8c0 2024-04-23T14:34:34.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.499+0000 7fc64d600640 1 --2- 172.21.15.100:0/3604657540 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc630068380 0x7fc63006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.499+0000 7fc64d600640 1 -- 172.21.15.100:0/3604657540 <== mon.2 v2:172.21.15.151:3301/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fc644121a90 con 0x7fc65006f8c0 2024-04-23T14:34:34.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.499+0000 7fc655200640 1 --2- 172.21.15.100:0/3604657540 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc630068380 0x7fc63006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.499+0000 7fc655200640 1 --2- 172.21.15.100:0/3604657540 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc630068380 0x7fc63006a840 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc65015c9a0 tx=0x7fc63c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:34.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.503+0000 7fc64d600640 1 -- 172.21.15.100:0/3604657540 <== mon.2 v2:172.21.15.151:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc6440c3060 con 0x7fc65006f8c0 2024-04-23T14:34:34.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.659+0000 7fc658600640 1 -- 172.21.15.100:0/3604657540 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc65006cc00 con 0x7fc65006f8c0 2024-04-23T14:34:34.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.659+0000 7fc64d600640 1 -- 172.21.15.100:0/3604657540 <== mon.2 v2:172.21.15.151:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) ==== 93+0+10 (secure 0 0 0) 0x7fc6440af020 con 0x7fc65006f8c0 2024-04-23T14:34:34.662 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:34:34.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.663+0000 7fc658600640 1 -- 172.21.15.100:0/3604657540 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc630068380 msgr2=0x7fc63006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.663+0000 7fc658600640 1 --2- 172.21.15.100:0/3604657540 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc630068380 0x7fc63006a840 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc65015c9a0 tx=0x7fc63c009190 comp rx=0 tx=0).stop 2024-04-23T14:34:34.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.663+0000 7fc658600640 1 -- 172.21.15.100:0/3604657540 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc65006f8c0 msgr2=0x7fc6500c9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.663+0000 7fc658600640 1 --2- 172.21.15.100:0/3604657540 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc65006f8c0 0x7fc6500c9f00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc6440ad040 tx=0x7fc64401cfb0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.663+0000 7fc658600640 1 -- 172.21.15.100:0/3604657540 shutdown_connections 2024-04-23T14:34:34.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.663+0000 7fc658600640 1 --2- 172.21.15.100:0/3604657540 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc630068380 0x7fc63006a840 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.663+0000 7fc658600640 1 --2- 172.21.15.100:0/3604657540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6500d0170 0x7fc65009d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.663+0000 7fc658600640 1 --2- 172.21.15.100:0/3604657540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6500ce620 0x7fc6500c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.663+0000 7fc658600640 1 --2- 172.21.15.100:0/3604657540 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc65006f8c0 0x7fc6500c9f00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.663+0000 7fc658600640 1 -- 172.21.15.100:0/3604657540 >> 172.21.15.100:0/3604657540 conn(0x7fc6500b5200 msgr2=0x7fc6501650b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:34.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.663+0000 7fc658600640 1 -- 172.21.15.100:0/3604657540 shutdown_connections 2024-04-23T14:34:34.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.663+0000 7fc658600640 1 -- 172.21.15.100:0/3604657540 wait complete. 2024-04-23T14:34:34.684 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:34:34.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.819+0000 7f4cf5800640 1 Processor -- start 2024-04-23T14:34:34.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.819+0000 7f4cf5800640 1 -- start start 2024-04-23T14:34:34.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.823+0000 7f4cf5800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4cf006bb20 0x7f4cf00ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.823+0000 7f4cf5800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cf00cd540 con 0x7f4cf00cd130 2024-04-23T14:34:34.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.823+0000 7f4cf5800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cf006cbb0 con 0x7f4cf015c7a0 2024-04-23T14:34:34.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.823+0000 7f4cf5800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f4cf006d450 con 0x7f4cf006bb20 2024-04-23T14:34:34.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.823+0000 7f4cef400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f4cf00cd130 0x7f4cf00ad430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:39876/0 (socket says 172.21.15.100:39876) 2024-04-23T14:34:34.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.823+0000 7f4cef400640 1 -- 172.21.15.100:0/630045338 learned_addr learned my addr 172.21.15.100:0/630045338 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:34.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.823+0000 7f4cee000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4cf006bb20 0x7f4cf00ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.823+0000 7f4cee000640 1 -- 172.21.15.100:0/630045338 >> v1:172.21.15.151:6789/0 conn(0x7f4cf015c7a0 legacy=0x7f4cf00af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:34.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.823+0000 7f4cee000640 1 -- 172.21.15.100:0/630045338 >> v1:172.21.15.100:6789/0 conn(0x7f4cf00cd130 legacy=0x7f4cf00ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.823+0000 7f4cee000640 1 -- 172.21.15.100:0/630045338 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf006ce90 con 0x7f4cf006bb20 2024-04-23T14:34:34.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.823+0000 7f4ced600640 1 -- 172.21.15.100:0/630045338 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 653831027 0 0) 0x7f4ce80014a0 con 0x7f4cf00cd130 2024-04-23T14:34:34.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.827+0000 7f4cee000640 1 --2- 172.21.15.100:0/630045338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cf006bb20 0x7f4cf00ccbf0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0097c0 tx=0x7f4cdc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eea61adbfbd5c32 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:34.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.827+0000 7f4ced600640 1 -- 172.21.15.100:0/630045338 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc0b0070 con 0x7f4cf006bb20 2024-04-23T14:34:34.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.827+0000 7f4ced600640 1 -- 172.21.15.100:0/630045338 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cdc0ab600 con 0x7f4cf006bb20 2024-04-23T14:34:34.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.827+0000 7f4ced600640 1 -- 172.21.15.100:0/630045338 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc0ab9d0 con 0x7f4cf006bb20 2024-04-23T14:34:34.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.827+0000 7f4cf5800640 1 -- 172.21.15.100:0/630045338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cf006bb20 msgr2=0x7f4cf00ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.827+0000 7f4cf5800640 1 --2- 172.21.15.100:0/630045338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cf006bb20 0x7f4cf00ccbf0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0097c0 tx=0x7f4cdc01c410 comp rx=0 tx=0).stop 2024-04-23T14:34:34.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.827+0000 7f4cf5800640 1 -- 172.21.15.100:0/630045338 shutdown_connections 2024-04-23T14:34:34.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.827+0000 7f4cf5800640 1 --2- 172.21.15.100:0/630045338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cf006bb20 0x7f4cf00ccbf0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.827+0000 7f4cf5800640 1 -- 172.21.15.100:0/630045338 >> 172.21.15.100:0/630045338 conn(0x7f4cf00b5200 msgr2=0x7f4cf00b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:34.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.827+0000 7f4cf5800640 1 -- 172.21.15.100:0/630045338 shutdown_connections 2024-04-23T14:34:34.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.827+0000 7f4cf5800640 1 -- 172.21.15.100:0/630045338 wait complete. 2024-04-23T14:34:34.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.827+0000 7f4cf5800640 1 Processor -- start 2024-04-23T14:34:34.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.827+0000 7f4cf5800640 1 -- start start 2024-04-23T14:34:34.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.827+0000 7f4cf5800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cf006bb20 0x7f4cf00c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.827+0000 7f4cf5800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4cf00cd130 0x7f4cf0150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.827+0000 7f4cf5800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4cf015c7a0 0x7f4cf0072d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.827+0000 7f4cf5800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf00ae720 con 0x7f4cf006bb20 2024-04-23T14:34:34.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.827+0000 7f4cf5800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf00ae5a0 con 0x7f4cf00cd130 2024-04-23T14:34:34.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.827+0000 7f4cf5800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf016a210 con 0x7f4cf015c7a0 2024-04-23T14:34:34.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.831+0000 7f4ceea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cf006bb20 0x7f4cf00c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.831+0000 7f4ceea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cf006bb20 0x7f4cf00c72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48478/0 (socket says 172.21.15.100:48478) 2024-04-23T14:34:34.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.831+0000 7f4ceea00640 1 -- 172.21.15.100:0/3900948531 learned_addr learned my addr 172.21.15.100:0/3900948531 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:34.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.831+0000 7f4cee000640 1 --2- 172.21.15.100:0/3900948531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4cf00cd130 0x7f4cf0150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.831+0000 7f4cef400640 1 --2- 172.21.15.100:0/3900948531 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4cf015c7a0 0x7f4cf0072d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.831+0000 7f4ceea00640 1 -- 172.21.15.100:0/3900948531 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4cf015c7a0 msgr2=0x7f4cf0072d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.831+0000 7f4ceea00640 1 --2- 172.21.15.100:0/3900948531 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4cf015c7a0 0x7f4cf0072d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.831+0000 7f4ceea00640 1 -- 172.21.15.100:0/3900948531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4cf00cd130 msgr2=0x7f4cf0150e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.831+0000 7f4ceea00640 1 --2- 172.21.15.100:0/3900948531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4cf00cd130 0x7f4cf0150e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.831+0000 7f4ceea00640 1 -- 172.21.15.100:0/3900948531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf00ca9d0 con 0x7f4cf006bb20 2024-04-23T14:34:34.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.831+0000 7f4ceea00640 1 --2- 172.21.15.100:0/3900948531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cf006bb20 0x7f4cf00c72b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000c3f0 tx=0x7f4ce000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:34.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.831+0000 7f4ce6a00640 1 -- 172.21.15.100:0/3900948531 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0017070 con 0x7f4cf006bb20 2024-04-23T14:34:34.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.831+0000 7f4ce6a00640 1 -- 172.21.15.100:0/3900948531 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce00052d0 con 0x7f4cf006bb20 2024-04-23T14:34:34.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.831+0000 7f4ce6a00640 1 -- 172.21.15.100:0/3900948531 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0003840 con 0x7f4cf006bb20 2024-04-23T14:34:34.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.831+0000 7f4cf5800640 1 -- 172.21.15.100:0/3900948531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf00cc120 con 0x7f4cf006bb20 2024-04-23T14:34:34.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.831+0000 7f4cf5800640 1 -- 172.21.15.100:0/3900948531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf00cc3c0 con 0x7f4cf006bb20 2024-04-23T14:34:34.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.835+0000 7f4cf5800640 1 -- 172.21.15.100:0/3900948531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cf0068ad0 con 0x7f4cf006bb20 2024-04-23T14:34:34.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.835+0000 7f4ce6a00640 1 -- 172.21.15.100:0/3900948531 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4ce001a070 con 0x7f4cf006bb20 2024-04-23T14:34:34.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.835+0000 7f4ce6a00640 1 --2- 172.21.15.100:0/3900948531 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4cd00683b0 0x7f4cd006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.835+0000 7f4ce6a00640 1 -- 172.21.15.100:0/3900948531 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f4ce0085ed0 con 0x7f4cf006bb20 2024-04-23T14:34:34.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.835+0000 7f4cee000640 1 --2- 172.21.15.100:0/3900948531 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4cd00683b0 0x7f4cd006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.835+0000 7f4cee000640 1 --2- 172.21.15.100:0/3900948531 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4cd00683b0 0x7f4cd006a870 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0ad000 tx=0x7f4cdc0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:34.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.839+0000 7f4ce6a00640 1 -- 172.21.15.100:0/3900948531 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4ce0005480 con 0x7f4cf006bb20 2024-04-23T14:34:34.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.995+0000 7f4cf5800640 1 -- 172.21.15.100:0/3900948531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4cf0155930 con 0x7f4cf006bb20 2024-04-23T14:34:34.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.999+0000 7f4ce6a00640 1 -- 172.21.15.100:0/3900948531 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) ==== 104+0+11 (secure 0 0 0) 0x7f4ce0055c10 con 0x7f4cf006bb20 2024-04-23T14:34:34.999 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:34:35.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.999+0000 7f4cf5800640 1 -- 172.21.15.100:0/3900948531 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4cd00683b0 msgr2=0x7f4cd006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.999+0000 7f4cf5800640 1 --2- 172.21.15.100:0/3900948531 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4cd00683b0 0x7f4cd006a870 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0ad000 tx=0x7f4cdc0a6000 comp rx=0 tx=0).stop 2024-04-23T14:34:35.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.999+0000 7f4cf5800640 1 -- 172.21.15.100:0/3900948531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cf006bb20 msgr2=0x7f4cf00c72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.999+0000 7f4cf5800640 1 --2- 172.21.15.100:0/3900948531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cf006bb20 0x7f4cf00c72b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000c3f0 tx=0x7f4ce000c8b0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.999+0000 7f4cf5800640 1 -- 172.21.15.100:0/3900948531 shutdown_connections 2024-04-23T14:34:35.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.999+0000 7f4cf5800640 1 --2- 172.21.15.100:0/3900948531 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4cd00683b0 0x7f4cd006a870 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.999+0000 7f4cf5800640 1 --2- 172.21.15.100:0/3900948531 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4cf015c7a0 0x7f4cf0072d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.999+0000 7f4cf5800640 1 --2- 172.21.15.100:0/3900948531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4cf00cd130 0x7f4cf0150e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.999+0000 7f4cf5800640 1 --2- 172.21.15.100:0/3900948531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cf006bb20 0x7f4cf00c72b0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.999+0000 7f4cf5800640 1 -- 172.21.15.100:0/3900948531 >> 172.21.15.100:0/3900948531 conn(0x7f4cf00b5200 msgr2=0x7f4cf00cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:35.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.999+0000 7f4cf5800640 1 -- 172.21.15.100:0/3900948531 shutdown_connections 2024-04-23T14:34:35.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:34.999+0000 7f4cf5800640 1 -- 172.21.15.100:0/3900948531 wait complete. 2024-04-23T14:34:35.019 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:34:35.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.155+0000 7f2674200640 1 Processor -- start 2024-04-23T14:34:35.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.155+0000 7f2674200640 1 -- start start 2024-04-23T14:34:35.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.155+0000 7f2674200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f266c0ce7f0 0x7f266c1663f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.155+0000 7f2674200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f266c06dd60 con 0x7f266c06c6c0 2024-04-23T14:34:35.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.155+0000 7f2674200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f266c0d09c0 con 0x7f266c06f8c0 2024-04-23T14:34:35.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.155+0000 7f2674200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f266c0ad050 con 0x7f266c0ce7f0 2024-04-23T14:34:35.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.155+0000 7f2670e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f266c06c6c0 0x7f266c0ce0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:39882/0 (socket says 172.21.15.100:39882) 2024-04-23T14:34:35.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.155+0000 7f2670e00640 1 -- 172.21.15.100:0/1908144178 learned_addr learned my addr 172.21.15.100:0/1908144178 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:35.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.159+0000 7f2672200640 1 --2- 172.21.15.100:0/1908144178 >> v2:172.21.15.151:3300/0 conn(0x7f266c0ce7f0 0x7f266c1663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.159+0000 7f266be00640 1 -- 172.21.15.100:0/1908144178 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 93679686 0 0) 0x7f2658001120 con 0x7f266c06c6c0 2024-04-23T14:34:35.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.159+0000 7f266be00640 1 -- 172.21.15.100:0/1908144178 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2654003530 con 0x7f266c06c6c0 2024-04-23T14:34:35.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.159+0000 7f266be00640 1 -- 172.21.15.100:0/1908144178 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3280422190 0 0) 0x7f266c0d09c0 con 0x7f266c06f8c0 2024-04-23T14:34:35.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.159+0000 7f266be00640 1 -- 172.21.15.100:0/1908144178 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2658001120 con 0x7f266c06f8c0 2024-04-23T14:34:35.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.159+0000 7f2672200640 1 -- 172.21.15.100:0/1908144178 >> v1:172.21.15.151:6789/0 conn(0x7f266c06f8c0 legacy=0x7f266c0d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.159+0000 7f2672200640 1 -- 172.21.15.100:0/1908144178 >> v1:172.21.15.100:6789/0 conn(0x7f266c06c6c0 legacy=0x7f266c0ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.159+0000 7f266be00640 1 -- 172.21.15.100:0/1908144178 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2614587053 0 0) 0x7f2654003530 con 0x7f266c06c6c0 2024-04-23T14:34:35.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.159+0000 7f2672200640 1 -- 172.21.15.100:0/1908144178 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266c0ad2a0 con 0x7f266c0ce7f0 2024-04-23T14:34:35.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.159+0000 7f2672200640 1 --2- 172.21.15.100:0/1908144178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f266c0ce7f0 0x7f266c1663f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2664009d90 tx=0x7f266401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d542a888c18c657 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:35.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f266be00640 1 -- 172.21.15.100:0/1908144178 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658001850 con 0x7f266c0ce7f0 2024-04-23T14:34:35.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f266be00640 1 -- 172.21.15.100:0/1908144178 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26640a64f0 con 0x7f266c0ce7f0 2024-04-23T14:34:35.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f266be00640 1 -- 172.21.15.100:0/1908144178 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26640a68a0 con 0x7f266c0ce7f0 2024-04-23T14:34:35.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f2674200640 1 -- 172.21.15.100:0/1908144178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f266c0ce7f0 msgr2=0x7f266c1663f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f2674200640 1 --2- 172.21.15.100:0/1908144178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f266c0ce7f0 0x7f266c1663f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2664009d90 tx=0x7f266401c890 comp rx=0 tx=0).stop 2024-04-23T14:34:35.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f2674200640 1 -- 172.21.15.100:0/1908144178 shutdown_connections 2024-04-23T14:34:35.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f2674200640 1 --2- 172.21.15.100:0/1908144178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f266c0ce7f0 0x7f266c1663f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f2674200640 1 -- 172.21.15.100:0/1908144178 >> 172.21.15.100:0/1908144178 conn(0x7f266c0b5200 msgr2=0x7f266c0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:35.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f2674200640 1 -- 172.21.15.100:0/1908144178 shutdown_connections 2024-04-23T14:34:35.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f2674200640 1 -- 172.21.15.100:0/1908144178 wait complete. 2024-04-23T14:34:35.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f2674200640 1 Processor -- start 2024-04-23T14:34:35.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f2674200640 1 -- start start 2024-04-23T14:34:35.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f2674200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f266c06c6c0 0x7f266c156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f2674200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f266c06f8c0 0x7f266c0c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f2674200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f266c0ce7f0 0x7f266c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f2674200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f266c0cd800 con 0x7f266c06f8c0 2024-04-23T14:34:35.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f2674200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f266c0cd680 con 0x7f266c06c6c0 2024-04-23T14:34:35.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f2674200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f266c0cd980 con 0x7f266c0ce7f0 2024-04-23T14:34:35.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f2670e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f266c06f8c0 0x7f266c0c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f2670e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f266c06f8c0 0x7f266c0c9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48484/0 (socket says 172.21.15.100:48484) 2024-04-23T14:34:35.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f2670e00640 1 -- 172.21.15.100:0/1790555391 learned_addr learned my addr 172.21.15.100:0/1790555391 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:35.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f2671800640 1 --2- 172.21.15.100:0/1790555391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f266c06c6c0 0x7f266c156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.163+0000 7f2672200640 1 --2- 172.21.15.100:0/1790555391 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f266c0ce7f0 0x7f266c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.167+0000 7f2670e00640 1 -- 172.21.15.100:0/1790555391 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f266c0ce7f0 msgr2=0x7f266c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.167+0000 7f2670e00640 1 --2- 172.21.15.100:0/1790555391 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f266c0ce7f0 0x7f266c0c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.167+0000 7f2670e00640 1 -- 172.21.15.100:0/1790555391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f266c06c6c0 msgr2=0x7f266c156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.167+0000 7f2670e00640 1 --2- 172.21.15.100:0/1790555391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f266c06c6c0 0x7f266c156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.167+0000 7f2670e00640 1 -- 172.21.15.100:0/1790555391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266c157800 con 0x7f266c06f8c0 2024-04-23T14:34:35.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.167+0000 7f2672200640 1 --2- 172.21.15.100:0/1790555391 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f266c0ce7f0 0x7f266c0c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:35.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.167+0000 7f2670e00640 1 --2- 172.21.15.100:0/1790555391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f266c06f8c0 0x7f266c0c9c40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f265800c750 tx=0x7f265800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:35.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.167+0000 7f2669600640 1 -- 172.21.15.100:0/1790555391 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658017070 con 0x7f266c06f8c0 2024-04-23T14:34:35.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.167+0000 7f2669600640 1 -- 172.21.15.100:0/1790555391 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2658005af0 con 0x7f266c06f8c0 2024-04-23T14:34:35.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.167+0000 7f2669600640 1 -- 172.21.15.100:0/1790555391 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658007070 con 0x7f266c06f8c0 2024-04-23T14:34:35.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.167+0000 7f2674200640 1 -- 172.21.15.100:0/1790555391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f266c0ca8c0 con 0x7f266c06f8c0 2024-04-23T14:34:35.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.167+0000 7f2674200640 1 -- 172.21.15.100:0/1790555391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f266c0cae80 con 0x7f266c06f8c0 2024-04-23T14:34:35.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.167+0000 7f2674200640 1 -- 172.21.15.100:0/1790555391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f266c068ad0 con 0x7f266c06f8c0 2024-04-23T14:34:35.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.167+0000 7f2669600640 1 -- 172.21.15.100:0/1790555391 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f265801a070 con 0x7f266c06f8c0 2024-04-23T14:34:35.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.167+0000 7f2669600640 1 --2- 172.21.15.100:0/1790555391 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f264c0683b0 0x7f264c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.167+0000 7f2669600640 1 -- 172.21.15.100:0/1790555391 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f26580861f0 con 0x7f266c06f8c0 2024-04-23T14:34:35.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.167+0000 7f2671800640 1 --2- 172.21.15.100:0/1790555391 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f264c0683b0 0x7f264c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.171+0000 7f2671800640 1 --2- 172.21.15.100:0/1790555391 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f264c0683b0 0x7f264c06a870 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f26600011f0 tx=0x7f2660009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:35.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.171+0000 7f2669600640 1 -- 172.21.15.100:0/1790555391 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f266c157800 con 0x7f266c06f8c0 2024-04-23T14:34:35.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.327+0000 7f2674200640 1 -- 172.21.15.100:0/1790555391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f266c157a20 con 0x7f266c06f8c0 2024-04-23T14:34:35.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.327+0000 7f2669600640 1 -- 172.21.15.100:0/1790555391 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) ==== 100+0+11 (secure 0 0 0) 0x7f266c06dd60 con 0x7f266c06f8c0 2024-04-23T14:34:35.330 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:34:35.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.331+0000 7f2674200640 1 -- 172.21.15.100:0/1790555391 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f264c0683b0 msgr2=0x7f264c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.331+0000 7f2674200640 1 --2- 172.21.15.100:0/1790555391 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f264c0683b0 0x7f264c06a870 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f26600011f0 tx=0x7f2660009040 comp rx=0 tx=0).stop 2024-04-23T14:34:35.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.331+0000 7f2674200640 1 -- 172.21.15.100:0/1790555391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f266c06f8c0 msgr2=0x7f266c0c9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.331+0000 7f2674200640 1 --2- 172.21.15.100:0/1790555391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f266c06f8c0 0x7f266c0c9c40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f265800c750 tx=0x7f265800cc10 comp rx=0 tx=0).stop 2024-04-23T14:34:35.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.331+0000 7f2674200640 1 -- 172.21.15.100:0/1790555391 shutdown_connections 2024-04-23T14:34:35.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.331+0000 7f2674200640 1 --2- 172.21.15.100:0/1790555391 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f264c0683b0 0x7f264c06a870 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.331+0000 7f2674200640 1 --2- 172.21.15.100:0/1790555391 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f266c0ce7f0 0x7f266c0c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.331+0000 7f2674200640 1 --2- 172.21.15.100:0/1790555391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f266c06f8c0 0x7f266c0c9c40 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.331+0000 7f2674200640 1 --2- 172.21.15.100:0/1790555391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f266c06c6c0 0x7f266c156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.331+0000 7f2674200640 1 -- 172.21.15.100:0/1790555391 >> 172.21.15.100:0/1790555391 conn(0x7f266c0b5200 msgr2=0x7f266c0afc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:35.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.331+0000 7f2674200640 1 -- 172.21.15.100:0/1790555391 shutdown_connections 2024-04-23T14:34:35.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.331+0000 7f2674200640 1 -- 172.21.15.100:0/1790555391 wait complete. 2024-04-23T14:34:35.350 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:34:35.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.483+0000 7fad01400640 1 Processor -- start 2024-04-23T14:34:35.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.483+0000 7fad01400640 1 -- start start 2024-04-23T14:34:35.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.487+0000 7fad01400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7facfc15d0c0 0x7facfc0cde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.487+0000 7fad01400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7facfc0cf160 0x7facfc0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.487+0000 7fad01400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facfc0cfab0 con 0x7facfc15d4a0 2024-04-23T14:34:35.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.487+0000 7fad01400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7facfc0cff50 con 0x7facfc0cf160 2024-04-23T14:34:35.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.487+0000 7fad01400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7facfc0ceda0 con 0x7facfc15d0c0 2024-04-23T14:34:35.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.487+0000 7facfb200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7facfc0cf160 0x7facfc0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.487+0000 7facf9e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7facfc15d4a0 0x7facfc0cb9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:39896/0 (socket says 172.21.15.100:39896) 2024-04-23T14:34:35.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.487+0000 7facf9e00640 1 -- 172.21.15.100:0/350537910 learned_addr learned my addr 172.21.15.100:0/350537910 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:35.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.487+0000 7facfa800640 1 --2- 172.21.15.100:0/350537910 >> v2:172.21.15.151:3300/0 conn(0x7facfc15d0c0 0x7facfc0cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.487+0000 7facfb200640 1 -- 172.21.15.100:0/350537910 >> v2:172.21.15.151:3300/0 conn(0x7facfc15d0c0 msgr2=0x7facfc0cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:35.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.487+0000 7facfb200640 1 --2- 172.21.15.100:0/350537910 >> v2:172.21.15.151:3300/0 conn(0x7facfc15d0c0 0x7facfc0cde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.487+0000 7facfb200640 1 -- 172.21.15.100:0/350537910 >> v1:172.21.15.100:6789/0 conn(0x7facfc15d4a0 legacy=0x7facfc0cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.487+0000 7facfb200640 1 -- 172.21.15.100:0/350537910 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facfc0ce500 con 0x7facfc0cf160 2024-04-23T14:34:35.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.491+0000 7facfb200640 1 --2- 172.21.15.100:0/350537910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facfc0cf160 0x7facfc0cf570 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7facfc0b6210 tx=0x7facf401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41789a87a3f14b6c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:35.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.491+0000 7facf9400640 1 -- 172.21.15.100:0/350537910 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf40b1070 con 0x7facfc0cf160 2024-04-23T14:34:35.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.491+0000 7facf9400640 1 -- 172.21.15.100:0/350537910 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facf40ac620 con 0x7facfc0cf160 2024-04-23T14:34:35.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.491+0000 7facf9400640 1 -- 172.21.15.100:0/350537910 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf40ac9f0 con 0x7facfc0cf160 2024-04-23T14:34:35.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.491+0000 7fad01400640 1 -- 172.21.15.100:0/350537910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facfc0cf160 msgr2=0x7facfc0cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.491+0000 7fad01400640 1 --2- 172.21.15.100:0/350537910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facfc0cf160 0x7facfc0cf570 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7facfc0b6210 tx=0x7facf401c3c0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.491+0000 7fad01400640 1 -- 172.21.15.100:0/350537910 shutdown_connections 2024-04-23T14:34:35.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.491+0000 7fad01400640 1 --2- 172.21.15.100:0/350537910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facfc0cf160 0x7facfc0cf570 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.491+0000 7fad01400640 1 --2- 172.21.15.100:0/350537910 >> v2:172.21.15.151:3300/0 conn(0x7facfc15d0c0 0x7facfc0cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.491+0000 7fad01400640 1 -- 172.21.15.100:0/350537910 >> 172.21.15.100:0/350537910 conn(0x7facfc0b5310 msgr2=0x7facfc0b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:35.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.491+0000 7fad01400640 1 -- 172.21.15.100:0/350537910 shutdown_connections 2024-04-23T14:34:35.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.491+0000 7fad01400640 1 -- 172.21.15.100:0/350537910 wait complete. 2024-04-23T14:34:35.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.491+0000 7fad01400640 1 Processor -- start 2024-04-23T14:34:35.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.491+0000 7fad01400640 1 -- start start 2024-04-23T14:34:35.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7fad01400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facfc0cf160 0x7facfc0aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7fad01400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7facfc15d0c0 0x7facfc14d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7fad01400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facfc15d4a0 0x7facfc14d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7fad01400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7facfc0cc230 con 0x7facfc15d4a0 2024-04-23T14:34:35.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7fad01400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7facfc0cc0b0 con 0x7facfc0cf160 2024-04-23T14:34:35.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7fad01400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7facfc152e30 con 0x7facfc15d0c0 2024-04-23T14:34:35.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7facfb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facfc15d4a0 0x7facfc14d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7facfb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facfc15d4a0 0x7facfc14d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48508/0 (socket says 172.21.15.100:48508) 2024-04-23T14:34:35.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7facfb200640 1 -- 172.21.15.100:0/4114391658 learned_addr learned my addr 172.21.15.100:0/4114391658 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:35.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7facfa800640 1 --2- 172.21.15.100:0/4114391658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facfc0cf160 0x7facfc0aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7facf9e00640 1 --2- 172.21.15.100:0/4114391658 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7facfc15d0c0 0x7facfc14d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7facfb200640 1 -- 172.21.15.100:0/4114391658 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7facfc15d0c0 msgr2=0x7facfc14d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7facfb200640 1 --2- 172.21.15.100:0/4114391658 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7facfc15d0c0 0x7facfc14d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7facfb200640 1 -- 172.21.15.100:0/4114391658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facfc0cf160 msgr2=0x7facfc0aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7facfb200640 1 --2- 172.21.15.100:0/4114391658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facfc0cf160 0x7facfc0aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7facfb200640 1 -- 172.21.15.100:0/4114391658 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facfc14df60 con 0x7facfc15d4a0 2024-04-23T14:34:35.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7facfb200640 1 --2- 172.21.15.100:0/4114391658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facfc15d4a0 0x7facfc14d540 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7facf40af040 tx=0x7facf4019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:35.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7facf2a00640 1 -- 172.21.15.100:0/4114391658 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf40b9020 con 0x7facfc15d4a0 2024-04-23T14:34:35.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7facf2a00640 1 -- 172.21.15.100:0/4114391658 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facf40b5410 con 0x7facfc15d4a0 2024-04-23T14:34:35.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7fad01400640 1 -- 172.21.15.100:0/4114391658 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facfc159710 con 0x7facfc15d4a0 2024-04-23T14:34:35.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7facf2a00640 1 -- 172.21.15.100:0/4114391658 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf40b1040 con 0x7facfc15d4a0 2024-04-23T14:34:35.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7fad01400640 1 -- 172.21.15.100:0/4114391658 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facfc159cd0 con 0x7facfc15d4a0 2024-04-23T14:34:35.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7fad01400640 1 -- 172.21.15.100:0/4114391658 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facfc068ad0 con 0x7facfc15d4a0 2024-04-23T14:34:35.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.495+0000 7facf2a00640 1 -- 172.21.15.100:0/4114391658 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7facf40bc020 con 0x7facfc15d4a0 2024-04-23T14:34:35.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.499+0000 7facf2a00640 1 --2- 172.21.15.100:0/4114391658 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7facdc068380 0x7facdc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.499+0000 7facf2a00640 1 -- 172.21.15.100:0/4114391658 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7facf4120670 con 0x7facfc15d4a0 2024-04-23T14:34:35.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.499+0000 7facfa800640 1 --2- 172.21.15.100:0/4114391658 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7facdc068380 0x7facdc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.499+0000 7facfa800640 1 --2- 172.21.15.100:0/4114391658 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7facdc068380 0x7facdc06a840 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7facfc0cd480 tx=0x7facec0098d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:35.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.503+0000 7facf2a00640 1 -- 172.21.15.100:0/4114391658 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7facf40c3030 con 0x7facfc15d4a0 2024-04-23T14:34:35.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.671+0000 7fad01400640 1 -- 172.21.15.100:0/4114391658 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7facfc06c730 con 0x7facfc15d4a0 2024-04-23T14:34:35.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.671+0000 7facf2a00640 1 -- 172.21.15.100:0/4114391658 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) ==== 80+0+0 (secure 0 0 0) 0x7facf4016070 con 0x7facfc15d4a0 2024-04-23T14:34:35.674 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-04-23T14:34:35.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.675+0000 7fad01400640 1 -- 172.21.15.100:0/4114391658 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7facdc068380 msgr2=0x7facdc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.675+0000 7fad01400640 1 --2- 172.21.15.100:0/4114391658 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7facdc068380 0x7facdc06a840 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7facfc0cd480 tx=0x7facec0098d0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.675+0000 7fad01400640 1 -- 172.21.15.100:0/4114391658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facfc15d4a0 msgr2=0x7facfc14d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.675+0000 7fad01400640 1 --2- 172.21.15.100:0/4114391658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facfc15d4a0 0x7facfc14d540 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7facf40af040 tx=0x7facf4019160 comp rx=0 tx=0).stop 2024-04-23T14:34:35.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.675+0000 7fad01400640 1 -- 172.21.15.100:0/4114391658 shutdown_connections 2024-04-23T14:34:35.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.675+0000 7fad01400640 1 --2- 172.21.15.100:0/4114391658 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7facdc068380 0x7facdc06a840 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.675+0000 7fad01400640 1 --2- 172.21.15.100:0/4114391658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facfc15d4a0 0x7facfc14d540 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.675+0000 7fad01400640 1 --2- 172.21.15.100:0/4114391658 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7facfc15d0c0 0x7facfc14d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.675+0000 7fad01400640 1 --2- 172.21.15.100:0/4114391658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facfc0cf160 0x7facfc0aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.675+0000 7fad01400640 1 -- 172.21.15.100:0/4114391658 >> 172.21.15.100:0/4114391658 conn(0x7facfc0b5310 msgr2=0x7facfc15e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:35.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.675+0000 7fad01400640 1 -- 172.21.15.100:0/4114391658 shutdown_connections 2024-04-23T14:34:35.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.675+0000 7fad01400640 1 -- 172.21.15.100:0/4114391658 wait complete. 2024-04-23T14:34:35.695 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:34:35.695 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:35.704 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:35.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.879+0000 7fda77c00640 1 Processor -- start 2024-04-23T14:34:35.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.879+0000 7fda77c00640 1 -- start start 2024-04-23T14:34:35.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.883+0000 7fda77c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fda7015e820 0x7fda700b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.883+0000 7fda77c00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda700b5820 con 0x7fda700d0200 2024-04-23T14:34:35.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.883+0000 7fda77c00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda700b5b00 con 0x7fda700b52c0 2024-04-23T14:34:35.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.883+0000 7fda77c00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fda7015ec00 con 0x7fda7015e820 2024-04-23T14:34:35.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.883+0000 7fda75200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fda7015e820 0x7fda700b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.883+0000 7fda75200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fda7015e820 0x7fda700b4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48520/0 (socket says 172.21.15.100:48520) 2024-04-23T14:34:35.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.883+0000 7fda75200640 1 -- 172.21.15.100:0/2207360570 learned_addr learned my addr 172.21.15.100:0/2207360570 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:35.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.883+0000 7fda75200640 1 -- 172.21.15.100:0/2207360570 >> v1:172.21.15.151:6789/0 conn(0x7fda700b52c0 legacy=0x7fda700cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:35.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.883+0000 7fda75200640 1 -- 172.21.15.100:0/2207360570 >> v1:172.21.15.100:6789/0 conn(0x7fda700d0200 legacy=0x7fda7006cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.883+0000 7fda75200640 1 -- 172.21.15.100:0/2207360570 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda700b5de0 con 0x7fda7015e820 2024-04-23T14:34:35.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.883+0000 7fda75200640 1 --2- 172.21.15.100:0/2207360570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda7015e820 0x7fda700b4d80 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fda60009870 tx=0x7fda6001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544a52a765c8f7fa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:35.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.883+0000 7fda6f400640 1 -- 172.21.15.100:0/2207360570 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda600aa650 con 0x7fda7015e820 2024-04-23T14:34:35.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.883+0000 7fda6f400640 1 -- 172.21.15.100:0/2207360570 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda600af070 con 0x7fda7015e820 2024-04-23T14:34:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.883+0000 7fda6f400640 1 -- 172.21.15.100:0/2207360570 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda600aaa10 con 0x7fda7015e820 2024-04-23T14:34:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda77c00640 1 -- 172.21.15.100:0/2207360570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda7015e820 msgr2=0x7fda700b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda77c00640 1 --2- 172.21.15.100:0/2207360570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda7015e820 0x7fda700b4d80 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fda60009870 tx=0x7fda6001c4f0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda77c00640 1 -- 172.21.15.100:0/2207360570 shutdown_connections 2024-04-23T14:34:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda77c00640 1 --2- 172.21.15.100:0/2207360570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda7015e820 0x7fda700b4d80 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda77c00640 1 -- 172.21.15.100:0/2207360570 >> 172.21.15.100:0/2207360570 conn(0x7fda70070690 msgr2=0x7fda70070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda77c00640 1 -- 172.21.15.100:0/2207360570 shutdown_connections 2024-04-23T14:34:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda77c00640 1 -- 172.21.15.100:0/2207360570 wait complete. 2024-04-23T14:34:35.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda77c00640 1 Processor -- start 2024-04-23T14:34:35.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda77c00640 1 -- start start 2024-04-23T14:34:35.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda77c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fda700b52c0 0x7fda700ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda77c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda700d0200 0x7fda700b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda77c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda7015e820 0x7fda700b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda77c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fda7006dbe0 con 0x7fda700d0200 2024-04-23T14:34:35.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda77c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fda7006da60 con 0x7fda7015e820 2024-04-23T14:34:35.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda77c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fda7006dd60 con 0x7fda700b52c0 2024-04-23T14:34:35.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda6fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda700d0200 0x7fda700b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda6fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda700d0200 0x7fda700b2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48524/0 (socket says 172.21.15.100:48524) 2024-04-23T14:34:35.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda6fe00640 1 -- 172.21.15.100:0/843337987 learned_addr learned my addr 172.21.15.100:0/843337987 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:35.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda75200640 1 --2- 172.21.15.100:0/843337987 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fda700b52c0 0x7fda700ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda75c00640 1 --2- 172.21.15.100:0/843337987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda7015e820 0x7fda700b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda6fe00640 1 -- 172.21.15.100:0/843337987 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fda700b52c0 msgr2=0x7fda700ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda6fe00640 1 --2- 172.21.15.100:0/843337987 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fda700b52c0 0x7fda700ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda6fe00640 1 -- 172.21.15.100:0/843337987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda7015e820 msgr2=0x7fda700b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda6fe00640 1 --2- 172.21.15.100:0/843337987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda7015e820 0x7fda700b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.887+0000 7fda6fe00640 1 -- 172.21.15.100:0/843337987 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda700cea20 con 0x7fda700d0200 2024-04-23T14:34:35.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.891+0000 7fda6fe00640 1 --2- 172.21.15.100:0/843337987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda700d0200 0x7fda700b2220 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00b410 tx=0x7fda5c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:35.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.891+0000 7fda6cc00640 1 -- 172.21.15.100:0/843337987 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c015030 con 0x7fda700d0200 2024-04-23T14:34:35.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.891+0000 7fda6cc00640 1 -- 172.21.15.100:0/843337987 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda5c004020 con 0x7fda700d0200 2024-04-23T14:34:35.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.891+0000 7fda6cc00640 1 -- 172.21.15.100:0/843337987 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c004450 con 0x7fda700d0200 2024-04-23T14:34:35.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.891+0000 7fda77c00640 1 -- 172.21.15.100:0/843337987 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7014f920 con 0x7fda700d0200 2024-04-23T14:34:35.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.891+0000 7fda77c00640 1 -- 172.21.15.100:0/843337987 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda7014fee0 con 0x7fda700d0200 2024-04-23T14:34:35.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.891+0000 7fda77c00640 1 -- 172.21.15.100:0/843337987 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda70068ad0 con 0x7fda700d0200 2024-04-23T14:34:35.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.891+0000 7fda6cc00640 1 -- 172.21.15.100:0/843337987 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fda5c018070 con 0x7fda700d0200 2024-04-23T14:34:35.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.891+0000 7fda6cc00640 1 --2- 172.21.15.100:0/843337987 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fda50068380 0x7fda5006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.891+0000 7fda6cc00640 1 -- 172.21.15.100:0/843337987 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fda5c086640 con 0x7fda700d0200 2024-04-23T14:34:35.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.891+0000 7fda75200640 1 --2- 172.21.15.100:0/843337987 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fda50068380 0x7fda5006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.891+0000 7fda75200640 1 --2- 172.21.15.100:0/843337987 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fda50068380 0x7fda5006a840 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fda600ae000 tx=0x7fda600a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:35.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:35.895+0000 7fda6cc00640 1 -- 172.21.15.100:0/843337987 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fda5c056380 con 0x7fda700d0200 2024-04-23T14:34:36.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.055+0000 7fda77c00640 1 -- 172.21.15.100:0/843337987 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda7006fb40 con 0x7fda700d0200 2024-04-23T14:34:36.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.059+0000 7fda6cc00640 1 -- 172.21.15.100:0/843337987 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16950 (secure 0 0 0) 0x7fda5c05a680 con 0x7fda700d0200 2024-04-23T14:34:36.059 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:36.059 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":53,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:34:25.795515+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:34:22.491379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-23T14:34:23.120342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:36.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.059+0000 7fda77c00640 1 -- 172.21.15.100:0/843337987 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fda50068380 msgr2=0x7fda5006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.059+0000 7fda77c00640 1 --2- 172.21.15.100:0/843337987 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fda50068380 0x7fda5006a840 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fda600ae000 tx=0x7fda600a7000 comp rx=0 tx=0).stop 2024-04-23T14:34:36.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.059+0000 7fda77c00640 1 -- 172.21.15.100:0/843337987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda700d0200 msgr2=0x7fda700b2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.059+0000 7fda77c00640 1 --2- 172.21.15.100:0/843337987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda700d0200 0x7fda700b2220 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00b410 tx=0x7fda5c00b8d0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.059+0000 7fda77c00640 1 -- 172.21.15.100:0/843337987 shutdown_connections 2024-04-23T14:34:36.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.059+0000 7fda77c00640 1 --2- 172.21.15.100:0/843337987 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fda50068380 0x7fda5006a840 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.059+0000 7fda77c00640 1 --2- 172.21.15.100:0/843337987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda7015e820 0x7fda700b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.059+0000 7fda77c00640 1 --2- 172.21.15.100:0/843337987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda700d0200 0x7fda700b2220 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.059+0000 7fda77c00640 1 --2- 172.21.15.100:0/843337987 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fda700b52c0 0x7fda700ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.059+0000 7fda77c00640 1 -- 172.21.15.100:0/843337987 >> 172.21.15.100:0/843337987 conn(0x7fda70070690 msgr2=0x7fda7015b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:36.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.059+0000 7fda77c00640 1 -- 172.21.15.100:0/843337987 shutdown_connections 2024-04-23T14:34:36.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.059+0000 7fda77c00640 1 -- 172.21.15.100:0/843337987 wait complete. 2024-04-23T14:34:36.080 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:34:22.491379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:34:23.120342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:34:36.080 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:34:36.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.215+0000 7fc291000640 1 Processor -- start 2024-04-23T14:34:36.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.215+0000 7fc291000640 1 -- start start 2024-04-23T14:34:36.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.219+0000 7fc291000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc28c06c6c0 0x7fc28c0ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.219+0000 7fc291000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc28c0ce620 0x7fc28c0add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.219+0000 7fc291000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc28c06dd60 con 0x7fc28c06f8c0 2024-04-23T14:34:36.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.219+0000 7fc291000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc28c0ad050 con 0x7fc28c06c6c0 2024-04-23T14:34:36.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.219+0000 7fc291000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fc28c0ad8f0 con 0x7fc28c0ce620 2024-04-23T14:34:36.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.219+0000 7fc289e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fc28c06f8c0 0x7fc28c0d0170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:39904/0 (socket says 172.21.15.100:39904) 2024-04-23T14:34:36.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.219+0000 7fc289e00640 1 -- 172.21.15.100:0/909932429 learned_addr learned my addr 172.21.15.100:0/909932429 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:36.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.219+0000 7fc281400640 1 --2- 172.21.15.100:0/909932429 >> v2:172.21.15.100:3300/0 conn(0x7fc28c06c6c0 0x7fc28c0ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.219+0000 7fc28a800640 1 --2- 172.21.15.100:0/909932429 >> v2:172.21.15.151:3300/0 conn(0x7fc28c0ce620 0x7fc28c0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.219+0000 7fc289400640 1 -- 172.21.15.100:0/909932429 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 546074173 0 0) 0x7fc27c001120 con 0x7fc28c06f8c0 2024-04-23T14:34:36.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.219+0000 7fc289400640 1 -- 172.21.15.100:0/909932429 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc270005740 con 0x7fc28c06f8c0 2024-04-23T14:34:36.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.219+0000 7fc281400640 1 -- 172.21.15.100:0/909932429 >> v2:172.21.15.151:3300/0 conn(0x7fc28c0ce620 msgr2=0x7fc28c0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:36.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.219+0000 7fc281400640 1 --2- 172.21.15.100:0/909932429 >> v2:172.21.15.151:3300/0 conn(0x7fc28c0ce620 0x7fc28c0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.219+0000 7fc281400640 1 -- 172.21.15.100:0/909932429 >> v1:172.21.15.100:6789/0 conn(0x7fc28c06f8c0 legacy=0x7fc28c0d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.219+0000 7fc281400640 1 -- 172.21.15.100:0/909932429 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28c0d09c0 con 0x7fc28c06c6c0 2024-04-23T14:34:36.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.223+0000 7fc281400640 1 --2- 172.21.15.100:0/909932429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28c06c6c0 0x7fc28c0ce0e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0b3120 tx=0x7fc27401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc123cb6326d3bb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:36.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.223+0000 7fc289400640 1 -- 172.21.15.100:0/909932429 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2740b1070 con 0x7fc28c06c6c0 2024-04-23T14:34:36.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.223+0000 7fc289400640 1 -- 172.21.15.100:0/909932429 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2740ac5f0 con 0x7fc28c06c6c0 2024-04-23T14:34:36.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.223+0000 7fc289400640 1 -- 172.21.15.100:0/909932429 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2740ac9c0 con 0x7fc28c06c6c0 2024-04-23T14:34:36.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.223+0000 7fc291000640 1 -- 172.21.15.100:0/909932429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28c06c6c0 msgr2=0x7fc28c0ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.223+0000 7fc291000640 1 --2- 172.21.15.100:0/909932429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28c06c6c0 0x7fc28c0ce0e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0b3120 tx=0x7fc27401c480 comp rx=0 tx=0).stop 2024-04-23T14:34:36.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.223+0000 7fc291000640 1 -- 172.21.15.100:0/909932429 shutdown_connections 2024-04-23T14:34:36.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.223+0000 7fc291000640 1 --2- 172.21.15.100:0/909932429 >> v2:172.21.15.151:3300/0 conn(0x7fc28c0ce620 0x7fc28c0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.223+0000 7fc291000640 1 --2- 172.21.15.100:0/909932429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28c06c6c0 0x7fc28c0ce0e0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.223+0000 7fc291000640 1 -- 172.21.15.100:0/909932429 >> 172.21.15.100:0/909932429 conn(0x7fc28c0b5200 msgr2=0x7fc28c0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:36.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.223+0000 7fc291000640 1 -- 172.21.15.100:0/909932429 shutdown_connections 2024-04-23T14:34:36.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.223+0000 7fc291000640 1 -- 172.21.15.100:0/909932429 wait complete. 2024-04-23T14:34:36.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.223+0000 7fc291000640 1 Processor -- start 2024-04-23T14:34:36.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.223+0000 7fc291000640 1 -- start start 2024-04-23T14:34:36.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc291000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc28c06c6c0 0x7fc28c15c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc291000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28c06f8c0 0x7fc28c071e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc291000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc28c0ce620 0x7fc28c0723b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc291000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c0aee60 con 0x7fc28c06f8c0 2024-04-23T14:34:36.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc291000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c0aece0 con 0x7fc28c06c6c0 2024-04-23T14:34:36.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc291000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc28c0ccc60 con 0x7fc28c0ce620 2024-04-23T14:34:36.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc281400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28c06f8c0 0x7fc28c071e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc281400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28c06f8c0 0x7fc28c071e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48536/0 (socket says 172.21.15.100:48536) 2024-04-23T14:34:36.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc281400640 1 -- 172.21.15.100:0/1333914739 learned_addr learned my addr 172.21.15.100:0/1333914739 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:36.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc289e00640 1 --2- 172.21.15.100:0/1333914739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc28c06c6c0 0x7fc28c15c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc28a800640 1 --2- 172.21.15.100:0/1333914739 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc28c0ce620 0x7fc28c0723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc281400640 1 -- 172.21.15.100:0/1333914739 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc28c0ce620 msgr2=0x7fc28c0723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc281400640 1 --2- 172.21.15.100:0/1333914739 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc28c0ce620 0x7fc28c0723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc281400640 1 -- 172.21.15.100:0/1333914739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc28c06c6c0 msgr2=0x7fc28c15c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc281400640 1 --2- 172.21.15.100:0/1333914739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc28c06c6c0 0x7fc28c15c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc281400640 1 -- 172.21.15.100:0/1333914739 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28c15fc10 con 0x7fc28c06f8c0 2024-04-23T14:34:36.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc281400640 1 --2- 172.21.15.100:0/1333914739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28c06f8c0 0x7fc28c071e70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc27401c960 tx=0x7fc274019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:36.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc282a00640 1 -- 172.21.15.100:0/1333914739 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2740b1040 con 0x7fc28c06f8c0 2024-04-23T14:34:36.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc282a00640 1 -- 172.21.15.100:0/1333914739 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2740ba070 con 0x7fc28c06f8c0 2024-04-23T14:34:36.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc282a00640 1 -- 172.21.15.100:0/1333914739 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2740b5670 con 0x7fc28c06f8c0 2024-04-23T14:34:36.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc291000640 1 -- 172.21.15.100:0/1333914739 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc28c0c7450 con 0x7fc28c06f8c0 2024-04-23T14:34:36.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc291000640 1 -- 172.21.15.100:0/1333914739 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc28c0c7a10 con 0x7fc28c06f8c0 2024-04-23T14:34:36.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc291000640 1 -- 172.21.15.100:0/1333914739 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc28c068ad0 con 0x7fc28c06f8c0 2024-04-23T14:34:36.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.227+0000 7fc282a00640 1 -- 172.21.15.100:0/1333914739 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc2740b5810 con 0x7fc28c06f8c0 2024-04-23T14:34:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.231+0000 7fc282a00640 1 --2- 172.21.15.100:0/1333914739 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc268068380 0x7fc26806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.231+0000 7fc282a00640 1 -- 172.21.15.100:0/1333914739 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fc274120740 con 0x7fc28c06f8c0 2024-04-23T14:34:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.231+0000 7fc289e00640 1 --2- 172.21.15.100:0/1333914739 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc268068380 0x7fc26806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.231+0000 7fc289e00640 1 --2- 172.21.15.100:0/1333914739 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc268068380 0x7fc26806a840 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc27c005890 tx=0x7fc27c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:36.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.235+0000 7fc282a00640 1 -- 172.21.15.100:0/1333914739 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc2740a9070 con 0x7fc28c06f8c0 2024-04-23T14:34:36.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.391+0000 7fc291000640 1 -- 172.21.15.100:0/1333914739 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc28c0c7be0 con 0x7fc28c06f8c0 2024-04-23T14:34:36.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.391+0000 7fc282a00640 1 -- 172.21.15.100:0/1333914739 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) ==== 93+0+10 (secure 0 0 0) 0x7fc274016070 con 0x7fc28c06f8c0 2024-04-23T14:34:36.394 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:34:36.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.395+0000 7fc291000640 1 -- 172.21.15.100:0/1333914739 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc268068380 msgr2=0x7fc26806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.395+0000 7fc291000640 1 --2- 172.21.15.100:0/1333914739 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc268068380 0x7fc26806a840 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc27c005890 tx=0x7fc27c009040 comp rx=0 tx=0).stop 2024-04-23T14:34:36.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.395+0000 7fc291000640 1 -- 172.21.15.100:0/1333914739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28c06f8c0 msgr2=0x7fc28c071e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.395+0000 7fc291000640 1 --2- 172.21.15.100:0/1333914739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28c06f8c0 0x7fc28c071e70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc27401c960 tx=0x7fc274019220 comp rx=0 tx=0).stop 2024-04-23T14:34:36.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.395+0000 7fc291000640 1 -- 172.21.15.100:0/1333914739 shutdown_connections 2024-04-23T14:34:36.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.395+0000 7fc291000640 1 --2- 172.21.15.100:0/1333914739 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc268068380 0x7fc26806a840 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.395+0000 7fc291000640 1 --2- 172.21.15.100:0/1333914739 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc28c0ce620 0x7fc28c0723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.395+0000 7fc291000640 1 --2- 172.21.15.100:0/1333914739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28c06f8c0 0x7fc28c071e70 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.395+0000 7fc291000640 1 --2- 172.21.15.100:0/1333914739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc28c06c6c0 0x7fc28c15c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.395+0000 7fc291000640 1 -- 172.21.15.100:0/1333914739 >> 172.21.15.100:0/1333914739 conn(0x7fc28c0b5200 msgr2=0x7fc28c0af280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:36.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.395+0000 7fc291000640 1 -- 172.21.15.100:0/1333914739 shutdown_connections 2024-04-23T14:34:36.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.395+0000 7fc291000640 1 -- 172.21.15.100:0/1333914739 wait complete. 2024-04-23T14:34:36.414 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:34:36.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.551+0000 7ff3f9400640 1 Processor -- start 2024-04-23T14:34:36.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.551+0000 7ff3f9400640 1 -- start start 2024-04-23T14:34:36.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.551+0000 7ff3f9400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff3f406c6c0 0x7ff3f40ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.551+0000 7ff3f9400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff3f40ce620 0x7ff3f40add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.551+0000 7ff3f9400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3f406dd60 con 0x7ff3f406f8c0 2024-04-23T14:34:36.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.551+0000 7ff3f9400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7ff3f40ad050 con 0x7ff3f40ce620 2024-04-23T14:34:36.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.551+0000 7ff3f9400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff3f40ad8f0 con 0x7ff3f406c6c0 2024-04-23T14:34:36.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.551+0000 7ff3f1e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff3f406c6c0 0x7ff3f40ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.551+0000 7ff3f2800640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7ff3f406f8c0 0x7ff3f40d0170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:59490/0 (socket says 172.21.15.100:59490) 2024-04-23T14:34:36.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.551+0000 7ff3f2800640 1 -- 172.21.15.100:0/2882097 learned_addr learned my addr 172.21.15.100:0/2882097 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:36.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.551+0000 7ff3f3200640 1 --2- 172.21.15.100:0/2882097 >> v2:172.21.15.151:3300/0 conn(0x7ff3f40ce620 0x7ff3f40add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.551+0000 7ff3f3200640 1 -- 172.21.15.100:0/2882097 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff3f406c6c0 msgr2=0x7ff3f40ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:36.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.551+0000 7ff3f3200640 1 --2- 172.21.15.100:0/2882097 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff3f406c6c0 0x7ff3f40ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.551+0000 7ff3f3200640 1 -- 172.21.15.100:0/2882097 >> v1:172.21.15.151:6789/0 conn(0x7ff3f406f8c0 legacy=0x7ff3f40d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.551+0000 7ff3f3200640 1 -- 172.21.15.100:0/2882097 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f40d09c0 con 0x7ff3f40ce620 2024-04-23T14:34:36.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.551+0000 7ff3f1400640 1 -- 172.21.15.100:0/2882097 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3180330204 0 0) 0x7ff3e4001120 con 0x7ff3f406f8c0 2024-04-23T14:34:36.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.551+0000 7ff3f1e00640 1 --2- 172.21.15.100:0/2882097 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff3f406c6c0 0x7ff3f40ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:34:36.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.555+0000 7ff3f3200640 1 --2- 172.21.15.100:0/2882097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3f40ce620 0x7ff3f40add80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009d90 tx=0x7ff3ec01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ede8cf01e73da2c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:36.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.555+0000 7ff3f1400640 1 -- 172.21.15.100:0/2882097 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec0b2070 con 0x7ff3f40ce620 2024-04-23T14:34:36.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.555+0000 7ff3f1400640 1 -- 172.21.15.100:0/2882097 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3ec0a5940 con 0x7ff3f40ce620 2024-04-23T14:34:36.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.555+0000 7ff3f1400640 1 -- 172.21.15.100:0/2882097 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec0a5d10 con 0x7ff3f40ce620 2024-04-23T14:34:36.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.555+0000 7ff3f9400640 1 -- 172.21.15.100:0/2882097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3f40ce620 msgr2=0x7ff3f40add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.555+0000 7ff3f9400640 1 --2- 172.21.15.100:0/2882097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3f40ce620 0x7ff3f40add80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009d90 tx=0x7ff3ec01c970 comp rx=0 tx=0).stop 2024-04-23T14:34:36.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.555+0000 7ff3f9400640 1 -- 172.21.15.100:0/2882097 shutdown_connections 2024-04-23T14:34:36.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.555+0000 7ff3f9400640 1 --2- 172.21.15.100:0/2882097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3f40ce620 0x7ff3f40add80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.555+0000 7ff3f9400640 1 --2- 172.21.15.100:0/2882097 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff3f406c6c0 0x7ff3f40ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.555+0000 7ff3f9400640 1 -- 172.21.15.100:0/2882097 >> 172.21.15.100:0/2882097 conn(0x7ff3f40b5200 msgr2=0x7ff3f40b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:36.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.555+0000 7ff3f9400640 1 -- 172.21.15.100:0/2882097 shutdown_connections 2024-04-23T14:34:36.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.555+0000 7ff3f9400640 1 -- 172.21.15.100:0/2882097 wait complete. 2024-04-23T14:34:36.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f9400640 1 Processor -- start 2024-04-23T14:34:36.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f9400640 1 -- start start 2024-04-23T14:34:36.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f406c6c0 0x7ff3f4167af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f9400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff3f406f8c0 0x7ff3f40c5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3f40ce620 0x7ff3f40cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f9400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f40aeea0 con 0x7ff3f406c6c0 2024-04-23T14:34:36.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f9400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f40aed20 con 0x7ff3f40ce620 2024-04-23T14:34:36.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f9400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff3f41663f0 con 0x7ff3f406f8c0 2024-04-23T14:34:36.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f406c6c0 0x7ff3f4167af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f406c6c0 0x7ff3f4167af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48550/0 (socket says 172.21.15.100:48550) 2024-04-23T14:34:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f2800640 1 -- 172.21.15.100:0/2692531822 learned_addr learned my addr 172.21.15.100:0/2692531822 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f1e00640 1 --2- 172.21.15.100:0/2692531822 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff3f406f8c0 0x7ff3f40c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f3200640 1 --2- 172.21.15.100:0/2692531822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3f40ce620 0x7ff3f40cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f2800640 1 -- 172.21.15.100:0/2692531822 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff3f406f8c0 msgr2=0x7ff3f40c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f2800640 1 --2- 172.21.15.100:0/2692531822 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff3f406f8c0 0x7ff3f40c5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f2800640 1 -- 172.21.15.100:0/2692531822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3f40ce620 msgr2=0x7ff3f40cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f2800640 1 --2- 172.21.15.100:0/2692531822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3f40ce620 0x7ff3f40cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f2800640 1 -- 172.21.15.100:0/2692531822 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f40c5ea0 con 0x7ff3f406c6c0 2024-04-23T14:34:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f3200640 1 --2- 172.21.15.100:0/2692531822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3f40ce620 0x7ff3f40cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f2800640 1 --2- 172.21.15.100:0/2692531822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f406c6c0 0x7ff3f4167af0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400c780 tx=0x7ff3e400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3eaa00640 1 -- 172.21.15.100:0/2692531822 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4018070 con 0x7ff3f406c6c0 2024-04-23T14:34:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3eaa00640 1 -- 172.21.15.100:0/2692531822 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e40072f0 con 0x7ff3f406c6c0 2024-04-23T14:34:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f9400640 1 -- 172.21.15.100:0/2692531822 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f4168260 con 0x7ff3f406c6c0 2024-04-23T14:34:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3eaa00640 1 -- 172.21.15.100:0/2692531822 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40047f0 con 0x7ff3f406c6c0 2024-04-23T14:34:36.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f9400640 1 -- 172.21.15.100:0/2692531822 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f4168820 con 0x7ff3f406c6c0 2024-04-23T14:34:36.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.559+0000 7ff3f9400640 1 -- 172.21.15.100:0/2692531822 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3f4068ad0 con 0x7ff3f406c6c0 2024-04-23T14:34:36.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.563+0000 7ff3eaa00640 1 -- 172.21.15.100:0/2692531822 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff3e4004990 con 0x7ff3f406c6c0 2024-04-23T14:34:36.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.563+0000 7ff3eaa00640 1 --2- 172.21.15.100:0/2692531822 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff3d4068380 0x7ff3d406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.563+0000 7ff3eaa00640 1 -- 172.21.15.100:0/2692531822 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7ff3e4085fe0 con 0x7ff3f406c6c0 2024-04-23T14:34:36.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.563+0000 7ff3f1e00640 1 --2- 172.21.15.100:0/2692531822 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff3d4068380 0x7ff3d406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.563+0000 7ff3f1e00640 1 --2- 172.21.15.100:0/2692531822 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff3d4068380 0x7ff3d406a840 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009580 tx=0x7ff3e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:36.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.567+0000 7ff3eaa00640 1 -- 172.21.15.100:0/2692531822 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff3f406dd60 con 0x7ff3f406c6c0 2024-04-23T14:34:36.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.723+0000 7ff3f9400640 1 -- 172.21.15.100:0/2692531822 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff3f406cc00 con 0x7ff3f406c6c0 2024-04-23T14:34:36.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.723+0000 7ff3eaa00640 1 -- 172.21.15.100:0/2692531822 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) ==== 104+0+11 (secure 0 0 0) 0x7ff3e4056390 con 0x7ff3f406c6c0 2024-04-23T14:34:36.723 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:34:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.723+0000 7ff3f9400640 1 -- 172.21.15.100:0/2692531822 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff3d4068380 msgr2=0x7ff3d406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.723+0000 7ff3f9400640 1 --2- 172.21.15.100:0/2692531822 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff3d4068380 0x7ff3d406a840 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009580 tx=0x7ff3e0008040 comp rx=0 tx=0).stop 2024-04-23T14:34:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.723+0000 7ff3f9400640 1 -- 172.21.15.100:0/2692531822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f406c6c0 msgr2=0x7ff3f4167af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.723+0000 7ff3f9400640 1 --2- 172.21.15.100:0/2692531822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f406c6c0 0x7ff3f4167af0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400c780 tx=0x7ff3e400cc40 comp rx=0 tx=0).stop 2024-04-23T14:34:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.723+0000 7ff3f9400640 1 -- 172.21.15.100:0/2692531822 shutdown_connections 2024-04-23T14:34:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.723+0000 7ff3f9400640 1 --2- 172.21.15.100:0/2692531822 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff3d4068380 0x7ff3d406a840 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.723+0000 7ff3f9400640 1 --2- 172.21.15.100:0/2692531822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3f40ce620 0x7ff3f40cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.723+0000 7ff3f9400640 1 --2- 172.21.15.100:0/2692531822 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff3f406f8c0 0x7ff3f40c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.723+0000 7ff3f9400640 1 --2- 172.21.15.100:0/2692531822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f406c6c0 0x7ff3f4167af0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.723+0000 7ff3f9400640 1 -- 172.21.15.100:0/2692531822 >> 172.21.15.100:0/2692531822 conn(0x7ff3f40b5200 msgr2=0x7ff3f40ca810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.723+0000 7ff3f9400640 1 -- 172.21.15.100:0/2692531822 shutdown_connections 2024-04-23T14:34:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.723+0000 7ff3f9400640 1 -- 172.21.15.100:0/2692531822 wait complete. 2024-04-23T14:34:36.745 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:34:36.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.883+0000 7fdf8ae00640 1 Processor -- start 2024-04-23T14:34:36.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.883+0000 7fdf8ae00640 1 -- start start 2024-04-23T14:34:36.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.887+0000 7fdf8ae00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fdf840b36c0 0x7fdf840b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.887+0000 7fdf8ae00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdf840b2900 0x7fdf8406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.887+0000 7fdf8ae00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf8415f420 con 0x7fdf840d03e0 2024-04-23T14:34:36.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.887+0000 7fdf8ae00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fdf840b3b40 con 0x7fdf840b2900 2024-04-23T14:34:36.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.887+0000 7fdf8ae00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fdf840b1c50 con 0x7fdf840b36c0 2024-04-23T14:34:36.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.887+0000 7fdf8a400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdf840b2900 0x7fdf8406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.887+0000 7fdf8a400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdf840b2900 0x7fdf8406d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48558/0 (socket says 172.21.15.100:48558) 2024-04-23T14:34:36.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.887+0000 7fdf8a400640 1 -- 172.21.15.100:0/3949309057 learned_addr learned my addr 172.21.15.100:0/3949309057 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:36.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.887+0000 7fdf89000640 1 --2- 172.21.15.100:0/3949309057 >> v2:172.21.15.151:3300/0 conn(0x7fdf840b36c0 0x7fdf840b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.887+0000 7fdf8a400640 1 -- 172.21.15.100:0/3949309057 >> v2:172.21.15.151:3300/0 conn(0x7fdf840b36c0 msgr2=0x7fdf840b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:36.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.887+0000 7fdf8a400640 1 --2- 172.21.15.100:0/3949309057 >> v2:172.21.15.151:3300/0 conn(0x7fdf840b36c0 0x7fdf840b23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.887+0000 7fdf8a400640 1 -- 172.21.15.100:0/3949309057 >> v1:172.21.15.151:6789/0 conn(0x7fdf840d03e0 legacy=0x7fdf8415ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.887+0000 7fdf8a400640 1 -- 172.21.15.100:0/3949309057 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8415f700 con 0x7fdf840b2900 2024-04-23T14:34:36.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.887+0000 7fdf8a400640 1 --2- 172.21.15.100:0/3949309057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf840b2900 0x7fdf8406d6f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fdf840cfba0 tx=0x7fdf7c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c100e09b3e7bd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:36.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf83e00640 1 -- 172.21.15.100:0/3949309057 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c0a5790 con 0x7fdf840b2900 2024-04-23T14:34:36.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf83e00640 1 -- 172.21.15.100:0/3949309057 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf7c0a5930 con 0x7fdf840b2900 2024-04-23T14:34:36.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf83e00640 1 -- 172.21.15.100:0/3949309057 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c0a5d00 con 0x7fdf840b2900 2024-04-23T14:34:36.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf8ae00640 1 -- 172.21.15.100:0/3949309057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf840b2900 msgr2=0x7fdf8406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf8ae00640 1 --2- 172.21.15.100:0/3949309057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf840b2900 0x7fdf8406d6f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fdf840cfba0 tx=0x7fdf7c01c9c0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf8ae00640 1 -- 172.21.15.100:0/3949309057 shutdown_connections 2024-04-23T14:34:36.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf8ae00640 1 --2- 172.21.15.100:0/3949309057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf840b2900 0x7fdf8406d6f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf8ae00640 1 --2- 172.21.15.100:0/3949309057 >> v2:172.21.15.151:3300/0 conn(0x7fdf840b36c0 0x7fdf840b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf8ae00640 1 -- 172.21.15.100:0/3949309057 >> 172.21.15.100:0/3949309057 conn(0x7fdf84072420 msgr2=0x7fdf840ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:36.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf8ae00640 1 -- 172.21.15.100:0/3949309057 shutdown_connections 2024-04-23T14:34:36.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf8ae00640 1 -- 172.21.15.100:0/3949309057 wait complete. 2024-04-23T14:34:36.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf8ae00640 1 Processor -- start 2024-04-23T14:34:36.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf8ae00640 1 -- start start 2024-04-23T14:34:36.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf8ae00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf840b2900 0x7fdf84148060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf8ae00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdf840b36c0 0x7fdf840cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf8ae00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf840d03e0 0x7fdf840c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf8ae00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdf84155050 con 0x7fdf840b2900 2024-04-23T14:34:36.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf8ae00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdf84154ed0 con 0x7fdf840d03e0 2024-04-23T14:34:36.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf8ae00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fdf841551d0 con 0x7fdf840b36c0 2024-04-23T14:34:36.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf89a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf840b2900 0x7fdf84148060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf89a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf840b2900 0x7fdf84148060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48562/0 (socket says 172.21.15.100:48562) 2024-04-23T14:34:36.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf89a00640 1 -- 172.21.15.100:0/2567170415 learned_addr learned my addr 172.21.15.100:0/2567170415 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:36.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf8a400640 1 --2- 172.21.15.100:0/2567170415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf840d03e0 0x7fdf840c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf89000640 1 --2- 172.21.15.100:0/2567170415 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdf840b36c0 0x7fdf840cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf89a00640 1 -- 172.21.15.100:0/2567170415 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdf840b36c0 msgr2=0x7fdf840cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf89a00640 1 --2- 172.21.15.100:0/2567170415 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdf840b36c0 0x7fdf840cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf89a00640 1 -- 172.21.15.100:0/2567170415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf840d03e0 msgr2=0x7fdf840c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf89a00640 1 --2- 172.21.15.100:0/2567170415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf840d03e0 0x7fdf840c76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.891+0000 7fdf89a00640 1 -- 172.21.15.100:0/2567170415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf840c9fe0 con 0x7fdf840b2900 2024-04-23T14:34:36.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.895+0000 7fdf89a00640 1 --2- 172.21.15.100:0/2567170415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf840b2900 0x7fdf84148060 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800c370 tx=0x7fdf7800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:36.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.895+0000 7fdf81600640 1 -- 172.21.15.100:0/2567170415 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf780052d0 con 0x7fdf840b2900 2024-04-23T14:34:36.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.895+0000 7fdf81600640 1 -- 172.21.15.100:0/2567170415 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf78005470 con 0x7fdf840b2900 2024-04-23T14:34:36.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.895+0000 7fdf81600640 1 -- 172.21.15.100:0/2567170415 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf78007260 con 0x7fdf840b2900 2024-04-23T14:34:36.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.895+0000 7fdf8ae00640 1 -- 172.21.15.100:0/2567170415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf840ca2c0 con 0x7fdf840b2900 2024-04-23T14:34:36.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.895+0000 7fdf8ae00640 1 -- 172.21.15.100:0/2567170415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf840ca880 con 0x7fdf840b2900 2024-04-23T14:34:36.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.895+0000 7fdf8ae00640 1 -- 172.21.15.100:0/2567170415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf84068ad0 con 0x7fdf840b2900 2024-04-23T14:34:36.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.895+0000 7fdf81600640 1 -- 172.21.15.100:0/2567170415 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdf78007400 con 0x7fdf840b2900 2024-04-23T14:34:36.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.895+0000 7fdf81600640 1 --2- 172.21.15.100:0/2567170415 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdf64068380 0x7fdf6406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.895+0000 7fdf81600640 1 -- 172.21.15.100:0/2567170415 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fdf78085fe0 con 0x7fdf840b2900 2024-04-23T14:34:36.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.895+0000 7fdf89000640 1 --2- 172.21.15.100:0/2567170415 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdf64068380 0x7fdf6406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.899+0000 7fdf89000640 1 --2- 172.21.15.100:0/2567170415 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdf64068380 0x7fdf6406a840 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdf840cc430 tx=0x7fdf70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:36.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:36.899+0000 7fdf81600640 1 -- 172.21.15.100:0/2567170415 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdf78056410 con 0x7fdf840b2900 2024-04-23T14:34:37.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.059+0000 7fdf8ae00640 1 -- 172.21.15.100:0/2567170415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdf840b8270 con 0x7fdf840b2900 2024-04-23T14:34:37.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.059+0000 7fdf81600640 1 -- 172.21.15.100:0/2567170415 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) ==== 100+0+11 (secure 0 0 0) 0x7fdf78055d10 con 0x7fdf840b2900 2024-04-23T14:34:37.062 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:34:37.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.063+0000 7fdf8ae00640 1 -- 172.21.15.100:0/2567170415 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdf64068380 msgr2=0x7fdf6406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:37.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.063+0000 7fdf8ae00640 1 --2- 172.21.15.100:0/2567170415 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdf64068380 0x7fdf6406a840 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdf840cc430 tx=0x7fdf70008040 comp rx=0 tx=0).stop 2024-04-23T14:34:37.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.063+0000 7fdf8ae00640 1 -- 172.21.15.100:0/2567170415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf840b2900 msgr2=0x7fdf84148060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:37.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.063+0000 7fdf8ae00640 1 --2- 172.21.15.100:0/2567170415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf840b2900 0x7fdf84148060 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800c370 tx=0x7fdf7800c830 comp rx=0 tx=0).stop 2024-04-23T14:34:37.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.063+0000 7fdf8ae00640 1 -- 172.21.15.100:0/2567170415 shutdown_connections 2024-04-23T14:34:37.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.063+0000 7fdf8ae00640 1 --2- 172.21.15.100:0/2567170415 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fdf64068380 0x7fdf6406a840 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.063+0000 7fdf8ae00640 1 --2- 172.21.15.100:0/2567170415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf840d03e0 0x7fdf840c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.063+0000 7fdf8ae00640 1 --2- 172.21.15.100:0/2567170415 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdf840b36c0 0x7fdf840cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.063+0000 7fdf8ae00640 1 --2- 172.21.15.100:0/2567170415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf840b2900 0x7fdf84148060 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.063+0000 7fdf8ae00640 1 -- 172.21.15.100:0/2567170415 >> 172.21.15.100:0/2567170415 conn(0x7fdf84072420 msgr2=0x7fdf84070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:37.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.063+0000 7fdf8ae00640 1 -- 172.21.15.100:0/2567170415 shutdown_connections 2024-04-23T14:34:37.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.063+0000 7fdf8ae00640 1 -- 172.21.15.100:0/2567170415 wait complete. 2024-04-23T14:34:37.083 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:34:37.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.215+0000 7f37bc000640 1 Processor -- start 2024-04-23T14:34:37.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.215+0000 7f37bc000640 1 -- start start 2024-04-23T14:34:37.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.219+0000 7f37bc000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f37b415c160 0x7f37b415f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:37.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.219+0000 7f37bc000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f37b4073cc0 0x7f37b40cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:37.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.219+0000 7f37bc000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37b40d0db0 con 0x7f37b40d0920 2024-04-23T14:34:37.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.219+0000 7f37bc000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f37b40d1090 con 0x7f37b415c160 2024-04-23T14:34:37.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.219+0000 7f37bc000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f37b4162360 con 0x7f37b4073cc0 2024-04-23T14:34:37.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.219+0000 7f37b9600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f37b415c160 0x7f37b415f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:37.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.219+0000 7f37b9600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f37b415c160 0x7f37b415f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48568/0 (socket says 172.21.15.100:48568) 2024-04-23T14:34:37.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.219+0000 7f37b9600640 1 -- 172.21.15.100:0/1477229493 learned_addr learned my addr 172.21.15.100:0/1477229493 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:37.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.219+0000 7f37ba000640 1 --2- 172.21.15.100:0/1477229493 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f37b4073cc0 0x7f37b40cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:37.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.219+0000 7f37b9600640 1 -- 172.21.15.100:0/1477229493 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f37b4073cc0 msgr2=0x7f37b40cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:37.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.219+0000 7f37b9600640 1 --2- 172.21.15.100:0/1477229493 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f37b4073cc0 0x7f37b40cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.219+0000 7f37b9600640 1 -- 172.21.15.100:0/1477229493 >> v1:172.21.15.151:6789/0 conn(0x7f37b40d0920 legacy=0x7f37b40735b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:37.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.219+0000 7f37b9600640 1 -- 172.21.15.100:0/1477229493 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b4162640 con 0x7f37b415c160 2024-04-23T14:34:37.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.223+0000 7f37b9600640 1 --2- 172.21.15.100:0/1477229493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b415c160 0x7f37b415f6c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009870 tx=0x7f37a801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2e9b8c331560eb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:37.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.223+0000 7f37b3e00640 1 -- 172.21.15.100:0/1477229493 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a80b2070 con 0x7f37b415c160 2024-04-23T14:34:37.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.223+0000 7f37b3e00640 1 -- 172.21.15.100:0/1477229493 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37a80a9070 con 0x7f37b415c160 2024-04-23T14:34:37.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.223+0000 7f37b3e00640 1 -- 172.21.15.100:0/1477229493 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a80ad880 con 0x7f37b415c160 2024-04-23T14:34:37.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.223+0000 7f37bc000640 1 -- 172.21.15.100:0/1477229493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b415c160 msgr2=0x7f37b415f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:37.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.223+0000 7f37bc000640 1 --2- 172.21.15.100:0/1477229493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b415c160 0x7f37b415f6c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009870 tx=0x7f37a801c4d0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.223+0000 7f37bc000640 1 -- 172.21.15.100:0/1477229493 shutdown_connections 2024-04-23T14:34:37.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.223+0000 7f37bc000640 1 --2- 172.21.15.100:0/1477229493 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f37b4073cc0 0x7f37b40cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.223+0000 7f37bc000640 1 --2- 172.21.15.100:0/1477229493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b415c160 0x7f37b415f6c0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.223+0000 7f37bc000640 1 -- 172.21.15.100:0/1477229493 >> 172.21.15.100:0/1477229493 conn(0x7f37b4071590 msgr2=0x7f37b4072ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:37.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.223+0000 7f37bc000640 1 -- 172.21.15.100:0/1477229493 shutdown_connections 2024-04-23T14:34:37.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.223+0000 7f37bc000640 1 -- 172.21.15.100:0/1477229493 wait complete. 2024-04-23T14:34:37.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.223+0000 7f37bc000640 1 Processor -- start 2024-04-23T14:34:37.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.223+0000 7f37bc000640 1 -- start start 2024-04-23T14:34:37.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.223+0000 7f37bc000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37b4073cc0 0x7f37b415a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:37.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37bc000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f37b415c160 0x7f37b40d0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:37.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37bc000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b4142aa0 0x7f37b413ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:37.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37bc000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f37b416b750 con 0x7f37b4142aa0 2024-04-23T14:34:37.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37bc000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f37b416b5d0 con 0x7f37b4073cc0 2024-04-23T14:34:37.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37bc000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f37b416b8d0 con 0x7f37b415c160 2024-04-23T14:34:37.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37ba000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b4142aa0 0x7f37b413ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:37.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37ba000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b4142aa0 0x7f37b413ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48574/0 (socket says 172.21.15.100:48574) 2024-04-23T14:34:37.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37ba000640 1 -- 172.21.15.100:0/3752630215 learned_addr learned my addr 172.21.15.100:0/3752630215 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:37.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37b8c00640 1 --2- 172.21.15.100:0/3752630215 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f37b415c160 0x7f37b40d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:37.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37b9600640 1 --2- 172.21.15.100:0/3752630215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37b4073cc0 0x7f37b415a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:37.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37ba000640 1 -- 172.21.15.100:0/3752630215 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f37b415c160 msgr2=0x7f37b40d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:37.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37ba000640 1 --2- 172.21.15.100:0/3752630215 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f37b415c160 0x7f37b40d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37ba000640 1 -- 172.21.15.100:0/3752630215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37b4073cc0 msgr2=0x7f37b415a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:37.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37ba000640 1 --2- 172.21.15.100:0/3752630215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37b4073cc0 0x7f37b415a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37ba000640 1 -- 172.21.15.100:0/3752630215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b40cf180 con 0x7f37b4142aa0 2024-04-23T14:34:37.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37ba000640 1 --2- 172.21.15.100:0/3752630215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b4142aa0 0x7f37b413ee30 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f37ac007e40 tx=0x7f37ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:37.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37b1600640 1 -- 172.21.15.100:0/3752630215 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac014620 con 0x7f37b4142aa0 2024-04-23T14:34:37.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37b1600640 1 -- 172.21.15.100:0/3752630215 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37ac019070 con 0x7f37b4142aa0 2024-04-23T14:34:37.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37bc000640 1 -- 172.21.15.100:0/3752630215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b40cf460 con 0x7f37b4142aa0 2024-04-23T14:34:37.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37b1600640 1 -- 172.21.15.100:0/3752630215 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac014a20 con 0x7f37b4142aa0 2024-04-23T14:34:37.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37bc000640 1 -- 172.21.15.100:0/3752630215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b406c530 con 0x7f37b4142aa0 2024-04-23T14:34:37.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37bc000640 1 -- 172.21.15.100:0/3752630215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37b406c730 con 0x7f37b4142aa0 2024-04-23T14:34:37.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37b1600640 1 -- 172.21.15.100:0/3752630215 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f37ac014bc0 con 0x7f37b4142aa0 2024-04-23T14:34:37.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.227+0000 7f37b1600640 1 --2- 172.21.15.100:0/3752630215 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f37940683b0 0x7f379406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:37.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.231+0000 7f37b1600640 1 -- 172.21.15.100:0/3752630215 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f37ac087310 con 0x7f37b4142aa0 2024-04-23T14:34:37.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.231+0000 7f37b9600640 1 --2- 172.21.15.100:0/3752630215 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f37940683b0 0x7f379406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:37.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.231+0000 7f37b9600640 1 --2- 172.21.15.100:0/3752630215 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f37940683b0 0x7f379406a870 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f37a80af000 tx=0x7f37a80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:37.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.231+0000 7f37b1600640 1 -- 172.21.15.100:0/3752630215 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f37b406c730 con 0x7f37b4142aa0 2024-04-23T14:34:37.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.399+0000 7f37bc000640 1 -- 172.21.15.100:0/3752630215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f37b40ce4b0 con 0x7f37b4142aa0 2024-04-23T14:34:37.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.399+0000 7f37b1600640 1 -- 172.21.15.100:0/3752630215 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) ==== 80+0+0 (secure 0 0 0) 0x7f37ac057050 con 0x7f37b4142aa0 2024-04-23T14:34:37.403 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-04-23T14:34:37.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.403+0000 7f37bc000640 1 -- 172.21.15.100:0/3752630215 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f37940683b0 msgr2=0x7f379406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:37.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.403+0000 7f37bc000640 1 --2- 172.21.15.100:0/3752630215 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f37940683b0 0x7f379406a870 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f37a80af000 tx=0x7f37a80a7000 comp rx=0 tx=0).stop 2024-04-23T14:34:37.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.403+0000 7f37bc000640 1 -- 172.21.15.100:0/3752630215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b4142aa0 msgr2=0x7f37b413ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:37.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.403+0000 7f37bc000640 1 --2- 172.21.15.100:0/3752630215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b4142aa0 0x7f37b413ee30 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f37ac007e40 tx=0x7f37ac00e4b0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.403+0000 7f37bc000640 1 -- 172.21.15.100:0/3752630215 shutdown_connections 2024-04-23T14:34:37.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.403+0000 7f37bc000640 1 --2- 172.21.15.100:0/3752630215 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f37940683b0 0x7f379406a870 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.403+0000 7f37bc000640 1 --2- 172.21.15.100:0/3752630215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b4142aa0 0x7f37b413ee30 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.403+0000 7f37bc000640 1 --2- 172.21.15.100:0/3752630215 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f37b415c160 0x7f37b40d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.403+0000 7f37bc000640 1 --2- 172.21.15.100:0/3752630215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37b4073cc0 0x7f37b415a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.403+0000 7f37bc000640 1 -- 172.21.15.100:0/3752630215 >> 172.21.15.100:0/3752630215 conn(0x7f37b4071590 msgr2=0x7f37b4072ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:37.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.403+0000 7f37bc000640 1 -- 172.21.15.100:0/3752630215 shutdown_connections 2024-04-23T14:34:37.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:37.403+0000 7f37bc000640 1 -- 172.21.15.100:0/3752630215 wait complete. 2024-04-23T14:34:37.424 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi100.front.sepia.ceph.com... 2024-04-23T14:34:37.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:37.424 DEBUG:teuthology.orchestra.run.smithi100:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:37.442 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[41994]: fuse finished with error 0 and tester_r 0 2024-04-23T14:34:37.444 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:34:37.491 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:34:37.494 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:34:43.545 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:34:43.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:43.546 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:34:43.556 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:34:43.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.743+0000 7fba90400640 1 Processor -- start 2024-04-23T14:34:43.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.743+0000 7fba90400640 1 -- start start 2024-04-23T14:34:43.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.747+0000 7fba90400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba8815c470 0x7fba880b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:43.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.747+0000 7fba90400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fba88160de0 0x7fba880ad160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:43.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.747+0000 7fba90400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fba880ad6a0 0x7fba8806bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:43.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.747+0000 7fba90400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fba880adad0 con 0x7fba880ad6a0 2024-04-23T14:34:43.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.747+0000 7fba90400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fba88160320 con 0x7fba88160de0 2024-04-23T14:34:43.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.747+0000 7fba90400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fba880cc7e0 con 0x7fba8815c470 2024-04-23T14:34:43.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.747+0000 7fba8e400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fba880ad6a0 0x7fba8806bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:43.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.747+0000 7fba8e400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fba880ad6a0 0x7fba8806bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35908/0 (socket says 172.21.15.100:35908) 2024-04-23T14:34:43.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.747+0000 7fba8e400640 1 -- 172.21.15.100:0/1025847625 learned_addr learned my addr 172.21.15.100:0/1025847625 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:43.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.747+0000 7fba8d000640 1 --2- 172.21.15.100:0/1025847625 >> v2:172.21.15.151:3300/0 conn(0x7fba88160de0 0x7fba880ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:43.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.747+0000 7fba8da00640 1 --2- 172.21.15.100:0/1025847625 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba8815c470 0x7fba880b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:43.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.747+0000 7fba8e400640 1 -- 172.21.15.100:0/1025847625 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba8815c470 msgr2=0x7fba880b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:43.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.747+0000 7fba8e400640 1 --2- 172.21.15.100:0/1025847625 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba8815c470 0x7fba880b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.747+0000 7fba8e400640 1 -- 172.21.15.100:0/1025847625 >> v2:172.21.15.151:3300/0 conn(0x7fba88160de0 msgr2=0x7fba880ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:43.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.747+0000 7fba8e400640 1 --2- 172.21.15.100:0/1025847625 >> v2:172.21.15.151:3300/0 conn(0x7fba88160de0 0x7fba880ad160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.747+0000 7fba8e400640 1 -- 172.21.15.100:0/1025847625 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba881605e0 con 0x7fba880ad6a0 2024-04-23T14:34:43.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba8e400640 1 --2- 172.21.15.100:0/1025847625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba880ad6a0 0x7fba8806bab0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fba88068700 tx=0x7fba8001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40159468112f6665 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:43.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba87e00640 1 -- 172.21.15.100:0/1025847625 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba800a56f0 con 0x7fba880ad6a0 2024-04-23T14:34:43.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba87e00640 1 -- 172.21.15.100:0/1025847625 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba800a5890 con 0x7fba880ad6a0 2024-04-23T14:34:43.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba87e00640 1 -- 172.21.15.100:0/1025847625 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba800a5c60 con 0x7fba880ad6a0 2024-04-23T14:34:43.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba90400640 1 -- 172.21.15.100:0/1025847625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba880ad6a0 msgr2=0x7fba8806bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:43.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba90400640 1 --2- 172.21.15.100:0/1025847625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba880ad6a0 0x7fba8806bab0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fba88068700 tx=0x7fba8001c960 comp rx=0 tx=0).stop 2024-04-23T14:34:43.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba90400640 1 -- 172.21.15.100:0/1025847625 shutdown_connections 2024-04-23T14:34:43.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba90400640 1 --2- 172.21.15.100:0/1025847625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba880ad6a0 0x7fba8806bab0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba90400640 1 --2- 172.21.15.100:0/1025847625 >> v2:172.21.15.151:3300/0 conn(0x7fba88160de0 0x7fba880ad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba90400640 1 --2- 172.21.15.100:0/1025847625 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba8815c470 0x7fba880b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba90400640 1 -- 172.21.15.100:0/1025847625 >> 172.21.15.100:0/1025847625 conn(0x7fba880b1e80 msgr2=0x7fba880b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:43.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba90400640 1 -- 172.21.15.100:0/1025847625 shutdown_connections 2024-04-23T14:34:43.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba90400640 1 -- 172.21.15.100:0/1025847625 wait complete. 2024-04-23T14:34:43.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba90400640 1 Processor -- start 2024-04-23T14:34:43.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba90400640 1 -- start start 2024-04-23T14:34:43.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba90400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba880ad6a0 0x7fba88099790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:43.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba90400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba8815c470 0x7fba88166730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:43.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba90400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba88160de0 0x7fba88143d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:43.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba90400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fba880cc370 con 0x7fba8815c470 2024-04-23T14:34:43.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba90400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fba8815caa0 con 0x7fba880ad6a0 2024-04-23T14:34:43.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba90400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fba880cc4f0 con 0x7fba88160de0 2024-04-23T14:34:43.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba8d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba8815c470 0x7fba88166730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:43.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba8d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba8815c470 0x7fba88166730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35914/0 (socket says 172.21.15.100:35914) 2024-04-23T14:34:43.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba8d000640 1 -- 172.21.15.100:0/1979759532 learned_addr learned my addr 172.21.15.100:0/1979759532 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:43.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.755+0000 7fba8e400640 1 --2- 172.21.15.100:0/1979759532 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba88160de0 0x7fba88143d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:43.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.751+0000 7fba8da00640 1 --2- 172.21.15.100:0/1979759532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba880ad6a0 0x7fba88099790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:43.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.755+0000 7fba8d000640 1 -- 172.21.15.100:0/1979759532 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba88160de0 msgr2=0x7fba88143d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:43.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.755+0000 7fba8d000640 1 --2- 172.21.15.100:0/1979759532 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba88160de0 0x7fba88143d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.755+0000 7fba8d000640 1 -- 172.21.15.100:0/1979759532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba880ad6a0 msgr2=0x7fba88099790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:43.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.755+0000 7fba8d000640 1 --2- 172.21.15.100:0/1979759532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba880ad6a0 0x7fba88099790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.755+0000 7fba8d000640 1 -- 172.21.15.100:0/1979759532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba880af640 con 0x7fba8815c470 2024-04-23T14:34:43.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.755+0000 7fba8d000640 1 --2- 172.21.15.100:0/1979759532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba8815c470 0x7fba88166730 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fba7400d800 tx=0x7fba7400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:43.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.755+0000 7fba85600640 1 -- 172.21.15.100:0/1979759532 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7400b6c0 con 0x7fba8815c470 2024-04-23T14:34:43.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.755+0000 7fba85600640 1 -- 172.21.15.100:0/1979759532 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba74019070 con 0x7fba8815c470 2024-04-23T14:34:43.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.755+0000 7fba90400640 1 -- 172.21.15.100:0/1979759532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba88099cd0 con 0x7fba8815c470 2024-04-23T14:34:43.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.755+0000 7fba85600640 1 -- 172.21.15.100:0/1979759532 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7400ba60 con 0x7fba8815c470 2024-04-23T14:34:43.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.755+0000 7fba90400640 1 -- 172.21.15.100:0/1979759532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba88143240 con 0x7fba8815c470 2024-04-23T14:34:43.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.755+0000 7fba90400640 1 -- 172.21.15.100:0/1979759532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba88068ad0 con 0x7fba8815c470 2024-04-23T14:34:43.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.755+0000 7fba85600640 1 -- 172.21.15.100:0/1979759532 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fba7400bc00 con 0x7fba8815c470 2024-04-23T14:34:43.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.759+0000 7fba85600640 1 --2- 172.21.15.100:0/1979759532 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fba68068380 0x7fba6806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:43.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.759+0000 7fba85600640 1 -- 172.21.15.100:0/1979759532 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fba74086f20 con 0x7fba8815c470 2024-04-23T14:34:43.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.759+0000 7fba8da00640 1 --2- 172.21.15.100:0/1979759532 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fba68068380 0x7fba6806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:43.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.759+0000 7fba8da00640 1 --2- 172.21.15.100:0/1979759532 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fba68068380 0x7fba6806a840 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fba881588f0 tx=0x7fba7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:43.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.763+0000 7fba85600640 1 -- 172.21.15.100:0/1979759532 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fba88099cd0 con 0x7fba8815c470 2024-04-23T14:34:43.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.955+0000 7fba90400640 1 -- 172.21.15.100:0/1979759532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fba880b5460 con 0x7fba8815c470 2024-04-23T14:34:43.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.955+0000 7fba85600640 1 -- 172.21.15.100:0/1979759532 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+73 (secure 0 0 0) 0x7fba74056c60 con 0x7fba8815c470 2024-04-23T14:34:43.958 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:34:43.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.959+0000 7fba90400640 1 -- 172.21.15.100:0/1979759532 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fba68068380 msgr2=0x7fba6806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:43.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.959+0000 7fba90400640 1 --2- 172.21.15.100:0/1979759532 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fba68068380 0x7fba6806a840 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fba881588f0 tx=0x7fba7c008040 comp rx=0 tx=0).stop 2024-04-23T14:34:43.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.959+0000 7fba90400640 1 -- 172.21.15.100:0/1979759532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba8815c470 msgr2=0x7fba88166730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:43.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.959+0000 7fba90400640 1 --2- 172.21.15.100:0/1979759532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba8815c470 0x7fba88166730 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fba7400d800 tx=0x7fba7400dcc0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.959+0000 7fba90400640 1 -- 172.21.15.100:0/1979759532 shutdown_connections 2024-04-23T14:34:43.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.959+0000 7fba90400640 1 --2- 172.21.15.100:0/1979759532 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fba68068380 0x7fba6806a840 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.959+0000 7fba90400640 1 --2- 172.21.15.100:0/1979759532 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba88160de0 0x7fba88143d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.959+0000 7fba90400640 1 --2- 172.21.15.100:0/1979759532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba8815c470 0x7fba88166730 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.959+0000 7fba90400640 1 --2- 172.21.15.100:0/1979759532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba880ad6a0 0x7fba88099790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.959+0000 7fba90400640 1 -- 172.21.15.100:0/1979759532 >> 172.21.15.100:0/1979759532 conn(0x7fba880b1e80 msgr2=0x7fba8815a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:43.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.959+0000 7fba90400640 1 -- 172.21.15.100:0/1979759532 shutdown_connections 2024-04-23T14:34:43.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:43.959+0000 7fba90400640 1 -- 172.21.15.100:0/1979759532 wait complete. 2024-04-23T14:34:43.980 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:34:43.980 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:34:43.980 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-04-23T14:34:43.980 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:34:43.980 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:43.980 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:34:43.980 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:34:43.980 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:34:43.980 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:34:43.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:43.981 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-04-23T14:34:43.989 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:34:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:34:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:34:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-23T14:34:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:43.990 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:34:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-04-23T14:34:43.990 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:34:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-04-23T14:34:43.991 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:34:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:43.991 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:34:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-23T14:34:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42300sec preferred_lft 42300sec 2024-04-23T14:34:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-04-23T14:34:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:43.991 INFO:teuthology.orchestra.run.smithi100.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-23T14:34:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 02:42:ff:fc:d1:cc brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-23T14:34:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:43.991 INFO:teuthology.orchestra.run.smithi100.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:34:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:34:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-23T14:34:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:43.992 INFO:teuthology.orchestra.run.smithi100.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:34:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:34:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-23T14:34:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:43.992 INFO:teuthology.orchestra.run.smithi100.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:34:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:34:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-23T14:34:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:43.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:43.993 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:34:44.052 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-04-23T14:34:44.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:44.054 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-04-23T14:34:44.110 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-04-23T14:34:44.110 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:34:44.111 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-23T14:34:44.111 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-23T14:34:44.111 INFO:teuthology.orchestra.run.smithi100.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-23T14:34:44.111 INFO:teuthology.orchestra.run.smithi100.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-23T14:34:44.111 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-23T14:34:44.111 INFO:teuthology.orchestra.run.smithi100.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:34:44.111 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:34:44.111 INFO:teuthology.orchestra.run.smithi100.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:34:44.111 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:34:44.111 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:34:44.111 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-23T14:34:44.112 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-23T14:34:44.112 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-23T14:34:44.112 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-23T14:34:44.112 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-23T14:34:44.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:34:44.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:34:44.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:44.247 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-23T14:34:44.256 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:34:44.256 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:34:44.256 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:34:44.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:44.256 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-04-23T14:34:44.317 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:34:44.317 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:34:44.317 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:44.366 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:34:44.366 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:34:44.366 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:44.414 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-23T14:34:44.474 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:34:44.523 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-23T14:34:44.523 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-23T14:34:44.523 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-23T14:34:44.524 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-23T14:34:44.524 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-23T14:34:44.524 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-23T14:34:44.524 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-23T14:34:44.524 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-23T14:34:44.524 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-23T14:34:44.524 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-23T14:34:44.524 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-23T14:34:44.524 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-23T14:34:44.524 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:34:44.524 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-23T14:34:44.524 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-23T14:34:44.524 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-23T14:34:44.524 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-23T14:34:44.524 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-23T14:34:44.525 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-23T14:34:44.525 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-23T14:34:44.525 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:34:44.525 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-23T14:34:44.525 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-23T14:34:44.525 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-23T14:34:44.525 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:34:44.525 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:34:44.525 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:34:44.525 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:34:44.525 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-23T14:34:44.525 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:34:44.525 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:34:44.525 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:34:44.525 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:34:44.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:44.526 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-23T14:34:44.578 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:34:44.578 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:34:44.620 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-23T14:34:44.640 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:34:44.692 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-23T14:34:44.692 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-23T14:34:44.692 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-23T14:34:44.692 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-23T14:34:44.692 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-23T14:34:44.692 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-23T14:34:44.692 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-23T14:34:44.692 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-23T14:34:44.692 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-23T14:34:44.692 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-23T14:34:44.692 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-23T14:34:44.692 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-23T14:34:44.692 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:34:44.693 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-23T14:34:44.693 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-23T14:34:44.693 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-23T14:34:44.693 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-23T14:34:44.693 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-23T14:34:44.693 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-23T14:34:44.693 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-23T14:34:44.693 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:34:44.693 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-23T14:34:44.693 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-23T14:34:44.693 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-23T14:34:44.693 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:34:44.693 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:34:44.694 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:34:44.694 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:34:44.694 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-23T14:34:44.694 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:34:44.694 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:34:44.694 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:34:44.694 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:34:44.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:44.695 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-23T14:34:44.713 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:2024-04-23T14:34:44.711+0000 7f25edb23580 -1 init, newargv = 0x5575e9c29e40 newargc=15 2024-04-23T14:34:44.713 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[42401]: starting ceph client 2024-04-23T14:34:44.720 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[42401]: starting fuse 2024-04-23T14:34:44.722 INFO:teuthology.orchestra.run.smithi100.stdout:44 2024-04-23T14:34:44.722 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-23T14:34:44.722 DEBUG:teuthology.orchestra.run.smithi100:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:34:44.723 DEBUG:teuthology.orchestra.run.smithi100:> import glob 2024-04-23T14:34:44.723 DEBUG:teuthology.orchestra.run.smithi100:> import re 2024-04-23T14:34:44.723 DEBUG:teuthology.orchestra.run.smithi100:> import os 2024-04-23T14:34:44.723 DEBUG:teuthology.orchestra.run.smithi100:> import subprocess 2024-04-23T14:34:44.723 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:34:44.723 DEBUG:teuthology.orchestra.run.smithi100:> def _find_admin_socket(client_name): 2024-04-23T14:34:44.723 DEBUG:teuthology.orchestra.run.smithi100:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:34:44.723 DEBUG:teuthology.orchestra.run.smithi100:> files = glob.glob(asok_path) 2024-04-23T14:34:44.723 DEBUG:teuthology.orchestra.run.smithi100:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:34:44.723 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:34:44.723 DEBUG:teuthology.orchestra.run.smithi100:> # Given a non-glob path, it better be there 2024-04-23T14:34:44.723 DEBUG:teuthology.orchestra.run.smithi100:> if "*" not in asok_path: 2024-04-23T14:34:44.724 DEBUG:teuthology.orchestra.run.smithi100:> assert(len(files) == 1) 2024-04-23T14:34:44.724 DEBUG:teuthology.orchestra.run.smithi100:> return files[0] 2024-04-23T14:34:44.724 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:34:44.724 DEBUG:teuthology.orchestra.run.smithi100:> for f in files: 2024-04-23T14:34:44.724 DEBUG:teuthology.orchestra.run.smithi100:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:34:44.724 DEBUG:teuthology.orchestra.run.smithi100:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:34:44.724 DEBUG:teuthology.orchestra.run.smithi100:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:34:44.724 DEBUG:teuthology.orchestra.run.smithi100:> contents = proc_f.read() 2024-04-23T14:34:44.724 DEBUG:teuthology.orchestra.run.smithi100:> if mountpoint in contents: 2024-04-23T14:34:44.724 DEBUG:teuthology.orchestra.run.smithi100:> return f 2024-04-23T14:34:44.724 DEBUG:teuthology.orchestra.run.smithi100:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:34:44.724 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:34:44.725 DEBUG:teuthology.orchestra.run.smithi100:> print(_find_admin_socket("client.0")) 2024-04-23T14:34:44.725 DEBUG:teuthology.orchestra.run.smithi100:> ' 2024-04-23T14:34:44.846 INFO:teuthology.orchestra.run.smithi100.stdout:/var/run/ceph/ceph-client.0.42401.asok 2024-04-23T14:34:44.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:34:44.857 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42401.asok 2024-04-23T14:34:44.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:44.858 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42401.asok status 2024-04-23T14:34:45.026 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:34:45.026 INFO:teuthology.orchestra.run.smithi100.stdout: "metadata": { 2024-04-23T14:34:45.026 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:34:45.026 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:34:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "entity_id": "0", 2024-04-23T14:34:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": "smithi100", 2024-04-23T14:34:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:34:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "pid": "42401", 2024-04-23T14:34:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "root": "/" 2024-04-23T14:34:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_count": 0, 2024-04-23T14:34:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_pinned_count": 0, 2024-04-23T14:34:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 6444, 2024-04-23T14:34:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "inst": { 2024-04-23T14:34:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "name": { 2024-04-23T14:34:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "client", 2024-04-23T14:34:45.027 INFO:teuthology.orchestra.run.smithi100.stdout: "num": 6444 2024-04-23T14:34:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-23T14:34:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-23T14:34:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:34:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3624311880 2024-04-23T14:34:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-23T14:34:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-23T14:34:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:34:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3624311880 2024-04-23T14:34:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: "inst_str": "client.6444 192.168.0.1:0/3624311880", 2024-04-23T14:34:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: "addr_str": "192.168.0.1:0/3624311880", 2024-04-23T14:34:45.028 INFO:teuthology.orchestra.run.smithi100.stdout: "inode_count": 1, 2024-04-23T14:34:45.029 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_epoch": 28, 2024-04-23T14:34:45.029 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch": 53, 2024-04-23T14:34:45.029 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:34:45.029 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklisted": false, 2024-04-23T14:34:45.029 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs" 2024-04-23T14:34:45.029 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:34:45.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:45.041 DEBUG:teuthology.orchestra.run.smithi100:> stat --file-system '--printf=%T 2024-04-23T14:34:45.041 DEBUG:teuthology.orchestra.run.smithi100:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:45.051 INFO:teuthology.orchestra.run.smithi100.stdout:fuseblk 2024-04-23T14:34:45.051 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:45.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:45.051 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:45.113 DEBUG:teuthology.orchestra.run.smithi100:> stdin-killer -- python3 -c ' 2024-04-23T14:34:45.113 DEBUG:teuthology.orchestra.run.smithi100:> import os 2024-04-23T14:34:45.113 DEBUG:teuthology.orchestra.run.smithi100:> import errno 2024-04-23T14:34:45.113 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:34:45.113 DEBUG:teuthology.orchestra.run.smithi100:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2024-04-23T14:34:45.113 DEBUG:teuthology.orchestra.run.smithi100:> try: 2024-04-23T14:34:45.113 DEBUG:teuthology.orchestra.run.smithi100:> if False: 2024-04-23T14:34:45.113 DEBUG:teuthology.orchestra.run.smithi100:> ret = os.read(fd, 1024) 2024-04-23T14:34:45.113 DEBUG:teuthology.orchestra.run.smithi100:> else: 2024-04-23T14:34:45.113 DEBUG:teuthology.orchestra.run.smithi100:> os.write(fd, b'"'"'content'"'"') 2024-04-23T14:34:45.114 DEBUG:teuthology.orchestra.run.smithi100:> except OSError as e: 2024-04-23T14:34:45.114 DEBUG:teuthology.orchestra.run.smithi100:> if e.errno != errno.EPERM: 2024-04-23T14:34:45.114 DEBUG:teuthology.orchestra.run.smithi100:> raise 2024-04-23T14:34:45.114 DEBUG:teuthology.orchestra.run.smithi100:> else: 2024-04-23T14:34:45.114 DEBUG:teuthology.orchestra.run.smithi100:> raise RuntimeError("client does not check permission of data pool") 2024-04-23T14:34:45.114 DEBUG:teuthology.orchestra.run.smithi100:> ' 2024-04-23T14:34:45.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:34:45.230 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow w pool=cephfs_data' 2024-04-23T14:34:45.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.415+0000 7fb36b800640 1 Processor -- start 2024-04-23T14:34:45.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.415+0000 7fb36b800640 1 -- start start 2024-04-23T14:34:45.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.415+0000 7fb36b800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb36415c7a0 0x7fb3640af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.415+0000 7fb36b800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb3640cd300 0x7fb3640ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.415+0000 7fb36b800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb36406cbb0 con 0x7fb36406bb20 2024-04-23T14:34:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.415+0000 7fb36b800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fb36406d450 con 0x7fb3640cd300 2024-04-23T14:34:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.415+0000 7fb36b800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb36406ce90 con 0x7fb36415c7a0 2024-04-23T14:34:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.415+0000 7fb369800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb3640cd300 0x7fb3640ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.415+0000 7fb369800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb3640cd300 0x7fb3640ad430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35926/0 (socket says 172.21.15.100:35926) 2024-04-23T14:34:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.415+0000 7fb369800640 1 -- 172.21.15.100:0/4280761255 learned_addr learned my addr 172.21.15.100:0/4280761255 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.415+0000 7fb368e00640 1 --2- 172.21.15.100:0/4280761255 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb36415c7a0 0x7fb3640af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.415+0000 7fb369800640 1 -- 172.21.15.100:0/4280761255 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb36415c7a0 msgr2=0x7fb3640af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.415+0000 7fb369800640 1 --2- 172.21.15.100:0/4280761255 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb36415c7a0 0x7fb3640af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.415+0000 7fb369800640 1 -- 172.21.15.100:0/4280761255 >> v1:172.21.15.151:6789/0 conn(0x7fb36406bb20 legacy=0x7fb3640ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:45.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.415+0000 7fb369800640 1 -- 172.21.15.100:0/4280761255 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36406d170 con 0x7fb3640cd300 2024-04-23T14:34:45.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.419+0000 7fb369800640 1 --2- 172.21.15.100:0/4280761255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3640cd300 0x7fb3640ad430 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009e50 tx=0x7fb35c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5382b22504f1c9f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:45.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.419+0000 7fb363400640 1 -- 172.21.15.100:0/4280761255 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c0a5660 con 0x7fb3640cd300 2024-04-23T14:34:45.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.419+0000 7fb363400640 1 -- 172.21.15.100:0/4280761255 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb35c0a5800 con 0x7fb3640cd300 2024-04-23T14:34:45.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.419+0000 7fb363400640 1 -- 172.21.15.100:0/4280761255 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c0a5bd0 con 0x7fb3640cd300 2024-04-23T14:34:45.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.419+0000 7fb36b800640 1 -- 172.21.15.100:0/4280761255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3640cd300 msgr2=0x7fb3640ad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:45.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.419+0000 7fb36b800640 1 --2- 172.21.15.100:0/4280761255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3640cd300 0x7fb3640ad430 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009e50 tx=0x7fb35c01c9a0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.419+0000 7fb36b800640 1 -- 172.21.15.100:0/4280761255 shutdown_connections 2024-04-23T14:34:45.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.419+0000 7fb36b800640 1 --2- 172.21.15.100:0/4280761255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3640cd300 0x7fb3640ad430 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.419+0000 7fb36b800640 1 --2- 172.21.15.100:0/4280761255 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb36415c7a0 0x7fb3640af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.419+0000 7fb36b800640 1 -- 172.21.15.100:0/4280761255 >> 172.21.15.100:0/4280761255 conn(0x7fb3640b5200 msgr2=0x7fb3640b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:45.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.419+0000 7fb36b800640 1 -- 172.21.15.100:0/4280761255 shutdown_connections 2024-04-23T14:34:45.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.419+0000 7fb36b800640 1 -- 172.21.15.100:0/4280761255 wait complete. 2024-04-23T14:34:45.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.419+0000 7fb36b800640 1 Processor -- start 2024-04-23T14:34:45.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.419+0000 7fb36b800640 1 -- start start 2024-04-23T14:34:45.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb36b800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb36406bb20 0x7fb3640992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:45.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb36b800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb3640cd300 0x7fb3640cf630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:45.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb36b800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb36415c7a0 0x7fb3641552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:45.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb36b800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb3640ae550 con 0x7fb36406bb20 2024-04-23T14:34:45.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb36b800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb3640ae3d0 con 0x7fb36415c7a0 2024-04-23T14:34:45.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb36b800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb3640ae6d0 con 0x7fb3640cd300 2024-04-23T14:34:45.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb368e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb36406bb20 0x7fb3640992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:45.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb368e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb36406bb20 0x7fb3640992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35938/0 (socket says 172.21.15.100:35938) 2024-04-23T14:34:45.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb368e00640 1 -- 172.21.15.100:0/2174609052 learned_addr learned my addr 172.21.15.100:0/2174609052 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:45.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb369800640 1 --2- 172.21.15.100:0/2174609052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb36415c7a0 0x7fb3641552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:45.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb363e00640 1 --2- 172.21.15.100:0/2174609052 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb3640cd300 0x7fb3640cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:45.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb368e00640 1 -- 172.21.15.100:0/2174609052 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb3640cd300 msgr2=0x7fb3640cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:45.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb368e00640 1 --2- 172.21.15.100:0/2174609052 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb3640cd300 0x7fb3640cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb368e00640 1 -- 172.21.15.100:0/2174609052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb36415c7a0 msgr2=0x7fb3641552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:45.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb368e00640 1 --2- 172.21.15.100:0/2174609052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb36415c7a0 0x7fb3641552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb368e00640 1 -- 172.21.15.100:0/2174609052 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb364069370 con 0x7fb36406bb20 2024-04-23T14:34:45.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb368e00640 1 --2- 172.21.15.100:0/2174609052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb36406bb20 0x7fb3640992b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb35800d800 tx=0x7fb35800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:45.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb360c00640 1 -- 172.21.15.100:0/2174609052 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35800b6c0 con 0x7fb36406bb20 2024-04-23T14:34:45.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb360c00640 1 -- 172.21.15.100:0/2174609052 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb358019070 con 0x7fb36406bb20 2024-04-23T14:34:45.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb36b800640 1 -- 172.21.15.100:0/2174609052 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb364154230 con 0x7fb36406bb20 2024-04-23T14:34:45.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb360c00640 1 -- 172.21.15.100:0/2174609052 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35800bae0 con 0x7fb36406bb20 2024-04-23T14:34:45.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb36b800640 1 -- 172.21.15.100:0/2174609052 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3641547f0 con 0x7fb36406bb20 2024-04-23T14:34:45.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb36b800640 1 -- 172.21.15.100:0/2174609052 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb364068ad0 con 0x7fb36406bb20 2024-04-23T14:34:45.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.423+0000 7fb360c00640 1 -- 172.21.15.100:0/2174609052 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb35800bc80 con 0x7fb36406bb20 2024-04-23T14:34:45.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.427+0000 7fb360c00640 1 --2- 172.21.15.100:0/2174609052 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb3440683b0 0x7fb34406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:45.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.427+0000 7fb360c00640 1 -- 172.21.15.100:0/2174609052 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fb358087090 con 0x7fb36406bb20 2024-04-23T14:34:45.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.427+0000 7fb363e00640 1 --2- 172.21.15.100:0/2174609052 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb3440683b0 0x7fb34406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:45.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.427+0000 7fb363e00640 1 --2- 172.21.15.100:0/2174609052 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb3440683b0 0x7fb34406a870 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb3640cbf80 tx=0x7fb350009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:45.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.431+0000 7fb360c00640 1 -- 172.21.15.100:0/2174609052 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb358057420 con 0x7fb36406bb20 2024-04-23T14:34:45.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.643+0000 7fb36b800640 1 -- 172.21.15.100:0/2174609052 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]} v 0) -- 0x7fb3640ca2e0 con 0x7fb36406bb20 2024-04-23T14:34:45.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.647+0000 7fb360c00640 1 -- 172.21.15.100:0/2174609052 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]}]=0 updated caps for client.0 v17) ==== 183+0+140 (secure 0 0 0) 0x7fb358056d20 con 0x7fb36406bb20 2024-04-23T14:34:45.647 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-04-23T14:34:45.648 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAtxidmyhImMBAAa49ApQEF2Lng6rY94F1HAQ== 2024-04-23T14:34:45.648 INFO:teuthology.orchestra.run.smithi100.stdout: caps mds = "allow" 2024-04-23T14:34:45.648 INFO:teuthology.orchestra.run.smithi100.stdout: caps mon = "allow r" 2024-04-23T14:34:45.648 INFO:teuthology.orchestra.run.smithi100.stdout: caps osd = "allow w pool=cephfs_data" 2024-04-23T14:34:45.648 INFO:teuthology.orchestra.run.smithi100.stderr:updated caps for client.0 2024-04-23T14:34:45.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.647+0000 7fb36b800640 1 -- 172.21.15.100:0/2174609052 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb3440683b0 msgr2=0x7fb34406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:45.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.647+0000 7fb36b800640 1 --2- 172.21.15.100:0/2174609052 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb3440683b0 0x7fb34406a870 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb3640cbf80 tx=0x7fb350009040 comp rx=0 tx=0).stop 2024-04-23T14:34:45.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.647+0000 7fb36b800640 1 -- 172.21.15.100:0/2174609052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb36406bb20 msgr2=0x7fb3640992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:45.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.647+0000 7fb36b800640 1 --2- 172.21.15.100:0/2174609052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb36406bb20 0x7fb3640992b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb35800d800 tx=0x7fb35800dcc0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.647+0000 7fb36b800640 1 -- 172.21.15.100:0/2174609052 shutdown_connections 2024-04-23T14:34:45.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.647+0000 7fb36b800640 1 --2- 172.21.15.100:0/2174609052 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb3440683b0 0x7fb34406a870 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.647+0000 7fb36b800640 1 --2- 172.21.15.100:0/2174609052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb36415c7a0 0x7fb3641552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.647+0000 7fb36b800640 1 --2- 172.21.15.100:0/2174609052 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb3640cd300 0x7fb3640cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.647+0000 7fb36b800640 1 --2- 172.21.15.100:0/2174609052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb36406bb20 0x7fb3640992b0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.647+0000 7fb36b800640 1 -- 172.21.15.100:0/2174609052 >> 172.21.15.100:0/2174609052 conn(0x7fb3640b5200 msgr2=0x7fb364166950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:45.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.647+0000 7fb36b800640 1 -- 172.21.15.100:0/2174609052 shutdown_connections 2024-04-23T14:34:45.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.647+0000 7fb36b800640 1 -- 172.21.15.100:0/2174609052 wait complete. 2024-04-23T14:34:45.669 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:34:45.669 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:45.678 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:45.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.851+0000 7ff1b1c00640 1 Processor -- start 2024-04-23T14:34:45.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.851+0000 7ff1b1c00640 1 -- start start 2024-04-23T14:34:45.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.851+0000 7ff1b1c00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff1ac06d350 0x7ff1ac073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:45.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.851+0000 7ff1b1c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff1ac06ef80 0x7ff1ac06f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:45.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.851+0000 7ff1b1c00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1ac073880 con 0x7ff1ac15fa90 2024-04-23T14:34:45.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.851+0000 7ff1b1c00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff1ac06eaf0 con 0x7ff1ac06ef80 2024-04-23T14:34:45.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.851+0000 7ff1b1c00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7ff1ac06e530 con 0x7ff1ac06d350 2024-04-23T14:34:45.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.855+0000 7ff1ab400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff1ac06ef80 0x7ff1ac06f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:45.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.855+0000 7ff1ab400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff1ac06ef80 0x7ff1ac06f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35940/0 (socket says 172.21.15.100:35940) 2024-04-23T14:34:45.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.855+0000 7ff1ab400640 1 -- 172.21.15.100:0/164850098 learned_addr learned my addr 172.21.15.100:0/164850098 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:45.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.855+0000 7ff1aaa00640 1 --2- 172.21.15.100:0/164850098 >> v2:172.21.15.151:3300/0 conn(0x7ff1ac06d350 0x7ff1ac073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:45.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.855+0000 7ff1ab400640 1 -- 172.21.15.100:0/164850098 >> v2:172.21.15.151:3300/0 conn(0x7ff1ac06d350 msgr2=0x7ff1ac073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:45.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.855+0000 7ff1ab400640 1 --2- 172.21.15.100:0/164850098 >> v2:172.21.15.151:3300/0 conn(0x7ff1ac06d350 0x7ff1ac073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.855+0000 7ff1ab400640 1 -- 172.21.15.100:0/164850098 >> v1:172.21.15.151:6789/0 conn(0x7ff1ac15fa90 legacy=0x7ff1ac06d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:45.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.855+0000 7ff1ab400640 1 -- 172.21.15.100:0/164850098 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ac0b47f0 con 0x7ff1ac06ef80 2024-04-23T14:34:45.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.855+0000 7ff1ab400640 1 --2- 172.21.15.100:0/164850098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1ac06ef80 0x7ff1ac06f430 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0b5960 tx=0x7ff1a401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123d6e2e3523b7a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:45.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.855+0000 7ff1aa000640 1 -- 172.21.15.100:0/164850098 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a40a57f0 con 0x7ff1ac06ef80 2024-04-23T14:34:45.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.855+0000 7ff1aa000640 1 -- 172.21.15.100:0/164850098 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a40a5990 con 0x7ff1ac06ef80 2024-04-23T14:34:45.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.855+0000 7ff1aa000640 1 -- 172.21.15.100:0/164850098 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a40a5d60 con 0x7ff1ac06ef80 2024-04-23T14:34:45.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1b1c00640 1 -- 172.21.15.100:0/164850098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1ac06ef80 msgr2=0x7ff1ac06f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:45.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1b1c00640 1 --2- 172.21.15.100:0/164850098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1ac06ef80 0x7ff1ac06f430 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0b5960 tx=0x7ff1a401c930 comp rx=0 tx=0).stop 2024-04-23T14:34:45.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1b1c00640 1 -- 172.21.15.100:0/164850098 shutdown_connections 2024-04-23T14:34:45.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1b1c00640 1 --2- 172.21.15.100:0/164850098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1ac06ef80 0x7ff1ac06f430 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1b1c00640 1 --2- 172.21.15.100:0/164850098 >> v2:172.21.15.151:3300/0 conn(0x7ff1ac06d350 0x7ff1ac073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1b1c00640 1 -- 172.21.15.100:0/164850098 >> 172.21.15.100:0/164850098 conn(0x7ff1ac072600 msgr2=0x7ff1ac072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:45.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1b1c00640 1 -- 172.21.15.100:0/164850098 shutdown_connections 2024-04-23T14:34:45.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1b1c00640 1 -- 172.21.15.100:0/164850098 wait complete. 2024-04-23T14:34:45.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1b1c00640 1 Processor -- start 2024-04-23T14:34:45.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1b1c00640 1 -- start start 2024-04-23T14:34:45.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1b1c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1ac06d350 0x7ff1ac157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:45.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1b1c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff1ac06ef80 0x7ff1ac157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:45.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1b1c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1ac15fa90 0x7ff1ac150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:45.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1b1c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac16bb10 con 0x7ff1ac06d350 2024-04-23T14:34:45.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1b1c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac16b990 con 0x7ff1ac15fa90 2024-04-23T14:34:45.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1b1c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff1ac16bc90 con 0x7ff1ac06ef80 2024-04-23T14:34:45.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1aaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1ac06d350 0x7ff1ac157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:45.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1aaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1ac06d350 0x7ff1ac157020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35952/0 (socket says 172.21.15.100:35952) 2024-04-23T14:34:45.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1aaa00640 1 -- 172.21.15.100:0/1888799645 learned_addr learned my addr 172.21.15.100:0/1888799645 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:45.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1ab400640 1 --2- 172.21.15.100:0/1888799645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1ac15fa90 0x7ff1ac150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:45.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1a2000640 1 --2- 172.21.15.100:0/1888799645 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff1ac06ef80 0x7ff1ac157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:45.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1aaa00640 1 -- 172.21.15.100:0/1888799645 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff1ac06ef80 msgr2=0x7ff1ac157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:45.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1aaa00640 1 --2- 172.21.15.100:0/1888799645 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff1ac06ef80 0x7ff1ac157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1aaa00640 1 -- 172.21.15.100:0/1888799645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1ac15fa90 msgr2=0x7ff1ac150500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:45.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1aaa00640 1 --2- 172.21.15.100:0/1888799645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1ac15fa90 0x7ff1ac150500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.859+0000 7ff1aaa00640 1 -- 172.21.15.100:0/1888799645 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ac151fd0 con 0x7ff1ac06d350 2024-04-23T14:34:45.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.863+0000 7ff1aaa00640 1 --2- 172.21.15.100:0/1888799645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1ac06d350 0x7ff1ac157020 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00d5f0 tx=0x7ff19c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:45.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.863+0000 7ff1a3400640 1 -- 172.21.15.100:0/1888799645 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c0125f0 con 0x7ff1ac06d350 2024-04-23T14:34:45.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.863+0000 7ff1a3400640 1 -- 172.21.15.100:0/1888799645 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff19c019070 con 0x7ff1ac06d350 2024-04-23T14:34:45.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.863+0000 7ff1a3400640 1 -- 172.21.15.100:0/1888799645 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c0129b0 con 0x7ff1ac06d350 2024-04-23T14:34:45.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.863+0000 7ff1b1c00640 1 -- 172.21.15.100:0/1888799645 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ac156260 con 0x7ff1ac06d350 2024-04-23T14:34:45.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.863+0000 7ff1b1c00640 1 -- 172.21.15.100:0/1888799645 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ac156820 con 0x7ff1ac06d350 2024-04-23T14:34:45.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.863+0000 7ff1b1c00640 1 -- 172.21.15.100:0/1888799645 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1ac068ad0 con 0x7ff1ac06d350 2024-04-23T14:34:45.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.863+0000 7ff1a3400640 1 -- 172.21.15.100:0/1888799645 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff19c012b50 con 0x7ff1ac06d350 2024-04-23T14:34:45.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.863+0000 7ff1a3400640 1 --2- 172.21.15.100:0/1888799645 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff188068380 0x7ff18806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:45.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.863+0000 7ff1a3400640 1 -- 172.21.15.100:0/1888799645 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7ff19c086dd0 con 0x7ff1ac06d350 2024-04-23T14:34:45.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.863+0000 7ff1a2000640 1 --2- 172.21.15.100:0/1888799645 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff188068380 0x7ff18806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:45.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.867+0000 7ff1a2000640 1 --2- 172.21.15.100:0/1888799645 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff188068380 0x7ff18806a840 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac157b00 tx=0x7ff194009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:45.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:45.867+0000 7ff1a3400640 1 -- 172.21.15.100:0/1888799645 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff19c056c20 con 0x7ff1ac06d350 2024-04-23T14:34:46.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.027+0000 7ff1b1c00640 1 -- 172.21.15.100:0/1888799645 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1ac073120 con 0x7ff1ac06d350 2024-04-23T14:34:46.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.031+0000 7ff1a3400640 1 -- 172.21.15.100:0/1888799645 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16950 (secure 0 0 0) 0x7ff19c088090 con 0x7ff1ac06d350 2024-04-23T14:34:46.031 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:46.031 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":53,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:34:25.795515+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:34:22.491379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-23T14:34:23.120342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:46.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.031+0000 7ff1b1c00640 1 -- 172.21.15.100:0/1888799645 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff188068380 msgr2=0x7ff18806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.031+0000 7ff1b1c00640 1 --2- 172.21.15.100:0/1888799645 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff188068380 0x7ff18806a840 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac157b00 tx=0x7ff194009040 comp rx=0 tx=0).stop 2024-04-23T14:34:46.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.031+0000 7ff1b1c00640 1 -- 172.21.15.100:0/1888799645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1ac06d350 msgr2=0x7ff1ac157020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.031+0000 7ff1b1c00640 1 --2- 172.21.15.100:0/1888799645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1ac06d350 0x7ff1ac157020 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00d5f0 tx=0x7ff19c00dab0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.031+0000 7ff1b1c00640 1 -- 172.21.15.100:0/1888799645 shutdown_connections 2024-04-23T14:34:46.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.031+0000 7ff1b1c00640 1 --2- 172.21.15.100:0/1888799645 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff188068380 0x7ff18806a840 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.031+0000 7ff1b1c00640 1 --2- 172.21.15.100:0/1888799645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1ac15fa90 0x7ff1ac150500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.031+0000 7ff1b1c00640 1 --2- 172.21.15.100:0/1888799645 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff1ac06ef80 0x7ff1ac157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.031+0000 7ff1b1c00640 1 --2- 172.21.15.100:0/1888799645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1ac06d350 0x7ff1ac157020 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.031+0000 7ff1b1c00640 1 -- 172.21.15.100:0/1888799645 >> 172.21.15.100:0/1888799645 conn(0x7ff1ac072600 msgr2=0x7ff1ac0b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:46.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.031+0000 7ff1b1c00640 1 -- 172.21.15.100:0/1888799645 shutdown_connections 2024-04-23T14:34:46.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.031+0000 7ff1b1c00640 1 -- 172.21.15.100:0/1888799645 wait complete. 2024-04-23T14:34:46.052 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:34:22.491379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:34:23.120342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:34:46.053 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:34:46.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.187+0000 7f23f9600640 1 Processor -- start 2024-04-23T14:34:46.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.187+0000 7f23f9600640 1 -- start start 2024-04-23T14:34:46.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.191+0000 7f23f9600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f23f415c7a0 0x7f23f40af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.191+0000 7f23f9600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f23f406bb20 0x7f23f40ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.191+0000 7f23f9600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23f406cbb0 con 0x7f23f40cd130 2024-04-23T14:34:46.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.191+0000 7f23f9600640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f23f40cd5a0 con 0x7f23f415c7a0 2024-04-23T14:34:46.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.191+0000 7f23f9600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f23f406d450 con 0x7f23f406bb20 2024-04-23T14:34:46.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.191+0000 7f23f2a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f23f415c7a0 0x7f23f40af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.191+0000 7f23f3400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f23f40cd130 0x7f23f406e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:59862/0 (socket says 172.21.15.100:59862) 2024-04-23T14:34:46.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.191+0000 7f23f3400640 1 -- 172.21.15.100:0/3670989568 learned_addr learned my addr 172.21.15.100:0/3670989568 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:46.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.191+0000 7f23ea000640 1 --2- 172.21.15.100:0/3670989568 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f23f406bb20 0x7f23f40ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.191+0000 7f23f2000640 1 -- 172.21.15.100:0/3670989568 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2271164152 0 0) 0x7f23f406cbb0 con 0x7f23f40cd130 2024-04-23T14:34:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.191+0000 7f23f2000640 1 -- 172.21.15.100:0/3670989568 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23d8003530 con 0x7f23f40cd130 2024-04-23T14:34:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.191+0000 7f23f2a00640 1 -- 172.21.15.100:0/3670989568 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f23f406bb20 msgr2=0x7f23f40ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.191+0000 7f23f2a00640 1 --2- 172.21.15.100:0/3670989568 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f23f406bb20 0x7f23f40ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.191+0000 7f23f2a00640 1 -- 172.21.15.100:0/3670989568 >> v1:172.21.15.100:6789/0 conn(0x7f23f40cd130 legacy=0x7f23f406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.191+0000 7f23f2a00640 1 -- 172.21.15.100:0/3670989568 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f406ce90 con 0x7f23f415c7a0 2024-04-23T14:34:46.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.195+0000 7f23f2a00640 1 --2- 172.21.15.100:0/3670989568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23f415c7a0 0x7f23f40af190 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f23e40097c0 tx=0x7f23e401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143098c334388960 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:46.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.195+0000 7f23f2000640 1 -- 172.21.15.100:0/3670989568 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e40b0070 con 0x7f23f415c7a0 2024-04-23T14:34:46.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.195+0000 7f23f2000640 1 -- 172.21.15.100:0/3670989568 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e40ab600 con 0x7f23f415c7a0 2024-04-23T14:34:46.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.195+0000 7f23f2000640 1 -- 172.21.15.100:0/3670989568 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e40ab9d0 con 0x7f23f415c7a0 2024-04-23T14:34:46.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.195+0000 7f23f9600640 1 -- 172.21.15.100:0/3670989568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23f415c7a0 msgr2=0x7f23f40af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.195+0000 7f23f9600640 1 --2- 172.21.15.100:0/3670989568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23f415c7a0 0x7f23f40af190 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f23e40097c0 tx=0x7f23e401c410 comp rx=0 tx=0).stop 2024-04-23T14:34:46.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.195+0000 7f23f9600640 1 -- 172.21.15.100:0/3670989568 shutdown_connections 2024-04-23T14:34:46.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.195+0000 7f23f9600640 1 --2- 172.21.15.100:0/3670989568 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f23f406bb20 0x7f23f40ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.195+0000 7f23f9600640 1 --2- 172.21.15.100:0/3670989568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23f415c7a0 0x7f23f40af190 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.195+0000 7f23f9600640 1 -- 172.21.15.100:0/3670989568 >> 172.21.15.100:0/3670989568 conn(0x7f23f40b5200 msgr2=0x7f23f40b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:46.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.195+0000 7f23f9600640 1 -- 172.21.15.100:0/3670989568 shutdown_connections 2024-04-23T14:34:46.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.195+0000 7f23f9600640 1 -- 172.21.15.100:0/3670989568 wait complete. 2024-04-23T14:34:46.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.195+0000 7f23f9600640 1 Processor -- start 2024-04-23T14:34:46.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.195+0000 7f23f9600640 1 -- start start 2024-04-23T14:34:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23f9600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f23f406bb20 0x7f23f40984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23f9600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23f40cd130 0x7f23f4098a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23f9600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23f415c7a0 0x7f23f4098f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23f9600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f23f4169590 con 0x7f23f415c7a0 2024-04-23T14:34:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23f9600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f23f4169410 con 0x7f23f40cd130 2024-04-23T14:34:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23f9600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f23f4169710 con 0x7f23f406bb20 2024-04-23T14:34:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23f3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23f415c7a0 0x7f23f4098f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23f3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23f415c7a0 0x7f23f4098f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35976/0 (socket says 172.21.15.100:35976) 2024-04-23T14:34:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23f3400640 1 -- 172.21.15.100:0/2654251088 learned_addr learned my addr 172.21.15.100:0/2654251088 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23f2a00640 1 --2- 172.21.15.100:0/2654251088 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f23f406bb20 0x7f23f40984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23ea000640 1 --2- 172.21.15.100:0/2654251088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23f40cd130 0x7f23f4098a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23f3400640 1 -- 172.21.15.100:0/2654251088 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f23f406bb20 msgr2=0x7f23f40984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23f3400640 1 --2- 172.21.15.100:0/2654251088 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f23f406bb20 0x7f23f40984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23f3400640 1 -- 172.21.15.100:0/2654251088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23f40cd130 msgr2=0x7f23f4098a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23f3400640 1 --2- 172.21.15.100:0/2654251088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23f40cd130 0x7f23f4098a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23f3400640 1 -- 172.21.15.100:0/2654251088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f40c3f60 con 0x7f23f415c7a0 2024-04-23T14:34:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23ea000640 1 --2- 172.21.15.100:0/2654251088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23f40cd130 0x7f23f4098a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23f2a00640 1 --2- 172.21.15.100:0/2654251088 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f23f406bb20 0x7f23f40984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23f3400640 1 --2- 172.21.15.100:0/2654251088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23f415c7a0 0x7f23f4098f50 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00cad0 tx=0x7f23ec00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23eb400640 1 -- 172.21.15.100:0/2654251088 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec017070 con 0x7f23f415c7a0 2024-04-23T14:34:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23eb400640 1 -- 172.21.15.100:0/2654251088 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23ec00ac30 con 0x7f23f415c7a0 2024-04-23T14:34:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23eb400640 1 -- 172.21.15.100:0/2654251088 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec012400 con 0x7f23f415c7a0 2024-04-23T14:34:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23f9600640 1 -- 172.21.15.100:0/2654251088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f40c4240 con 0x7f23f415c7a0 2024-04-23T14:34:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23f9600640 1 -- 172.21.15.100:0/2654251088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f40c4800 con 0x7f23f415c7a0 2024-04-23T14:34:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23f9600640 1 -- 172.21.15.100:0/2654251088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f4068ad0 con 0x7f23f415c7a0 2024-04-23T14:34:46.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.199+0000 7f23eb400640 1 -- 172.21.15.100:0/2654251088 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f23ec01a070 con 0x7f23f415c7a0 2024-04-23T14:34:46.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.203+0000 7f23eb400640 1 --2- 172.21.15.100:0/2654251088 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f23d0068380 0x7f23d006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.203+0000 7f23eb400640 1 -- 172.21.15.100:0/2654251088 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f23ec0869c0 con 0x7f23f415c7a0 2024-04-23T14:34:46.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.203+0000 7f23f2a00640 1 --2- 172.21.15.100:0/2654251088 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f23d0068380 0x7f23d006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.203+0000 7f23f2a00640 1 --2- 172.21.15.100:0/2654251088 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f23d0068380 0x7f23d006a840 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f23e40ad000 tx=0x7f23e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:46.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.207+0000 7f23eb400640 1 -- 172.21.15.100:0/2654251088 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f23ec056d50 con 0x7f23f415c7a0 2024-04-23T14:34:46.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.363+0000 7f23f9600640 1 -- 172.21.15.100:0/2654251088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f23f40740c0 con 0x7f23f415c7a0 2024-04-23T14:34:46.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.363+0000 7f23eb400640 1 -- 172.21.15.100:0/2654251088 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) ==== 93+0+10 (secure 0 0 0) 0x7f23ec056650 con 0x7f23f415c7a0 2024-04-23T14:34:46.363 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:34:46.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.363+0000 7f23f9600640 1 -- 172.21.15.100:0/2654251088 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f23d0068380 msgr2=0x7f23d006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.363+0000 7f23f9600640 1 --2- 172.21.15.100:0/2654251088 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f23d0068380 0x7f23d006a840 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f23e40ad000 tx=0x7f23e4008040 comp rx=0 tx=0).stop 2024-04-23T14:34:46.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.363+0000 7f23f9600640 1 -- 172.21.15.100:0/2654251088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23f415c7a0 msgr2=0x7f23f4098f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.363+0000 7f23f9600640 1 --2- 172.21.15.100:0/2654251088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23f415c7a0 0x7f23f4098f50 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00cad0 tx=0x7f23ec00cf90 comp rx=0 tx=0).stop 2024-04-23T14:34:46.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.363+0000 7f23f9600640 1 -- 172.21.15.100:0/2654251088 shutdown_connections 2024-04-23T14:34:46.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.363+0000 7f23f9600640 1 --2- 172.21.15.100:0/2654251088 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f23d0068380 0x7f23d006a840 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.363+0000 7f23f9600640 1 --2- 172.21.15.100:0/2654251088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23f415c7a0 0x7f23f4098f50 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.363+0000 7f23f9600640 1 --2- 172.21.15.100:0/2654251088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23f40cd130 0x7f23f4098a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.363+0000 7f23f9600640 1 --2- 172.21.15.100:0/2654251088 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f23f406bb20 0x7f23f40984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.363+0000 7f23f9600640 1 -- 172.21.15.100:0/2654251088 >> 172.21.15.100:0/2654251088 conn(0x7f23f40b5200 msgr2=0x7f23f40d0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:46.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.363+0000 7f23f9600640 1 -- 172.21.15.100:0/2654251088 shutdown_connections 2024-04-23T14:34:46.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.363+0000 7f23f9600640 1 -- 172.21.15.100:0/2654251088 wait complete. 2024-04-23T14:34:46.384 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:34:46.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.523+0000 7faf12000640 1 Processor -- start 2024-04-23T14:34:46.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.523+0000 7faf12000640 1 -- start start 2024-04-23T14:34:46.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.523+0000 7faf12000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7faf0c0cea60 0x7faf0c0d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.523+0000 7faf12000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf0c0b1f90 con 0x7faf0c06d450 2024-04-23T14:34:46.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.523+0000 7faf12000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf0c0b2270 con 0x7faf0c0cd460 2024-04-23T14:34:46.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.523+0000 7faf12000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7faf0c0b2550 con 0x7faf0c0cea60 2024-04-23T14:34:46.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.523+0000 7faf0b400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7faf0c06d450 0x7faf0c06d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:59864/0 (socket says 172.21.15.100:59864) 2024-04-23T14:34:46.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.523+0000 7faf0b400640 1 -- 172.21.15.100:0/3172280737 learned_addr learned my addr 172.21.15.100:0/3172280737 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:46.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.523+0000 7faf0aa00640 1 --2- 172.21.15.100:0/3172280737 >> v2:172.21.15.151:3300/0 conn(0x7faf0c0cea60 0x7faf0c0d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.523+0000 7faf0a000640 1 -- 172.21.15.100:0/3172280737 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1674601088 0 0) 0x7faf0c0b1f90 con 0x7faf0c06d450 2024-04-23T14:34:46.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.523+0000 7faf0a000640 1 -- 172.21.15.100:0/3172280737 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faef0003530 con 0x7faf0c06d450 2024-04-23T14:34:46.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.523+0000 7faf0a000640 1 -- 172.21.15.100:0/3172280737 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 944369804 0 0) 0x7faf0c0b2270 con 0x7faf0c0cd460 2024-04-23T14:34:46.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.523+0000 7faf0a000640 1 -- 172.21.15.100:0/3172280737 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf0c0b1f90 con 0x7faf0c0cd460 2024-04-23T14:34:46.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.523+0000 7faf0aa00640 1 -- 172.21.15.100:0/3172280737 >> v1:172.21.15.151:6789/0 conn(0x7faf0c0cd460 legacy=0x7faf0c0b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.523+0000 7faf0aa00640 1 -- 172.21.15.100:0/3172280737 >> v1:172.21.15.100:6789/0 conn(0x7faf0c06d450 legacy=0x7faf0c06d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.523+0000 7faf0aa00640 1 -- 172.21.15.100:0/3172280737 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0c0b2830 con 0x7faf0c0cea60 2024-04-23T14:34:46.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.527+0000 7faf0aa00640 1 --2- 172.21.15.100:0/3172280737 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf0c0cea60 0x7faf0c0d0300 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faef80097c0 tx=0x7faef801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88684735ca974d6d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:46.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.527+0000 7faf0a000640 1 -- 172.21.15.100:0/3172280737 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef80b0070 con 0x7faf0c0cea60 2024-04-23T14:34:46.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.527+0000 7faf0a000640 1 -- 172.21.15.100:0/3172280737 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faef80ab600 con 0x7faf0c0cea60 2024-04-23T14:34:46.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.527+0000 7faf0a000640 1 -- 172.21.15.100:0/3172280737 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef80ab9d0 con 0x7faf0c0cea60 2024-04-23T14:34:46.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.527+0000 7faf12000640 1 -- 172.21.15.100:0/3172280737 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf0c0cea60 msgr2=0x7faf0c0d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.527+0000 7faf12000640 1 --2- 172.21.15.100:0/3172280737 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf0c0cea60 0x7faf0c0d0300 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faef80097c0 tx=0x7faef801c410 comp rx=0 tx=0).stop 2024-04-23T14:34:46.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.527+0000 7faf12000640 1 -- 172.21.15.100:0/3172280737 shutdown_connections 2024-04-23T14:34:46.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.527+0000 7faf12000640 1 --2- 172.21.15.100:0/3172280737 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf0c0cea60 0x7faf0c0d0300 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.527+0000 7faf12000640 1 -- 172.21.15.100:0/3172280737 >> 172.21.15.100:0/3172280737 conn(0x7faf0c0cee80 msgr2=0x7faf0c0cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:46.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.527+0000 7faf12000640 1 -- 172.21.15.100:0/3172280737 shutdown_connections 2024-04-23T14:34:46.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.527+0000 7faf12000640 1 -- 172.21.15.100:0/3172280737 wait complete. 2024-04-23T14:34:46.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.527+0000 7faf12000640 1 Processor -- start 2024-04-23T14:34:46.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.527+0000 7faf12000640 1 -- start start 2024-04-23T14:34:46.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf12000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faf0c06d450 0x7faf0c155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf12000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf0c0cd460 0x7faf0c15bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf12000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf0c0cea60 0x7faf0c0c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf12000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faf0c16cd50 con 0x7faf0c0cd460 2024-04-23T14:34:46.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf12000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faf0c16cbd0 con 0x7faf0c0cea60 2024-04-23T14:34:46.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf12000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7faf0c16ced0 con 0x7faf0c06d450 2024-04-23T14:34:46.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf0aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf0c0cd460 0x7faf0c15bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf0aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf0c0cd460 0x7faf0c15bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35982/0 (socket says 172.21.15.100:35982) 2024-04-23T14:34:46.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf0aa00640 1 -- 172.21.15.100:0/658673186 learned_addr learned my addr 172.21.15.100:0/658673186 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:46.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf0b400640 1 --2- 172.21.15.100:0/658673186 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faf0c06d450 0x7faf0c155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf0be00640 1 --2- 172.21.15.100:0/658673186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf0c0cea60 0x7faf0c0c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf0aa00640 1 -- 172.21.15.100:0/658673186 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faf0c06d450 msgr2=0x7faf0c155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf0aa00640 1 --2- 172.21.15.100:0/658673186 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faf0c06d450 0x7faf0c155850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf0aa00640 1 -- 172.21.15.100:0/658673186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf0c0cea60 msgr2=0x7faf0c0c81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf0aa00640 1 --2- 172.21.15.100:0/658673186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf0c0cea60 0x7faf0c0c81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf0aa00640 1 -- 172.21.15.100:0/658673186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0c155d90 con 0x7faf0c0cd460 2024-04-23T14:34:46.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf0aa00640 1 --2- 172.21.15.100:0/658673186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf0c0cd460 0x7faf0c15bac0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7faef8019020 tx=0x7faef80191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:46.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf03400640 1 -- 172.21.15.100:0/658673186 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef80b0040 con 0x7faf0c0cd460 2024-04-23T14:34:46.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf03400640 1 -- 172.21.15.100:0/658673186 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faef80b9070 con 0x7faf0c0cd460 2024-04-23T14:34:46.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf03400640 1 -- 172.21.15.100:0/658673186 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef80b4650 con 0x7faf0c0cd460 2024-04-23T14:34:46.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf12000640 1 -- 172.21.15.100:0/658673186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf0c141950 con 0x7faf0c0cd460 2024-04-23T14:34:46.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf12000640 1 -- 172.21.15.100:0/658673186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf0c141f10 con 0x7faf0c0cd460 2024-04-23T14:34:46.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.531+0000 7faf12000640 1 -- 172.21.15.100:0/658673186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf0c068ad0 con 0x7faf0c0cd460 2024-04-23T14:34:46.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.535+0000 7faf03400640 1 -- 172.21.15.100:0/658673186 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7faef80b47f0 con 0x7faf0c0cd460 2024-04-23T14:34:46.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.535+0000 7faf03400640 1 --2- 172.21.15.100:0/658673186 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faeec068380 0x7faeec06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.535+0000 7faf03400640 1 -- 172.21.15.100:0/658673186 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7faef8120580 con 0x7faf0c0cd460 2024-04-23T14:34:46.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.535+0000 7faf0b400640 1 --2- 172.21.15.100:0/658673186 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faeec068380 0x7faeec06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.535+0000 7faf0b400640 1 --2- 172.21.15.100:0/658673186 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faeec068380 0x7faeec06a840 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7faefc007fd0 tx=0x7faefc007d80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:46.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.539+0000 7faf03400640 1 -- 172.21.15.100:0/658673186 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faef8016090 con 0x7faf0c0cd460 2024-04-23T14:34:46.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.695+0000 7faf12000640 1 -- 172.21.15.100:0/658673186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faf0c15de70 con 0x7faf0c0cd460 2024-04-23T14:34:46.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.695+0000 7faf03400640 1 -- 172.21.15.100:0/658673186 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) ==== 104+0+11 (secure 0 0 0) 0x7faef80f03f0 con 0x7faf0c0cd460 2024-04-23T14:34:46.697 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:34:46.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.695+0000 7faf12000640 1 -- 172.21.15.100:0/658673186 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faeec068380 msgr2=0x7faeec06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.695+0000 7faf12000640 1 --2- 172.21.15.100:0/658673186 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faeec068380 0x7faeec06a840 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7faefc007fd0 tx=0x7faefc007d80 comp rx=0 tx=0).stop 2024-04-23T14:34:46.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.695+0000 7faf12000640 1 -- 172.21.15.100:0/658673186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf0c0cd460 msgr2=0x7faf0c15bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.695+0000 7faf12000640 1 --2- 172.21.15.100:0/658673186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf0c0cd460 0x7faf0c15bac0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7faef8019020 tx=0x7faef80191b0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.695+0000 7faf12000640 1 -- 172.21.15.100:0/658673186 shutdown_connections 2024-04-23T14:34:46.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.695+0000 7faf12000640 1 --2- 172.21.15.100:0/658673186 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7faeec068380 0x7faeec06a840 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.695+0000 7faf12000640 1 --2- 172.21.15.100:0/658673186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf0c0cea60 0x7faf0c0c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.695+0000 7faf12000640 1 --2- 172.21.15.100:0/658673186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf0c0cd460 0x7faf0c15bac0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.695+0000 7faf12000640 1 --2- 172.21.15.100:0/658673186 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faf0c06d450 0x7faf0c155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.695+0000 7faf12000640 1 -- 172.21.15.100:0/658673186 >> 172.21.15.100:0/658673186 conn(0x7faf0c0cee80 msgr2=0x7faf0c0cdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:46.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.699+0000 7faf12000640 1 -- 172.21.15.100:0/658673186 shutdown_connections 2024-04-23T14:34:46.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.699+0000 7faf12000640 1 -- 172.21.15.100:0/658673186 wait complete. 2024-04-23T14:34:46.717 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:34:46.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.855+0000 7f8cace00640 1 Processor -- start 2024-04-23T14:34:46.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.855+0000 7f8cace00640 1 -- start start 2024-04-23T14:34:46.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.855+0000 7f8cace00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8ca806cd50 0x7f8ca806d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.855+0000 7f8cace00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f8ca80cedf0 0x7f8ca80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.855+0000 7f8cace00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ca80cdab0 con 0x7f8ca8145e90 2024-04-23T14:34:46.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.855+0000 7f8cace00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f8ca806d6d0 con 0x7f8ca80cedf0 2024-04-23T14:34:46.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.855+0000 7f8cace00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca8146270 con 0x7f8ca806cd50 2024-04-23T14:34:46.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.855+0000 7f8c9d400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8ca806cd50 0x7f8ca806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.855+0000 7f8ca5e00640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f8ca8145e90 0x7f8ca806c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:54406/0 (socket says 172.21.15.100:54406) 2024-04-23T14:34:46.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.855+0000 7f8ca5e00640 1 -- 172.21.15.100:0/1704576856 learned_addr learned my addr 172.21.15.100:0/1704576856 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:46.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.855+0000 7f8ca6800640 1 --2- 172.21.15.100:0/1704576856 >> v2:172.21.15.151:3300/0 conn(0x7f8ca80cedf0 0x7f8ca80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.855+0000 7f8ca5400640 1 -- 172.21.15.100:0/1704576856 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 817360311 0 0) 0x7f8ca80cdab0 con 0x7f8ca8145e90 2024-04-23T14:34:46.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.855+0000 7f8ca5400640 1 -- 172.21.15.100:0/1704576856 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c8c003530 con 0x7f8ca8145e90 2024-04-23T14:34:46.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.855+0000 7f8c9d400640 1 -- 172.21.15.100:0/1704576856 >> v2:172.21.15.151:3300/0 conn(0x7f8ca80cedf0 msgr2=0x7f8ca80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:46.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.855+0000 7f8c9d400640 1 --2- 172.21.15.100:0/1704576856 >> v2:172.21.15.151:3300/0 conn(0x7f8ca80cedf0 0x7f8ca80cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.855+0000 7f8c9d400640 1 -- 172.21.15.100:0/1704576856 >> v1:172.21.15.151:6789/0 conn(0x7f8ca8145e90 legacy=0x7f8ca806c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.855+0000 7f8c9d400640 1 -- 172.21.15.100:0/1704576856 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca80cdd50 con 0x7f8ca806cd50 2024-04-23T14:34:46.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.855+0000 7f8ca6800640 1 --2- 172.21.15.100:0/1704576856 >> v2:172.21.15.151:3300/0 conn(0x7f8ca80cedf0 0x7f8ca80cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:34:46.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.859+0000 7f8c9d400640 1 --2- 172.21.15.100:0/1704576856 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8ca806cd50 0x7f8ca806d190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8c900097c0 tx=0x7f8c9001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dd5af99b8a259d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:46.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.859+0000 7f8ca5400640 1 -- 172.21.15.100:0/1704576856 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c900b0070 con 0x7f8ca806cd50 2024-04-23T14:34:46.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.859+0000 7f8ca5400640 1 -- 172.21.15.100:0/1704576856 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c900ab600 con 0x7f8ca806cd50 2024-04-23T14:34:46.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.859+0000 7f8ca5400640 1 -- 172.21.15.100:0/1704576856 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c900ab9d0 con 0x7f8ca806cd50 2024-04-23T14:34:46.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.859+0000 7f8cace00640 1 -- 172.21.15.100:0/1704576856 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8ca806cd50 msgr2=0x7f8ca806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.859+0000 7f8cace00640 1 --2- 172.21.15.100:0/1704576856 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8ca806cd50 0x7f8ca806d190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8c900097c0 tx=0x7f8c9001c410 comp rx=0 tx=0).stop 2024-04-23T14:34:46.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.859+0000 7f8cace00640 1 -- 172.21.15.100:0/1704576856 shutdown_connections 2024-04-23T14:34:46.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.859+0000 7f8cace00640 1 --2- 172.21.15.100:0/1704576856 >> v2:172.21.15.151:3300/0 conn(0x7f8ca80cedf0 0x7f8ca80cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.859+0000 7f8cace00640 1 --2- 172.21.15.100:0/1704576856 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8ca806cd50 0x7f8ca806d190 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.859+0000 7f8cace00640 1 -- 172.21.15.100:0/1704576856 >> 172.21.15.100:0/1704576856 conn(0x7f8ca80704d0 msgr2=0x7f8ca80708e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:46.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8cace00640 1 -- 172.21.15.100:0/1704576856 shutdown_connections 2024-04-23T14:34:46.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8cace00640 1 -- 172.21.15.100:0/1704576856 wait complete. 2024-04-23T14:34:46.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8cace00640 1 Processor -- start 2024-04-23T14:34:46.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8cace00640 1 -- start start 2024-04-23T14:34:46.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8cace00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ca806cd50 0x7f8ca815be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8cace00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ca80cedf0 0x7f8ca81f6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8cace00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8ca8145e90 0x7f8ca813e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8cace00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca806ca50 con 0x7f8ca80cedf0 2024-04-23T14:34:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8cace00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca80ce8d0 con 0x7f8ca806cd50 2024-04-23T14:34:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8cace00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca8169450 con 0x7f8ca8145e90 2024-04-23T14:34:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8c9d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ca80cedf0 0x7f8ca81f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8c9d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ca80cedf0 0x7f8ca81f6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35998/0 (socket says 172.21.15.100:35998) 2024-04-23T14:34:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8c9d400640 1 -- 172.21.15.100:0/2089214079 learned_addr learned my addr 172.21.15.100:0/2089214079 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8ca5e00640 1 --2- 172.21.15.100:0/2089214079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ca806cd50 0x7f8ca815be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8ca6800640 1 --2- 172.21.15.100:0/2089214079 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8ca8145e90 0x7f8ca813e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8c9d400640 1 -- 172.21.15.100:0/2089214079 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8ca8145e90 msgr2=0x7f8ca813e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8c9d400640 1 --2- 172.21.15.100:0/2089214079 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8ca8145e90 0x7f8ca813e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8c9d400640 1 -- 172.21.15.100:0/2089214079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ca806cd50 msgr2=0x7f8ca815be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8c9d400640 1 --2- 172.21.15.100:0/2089214079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ca806cd50 0x7f8ca815be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8c9d400640 1 -- 172.21.15.100:0/2089214079 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca815c470 con 0x7f8ca80cedf0 2024-04-23T14:34:46.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8c9d400640 1 --2- 172.21.15.100:0/2089214079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ca80cedf0 0x7f8ca81f6200 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8c90019020 tx=0x7f8c900191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:46.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8c9ea00640 1 -- 172.21.15.100:0/2089214079 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c900b8020 con 0x7f8ca80cedf0 2024-04-23T14:34:46.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8c9ea00640 1 -- 172.21.15.100:0/2089214079 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c900b4600 con 0x7f8ca80cedf0 2024-04-23T14:34:46.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8cace00640 1 -- 172.21.15.100:0/2089214079 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca813ddb0 con 0x7f8ca80cedf0 2024-04-23T14:34:46.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.863+0000 7f8c9ea00640 1 -- 172.21.15.100:0/2089214079 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c900b0040 con 0x7f8ca80cedf0 2024-04-23T14:34:46.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.867+0000 7f8cace00640 1 -- 172.21.15.100:0/2089214079 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca813e370 con 0x7f8ca80cedf0 2024-04-23T14:34:46.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.867+0000 7f8cace00640 1 -- 172.21.15.100:0/2089214079 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca8068ad0 con 0x7f8ca80cedf0 2024-04-23T14:34:46.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.867+0000 7f8c9ea00640 1 -- 172.21.15.100:0/2089214079 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8c900bb020 con 0x7f8ca80cedf0 2024-04-23T14:34:46.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.867+0000 7f8c9ea00640 1 --2- 172.21.15.100:0/2089214079 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8c84068380 0x7f8c8406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.867+0000 7f8c9ea00640 1 -- 172.21.15.100:0/2089214079 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f8c90120870 con 0x7f8ca80cedf0 2024-04-23T14:34:46.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.867+0000 7f8ca5e00640 1 --2- 172.21.15.100:0/2089214079 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8c84068380 0x7f8c8406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.871+0000 7f8ca5e00640 1 --2- 172.21.15.100:0/2089214079 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8c84068380 0x7f8c8406a840 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8c98005c10 tx=0x7f8c98009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:46.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:46.871+0000 7f8c9ea00640 1 -- 172.21.15.100:0/2089214079 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8c90016090 con 0x7f8ca80cedf0 2024-04-23T14:34:47.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.027+0000 7f8cace00640 1 -- 172.21.15.100:0/2089214079 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8ca813fbf0 con 0x7f8ca80cedf0 2024-04-23T14:34:47.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.027+0000 7f8c9ea00640 1 -- 172.21.15.100:0/2089214079 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) ==== 100+0+11 (secure 0 0 0) 0x7f8c900f0c90 con 0x7f8ca80cedf0 2024-04-23T14:34:47.030 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:34:47.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.031+0000 7f8cace00640 1 -- 172.21.15.100:0/2089214079 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8c84068380 msgr2=0x7f8c8406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.031+0000 7f8cace00640 1 --2- 172.21.15.100:0/2089214079 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8c84068380 0x7f8c8406a840 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8c98005c10 tx=0x7f8c98009040 comp rx=0 tx=0).stop 2024-04-23T14:34:47.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.031+0000 7f8cace00640 1 -- 172.21.15.100:0/2089214079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ca80cedf0 msgr2=0x7f8ca81f6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.031+0000 7f8cace00640 1 --2- 172.21.15.100:0/2089214079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ca80cedf0 0x7f8ca81f6200 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8c90019020 tx=0x7f8c900191b0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.031+0000 7f8cace00640 1 -- 172.21.15.100:0/2089214079 shutdown_connections 2024-04-23T14:34:47.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.031+0000 7f8cace00640 1 --2- 172.21.15.100:0/2089214079 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f8c84068380 0x7f8c8406a840 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.031+0000 7f8cace00640 1 --2- 172.21.15.100:0/2089214079 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8ca8145e90 0x7f8ca813e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.031+0000 7f8cace00640 1 --2- 172.21.15.100:0/2089214079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ca80cedf0 0x7f8ca81f6200 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.031+0000 7f8cace00640 1 --2- 172.21.15.100:0/2089214079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ca806cd50 0x7f8ca815be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.031+0000 7f8cace00640 1 -- 172.21.15.100:0/2089214079 >> 172.21.15.100:0/2089214079 conn(0x7f8ca80704d0 msgr2=0x7f8ca8158d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:47.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.031+0000 7f8cace00640 1 -- 172.21.15.100:0/2089214079 shutdown_connections 2024-04-23T14:34:47.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.031+0000 7f8cace00640 1 -- 172.21.15.100:0/2089214079 wait complete. 2024-04-23T14:34:47.051 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:34:47.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.187+0000 7f70da200640 1 Processor -- start 2024-04-23T14:34:47.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.187+0000 7f70da200640 1 -- start start 2024-04-23T14:34:47.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.187+0000 7f70da200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f70d415b660 0x7f70d40c6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:47.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.187+0000 7f70da200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f70d40c6d60 0x7f70d40d0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:47.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.187+0000 7f70da200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70d40cee00 con 0x7f70d40d06d0 2024-04-23T14:34:47.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.187+0000 7f70da200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f70d40d0b60 con 0x7f70d40c6d60 2024-04-23T14:34:47.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.187+0000 7f70da200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f70d415ba40 con 0x7f70d415b660 2024-04-23T14:34:47.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.187+0000 7f70d2a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f70d40c6d60 0x7f70d40d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:47.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.187+0000 7f70d3e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f70d40d06d0 0x7f70d415f0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:59870/0 (socket says 172.21.15.100:59870) 2024-04-23T14:34:47.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.187+0000 7f70d3e00640 1 -- 172.21.15.100:0/954718878 learned_addr learned my addr 172.21.15.100:0/954718878 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:47.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.187+0000 7f70d3400640 1 --2- 172.21.15.100:0/954718878 >> v2:172.21.15.151:3300/0 conn(0x7f70d415b660 0x7f70d40c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:47.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.187+0000 7f70d2a00640 1 -- 172.21.15.100:0/954718878 >> v2:172.21.15.151:3300/0 conn(0x7f70d415b660 msgr2=0x7f70d40c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:47.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.187+0000 7f70d2a00640 1 --2- 172.21.15.100:0/954718878 >> v2:172.21.15.151:3300/0 conn(0x7f70d415b660 0x7f70d40c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.187+0000 7f70d2a00640 1 -- 172.21.15.100:0/954718878 >> v1:172.21.15.100:6789/0 conn(0x7f70d40d06d0 legacy=0x7f70d415f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.187+0000 7f70d2a00640 1 -- 172.21.15.100:0/954718878 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d406bc50 con 0x7f70d40c6d60 2024-04-23T14:34:47.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.187+0000 7f70d2000640 1 -- 172.21.15.100:0/954718878 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2437447273 0 0) 0x7f70cc0014a0 con 0x7f70d40d06d0 2024-04-23T14:34:47.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.187+0000 7f70d3400640 1 --2- 172.21.15.100:0/954718878 >> v2:172.21.15.151:3300/0 conn(0x7f70d415b660 0x7f70d40c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:47.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.191+0000 7f70d2a00640 1 --2- 172.21.15.100:0/954718878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d40c6d60 0x7f70d40d0190 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f70d40b6080 tx=0x7f70c401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c46afdb586c869b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:47.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.191+0000 7f70d2000640 1 -- 172.21.15.100:0/954718878 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c40ab650 con 0x7f70d40c6d60 2024-04-23T14:34:47.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.191+0000 7f70d2000640 1 -- 172.21.15.100:0/954718878 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c40b0070 con 0x7f70d40c6d60 2024-04-23T14:34:47.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.191+0000 7f70d2000640 1 -- 172.21.15.100:0/954718878 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c40aba10 con 0x7f70d40c6d60 2024-04-23T14:34:47.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.191+0000 7f70da200640 1 -- 172.21.15.100:0/954718878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d40c6d60 msgr2=0x7f70d40d0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.191+0000 7f70da200640 1 --2- 172.21.15.100:0/954718878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d40c6d60 0x7f70d40d0190 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f70d40b6080 tx=0x7f70c401c450 comp rx=0 tx=0).stop 2024-04-23T14:34:47.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.191+0000 7f70da200640 1 -- 172.21.15.100:0/954718878 shutdown_connections 2024-04-23T14:34:47.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.191+0000 7f70da200640 1 --2- 172.21.15.100:0/954718878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d40c6d60 0x7f70d40d0190 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.191+0000 7f70da200640 1 --2- 172.21.15.100:0/954718878 >> v2:172.21.15.151:3300/0 conn(0x7f70d415b660 0x7f70d40c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.191+0000 7f70da200640 1 -- 172.21.15.100:0/954718878 >> 172.21.15.100:0/954718878 conn(0x7f70d40b5310 msgr2=0x7f70d40b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:47.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.191+0000 7f70da200640 1 -- 172.21.15.100:0/954718878 shutdown_connections 2024-04-23T14:34:47.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.191+0000 7f70da200640 1 -- 172.21.15.100:0/954718878 wait complete. 2024-04-23T14:34:47.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.191+0000 7f70da200640 1 Processor -- start 2024-04-23T14:34:47.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.191+0000 7f70da200640 1 -- start start 2024-04-23T14:34:47.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70da200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f70d40c6d60 0x7f70d406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:47.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70da200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d40d06d0 0x7f70d414db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:47.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70da200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d415b660 0x7f70d40ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:47.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70da200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f70d4168410 con 0x7f70d40d06d0 2024-04-23T14:34:47.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70da200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f70d406c7d0 con 0x7f70d415b660 2024-04-23T14:34:47.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70da200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f70d4168590 con 0x7f70d40c6d60 2024-04-23T14:34:47.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70d2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d40d06d0 0x7f70d414db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:47.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70d2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d40d06d0 0x7f70d414db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36026/0 (socket says 172.21.15.100:36026) 2024-04-23T14:34:47.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70d2a00640 1 -- 172.21.15.100:0/322239920 learned_addr learned my addr 172.21.15.100:0/322239920 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:47.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70d3e00640 1 --2- 172.21.15.100:0/322239920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d415b660 0x7f70d40ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:47.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70d3400640 1 --2- 172.21.15.100:0/322239920 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f70d40c6d60 0x7f70d406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:47.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70d2a00640 1 -- 172.21.15.100:0/322239920 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f70d40c6d60 msgr2=0x7f70d406dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70d2a00640 1 --2- 172.21.15.100:0/322239920 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f70d40c6d60 0x7f70d406dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70d2a00640 1 -- 172.21.15.100:0/322239920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d415b660 msgr2=0x7f70d40ae0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70d2a00640 1 --2- 172.21.15.100:0/322239920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d415b660 0x7f70d40ae0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70d2a00640 1 -- 172.21.15.100:0/322239920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d40adc10 con 0x7f70d40d06d0 2024-04-23T14:34:47.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70d2a00640 1 --2- 172.21.15.100:0/322239920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d40d06d0 0x7f70d414db90 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f70c4019060 tx=0x7f70c401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:47.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70cb400640 1 -- 172.21.15.100:0/322239920 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c40b5610 con 0x7f70d40d06d0 2024-04-23T14:34:47.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70cb400640 1 -- 172.21.15.100:0/322239920 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c40b57d0 con 0x7f70d40d06d0 2024-04-23T14:34:47.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70da200640 1 -- 172.21.15.100:0/322239920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d40b04b0 con 0x7f70d40d06d0 2024-04-23T14:34:47.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70cb400640 1 -- 172.21.15.100:0/322239920 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c40b5bc0 con 0x7f70d40d06d0 2024-04-23T14:34:47.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70da200640 1 -- 172.21.15.100:0/322239920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d40b0a70 con 0x7f70d40d06d0 2024-04-23T14:34:47.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70da200640 1 -- 172.21.15.100:0/322239920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d4068ad0 con 0x7f70d40d06d0 2024-04-23T14:34:47.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.195+0000 7f70cb400640 1 -- 172.21.15.100:0/322239920 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f70c40bd020 con 0x7f70d40d06d0 2024-04-23T14:34:47.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.199+0000 7f70cb400640 1 --2- 172.21.15.100:0/322239920 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f70b00683b0 0x7f70b006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:47.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.199+0000 7f70cb400640 1 -- 172.21.15.100:0/322239920 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f70c4121770 con 0x7f70d40d06d0 2024-04-23T14:34:47.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.199+0000 7f70d3400640 1 --2- 172.21.15.100:0/322239920 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f70b00683b0 0x7f70b006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:47.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.199+0000 7f70d3400640 1 --2- 172.21.15.100:0/322239920 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f70b00683b0 0x7f70b006a870 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f70bc006fd0 tx=0x7f70bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:47.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.203+0000 7f70cb400640 1 -- 172.21.15.100:0/322239920 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f70c4016070 con 0x7f70d40d06d0 2024-04-23T14:34:47.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.371+0000 7f70da200640 1 -- 172.21.15.100:0/322239920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f70d40cca00 con 0x7f70d40d06d0 2024-04-23T14:34:47.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.371+0000 7f70cb400640 1 -- 172.21.15.100:0/322239920 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) ==== 80+0+0 (secure 0 0 0) 0x7f70c40f1b90 con 0x7f70d40d06d0 2024-04-23T14:34:47.374 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-04-23T14:34:47.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.375+0000 7f70da200640 1 -- 172.21.15.100:0/322239920 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f70b00683b0 msgr2=0x7f70b006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.375+0000 7f70da200640 1 --2- 172.21.15.100:0/322239920 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f70b00683b0 0x7f70b006a870 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f70bc006fd0 tx=0x7f70bc008040 comp rx=0 tx=0).stop 2024-04-23T14:34:47.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.375+0000 7f70da200640 1 -- 172.21.15.100:0/322239920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d40d06d0 msgr2=0x7f70d414db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.375+0000 7f70da200640 1 --2- 172.21.15.100:0/322239920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d40d06d0 0x7f70d414db90 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f70c4019060 tx=0x7f70c401ac90 comp rx=0 tx=0).stop 2024-04-23T14:34:47.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.375+0000 7f70da200640 1 -- 172.21.15.100:0/322239920 shutdown_connections 2024-04-23T14:34:47.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.375+0000 7f70da200640 1 --2- 172.21.15.100:0/322239920 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f70b00683b0 0x7f70b006a870 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.375+0000 7f70da200640 1 --2- 172.21.15.100:0/322239920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d415b660 0x7f70d40ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.375+0000 7f70da200640 1 --2- 172.21.15.100:0/322239920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d40d06d0 0x7f70d414db90 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.375+0000 7f70da200640 1 --2- 172.21.15.100:0/322239920 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f70d40c6d60 0x7f70d406dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.375+0000 7f70da200640 1 -- 172.21.15.100:0/322239920 >> 172.21.15.100:0/322239920 conn(0x7f70d40b5310 msgr2=0x7f70d40c50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:47.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.375+0000 7f70da200640 1 -- 172.21.15.100:0/322239920 shutdown_connections 2024-04-23T14:34:47.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.375+0000 7f70da200640 1 -- 172.21.15.100:0/322239920 wait complete. 2024-04-23T14:34:47.395 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:34:47.395 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:47.404 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.579+0000 7f1c5e200640 1 Processor -- start 2024-04-23T14:34:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.579+0000 7f1c5e200640 1 -- start start 2024-04-23T14:34:47.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.579+0000 7f1c5e200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1c5815b640 0x7f1c580adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:47.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.579+0000 7f1c5e200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c580af080 con 0x7f1c58073610 2024-04-23T14:34:47.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.579+0000 7f1c5e200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c580afbc0 con 0x7f1c580b2a10 2024-04-23T14:34:47.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.579+0000 7f1c5e200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f1c5815ba20 con 0x7f1c5815b640 2024-04-23T14:34:47.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.579+0000 7f1c57e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1c58073610 0x7f1c5806df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:59886/0 (socket says 172.21.15.100:59886) 2024-04-23T14:34:47.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.579+0000 7f1c57400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1c5815b640 0x7f1c580adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:47.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.579+0000 7f1c57e00640 1 -- 172.21.15.100:0/3282205236 learned_addr learned my addr 172.21.15.100:0/3282205236 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:47.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.579+0000 7f1c57400640 1 -- 172.21.15.100:0/3282205236 >> v1:172.21.15.151:6789/0 conn(0x7f1c580b2a10 legacy=0x7f1c58072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.579+0000 7f1c57400640 1 -- 172.21.15.100:0/3282205236 >> v1:172.21.15.100:6789/0 conn(0x7f1c58073610 legacy=0x7f1c5806df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.579+0000 7f1c57400640 1 -- 172.21.15.100:0/3282205236 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c580af320 con 0x7f1c5815b640 2024-04-23T14:34:47.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.579+0000 7f1c56000640 1 -- 172.21.15.100:0/3282205236 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 613484962 0 0) 0x7f1c580af080 con 0x7f1c58073610 2024-04-23T14:34:47.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.583+0000 7f1c57400640 1 --2- 172.21.15.100:0/3282205236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c5815b640 0x7f1c580adfc0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009d90 tx=0x7f1c4801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4da809656f999d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:47.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.583+0000 7f1c56000640 1 -- 172.21.15.100:0/3282205236 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c480b2070 con 0x7f1c5815b640 2024-04-23T14:34:47.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.583+0000 7f1c56000640 1 -- 172.21.15.100:0/3282205236 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c480a5970 con 0x7f1c5815b640 2024-04-23T14:34:47.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.583+0000 7f1c56000640 1 -- 172.21.15.100:0/3282205236 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c480a5d40 con 0x7f1c5815b640 2024-04-23T14:34:47.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.583+0000 7f1c5e200640 1 -- 172.21.15.100:0/3282205236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c5815b640 msgr2=0x7f1c580adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.583+0000 7f1c5e200640 1 --2- 172.21.15.100:0/3282205236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c5815b640 0x7f1c580adfc0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009d90 tx=0x7f1c4801c8b0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.583+0000 7f1c5e200640 1 -- 172.21.15.100:0/3282205236 shutdown_connections 2024-04-23T14:34:47.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.583+0000 7f1c5e200640 1 --2- 172.21.15.100:0/3282205236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c5815b640 0x7f1c580adfc0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.583+0000 7f1c5e200640 1 -- 172.21.15.100:0/3282205236 >> 172.21.15.100:0/3282205236 conn(0x7f1c580b1e80 msgr2=0x7f1c580b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:47.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.583+0000 7f1c5e200640 1 -- 172.21.15.100:0/3282205236 shutdown_connections 2024-04-23T14:34:47.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.583+0000 7f1c5e200640 1 -- 172.21.15.100:0/3282205236 wait complete. 2024-04-23T14:34:47.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c5e200640 1 Processor -- start 2024-04-23T14:34:47.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c5e200640 1 -- start start 2024-04-23T14:34:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c5e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c58073610 0x7f1c58147a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c5e200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c580b2a10 0x7f1c581f78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c5e200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1c5815b640 0x7f1c58151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c5e200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5806f2a0 con 0x7f1c58073610 2024-04-23T14:34:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c5e200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5806f120 con 0x7f1c580b2a10 2024-04-23T14:34:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c5e200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1c581691c0 con 0x7f1c5815b640 2024-04-23T14:34:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c57400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c58073610 0x7f1c58147a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c57400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c58073610 0x7f1c58147a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36050/0 (socket says 172.21.15.100:36050) 2024-04-23T14:34:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c57400640 1 -- 172.21.15.100:0/1125120891 learned_addr learned my addr 172.21.15.100:0/1125120891 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c56a00640 1 --2- 172.21.15.100:0/1125120891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c580b2a10 0x7f1c581f78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c57e00640 1 --2- 172.21.15.100:0/1125120891 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1c5815b640 0x7f1c58151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c57400640 1 -- 172.21.15.100:0/1125120891 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1c5815b640 msgr2=0x7f1c58151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c57400640 1 --2- 172.21.15.100:0/1125120891 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1c5815b640 0x7f1c58151630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c57400640 1 -- 172.21.15.100:0/1125120891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c580b2a10 msgr2=0x7f1c581f78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c57400640 1 --2- 172.21.15.100:0/1125120891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c580b2a10 0x7f1c581f78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c57400640 1 -- 172.21.15.100:0/1125120891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c581f8180 con 0x7f1c58073610 2024-04-23T14:34:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c57e00640 1 --2- 172.21.15.100:0/1125120891 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1c5815b640 0x7f1c58151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c57400640 1 --2- 172.21.15.100:0/1125120891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c58073610 0x7f1c58147a00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1c480194c0 tx=0x7f1c48019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c4f400640 1 -- 172.21.15.100:0/1125120891 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c480bd020 con 0x7f1c58073610 2024-04-23T14:34:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c4f400640 1 -- 172.21.15.100:0/1125120891 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c480ad740 con 0x7f1c58073610 2024-04-23T14:34:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c4f400640 1 -- 172.21.15.100:0/1125120891 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c480b2040 con 0x7f1c58073610 2024-04-23T14:34:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c5e200640 1 -- 172.21.15.100:0/1125120891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c581f8460 con 0x7f1c58073610 2024-04-23T14:34:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c5e200640 1 -- 172.21.15.100:0/1125120891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c581f8a70 con 0x7f1c58073610 2024-04-23T14:34:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.587+0000 7f1c5e200640 1 -- 172.21.15.100:0/1125120891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c58068ad0 con 0x7f1c58073610 2024-04-23T14:34:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.591+0000 7f1c4f400640 1 -- 172.21.15.100:0/1125120891 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1c480c0020 con 0x7f1c58073610 2024-04-23T14:34:47.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.591+0000 7f1c4f400640 1 --2- 172.21.15.100:0/1125120891 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1c38068380 0x7f1c3806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:47.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.591+0000 7f1c4f400640 1 -- 172.21.15.100:0/1125120891 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f1c48121d70 con 0x7f1c58073610 2024-04-23T14:34:47.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.591+0000 7f1c56a00640 1 --2- 172.21.15.100:0/1125120891 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1c38068380 0x7f1c3806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:47.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.591+0000 7f1c56a00640 1 --2- 172.21.15.100:0/1125120891 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1c38068380 0x7f1c3806a840 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1c44000f70 tx=0x7f1c44009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:47.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.595+0000 7f1c4f400640 1 -- 172.21.15.100:0/1125120891 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1c480c7030 con 0x7f1c58073610 2024-04-23T14:34:47.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.751+0000 7f1c5e200640 1 -- 172.21.15.100:0/1125120891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c580d10e0 con 0x7f1c58073610 2024-04-23T14:34:47.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.751+0000 7f1c4f400640 1 -- 172.21.15.100:0/1125120891 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16950 (secure 0 0 0) 0x7f1c48016560 con 0x7f1c58073610 2024-04-23T14:34:47.754 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:47.754 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":53,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:34:25.795515+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:34:22.491379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-23T14:34:23.120342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:47.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.755+0000 7f1c5e200640 1 -- 172.21.15.100:0/1125120891 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1c38068380 msgr2=0x7f1c3806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.755+0000 7f1c5e200640 1 --2- 172.21.15.100:0/1125120891 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1c38068380 0x7f1c3806a840 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1c44000f70 tx=0x7f1c44009040 comp rx=0 tx=0).stop 2024-04-23T14:34:47.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.755+0000 7f1c5e200640 1 -- 172.21.15.100:0/1125120891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c58073610 msgr2=0x7f1c58147a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.755+0000 7f1c5e200640 1 --2- 172.21.15.100:0/1125120891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c58073610 0x7f1c58147a00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1c480194c0 tx=0x7f1c48019650 comp rx=0 tx=0).stop 2024-04-23T14:34:47.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.755+0000 7f1c5e200640 1 -- 172.21.15.100:0/1125120891 shutdown_connections 2024-04-23T14:34:47.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.755+0000 7f1c5e200640 1 --2- 172.21.15.100:0/1125120891 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1c38068380 0x7f1c3806a840 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.755+0000 7f1c5e200640 1 --2- 172.21.15.100:0/1125120891 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1c5815b640 0x7f1c58151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.755+0000 7f1c5e200640 1 --2- 172.21.15.100:0/1125120891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c580b2a10 0x7f1c581f78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.755+0000 7f1c5e200640 1 --2- 172.21.15.100:0/1125120891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c58073610 0x7f1c58147a00 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.755+0000 7f1c5e200640 1 -- 172.21.15.100:0/1125120891 >> 172.21.15.100:0/1125120891 conn(0x7f1c580b1e80 msgr2=0x7f1c581521c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:47.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.755+0000 7f1c5e200640 1 -- 172.21.15.100:0/1125120891 shutdown_connections 2024-04-23T14:34:47.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.755+0000 7f1c5e200640 1 -- 172.21.15.100:0/1125120891 wait complete. 2024-04-23T14:34:47.775 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:34:22.491379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:34:23.120342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:34:47.776 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:34:47.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.915+0000 7fe6b6800640 1 Processor -- start 2024-04-23T14:34:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.915+0000 7fe6b6800640 1 -- start start 2024-04-23T14:34:47.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.915+0000 7fe6b6800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe6b00adb90 0x7fe6b00adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:47.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.915+0000 7fe6b6800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6b006e5f0 con 0x7fe6b015e860 2024-04-23T14:34:47.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.915+0000 7fe6b6800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6b006e8b0 con 0x7fe6b006f950 2024-04-23T14:34:47.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.915+0000 7fe6b6800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fe6b006eb90 con 0x7fe6b00adb90 2024-04-23T14:34:47.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.915+0000 7fe6aea00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe6b00adb90 0x7fe6b00adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:47.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.915+0000 7fe6af400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe6b015e860 0x7fe6b00b3ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:59898/0 (socket says 172.21.15.100:59898) 2024-04-23T14:34:47.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.915+0000 7fe6af400640 1 -- 172.21.15.100:0/3198283044 learned_addr learned my addr 172.21.15.100:0/3198283044 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:47.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.915+0000 7fe6aea00640 1 -- 172.21.15.100:0/3198283044 >> v1:172.21.15.151:6789/0 conn(0x7fe6b006f950 legacy=0x7fe6b006d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.915+0000 7fe6aea00640 1 -- 172.21.15.100:0/3198283044 >> v1:172.21.15.100:6789/0 conn(0x7fe6b015e860 legacy=0x7fe6b00b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.915+0000 7fe6ae000640 1 -- 172.21.15.100:0/3198283044 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2331034533 0 0) 0x7fe6b006e5f0 con 0x7fe6b015e860 2024-04-23T14:34:47.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.919+0000 7fe6aea00640 1 -- 172.21.15.100:0/3198283044 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b006ee70 con 0x7fe6b00adb90 2024-04-23T14:34:47.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.919+0000 7fe6aea00640 1 --2- 172.21.15.100:0/3198283044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b00adb90 0x7fe6b00adfe0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0097c0 tx=0x7fe69c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2426b3ea41416805 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:47.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.919+0000 7fe6ae000640 1 -- 172.21.15.100:0/3198283044 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c0a7070 con 0x7fe6b00adb90 2024-04-23T14:34:47.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.919+0000 7fe6ae000640 1 -- 172.21.15.100:0/3198283044 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe69c0b0070 con 0x7fe6b00adb90 2024-04-23T14:34:47.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.919+0000 7fe6ae000640 1 -- 172.21.15.100:0/3198283044 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c0ab890 con 0x7fe6b00adb90 2024-04-23T14:34:47.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.919+0000 7fe6b6800640 1 -- 172.21.15.100:0/3198283044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b00adb90 msgr2=0x7fe6b00adfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.919+0000 7fe6b6800640 1 --2- 172.21.15.100:0/3198283044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b00adb90 0x7fe6b00adfe0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0097c0 tx=0x7fe69c01c470 comp rx=0 tx=0).stop 2024-04-23T14:34:47.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.919+0000 7fe6b6800640 1 -- 172.21.15.100:0/3198283044 shutdown_connections 2024-04-23T14:34:47.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.919+0000 7fe6b6800640 1 --2- 172.21.15.100:0/3198283044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b00adb90 0x7fe6b00adfe0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.919+0000 7fe6b6800640 1 -- 172.21.15.100:0/3198283044 >> 172.21.15.100:0/3198283044 conn(0x7fe6b0072590 msgr2=0x7fe6b00729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:47.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.919+0000 7fe6b6800640 1 -- 172.21.15.100:0/3198283044 shutdown_connections 2024-04-23T14:34:47.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.919+0000 7fe6b6800640 1 -- 172.21.15.100:0/3198283044 wait complete. 2024-04-23T14:34:47.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6b6800640 1 Processor -- start 2024-04-23T14:34:47.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6b6800640 1 -- start start 2024-04-23T14:34:47.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6b6800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6b006f950 0x7fe6b013c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:47.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6b6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6b00adb90 0x7fe6b0146da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:47.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6b6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b015e860 0x7fe6b01472e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:47.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6b6800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b006da60 con 0x7fe6b015e860 2024-04-23T14:34:47.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6b6800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b015ec40 con 0x7fe6b00adb90 2024-04-23T14:34:47.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6b6800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe6b006dbe0 con 0x7fe6b006f950 2024-04-23T14:34:47.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6afe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b015e860 0x7fe6b01472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6afe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b015e860 0x7fe6b01472e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36066/0 (socket says 172.21.15.100:36066) 2024-04-23T14:34:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6afe00640 1 -- 172.21.15.100:0/808481234 learned_addr learned my addr 172.21.15.100:0/808481234 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6aea00640 1 --2- 172.21.15.100:0/808481234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6b00adb90 0x7fe6b0146da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6af400640 1 --2- 172.21.15.100:0/808481234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6b006f950 0x7fe6b013c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6afe00640 1 -- 172.21.15.100:0/808481234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6b006f950 msgr2=0x7fe6b013c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6afe00640 1 --2- 172.21.15.100:0/808481234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6b006f950 0x7fe6b013c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6afe00640 1 -- 172.21.15.100:0/808481234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6b00adb90 msgr2=0x7fe6b0146da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6afe00640 1 --2- 172.21.15.100:0/808481234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6b00adb90 0x7fe6b0146da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6afe00640 1 -- 172.21.15.100:0/808481234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b0158e50 con 0x7fe6b015e860 2024-04-23T14:34:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6af400640 1 --2- 172.21.15.100:0/808481234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6b006f950 0x7fe6b013c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6afe00640 1 --2- 172.21.15.100:0/808481234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b015e860 0x7fe6b01472e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800c770 tx=0x7fe6a800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6a7400640 1 -- 172.21.15.100:0/808481234 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8003b80 con 0x7fe6b015e860 2024-04-23T14:34:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6a7400640 1 -- 172.21.15.100:0/808481234 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6a8003d20 con 0x7fe6b015e860 2024-04-23T14:34:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6b6800640 1 -- 172.21.15.100:0/808481234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b0152030 con 0x7fe6b015e860 2024-04-23T14:34:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6a7400640 1 -- 172.21.15.100:0/808481234 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a800aa90 con 0x7fe6b015e860 2024-04-23T14:34:47.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.923+0000 7fe6b6800640 1 -- 172.21.15.100:0/808481234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b013c8e0 con 0x7fe6b015e860 2024-04-23T14:34:47.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.927+0000 7fe6b6800640 1 -- 172.21.15.100:0/808481234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6b0068ad0 con 0x7fe6b015e860 2024-04-23T14:34:47.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.927+0000 7fe6a7400640 1 -- 172.21.15.100:0/808481234 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe6a800ac30 con 0x7fe6b015e860 2024-04-23T14:34:47.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.927+0000 7fe6a7400640 1 --2- 172.21.15.100:0/808481234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe690068380 0x7fe69006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:47.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.927+0000 7fe6a7400640 1 -- 172.21.15.100:0/808481234 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fe6a8086920 con 0x7fe6b015e860 2024-04-23T14:34:47.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.927+0000 7fe6af400640 1 --2- 172.21.15.100:0/808481234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe690068380 0x7fe69006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:47.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.927+0000 7fe6af400640 1 --2- 172.21.15.100:0/808481234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe690068380 0x7fe69006a840 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0000f70 tx=0x7fe6a0007dd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:47.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:47.931+0000 7fe6a7400640 1 -- 172.21.15.100:0/808481234 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe6a8056de0 con 0x7fe6b015e860 2024-04-23T14:34:48.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.087+0000 7fe6b6800640 1 -- 172.21.15.100:0/808481234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe6b01621e0 con 0x7fe6b015e860 2024-04-23T14:34:48.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.087+0000 7fe6a7400640 1 -- 172.21.15.100:0/808481234 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) ==== 93+0+10 (secure 0 0 0) 0x7fe6a80566e0 con 0x7fe6b015e860 2024-04-23T14:34:48.091 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:34:48.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.091+0000 7fe6b6800640 1 -- 172.21.15.100:0/808481234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe690068380 msgr2=0x7fe69006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.091+0000 7fe6b6800640 1 --2- 172.21.15.100:0/808481234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe690068380 0x7fe69006a840 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0000f70 tx=0x7fe6a0007dd0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.091+0000 7fe6b6800640 1 -- 172.21.15.100:0/808481234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b015e860 msgr2=0x7fe6b01472e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.091+0000 7fe6b6800640 1 --2- 172.21.15.100:0/808481234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b015e860 0x7fe6b01472e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800c770 tx=0x7fe6a800cc30 comp rx=0 tx=0).stop 2024-04-23T14:34:48.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.091+0000 7fe6b6800640 1 -- 172.21.15.100:0/808481234 shutdown_connections 2024-04-23T14:34:48.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.091+0000 7fe6b6800640 1 --2- 172.21.15.100:0/808481234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe690068380 0x7fe69006a840 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.091+0000 7fe6b6800640 1 --2- 172.21.15.100:0/808481234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b015e860 0x7fe6b01472e0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.091+0000 7fe6b6800640 1 --2- 172.21.15.100:0/808481234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6b00adb90 0x7fe6b0146da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.091+0000 7fe6b6800640 1 --2- 172.21.15.100:0/808481234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6b006f950 0x7fe6b013c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.091+0000 7fe6b6800640 1 -- 172.21.15.100:0/808481234 >> 172.21.15.100:0/808481234 conn(0x7fe6b0072590 msgr2=0x7fe6b015be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:48.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.091+0000 7fe6b6800640 1 -- 172.21.15.100:0/808481234 shutdown_connections 2024-04-23T14:34:48.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.091+0000 7fe6b6800640 1 -- 172.21.15.100:0/808481234 wait complete. 2024-04-23T14:34:48.110 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:34:48.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe492e00640 1 Processor -- start 2024-04-23T14:34:48.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe492e00640 1 -- start start 2024-04-23T14:34:48.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe492e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fe48c0cca50 0x7fe48c0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe492e00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe48c160f90 con 0x7fe48c15c7e0 2024-04-23T14:34:48.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe492e00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe48c0cd370 con 0x7fe48c15d600 2024-04-23T14:34:48.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe492e00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fe48c0d2270 con 0x7fe48c0cca50 2024-04-23T14:34:48.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe48b400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe48c15c7e0 0x7fe48c160170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:59906/0 (socket says 172.21.15.100:59906) 2024-04-23T14:34:48.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe48b400640 1 -- 172.21.15.100:0/3811763852 learned_addr learned my addr 172.21.15.100:0/3811763852 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:48.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe48be00640 1 --2- 172.21.15.100:0/3811763852 >> v2:172.21.15.151:3300/0 conn(0x7fe48c0cca50 0x7fe48c0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe48aa00640 1 -- 172.21.15.100:0/3811763852 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1689092619 0 0) 0x7fe48c160f90 con 0x7fe48c15c7e0 2024-04-23T14:34:48.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe48aa00640 1 -- 172.21.15.100:0/3811763852 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe470003530 con 0x7fe48c15c7e0 2024-04-23T14:34:48.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe48aa00640 1 -- 172.21.15.100:0/3811763852 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3113788031 0 0) 0x7fe470003530 con 0x7fe48c15c7e0 2024-04-23T14:34:48.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe48aa00640 1 -- 172.21.15.100:0/3811763852 >> v2:172.21.15.151:3300/0 conn(0x7fe48c0cca50 msgr2=0x7fe48c0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:48.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe48aa00640 1 --2- 172.21.15.100:0/3811763852 >> v2:172.21.15.151:3300/0 conn(0x7fe48c0cca50 0x7fe48c0cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe48aa00640 1 -- 172.21.15.100:0/3811763852 >> v1:172.21.15.151:6789/0 conn(0x7fe48c15d600 legacy=0x7fe48c160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe48aa00640 1 -- 172.21.15.100:0/3811763852 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48c0d04a0 con 0x7fe48c15c7e0 2024-04-23T14:34:48.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe48aa00640 1 -- 172.21.15.100:0/3811763852 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7fe478001680 con 0x7fe48c15c7e0 2024-04-23T14:34:48.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe48aa00640 1 -- 172.21.15.100:0/3811763852 >> v1:172.21.15.100:6789/0 conn(0x7fe48c15c7e0 legacy=0x7fe48c160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe48aa00640 1 --2- 172.21.15.100:0/3811763852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4700039c0 0x7fe470003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe48aa00640 1 --2- 172.21.15.100:0/3811763852 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe470004810 0x7fe470004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe48aa00640 1 --2- 172.21.15.100:0/3811763852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4700051d0 0x7fe47000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe48aa00640 1 -- 172.21.15.100:0/3811763852 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe47000c140 con 0x7fe4700039c0 2024-04-23T14:34:48.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe48aa00640 1 -- 172.21.15.100:0/3811763852 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe47000c2f0 con 0x7fe4700051d0 2024-04-23T14:34:48.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.247+0000 7fe48aa00640 1 -- 172.21.15.100:0/3811763852 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe47000c4f0 con 0x7fe470004810 2024-04-23T14:34:48.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.251+0000 7fe48be00640 1 --2- 172.21.15.100:0/3811763852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4700039c0 0x7fe470003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.251+0000 7fe48b400640 1 --2- 172.21.15.100:0/3811763852 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe470004810 0x7fe470004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.251+0000 7fe490e00640 1 --2- 172.21.15.100:0/3811763852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4700051d0 0x7fe47000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.251+0000 7fe48be00640 1 -- 172.21.15.100:0/3811763852 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe470004810 msgr2=0x7fe470004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:48.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.251+0000 7fe48be00640 1 --2- 172.21.15.100:0/3811763852 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe470004810 0x7fe470004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.251+0000 7fe48be00640 1 -- 172.21.15.100:0/3811763852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4700051d0 msgr2=0x7fe47000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:48.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.251+0000 7fe48be00640 1 --2- 172.21.15.100:0/3811763852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4700051d0 0x7fe47000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.251+0000 7fe490e00640 1 -- 172.21.15.100:0/3811763852 reap_dead start 2024-04-23T14:34:48.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.251+0000 7fe48be00640 1 -- 172.21.15.100:0/3811763852 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe470003530 con 0x7fe4700039c0 2024-04-23T14:34:48.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.251+0000 7fe48b400640 1 --2- 172.21.15.100:0/3811763852 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe470004810 0x7fe470004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:48.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.251+0000 7fe48be00640 1 --2- 172.21.15.100:0/3811763852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4700039c0 0x7fe470003de0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe47c009fc0 tx=0x7fe47c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bec7c7b1639179 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:48.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.251+0000 7fe48aa00640 1 -- 172.21.15.100:0/3811763852 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0b2070 con 0x7fe4700039c0 2024-04-23T14:34:48.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.251+0000 7fe48aa00640 1 -- 172.21.15.100:0/3811763852 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe47c0a5b60 con 0x7fe4700039c0 2024-04-23T14:34:48.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.251+0000 7fe48aa00640 1 -- 172.21.15.100:0/3811763852 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c009c10 con 0x7fe4700039c0 2024-04-23T14:34:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe492e00640 1 -- 172.21.15.100:0/3811763852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4700039c0 msgr2=0x7fe470003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe492e00640 1 --2- 172.21.15.100:0/3811763852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4700039c0 0x7fe470003de0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe47c009fc0 tx=0x7fe47c01cc80 comp rx=0 tx=0).stop 2024-04-23T14:34:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe492e00640 1 -- 172.21.15.100:0/3811763852 shutdown_connections 2024-04-23T14:34:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe492e00640 1 --2- 172.21.15.100:0/3811763852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4700039c0 0x7fe470003de0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe492e00640 1 -- 172.21.15.100:0/3811763852 >> 172.21.15.100:0/3811763852 conn(0x7fe48c0cee80 msgr2=0x7fe48c0cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe492e00640 1 -- 172.21.15.100:0/3811763852 shutdown_connections 2024-04-23T14:34:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe492e00640 1 -- 172.21.15.100:0/3811763852 wait complete. 2024-04-23T14:34:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe492e00640 1 Processor -- start 2024-04-23T14:34:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe492e00640 1 -- start start 2024-04-23T14:34:48.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe492e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4700039c0 0x7fe48c1f7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe492e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe48c157490 0x7fe48c15e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe492e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe48c157910 0x7fe48c15ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe492e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe48c0b5430 con 0x7fe48c157910 2024-04-23T14:34:48.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe492e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe48c0b52b0 con 0x7fe4700039c0 2024-04-23T14:34:48.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe492e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe48c0b3bf0 con 0x7fe48c157490 2024-04-23T14:34:48.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe490e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe48c157910 0x7fe48c15ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe490e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe48c157910 0x7fe48c15ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36098/0 (socket says 172.21.15.100:36098) 2024-04-23T14:34:48.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe490e00640 1 -- 172.21.15.100:0/1072495564 learned_addr learned my addr 172.21.15.100:0/1072495564 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:48.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe48be00640 1 --2- 172.21.15.100:0/1072495564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4700039c0 0x7fe48c1f7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe48b400640 1 --2- 172.21.15.100:0/1072495564 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe48c157490 0x7fe48c15e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe490e00640 1 -- 172.21.15.100:0/1072495564 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe48c157490 msgr2=0x7fe48c15e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe490e00640 1 --2- 172.21.15.100:0/1072495564 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe48c157490 0x7fe48c15e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe490e00640 1 -- 172.21.15.100:0/1072495564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4700039c0 msgr2=0x7fe48c1f7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe490e00640 1 --2- 172.21.15.100:0/1072495564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4700039c0 0x7fe48c1f7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.255+0000 7fe490e00640 1 -- 172.21.15.100:0/1072495564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48c0c5f30 con 0x7fe48c157910 2024-04-23T14:34:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.259+0000 7fe490e00640 1 --2- 172.21.15.100:0/1072495564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe48c157910 0x7fe48c15ed50 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe484007fd0 tx=0x7fe48400f8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.259+0000 7fe483e00640 1 -- 172.21.15.100:0/1072495564 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4840053a0 con 0x7fe48c157910 2024-04-23T14:34:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.259+0000 7fe483e00640 1 -- 172.21.15.100:0/1072495564 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe484018070 con 0x7fe48c157910 2024-04-23T14:34:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.259+0000 7fe492e00640 1 -- 172.21.15.100:0/1072495564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe48c0c6210 con 0x7fe48c157910 2024-04-23T14:34:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.259+0000 7fe483e00640 1 -- 172.21.15.100:0/1072495564 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484003840 con 0x7fe48c157910 2024-04-23T14:34:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.259+0000 7fe492e00640 1 -- 172.21.15.100:0/1072495564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe48c0c67d0 con 0x7fe48c157910 2024-04-23T14:34:48.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.259+0000 7fe492e00640 1 -- 172.21.15.100:0/1072495564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe48c068ad0 con 0x7fe48c157910 2024-04-23T14:34:48.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.259+0000 7fe483e00640 1 -- 172.21.15.100:0/1072495564 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe4840039e0 con 0x7fe48c157910 2024-04-23T14:34:48.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.259+0000 7fe483e00640 1 --2- 172.21.15.100:0/1072495564 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe46c068380 0x7fe46c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.259+0000 7fe483e00640 1 -- 172.21.15.100:0/1072495564 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fe484086680 con 0x7fe48c157910 2024-04-23T14:34:48.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.259+0000 7fe48be00640 1 --2- 172.21.15.100:0/1072495564 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe46c068380 0x7fe46c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.263+0000 7fe48be00640 1 --2- 172.21.15.100:0/1072495564 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe46c068380 0x7fe46c06a840 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe47c0af000 tx=0x7fe47c004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:48.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.263+0000 7fe483e00640 1 -- 172.21.15.100:0/1072495564 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe484056aa0 con 0x7fe48c157910 2024-04-23T14:34:48.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.427+0000 7fe492e00640 1 -- 172.21.15.100:0/1072495564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe48c1f79b0 con 0x7fe48c157910 2024-04-23T14:34:48.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.427+0000 7fe483e00640 1 -- 172.21.15.100:0/1072495564 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) ==== 104+0+11 (secure 0 0 0) 0x7fe4840563a0 con 0x7fe48c157910 2024-04-23T14:34:48.430 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:34:48.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.431+0000 7fe492e00640 1 -- 172.21.15.100:0/1072495564 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe46c068380 msgr2=0x7fe46c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.431+0000 7fe492e00640 1 --2- 172.21.15.100:0/1072495564 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe46c068380 0x7fe46c06a840 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe47c0af000 tx=0x7fe47c004ac0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.431+0000 7fe492e00640 1 -- 172.21.15.100:0/1072495564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe48c157910 msgr2=0x7fe48c15ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.431+0000 7fe492e00640 1 --2- 172.21.15.100:0/1072495564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe48c157910 0x7fe48c15ed50 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe484007fd0 tx=0x7fe48400f8e0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.431+0000 7fe492e00640 1 -- 172.21.15.100:0/1072495564 shutdown_connections 2024-04-23T14:34:48.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.431+0000 7fe492e00640 1 --2- 172.21.15.100:0/1072495564 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fe46c068380 0x7fe46c06a840 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.431+0000 7fe492e00640 1 --2- 172.21.15.100:0/1072495564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe48c157910 0x7fe48c15ed50 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.431+0000 7fe492e00640 1 --2- 172.21.15.100:0/1072495564 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe48c157490 0x7fe48c15e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.431+0000 7fe492e00640 1 --2- 172.21.15.100:0/1072495564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4700039c0 0x7fe48c1f7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.431+0000 7fe492e00640 1 -- 172.21.15.100:0/1072495564 >> 172.21.15.100:0/1072495564 conn(0x7fe48c0cee80 msgr2=0x7fe48c0cbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:48.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.431+0000 7fe492e00640 1 -- 172.21.15.100:0/1072495564 shutdown_connections 2024-04-23T14:34:48.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.431+0000 7fe492e00640 1 -- 172.21.15.100:0/1072495564 wait complete. 2024-04-23T14:34:48.450 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:34:48.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.587+0000 7f4a47e00640 1 Processor -- start 2024-04-23T14:34:48.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.587+0000 7f4a47e00640 1 -- start start 2024-04-23T14:34:48.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.587+0000 7f4a47e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4a480d03e0 0x7f4a480b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.587+0000 7f4a47e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4a4815ec40 0x7f4a480b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.587+0000 7f4a47e00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a4815f420 con 0x7f4a480b2900 2024-04-23T14:34:48.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.587+0000 7f4a47e00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f4a480b1c50 con 0x7f4a4815ec40 2024-04-23T14:34:48.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.587+0000 7f4a47e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4a4815f700 con 0x7f4a480d03e0 2024-04-23T14:34:48.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.587+0000 7f4a46000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4a4815ec40 0x7f4a480b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.591+0000 7f4a46000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4a4815ec40 0x7f4a480b23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54072/0 (socket says 172.21.15.100:54072) 2024-04-23T14:34:48.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.591+0000 7f4a46000640 1 -- 172.21.15.100:0/379311775 learned_addr learned my addr 172.21.15.100:0/379311775 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:48.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.591+0000 7f4a46a00640 1 --2- 172.21.15.100:0/379311775 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4a480d03e0 0x7f4a480b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.591+0000 7f4a46000640 1 -- 172.21.15.100:0/379311775 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4a480d03e0 msgr2=0x7f4a480b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:48.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.591+0000 7f4a46000640 1 --2- 172.21.15.100:0/379311775 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4a480d03e0 0x7f4a480b36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.591+0000 7f4a46000640 1 -- 172.21.15.100:0/379311775 >> v1:172.21.15.151:6789/0 conn(0x7f4a480b2900 legacy=0x7f4a4806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.591+0000 7f4a46000640 1 -- 172.21.15.100:0/379311775 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4815f9e0 con 0x7f4a4815ec40 2024-04-23T14:34:48.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.591+0000 7f4a46a00640 1 --2- 172.21.15.100:0/379311775 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4a480d03e0 0x7f4a480b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:48.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.591+0000 7f4a46000640 1 --2- 172.21.15.100:0/379311775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a4815ec40 0x7f4a480b23c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4a34009870 tx=0x7f4a3401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438131789faa422c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.591+0000 7f4a45600640 1 -- 172.21.15.100:0/379311775 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a340a9610 con 0x7f4a4815ec40 2024-04-23T14:34:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.591+0000 7f4a45600640 1 -- 172.21.15.100:0/379311775 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a340a97b0 con 0x7f4a4815ec40 2024-04-23T14:34:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.591+0000 7f4a45600640 1 -- 172.21.15.100:0/379311775 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a340a9b80 con 0x7f4a4815ec40 2024-04-23T14:34:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47e00640 1 -- 172.21.15.100:0/379311775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a4815ec40 msgr2=0x7f4a480b23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47e00640 1 --2- 172.21.15.100:0/379311775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a4815ec40 0x7f4a480b23c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4a34009870 tx=0x7f4a3401c530 comp rx=0 tx=0).stop 2024-04-23T14:34:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47e00640 1 -- 172.21.15.100:0/379311775 shutdown_connections 2024-04-23T14:34:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47e00640 1 --2- 172.21.15.100:0/379311775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a4815ec40 0x7f4a480b23c0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47e00640 1 --2- 172.21.15.100:0/379311775 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4a480d03e0 0x7f4a480b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47e00640 1 -- 172.21.15.100:0/379311775 >> 172.21.15.100:0/379311775 conn(0x7f4a48072420 msgr2=0x7f4a480ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47e00640 1 -- 172.21.15.100:0/379311775 shutdown_connections 2024-04-23T14:34:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47e00640 1 -- 172.21.15.100:0/379311775 wait complete. 2024-04-23T14:34:48.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47e00640 1 Processor -- start 2024-04-23T14:34:48.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47e00640 1 -- start start 2024-04-23T14:34:48.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4a480b2900 0x7f4a48157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a480d03e0 0x7f4a48158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a4815ec40 0x7f4a48154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4a480d0940 con 0x7f4a4815ec40 2024-04-23T14:34:48.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a480d07c0 con 0x7f4a480d03e0 2024-04-23T14:34:48.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4a4806de00 con 0x7f4a480b2900 2024-04-23T14:34:48.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a4815ec40 0x7f4a48154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a4815ec40 0x7f4a48154b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54086/0 (socket says 172.21.15.100:54086) 2024-04-23T14:34:48.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47400640 1 -- 172.21.15.100:0/2484604097 learned_addr learned my addr 172.21.15.100:0/2484604097 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:48.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a46a00640 1 --2- 172.21.15.100:0/2484604097 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4a480b2900 0x7f4a48157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a46000640 1 --2- 172.21.15.100:0/2484604097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a480d03e0 0x7f4a48158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47400640 1 -- 172.21.15.100:0/2484604097 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4a480b2900 msgr2=0x7f4a48157c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47400640 1 --2- 172.21.15.100:0/2484604097 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4a480b2900 0x7f4a48157c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47400640 1 -- 172.21.15.100:0/2484604097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a480d03e0 msgr2=0x7f4a48158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47400640 1 --2- 172.21.15.100:0/2484604097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a480d03e0 0x7f4a48158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47400640 1 -- 172.21.15.100:0/2484604097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a480b7a50 con 0x7f4a4815ec40 2024-04-23T14:34:48.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a46000640 1 --2- 172.21.15.100:0/2484604097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a480d03e0 0x7f4a48158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:48.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.595+0000 7f4a47400640 1 --2- 172.21.15.100:0/2484604097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a4815ec40 0x7f4a48154b60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f4a4000c730 tx=0x7f4a4000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:48.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.599+0000 7f4a3f400640 1 -- 172.21.15.100:0/2484604097 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a40003b80 con 0x7f4a4815ec40 2024-04-23T14:34:48.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.599+0000 7f4a3f400640 1 -- 172.21.15.100:0/2484604097 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a40003d20 con 0x7f4a4815ec40 2024-04-23T14:34:48.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.599+0000 7f4a3f400640 1 -- 172.21.15.100:0/2484604097 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4000aa90 con 0x7f4a4815ec40 2024-04-23T14:34:48.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.599+0000 7f4a47e00640 1 -- 172.21.15.100:0/2484604097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a480b7d30 con 0x7f4a4815ec40 2024-04-23T14:34:48.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.599+0000 7f4a47e00640 1 -- 172.21.15.100:0/2484604097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a48158ed0 con 0x7f4a4815ec40 2024-04-23T14:34:48.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.599+0000 7f4a47e00640 1 -- 172.21.15.100:0/2484604097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a48068ad0 con 0x7f4a4815ec40 2024-04-23T14:34:48.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.599+0000 7f4a3f400640 1 -- 172.21.15.100:0/2484604097 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4a4000ac30 con 0x7f4a4815ec40 2024-04-23T14:34:48.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.599+0000 7f4a3f400640 1 --2- 172.21.15.100:0/2484604097 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4a28068380 0x7f4a2806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.599+0000 7f4a3f400640 1 -- 172.21.15.100:0/2484604097 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f4a400869d0 con 0x7f4a4815ec40 2024-04-23T14:34:48.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.599+0000 7f4a46a00640 1 --2- 172.21.15.100:0/2484604097 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4a28068380 0x7f4a2806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.599+0000 7f4a46a00640 1 --2- 172.21.15.100:0/2484604097 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4a28068380 0x7f4a2806a840 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800c900 tx=0x7f4a38008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:48.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.603+0000 7f4a3f400640 1 -- 172.21.15.100:0/2484604097 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4a40056e00 con 0x7f4a4815ec40 2024-04-23T14:34:48.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.763+0000 7f4a47e00640 1 -- 172.21.15.100:0/2484604097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4a481591c0 con 0x7f4a4815ec40 2024-04-23T14:34:48.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.763+0000 7f4a3f400640 1 -- 172.21.15.100:0/2484604097 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) ==== 100+0+11 (secure 0 0 0) 0x7f4a40056700 con 0x7f4a4815ec40 2024-04-23T14:34:48.764 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:34:48.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.763+0000 7f4a47e00640 1 -- 172.21.15.100:0/2484604097 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4a28068380 msgr2=0x7f4a2806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.763+0000 7f4a47e00640 1 --2- 172.21.15.100:0/2484604097 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4a28068380 0x7f4a2806a840 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800c900 tx=0x7f4a38008040 comp rx=0 tx=0).stop 2024-04-23T14:34:48.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.763+0000 7f4a47e00640 1 -- 172.21.15.100:0/2484604097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a4815ec40 msgr2=0x7f4a48154b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.763+0000 7f4a47e00640 1 --2- 172.21.15.100:0/2484604097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a4815ec40 0x7f4a48154b60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f4a4000c730 tx=0x7f4a4000cbf0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.763+0000 7f4a47e00640 1 -- 172.21.15.100:0/2484604097 shutdown_connections 2024-04-23T14:34:48.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.763+0000 7f4a47e00640 1 --2- 172.21.15.100:0/2484604097 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4a28068380 0x7f4a2806a840 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.763+0000 7f4a47e00640 1 --2- 172.21.15.100:0/2484604097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a4815ec40 0x7f4a48154b60 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.763+0000 7f4a47e00640 1 --2- 172.21.15.100:0/2484604097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a480d03e0 0x7f4a48158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.763+0000 7f4a47e00640 1 --2- 172.21.15.100:0/2484604097 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4a480b2900 0x7f4a48157c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.763+0000 7f4a47e00640 1 -- 172.21.15.100:0/2484604097 >> 172.21.15.100:0/2484604097 conn(0x7f4a48072420 msgr2=0x7f4a4815d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:48.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.763+0000 7f4a47e00640 1 -- 172.21.15.100:0/2484604097 shutdown_connections 2024-04-23T14:34:48.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.763+0000 7f4a47e00640 1 -- 172.21.15.100:0/2484604097 wait complete. 2024-04-23T14:34:48.786 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:34:48.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.923+0000 7f4fa8200640 1 Processor -- start 2024-04-23T14:34:48.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.923+0000 7f4fa8200640 1 -- start start 2024-04-23T14:34:48.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.927+0000 7f4fa8200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f4fa0159180 0x7f4fa006f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.927+0000 7f4fa8200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4fa006fbd0 0x7f4fa00b3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.927+0000 7f4fa8200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4fa00b3ea0 0x7f4fa006c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.927+0000 7f4fa8200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f4fa006e5f0 con 0x7f4fa006fbd0 2024-04-23T14:34:48.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.927+0000 7f4fa8200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f4fa006e8d0 con 0x7f4fa0159180 2024-04-23T14:34:48.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.927+0000 7f4fa8200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4fa006ebb0 con 0x7f4fa00b3ea0 2024-04-23T14:34:48.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.927+0000 7f4fa4e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4fa006fbd0 0x7f4fa00b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.927+0000 7f4fa4e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4fa006fbd0 0x7f4fa00b3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54100/0 (socket says 172.21.15.100:54100) 2024-04-23T14:34:48.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.927+0000 7f4fa4e00640 1 -- 172.21.15.100:0/3876524423 learned_addr learned my addr 172.21.15.100:0/3876524423 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:48.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.927+0000 7f4fa6200640 1 --2- 172.21.15.100:0/3876524423 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4fa00b3ea0 0x7f4fa006c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.927+0000 7f4fa5800640 1 --2- 172.21.15.100:0/3876524423 >> v2:172.21.15.151:3300/0 conn(0x7f4fa0159180 0x7f4fa006f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.927+0000 7f4fa4e00640 1 -- 172.21.15.100:0/3876524423 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4fa00b3ea0 msgr2=0x7f4fa006c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:48.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.927+0000 7f4fa4e00640 1 --2- 172.21.15.100:0/3876524423 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4fa00b3ea0 0x7f4fa006c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.927+0000 7f4fa4e00640 1 -- 172.21.15.100:0/3876524423 >> v2:172.21.15.151:3300/0 conn(0x7f4fa0159180 msgr2=0x7f4fa006f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:48.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.927+0000 7f4fa4e00640 1 --2- 172.21.15.100:0/3876524423 >> v2:172.21.15.151:3300/0 conn(0x7f4fa0159180 0x7f4fa006f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.927+0000 7f4fa4e00640 1 -- 172.21.15.100:0/3876524423 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa006ee90 con 0x7f4fa006fbd0 2024-04-23T14:34:48.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.927+0000 7f4fa5800640 1 --2- 172.21.15.100:0/3876524423 >> v2:172.21.15.151:3300/0 conn(0x7f4fa0159180 0x7f4fa006f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:48.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.927+0000 7f4fa4e00640 1 --2- 172.21.15.100:0/3876524423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa006fbd0 0x7f4fa00b3960 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4fa006bcc0 tx=0x7f4f8c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f8407758c28fdd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:48.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4f9fe00640 1 -- 172.21.15.100:0/3876524423 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c0ab5f0 con 0x7f4fa006fbd0 2024-04-23T14:34:48.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4f9fe00640 1 -- 172.21.15.100:0/3876524423 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f8c0b0070 con 0x7f4fa006fbd0 2024-04-23T14:34:48.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4f9fe00640 1 -- 172.21.15.100:0/3876524423 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c0ab9b0 con 0x7f4fa006fbd0 2024-04-23T14:34:48.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4fa8200640 1 -- 172.21.15.100:0/3876524423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa006fbd0 msgr2=0x7f4fa00b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4fa8200640 1 --2- 172.21.15.100:0/3876524423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa006fbd0 0x7f4fa00b3960 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4fa006bcc0 tx=0x7f4f8c01c520 comp rx=0 tx=0).stop 2024-04-23T14:34:48.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4fa8200640 1 -- 172.21.15.100:0/3876524423 shutdown_connections 2024-04-23T14:34:48.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4fa8200640 1 --2- 172.21.15.100:0/3876524423 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4fa00b3ea0 0x7f4fa006c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4fa8200640 1 --2- 172.21.15.100:0/3876524423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa006fbd0 0x7f4fa00b3960 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4fa8200640 1 --2- 172.21.15.100:0/3876524423 >> v2:172.21.15.151:3300/0 conn(0x7f4fa0159180 0x7f4fa006f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4fa8200640 1 -- 172.21.15.100:0/3876524423 >> 172.21.15.100:0/3876524423 conn(0x7f4fa00d0a50 msgr2=0x7f4fa00d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:48.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4fa8200640 1 -- 172.21.15.100:0/3876524423 shutdown_connections 2024-04-23T14:34:48.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4fa8200640 1 -- 172.21.15.100:0/3876524423 wait complete. 2024-04-23T14:34:48.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4fa8200640 1 Processor -- start 2024-04-23T14:34:48.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4fa8200640 1 -- start start 2024-04-23T14:34:48.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4fa8200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4fa006fbd0 0x7f4fa0149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4fa8200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4fa00b3ea0 0x7f4fa0149560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4fa8200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa0159180 0x7f4fa014aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4fa8200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa0162940 con 0x7f4fa0159180 2024-04-23T14:34:48.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4fa8200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa01627c0 con 0x7f4fa00b3ea0 2024-04-23T14:34:48.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4fa8200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4fa0162ac0 con 0x7f4fa006fbd0 2024-04-23T14:34:48.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4fa6200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa0159180 0x7f4fa014aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4fa6200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa0159180 0x7f4fa014aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54106/0 (socket says 172.21.15.100:54106) 2024-04-23T14:34:48.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4fa6200640 1 -- 172.21.15.100:0/3324616330 learned_addr learned my addr 172.21.15.100:0/3324616330 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:48.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4fa4e00640 1 --2- 172.21.15.100:0/3324616330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4fa00b3ea0 0x7f4fa0149560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.931+0000 7f4fa5800640 1 --2- 172.21.15.100:0/3324616330 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4fa006fbd0 0x7f4fa0149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.935+0000 7f4fa6200640 1 -- 172.21.15.100:0/3324616330 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4fa006fbd0 msgr2=0x7f4fa0149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.935+0000 7f4fa6200640 1 --2- 172.21.15.100:0/3324616330 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4fa006fbd0 0x7f4fa0149c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.935+0000 7f4fa6200640 1 -- 172.21.15.100:0/3324616330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4fa00b3ea0 msgr2=0x7f4fa0149560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.935+0000 7f4fa6200640 1 --2- 172.21.15.100:0/3324616330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4fa00b3ea0 0x7f4fa0149560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.935+0000 7f4fa6200640 1 -- 172.21.15.100:0/3324616330 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa0151c00 con 0x7f4fa0159180 2024-04-23T14:34:48.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.935+0000 7f4fa5800640 1 --2- 172.21.15.100:0/3324616330 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4fa006fbd0 0x7f4fa0149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:48.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.935+0000 7f4fa6200640 1 --2- 172.21.15.100:0/3324616330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa0159180 0x7f4fa014aef0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800eed0 tx=0x7f4f9800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:48.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.935+0000 7f4f9d600640 1 -- 172.21.15.100:0/3324616330 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98018020 con 0x7f4fa0159180 2024-04-23T14:34:48.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.935+0000 7f4f9d600640 1 -- 172.21.15.100:0/3324616330 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f98014400 con 0x7f4fa0159180 2024-04-23T14:34:48.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.935+0000 7f4f9d600640 1 -- 172.21.15.100:0/3324616330 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f980147f0 con 0x7f4fa0159180 2024-04-23T14:34:48.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.935+0000 7f4fa8200640 1 -- 172.21.15.100:0/3324616330 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa0151ee0 con 0x7f4fa0159180 2024-04-23T14:34:48.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.935+0000 7f4fa8200640 1 -- 172.21.15.100:0/3324616330 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa014a310 con 0x7f4fa0159180 2024-04-23T14:34:48.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.935+0000 7f4fa8200640 1 -- 172.21.15.100:0/3324616330 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fa014da00 con 0x7f4fa0159180 2024-04-23T14:34:48.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.935+0000 7f4f9d600640 1 -- 172.21.15.100:0/3324616330 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4f9801b020 con 0x7f4fa0159180 2024-04-23T14:34:48.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.935+0000 7f4f9d600640 1 --2- 172.21.15.100:0/3324616330 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4f800683b0 0x7f4f8006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.935+0000 7f4f9d600640 1 -- 172.21.15.100:0/3324616330 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f4f98087570 con 0x7f4fa0159180 2024-04-23T14:34:48.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.935+0000 7f4fa5800640 1 --2- 172.21.15.100:0/3324616330 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4f800683b0 0x7f4f8006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.939+0000 7f4fa5800640 1 --2- 172.21.15.100:0/3324616330 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4f800683b0 0x7f4f8006a870 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4f9400b990 tx=0x7f4f94008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:48.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:48.943+0000 7f4f9d600640 1 -- 172.21.15.100:0/3324616330 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4f98022030 con 0x7f4fa0159180 2024-04-23T14:34:49.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:49.111+0000 7f4fa8200640 1 -- 172.21.15.100:0/3324616330 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4fa0068ad0 con 0x7f4fa0159180 2024-04-23T14:34:49.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:49.111+0000 7f4f9d600640 1 -- 172.21.15.100:0/3324616330 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) ==== 80+0+0 (secure 0 0 0) 0x7f4f98057900 con 0x7f4fa0159180 2024-04-23T14:34:49.111 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-04-23T14:34:49.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:49.111+0000 7f4fa8200640 1 -- 172.21.15.100:0/3324616330 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4f800683b0 msgr2=0x7f4f8006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:49.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:49.111+0000 7f4fa8200640 1 --2- 172.21.15.100:0/3324616330 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4f800683b0 0x7f4f8006a870 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4f9400b990 tx=0x7f4f94008040 comp rx=0 tx=0).stop 2024-04-23T14:34:49.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:49.111+0000 7f4fa8200640 1 -- 172.21.15.100:0/3324616330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa0159180 msgr2=0x7f4fa014aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:49.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:49.111+0000 7f4fa8200640 1 --2- 172.21.15.100:0/3324616330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa0159180 0x7f4fa014aef0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800eed0 tx=0x7f4f9800b480 comp rx=0 tx=0).stop 2024-04-23T14:34:49.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:49.111+0000 7f4fa8200640 1 -- 172.21.15.100:0/3324616330 shutdown_connections 2024-04-23T14:34:49.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:49.111+0000 7f4fa8200640 1 --2- 172.21.15.100:0/3324616330 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4f800683b0 0x7f4f8006a870 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:49.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:49.111+0000 7f4fa8200640 1 --2- 172.21.15.100:0/3324616330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa0159180 0x7f4fa014aef0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:49.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:49.111+0000 7f4fa8200640 1 --2- 172.21.15.100:0/3324616330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4fa00b3ea0 0x7f4fa0149560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:49.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:49.111+0000 7f4fa8200640 1 --2- 172.21.15.100:0/3324616330 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4fa006fbd0 0x7f4fa0149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:49.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:49.111+0000 7f4fa8200640 1 -- 172.21.15.100:0/3324616330 >> 172.21.15.100:0/3324616330 conn(0x7f4fa00d0a50 msgr2=0x7f4fa00b4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:49.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:49.111+0000 7f4fa8200640 1 -- 172.21.15.100:0/3324616330 shutdown_connections 2024-04-23T14:34:49.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:49.111+0000 7f4fa8200640 1 -- 172.21.15.100:0/3324616330 wait complete. 2024-04-23T14:34:49.131 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi100.front.sepia.ceph.com... 2024-04-23T14:34:49.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:49.132 DEBUG:teuthology.orchestra.run.smithi100:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:49.149 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[42401]: fuse finished with error 0 and tester_r 0 2024-04-23T14:34:49.153 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:34:55.254 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:34:55.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:55.255 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:34:55.266 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:34:55.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.451+0000 7f5c72000640 1 Processor -- start 2024-04-23T14:34:55.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.451+0000 7f5c72000640 1 -- start start 2024-04-23T14:34:55.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.451+0000 7f5c72000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5c6c0afe40 0x7f5c6c15d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:55.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.451+0000 7f5c72000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c6c15cac0 con 0x7f5c6c06da60 2024-04-23T14:34:55.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.451+0000 7f5c72000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c6c073450 con 0x7f5c6c153420 2024-04-23T14:34:55.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.451+0000 7f5c72000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f5c6c15cd60 con 0x7f5c6c0afe40 2024-04-23T14:34:55.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.451+0000 7f5c71600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5c6c0afe40 0x7f5c6c15d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:55.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.451+0000 7f5c71600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5c6c0afe40 0x7f5c6c15d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54108/0 (socket says 172.21.15.100:54108) 2024-04-23T14:34:55.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.451+0000 7f5c6be00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f5c6c06da60 0x7f5c6c0cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:56826/0 (socket says 172.21.15.100:56826) 2024-04-23T14:34:55.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.451+0000 7f5c71600640 1 -- 172.21.15.100:0/4259915317 learned_addr learned my addr 172.21.15.100:0/4259915317 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:55.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.451+0000 7f5c71600640 1 -- 172.21.15.100:0/4259915317 >> v1:172.21.15.151:6789/0 conn(0x7f5c6c153420 legacy=0x7f5c6c06d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:55.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.451+0000 7f5c71600640 1 -- 172.21.15.100:0/4259915317 >> v1:172.21.15.100:6789/0 conn(0x7f5c6c06da60 legacy=0x7f5c6c0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:55.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.451+0000 7f5c71600640 1 -- 172.21.15.100:0/4259915317 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6c15d040 con 0x7f5c6c0afe40 2024-04-23T14:34:55.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.455+0000 7f5c71600640 1 --2- 172.21.15.100:0/4259915317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c6c0afe40 0x7f5c6c15d560 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009e50 tx=0x7f5c6401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71119f7233b1292d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:55.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.455+0000 7f5c6b400640 1 -- 172.21.15.100:0/4259915317 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c640a5830 con 0x7f5c6c0afe40 2024-04-23T14:34:55.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.455+0000 7f5c6b400640 1 -- 172.21.15.100:0/4259915317 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c640a59d0 con 0x7f5c6c0afe40 2024-04-23T14:34:55.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.455+0000 7f5c6b400640 1 -- 172.21.15.100:0/4259915317 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c640a5da0 con 0x7f5c6c0afe40 2024-04-23T14:34:55.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.455+0000 7f5c72000640 1 -- 172.21.15.100:0/4259915317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c6c0afe40 msgr2=0x7f5c6c15d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:55.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.455+0000 7f5c72000640 1 --2- 172.21.15.100:0/4259915317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c6c0afe40 0x7f5c6c15d560 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009e50 tx=0x7f5c6401ca00 comp rx=0 tx=0).stop 2024-04-23T14:34:55.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.455+0000 7f5c72000640 1 -- 172.21.15.100:0/4259915317 shutdown_connections 2024-04-23T14:34:55.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.455+0000 7f5c72000640 1 --2- 172.21.15.100:0/4259915317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c6c0afe40 0x7f5c6c15d560 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:55.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.455+0000 7f5c72000640 1 -- 172.21.15.100:0/4259915317 >> 172.21.15.100:0/4259915317 conn(0x7f5c6c0724c0 msgr2=0x7f5c6c0728d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:55.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.455+0000 7f5c72000640 1 -- 172.21.15.100:0/4259915317 shutdown_connections 2024-04-23T14:34:55.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.455+0000 7f5c72000640 1 -- 172.21.15.100:0/4259915317 wait complete. 2024-04-23T14:34:55.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c72000640 1 Processor -- start 2024-04-23T14:34:55.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c72000640 1 -- start start 2024-04-23T14:34:55.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c72000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5c6c06da60 0x7f5c6c14c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:55.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c72000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c6c0afe40 0x7f5c6c1521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:55.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c72000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c6c153420 0x7f5c6c139d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:55.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c72000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c073160 con 0x7f5c6c0afe40 2024-04-23T14:34:55.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c72000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c072fe0 con 0x7f5c6c153420 2024-04-23T14:34:55.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c72000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5c6c16a120 con 0x7f5c6c06da60 2024-04-23T14:34:55.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c6be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c6c0afe40 0x7f5c6c1521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:55.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c6be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c6c0afe40 0x7f5c6c1521e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54114/0 (socket says 172.21.15.100:54114) 2024-04-23T14:34:55.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c6be00640 1 -- 172.21.15.100:0/4116309881 learned_addr learned my addr 172.21.15.100:0/4116309881 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:55.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c70c00640 1 --2- 172.21.15.100:0/4116309881 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5c6c06da60 0x7f5c6c14c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:55.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c71600640 1 --2- 172.21.15.100:0/4116309881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c6c153420 0x7f5c6c139d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:55.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c6be00640 1 -- 172.21.15.100:0/4116309881 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5c6c06da60 msgr2=0x7f5c6c14c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:55.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c6be00640 1 --2- 172.21.15.100:0/4116309881 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5c6c06da60 0x7f5c6c14c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:55.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c6be00640 1 -- 172.21.15.100:0/4116309881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c6c153420 msgr2=0x7f5c6c139d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:55.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c6be00640 1 --2- 172.21.15.100:0/4116309881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c6c153420 0x7f5c6c139d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:55.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c6be00640 1 -- 172.21.15.100:0/4116309881 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6c153ad0 con 0x7f5c6c0afe40 2024-04-23T14:34:55.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c6be00640 1 --2- 172.21.15.100:0/4116309881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c6c0afe40 0x7f5c6c1521e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800c490 tx=0x7f5c5800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:55.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c68c00640 1 -- 172.21.15.100:0/4116309881 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c58018070 con 0x7f5c6c0afe40 2024-04-23T14:34:55.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c68c00640 1 -- 172.21.15.100:0/4116309881 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c58007480 con 0x7f5c6c0afe40 2024-04-23T14:34:55.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c72000640 1 -- 172.21.15.100:0/4116309881 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c6c151ad0 con 0x7f5c6c0afe40 2024-04-23T14:34:55.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c68c00640 1 -- 172.21.15.100:0/4116309881 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c58004a10 con 0x7f5c6c0afe40 2024-04-23T14:34:55.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c72000640 1 -- 172.21.15.100:0/4116309881 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c6c14e4c0 con 0x7f5c6c0afe40 2024-04-23T14:34:55.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.459+0000 7f5c72000640 1 -- 172.21.15.100:0/4116309881 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c6c148750 con 0x7f5c6c0afe40 2024-04-23T14:34:55.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.463+0000 7f5c68c00640 1 -- 172.21.15.100:0/4116309881 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5c5801b070 con 0x7f5c6c0afe40 2024-04-23T14:34:55.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.463+0000 7f5c68c00640 1 --2- 172.21.15.100:0/4116309881 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5c4c068380 0x7f5c4c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:55.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.463+0000 7f5c68c00640 1 -- 172.21.15.100:0/4116309881 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f5c58085d90 con 0x7f5c6c0afe40 2024-04-23T14:34:55.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.463+0000 7f5c70c00640 1 --2- 172.21.15.100:0/4116309881 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5c4c068380 0x7f5c4c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:55.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.463+0000 7f5c70c00640 1 --2- 172.21.15.100:0/4116309881 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5c4c068380 0x7f5c4c06a840 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5c60001770 tx=0x7f5c60009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:55.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.467+0000 7f5c68c00640 1 -- 172.21.15.100:0/4116309881 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5c58055ad0 con 0x7f5c6c0afe40 2024-04-23T14:34:55.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.655+0000 7f5c72000640 1 -- 172.21.15.100:0/4116309881 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5c6c0678b0 con 0x7f5c6c0afe40 2024-04-23T14:34:55.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.655+0000 7f5c68c00640 1 -- 172.21.15.100:0/4116309881 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+73 (secure 0 0 0) 0x7f5c58059dd0 con 0x7f5c6c0afe40 2024-04-23T14:34:55.658 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:34:55.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.659+0000 7f5c72000640 1 -- 172.21.15.100:0/4116309881 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5c4c068380 msgr2=0x7f5c4c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:55.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.659+0000 7f5c72000640 1 --2- 172.21.15.100:0/4116309881 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5c4c068380 0x7f5c4c06a840 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5c60001770 tx=0x7f5c60009040 comp rx=0 tx=0).stop 2024-04-23T14:34:55.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.659+0000 7f5c72000640 1 -- 172.21.15.100:0/4116309881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c6c0afe40 msgr2=0x7f5c6c1521e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:55.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.659+0000 7f5c72000640 1 --2- 172.21.15.100:0/4116309881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c6c0afe40 0x7f5c6c1521e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800c490 tx=0x7f5c5800c950 comp rx=0 tx=0).stop 2024-04-23T14:34:55.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.659+0000 7f5c72000640 1 -- 172.21.15.100:0/4116309881 shutdown_connections 2024-04-23T14:34:55.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.659+0000 7f5c72000640 1 --2- 172.21.15.100:0/4116309881 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f5c4c068380 0x7f5c4c06a840 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:55.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.659+0000 7f5c72000640 1 --2- 172.21.15.100:0/4116309881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c6c153420 0x7f5c6c139d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:55.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.659+0000 7f5c72000640 1 --2- 172.21.15.100:0/4116309881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c6c0afe40 0x7f5c6c1521e0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:55.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.659+0000 7f5c72000640 1 --2- 172.21.15.100:0/4116309881 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5c6c06da60 0x7f5c6c14c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:55.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.659+0000 7f5c72000640 1 -- 172.21.15.100:0/4116309881 >> 172.21.15.100:0/4116309881 conn(0x7f5c6c0724c0 msgr2=0x7f5c6c0728a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:55.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.659+0000 7f5c72000640 1 -- 172.21.15.100:0/4116309881 shutdown_connections 2024-04-23T14:34:55.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55.659+0000 7f5c72000640 1 -- 172.21.15.100:0/4116309881 wait complete. 2024-04-23T14:34:55.679 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:34:55.679 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:34:55.679 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-04-23T14:34:55.679 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:34:55.679 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:55.679 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:34:55.680 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:34:55.680 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:34:55.680 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:34:55.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:55.680 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-04-23T14:34:55.689 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:34:55.689 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:34:55.689 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:34:55.689 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:55.690 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-23T14:34:55.690 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:55.690 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:34:55.690 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:55.690 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-04-23T14:34:55.690 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:34:55.690 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:55.690 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-04-23T14:34:55.690 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-23T14:34:55.690 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:55.690 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:34:55.690 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:55.690 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-23T14:34:55.690 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42288sec preferred_lft 42288sec 2024-04-23T14:34:55.690 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-04-23T14:34:55.691 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:55.691 INFO:teuthology.orchestra.run.smithi100.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-23T14:34:55.691 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 02:42:ff:fc:d1:cc brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:55.691 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-23T14:34:55.691 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:55.691 INFO:teuthology.orchestra.run.smithi100.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:34:55.691 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-23T14:34:55.691 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:34:55.691 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:55.691 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-23T14:34:55.691 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:55.691 INFO:teuthology.orchestra.run.smithi100.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:34:55.691 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:34:55.691 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-23T14:34:55.692 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:55.692 INFO:teuthology.orchestra.run.smithi100.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:34:55.692 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:34:55.692 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-23T14:34:55.692 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:34:55.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:55.692 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:34:55.751 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-04-23T14:34:55.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:55.753 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-04-23T14:34:55.809 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-04-23T14:34:55.810 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:34:55.810 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-23T14:34:55.810 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-23T14:34:55.810 INFO:teuthology.orchestra.run.smithi100.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-23T14:34:55.810 INFO:teuthology.orchestra.run.smithi100.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-23T14:34:55.810 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-23T14:34:55.810 INFO:teuthology.orchestra.run.smithi100.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:34:55.810 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:34:55.810 INFO:teuthology.orchestra.run.smithi100.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-23T14:34:55.810 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:34:55.811 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:34:55.811 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-23T14:34:55.811 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-23T14:34:55.811 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-23T14:34:55.811 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-23T14:34:55.811 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-23T14:34:55.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:34:55.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:34:55.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:55.943 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-23T14:34:55.952 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:34:55.952 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:34:55.953 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:34:55.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:55.953 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-04-23T14:34:56.017 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:34:56.017 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:34:56.017 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:56.070 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:34:56.070 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:34:56.070 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:56.117 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-23T14:34:56.182 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:34:56.236 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-23T14:34:56.236 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-23T14:34:56.236 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-23T14:34:56.236 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-23T14:34:56.236 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-23T14:34:56.236 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-23T14:34:56.236 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-23T14:34:56.236 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-23T14:34:56.236 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-23T14:34:56.236 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-23T14:34:56.236 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-23T14:34:56.237 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-23T14:34:56.237 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:34:56.237 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-23T14:34:56.237 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-23T14:34:56.237 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-23T14:34:56.237 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-23T14:34:56.237 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-23T14:34:56.237 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-23T14:34:56.237 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-23T14:34:56.237 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:34:56.237 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-23T14:34:56.237 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-23T14:34:56.238 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-23T14:34:56.238 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:34:56.238 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:34:56.238 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:34:56.238 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:34:56.238 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-23T14:34:56.238 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:34:56.238 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:34:56.238 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:34:56.238 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:34:56.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:56.239 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-23T14:34:56.294 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:34:56.294 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:34:56.344 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-23T14:34:56.365 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:34:56.419 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-23T14:34:56.419 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-23T14:34:56.419 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-23T14:34:56.420 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-23T14:34:56.420 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-23T14:34:56.420 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-23T14:34:56.420 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-23T14:34:56.420 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-23T14:34:56.420 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-23T14:34:56.420 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-23T14:34:56.421 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-23T14:34:56.421 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-23T14:34:56.421 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:34:56.421 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-23T14:34:56.421 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-23T14:34:56.421 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-23T14:34:56.422 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-23T14:34:56.422 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-23T14:34:56.422 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-23T14:34:56.422 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-23T14:34:56.422 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-23T14:34:56.422 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-23T14:34:56.423 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-23T14:34:56.423 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-23T14:34:56.423 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:34:56.423 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:34:56.423 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:34:56.423 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:34:56.423 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-23T14:34:56.424 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:34:56.424 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:34:56.424 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:34:56.424 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:34:56.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:56.426 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-23T14:34:56.440 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:2024-04-23T14:34:56.439+0000 7efece446580 -1 init, newargv = 0x55e1efb36c20 newargc=15 2024-04-23T14:34:56.440 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[42802]: starting ceph client 2024-04-23T14:34:56.447 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[42802]: starting fuse 2024-04-23T14:34:56.449 INFO:teuthology.orchestra.run.smithi100.stdout:44 2024-04-23T14:34:56.449 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-23T14:34:56.449 DEBUG:teuthology.orchestra.run.smithi100:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:34:56.450 DEBUG:teuthology.orchestra.run.smithi100:> import glob 2024-04-23T14:34:56.450 DEBUG:teuthology.orchestra.run.smithi100:> import re 2024-04-23T14:34:56.450 DEBUG:teuthology.orchestra.run.smithi100:> import os 2024-04-23T14:34:56.450 DEBUG:teuthology.orchestra.run.smithi100:> import subprocess 2024-04-23T14:34:56.450 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:34:56.450 DEBUG:teuthology.orchestra.run.smithi100:> def _find_admin_socket(client_name): 2024-04-23T14:34:56.450 DEBUG:teuthology.orchestra.run.smithi100:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:34:56.450 DEBUG:teuthology.orchestra.run.smithi100:> files = glob.glob(asok_path) 2024-04-23T14:34:56.450 DEBUG:teuthology.orchestra.run.smithi100:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:34:56.450 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:34:56.450 DEBUG:teuthology.orchestra.run.smithi100:> # Given a non-glob path, it better be there 2024-04-23T14:34:56.450 DEBUG:teuthology.orchestra.run.smithi100:> if "*" not in asok_path: 2024-04-23T14:34:56.451 DEBUG:teuthology.orchestra.run.smithi100:> assert(len(files) == 1) 2024-04-23T14:34:56.451 DEBUG:teuthology.orchestra.run.smithi100:> return files[0] 2024-04-23T14:34:56.451 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:34:56.451 DEBUG:teuthology.orchestra.run.smithi100:> for f in files: 2024-04-23T14:34:56.451 DEBUG:teuthology.orchestra.run.smithi100:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:34:56.451 DEBUG:teuthology.orchestra.run.smithi100:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:34:56.451 DEBUG:teuthology.orchestra.run.smithi100:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:34:56.451 DEBUG:teuthology.orchestra.run.smithi100:> contents = proc_f.read() 2024-04-23T14:34:56.451 DEBUG:teuthology.orchestra.run.smithi100:> if mountpoint in contents: 2024-04-23T14:34:56.451 DEBUG:teuthology.orchestra.run.smithi100:> return f 2024-04-23T14:34:56.451 DEBUG:teuthology.orchestra.run.smithi100:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:34:56.451 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:34:56.451 DEBUG:teuthology.orchestra.run.smithi100:> print(_find_admin_socket("client.0")) 2024-04-23T14:34:56.451 DEBUG:teuthology.orchestra.run.smithi100:> ' 2024-04-23T14:34:56.573 INFO:teuthology.orchestra.run.smithi100.stdout:/var/run/ceph/ceph-client.0.42802.asok 2024-04-23T14:34:56.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:34:56.584 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42802.asok 2024-04-23T14:34:56.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:56.584 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42802.asok status 2024-04-23T14:34:56.752 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:34:56.752 INFO:teuthology.orchestra.run.smithi100.stdout: "metadata": { 2024-04-23T14:34:56.752 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:34:56.752 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:34:56.752 INFO:teuthology.orchestra.run.smithi100.stdout: "entity_id": "0", 2024-04-23T14:34:56.752 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": "smithi100", 2024-04-23T14:34:56.753 INFO:teuthology.orchestra.run.smithi100.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:34:56.753 INFO:teuthology.orchestra.run.smithi100.stdout: "pid": "42802", 2024-04-23T14:34:56.753 INFO:teuthology.orchestra.run.smithi100.stdout: "root": "/" 2024-04-23T14:34:56.753 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:56.753 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_count": 0, 2024-04-23T14:34:56.753 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_pinned_count": 0, 2024-04-23T14:34:56.753 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 6534, 2024-04-23T14:34:56.753 INFO:teuthology.orchestra.run.smithi100.stdout: "inst": { 2024-04-23T14:34:56.753 INFO:teuthology.orchestra.run.smithi100.stdout: "name": { 2024-04-23T14:34:56.753 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "client", 2024-04-23T14:34:56.753 INFO:teuthology.orchestra.run.smithi100.stdout: "num": 6534 2024-04-23T14:34:56.753 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:56.753 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-23T14:34:56.753 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-23T14:34:56.753 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:34:56.754 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 974587438 2024-04-23T14:34:56.754 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-23T14:34:56.754 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:56.754 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-23T14:34:56.754 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-23T14:34:56.754 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:34:56.754 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 974587438 2024-04-23T14:34:56.754 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-23T14:34:56.754 INFO:teuthology.orchestra.run.smithi100.stdout: "inst_str": "client.6534 192.168.0.1:0/974587438", 2024-04-23T14:34:56.754 INFO:teuthology.orchestra.run.smithi100.stdout: "addr_str": "192.168.0.1:0/974587438", 2024-04-23T14:34:56.754 INFO:teuthology.orchestra.run.smithi100.stdout: "inode_count": 1, 2024-04-23T14:34:56.754 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_epoch": 28, 2024-04-23T14:34:56.754 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch": 53, 2024-04-23T14:34:56.754 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:34:56.754 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklisted": false, 2024-04-23T14:34:56.754 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs" 2024-04-23T14:34:56.755 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:34:56.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:56.767 DEBUG:teuthology.orchestra.run.smithi100:> stat --file-system '--printf=%T 2024-04-23T14:34:56.768 DEBUG:teuthology.orchestra.run.smithi100:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:56.776 INFO:teuthology.orchestra.run.smithi100.stdout:fuseblk 2024-04-23T14:34:56.776 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:56.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:56.777 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:56.837 DEBUG:teuthology.orchestra.run.smithi100:> stdin-killer -- python3 -c ' 2024-04-23T14:34:56.837 DEBUG:teuthology.orchestra.run.smithi100:> import os 2024-04-23T14:34:56.837 DEBUG:teuthology.orchestra.run.smithi100:> import errno 2024-04-23T14:34:56.837 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-23T14:34:56.837 DEBUG:teuthology.orchestra.run.smithi100:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2024-04-23T14:34:56.837 DEBUG:teuthology.orchestra.run.smithi100:> try: 2024-04-23T14:34:56.837 DEBUG:teuthology.orchestra.run.smithi100:> if True: 2024-04-23T14:34:56.838 DEBUG:teuthology.orchestra.run.smithi100:> ret = os.read(fd, 1024) 2024-04-23T14:34:56.838 DEBUG:teuthology.orchestra.run.smithi100:> else: 2024-04-23T14:34:56.838 DEBUG:teuthology.orchestra.run.smithi100:> os.write(fd, b'"'"'content'"'"') 2024-04-23T14:34:56.838 DEBUG:teuthology.orchestra.run.smithi100:> except OSError as e: 2024-04-23T14:34:56.838 DEBUG:teuthology.orchestra.run.smithi100:> if e.errno != errno.EPERM: 2024-04-23T14:34:56.838 DEBUG:teuthology.orchestra.run.smithi100:> raise 2024-04-23T14:34:56.838 DEBUG:teuthology.orchestra.run.smithi100:> else: 2024-04-23T14:34:56.838 DEBUG:teuthology.orchestra.run.smithi100:> raise RuntimeError("client does not check permission of data pool") 2024-04-23T14:34:56.838 DEBUG:teuthology.orchestra.run.smithi100:> ' 2024-04-23T14:34:56.943 INFO:teuthology.orchestra.run.smithi100.stderr:Traceback (most recent call last): 2024-04-23T14:34:56.943 INFO:teuthology.orchestra.run.smithi100.stderr: File "", line 15, in 2024-04-23T14:34:56.943 INFO:teuthology.orchestra.run.smithi100.stderr:RuntimeError: client does not check permission of data pool 2024-04-23T14:34:56.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:56 stdin-killer INFO: command exited with status 1: exiting normally with same code! 2024-04-23T14:34:56.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:34:56.952 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2024-04-23T14:34:57.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.135+0000 7fde65c00640 1 Processor -- start 2024-04-23T14:34:57.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.135+0000 7fde65c00640 1 -- start start 2024-04-23T14:34:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.135+0000 7fde65c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde600cd8b0 0x7fde600718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.135+0000 7fde65c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fde60071e30 0x7fde600d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.135+0000 7fde65c00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde600ccaf0 con 0x7fde6015e9c0 2024-04-23T14:34:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.135+0000 7fde65c00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fde6015eda0 con 0x7fde60071e30 2024-04-23T14:34:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.135+0000 7fde65c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fde600ccdd0 con 0x7fde600cd8b0 2024-04-23T14:34:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.135+0000 7fde5f400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fde60071e30 0x7fde600d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.135+0000 7fde5f400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fde60071e30 0x7fde600d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54126/0 (socket says 172.21.15.100:54126) 2024-04-23T14:34:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.135+0000 7fde5f400640 1 -- 172.21.15.100:0/2696124376 learned_addr learned my addr 172.21.15.100:0/2696124376 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.135+0000 7fde5e000640 1 --2- 172.21.15.100:0/2696124376 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde600cd8b0 0x7fde600718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.135+0000 7fde5f400640 1 -- 172.21.15.100:0/2696124376 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde600cd8b0 msgr2=0x7fde600718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.135+0000 7fde5f400640 1 --2- 172.21.15.100:0/2696124376 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde600cd8b0 0x7fde600718f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.135+0000 7fde5f400640 1 -- 172.21.15.100:0/2696124376 >> v1:172.21.15.151:6789/0 conn(0x7fde6015e9c0 legacy=0x7fde6006d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.135+0000 7fde5f400640 1 -- 172.21.15.100:0/2696124376 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde600cd0b0 con 0x7fde60071e30 2024-04-23T14:34:57.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.139+0000 7fde5f400640 1 --2- 172.21.15.100:0/2696124376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde60071e30 0x7fde600d0300 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fde58009870 tx=0x7fde5801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d38570646d77f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:57.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.139+0000 7fde5d600640 1 -- 172.21.15.100:0/2696124376 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde580b2070 con 0x7fde60071e30 2024-04-23T14:34:57.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.139+0000 7fde5d600640 1 -- 172.21.15.100:0/2696124376 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde580a9070 con 0x7fde60071e30 2024-04-23T14:34:57.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.139+0000 7fde5d600640 1 -- 172.21.15.100:0/2696124376 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde580ad880 con 0x7fde60071e30 2024-04-23T14:34:57.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.139+0000 7fde65c00640 1 -- 172.21.15.100:0/2696124376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde60071e30 msgr2=0x7fde600d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.139+0000 7fde65c00640 1 --2- 172.21.15.100:0/2696124376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde60071e30 0x7fde600d0300 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fde58009870 tx=0x7fde5801c560 comp rx=0 tx=0).stop 2024-04-23T14:34:57.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.139+0000 7fde65c00640 1 -- 172.21.15.100:0/2696124376 shutdown_connections 2024-04-23T14:34:57.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.139+0000 7fde65c00640 1 --2- 172.21.15.100:0/2696124376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde60071e30 0x7fde600d0300 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.139+0000 7fde65c00640 1 --2- 172.21.15.100:0/2696124376 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde600cd8b0 0x7fde600718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.139+0000 7fde65c00640 1 -- 172.21.15.100:0/2696124376 >> 172.21.15.100:0/2696124376 conn(0x7fde600d2400 msgr2=0x7fde600d2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:57.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.139+0000 7fde65c00640 1 -- 172.21.15.100:0/2696124376 shutdown_connections 2024-04-23T14:34:57.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.139+0000 7fde65c00640 1 -- 172.21.15.100:0/2696124376 wait complete. 2024-04-23T14:34:57.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde65c00640 1 Processor -- start 2024-04-23T14:34:57.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde65c00640 1 -- start start 2024-04-23T14:34:57.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde65c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde60071e30 0x7fde60156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde65c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde600cd8b0 0x7fde6015a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde65c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde6015e9c0 0x7fde60161fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde65c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fde6006db60 con 0x7fde600cd8b0 2024-04-23T14:34:57.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde65c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde6016b870 con 0x7fde6015e9c0 2024-04-23T14:34:57.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde65c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fde60157030 con 0x7fde60071e30 2024-04-23T14:34:57.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde5e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde600cd8b0 0x7fde6015a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde5e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde600cd8b0 0x7fde6015a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54130/0 (socket says 172.21.15.100:54130) 2024-04-23T14:34:57.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde5e000640 1 -- 172.21.15.100:0/3304004417 learned_addr learned my addr 172.21.15.100:0/3304004417 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:57.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde5ea00640 1 --2- 172.21.15.100:0/3304004417 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde60071e30 0x7fde60156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde5f400640 1 --2- 172.21.15.100:0/3304004417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde6015e9c0 0x7fde60161fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde5e000640 1 -- 172.21.15.100:0/3304004417 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde60071e30 msgr2=0x7fde60156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde5e000640 1 --2- 172.21.15.100:0/3304004417 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde60071e30 0x7fde60156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde5e000640 1 -- 172.21.15.100:0/3304004417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde6015e9c0 msgr2=0x7fde60161fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde5e000640 1 --2- 172.21.15.100:0/3304004417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde6015e9c0 0x7fde60161fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde5e000640 1 -- 172.21.15.100:0/3304004417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde60158540 con 0x7fde600cd8b0 2024-04-23T14:34:57.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde5f400640 1 --2- 172.21.15.100:0/3304004417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde6015e9c0 0x7fde60161fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:57.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde5e000640 1 --2- 172.21.15.100:0/3304004417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde600cd8b0 0x7fde6015a070 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00d800 tx=0x7fde4c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:57.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde56a00640 1 -- 172.21.15.100:0/3304004417 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c00b6c0 con 0x7fde600cd8b0 2024-04-23T14:34:57.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde56a00640 1 -- 172.21.15.100:0/3304004417 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde4c019070 con 0x7fde600cd8b0 2024-04-23T14:34:57.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde56a00640 1 -- 172.21.15.100:0/3304004417 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c00bae0 con 0x7fde600cd8b0 2024-04-23T14:34:57.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde65c00640 1 -- 172.21.15.100:0/3304004417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde601571e0 con 0x7fde600cd8b0 2024-04-23T14:34:57.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.143+0000 7fde65c00640 1 -- 172.21.15.100:0/3304004417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde60157780 con 0x7fde600cd8b0 2024-04-23T14:34:57.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.147+0000 7fde65c00640 1 -- 172.21.15.100:0/3304004417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde60068ad0 con 0x7fde600cd8b0 2024-04-23T14:34:57.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.147+0000 7fde56a00640 1 -- 172.21.15.100:0/3304004417 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fde4c00bc80 con 0x7fde600cd8b0 2024-04-23T14:34:57.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.147+0000 7fde56a00640 1 --2- 172.21.15.100:0/3304004417 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fde40068380 0x7fde4006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.147+0000 7fde56a00640 1 -- 172.21.15.100:0/3304004417 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fde4c0870e0 con 0x7fde600cd8b0 2024-04-23T14:34:57.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.147+0000 7fde5ea00640 1 --2- 172.21.15.100:0/3304004417 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fde40068380 0x7fde4006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.147+0000 7fde5ea00640 1 --2- 172.21.15.100:0/3304004417 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fde40068380 0x7fde4006a840 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fde6015e0e0 tx=0x7fde50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:57.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.151+0000 7fde56a00640 1 -- 172.21.15.100:0/3304004417 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fde4c057470 con 0x7fde600cd8b0 2024-04-23T14:34:57.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.363+0000 7fde65c00640 1 -- 172.21.15.100:0/3304004417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) -- 0x7fde600b36a0 con 0x7fde600cd8b0 2024-04-23T14:34:57.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.363+0000 7fde56a00640 1 -- 172.21.15.100:0/3304004417 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v18) ==== 184+0+141 (secure 0 0 0) 0x7fde4c056d70 con 0x7fde600cd8b0 2024-04-23T14:34:57.366 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-04-23T14:34:57.367 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAtxidmyhImMBAAa49ApQEF2Lng6rY94F1HAQ== 2024-04-23T14:34:57.367 INFO:teuthology.orchestra.run.smithi100.stdout: caps mds = "allow" 2024-04-23T14:34:57.367 INFO:teuthology.orchestra.run.smithi100.stdout: caps mon = "allow r" 2024-04-23T14:34:57.367 INFO:teuthology.orchestra.run.smithi100.stdout: caps osd = "allow rw pool=cephfs_data" 2024-04-23T14:34:57.367 INFO:teuthology.orchestra.run.smithi100.stderr:updated caps for client.0 2024-04-23T14:34:57.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.367+0000 7fde65c00640 1 -- 172.21.15.100:0/3304004417 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fde40068380 msgr2=0x7fde4006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.367+0000 7fde65c00640 1 --2- 172.21.15.100:0/3304004417 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fde40068380 0x7fde4006a840 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fde6015e0e0 tx=0x7fde50009040 comp rx=0 tx=0).stop 2024-04-23T14:34:57.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.367+0000 7fde65c00640 1 -- 172.21.15.100:0/3304004417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde600cd8b0 msgr2=0x7fde6015a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.367+0000 7fde65c00640 1 --2- 172.21.15.100:0/3304004417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde600cd8b0 0x7fde6015a070 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00d800 tx=0x7fde4c00dcc0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.367+0000 7fde65c00640 1 -- 172.21.15.100:0/3304004417 shutdown_connections 2024-04-23T14:34:57.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.367+0000 7fde65c00640 1 --2- 172.21.15.100:0/3304004417 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fde40068380 0x7fde4006a840 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.367+0000 7fde65c00640 1 --2- 172.21.15.100:0/3304004417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde6015e9c0 0x7fde60161fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.367+0000 7fde65c00640 1 --2- 172.21.15.100:0/3304004417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde600cd8b0 0x7fde6015a070 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.367+0000 7fde65c00640 1 --2- 172.21.15.100:0/3304004417 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde60071e30 0x7fde60156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.367+0000 7fde65c00640 1 -- 172.21.15.100:0/3304004417 >> 172.21.15.100:0/3304004417 conn(0x7fde600d2400 msgr2=0x7fde600d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:57.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.367+0000 7fde65c00640 1 -- 172.21.15.100:0/3304004417 shutdown_connections 2024-04-23T14:34:57.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.367+0000 7fde65c00640 1 -- 172.21.15.100:0/3304004417 wait complete. 2024-04-23T14:34:57.388 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-23T14:34:57.388 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:34:57.489 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:34:57.570 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-23T14:34:57.570 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:34:57.571 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:34:57.571 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:57.580 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:57.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.755+0000 7f12ffa00640 1 Processor -- start 2024-04-23T14:34:57.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.755+0000 7f12ffa00640 1 -- start start 2024-04-23T14:34:57.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.755+0000 7f12ffa00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12f815c470 0x7f12f80b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.755+0000 7f12ffa00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12f8160320 con 0x7f12f8160de0 2024-04-23T14:34:57.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.755+0000 7f12ffa00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12f80cc7e0 con 0x7f12f80ad870 2024-04-23T14:34:57.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.755+0000 7f12ffa00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f12f81605e0 con 0x7f12f815c470 2024-04-23T14:34:57.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.755+0000 7f12f7e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f12f8160de0 0x7f12f80ad160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:56840/0 (socket says 172.21.15.100:56840) 2024-04-23T14:34:57.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.755+0000 7f12f7e00640 1 -- 172.21.15.100:0/3569458403 learned_addr learned my addr 172.21.15.100:0/3569458403 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:57.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.755+0000 7f12f7400640 1 -- 172.21.15.100:0/3569458403 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3736178251 0 0) 0x7f12f8160320 con 0x7f12f8160de0 2024-04-23T14:34:57.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.755+0000 7f12fd000640 1 --2- 172.21.15.100:0/3569458403 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12f815c470 0x7f12f80b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.755+0000 7f12f7400640 1 -- 172.21.15.100:0/3569458403 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12e0003530 con 0x7f12f8160de0 2024-04-23T14:34:57.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.755+0000 7f12f7400640 1 -- 172.21.15.100:0/3569458403 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 431703583 0 0) 0x7f12e0003530 con 0x7f12f8160de0 2024-04-23T14:34:57.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.755+0000 7f12f7400640 1 -- 172.21.15.100:0/3569458403 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12f815c470 msgr2=0x7f12f80b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:57.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.755+0000 7f12f7400640 1 --2- 172.21.15.100:0/3569458403 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12f815c470 0x7f12f80b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.755+0000 7f12f7400640 1 -- 172.21.15.100:0/3569458403 >> v1:172.21.15.151:6789/0 conn(0x7f12f80ad870 legacy=0x7f12f806bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.755+0000 7f12f7400640 1 -- 172.21.15.100:0/3569458403 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f81608c0 con 0x7f12f8160de0 2024-04-23T14:34:57.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.755+0000 7f12f7400640 1 -- 172.21.15.100:0/3569458403 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f12e40016a0 con 0x7f12f8160de0 2024-04-23T14:34:57.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.759+0000 7f12f7400640 1 -- 172.21.15.100:0/3569458403 >> v1:172.21.15.100:6789/0 conn(0x7f12f8160de0 legacy=0x7f12f80ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.759+0000 7f12f7400640 1 --2- 172.21.15.100:0/3569458403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e00039f0 0x7f12e0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.759+0000 7f12f7400640 1 --2- 172.21.15.100:0/3569458403 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12f815c470 0x7f12e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.759+0000 7f12f7400640 1 --2- 172.21.15.100:0/3569458403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12e0004d50 0x7f12e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.759+0000 7f12f7400640 1 -- 172.21.15.100:0/3569458403 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f12f81605e0 con 0x7f12e00039f0 2024-04-23T14:34:57.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.759+0000 7f12f7400640 1 -- 172.21.15.100:0/3569458403 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f12e000bb00 con 0x7f12e0004d50 2024-04-23T14:34:57.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.759+0000 7f12f7400640 1 -- 172.21.15.100:0/3569458403 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f12e000bd00 con 0x7f12f815c470 2024-04-23T14:34:57.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.759+0000 7f12fd000640 1 --2- 172.21.15.100:0/3569458403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e00039f0 0x7f12e0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.759+0000 7f12f7e00640 1 --2- 172.21.15.100:0/3569458403 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12f815c470 0x7f12e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.759+0000 7f12fda00640 1 --2- 172.21.15.100:0/3569458403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12e0004d50 0x7f12e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.759+0000 7f12fd000640 1 -- 172.21.15.100:0/3569458403 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12f815c470 msgr2=0x7f12e0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:57.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.759+0000 7f12fd000640 1 --2- 172.21.15.100:0/3569458403 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12f815c470 0x7f12e0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.759+0000 7f12fd000640 1 -- 172.21.15.100:0/3569458403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12e0004d50 msgr2=0x7f12e00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:57.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.759+0000 7f12fd000640 1 --2- 172.21.15.100:0/3569458403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12e0004d50 0x7f12e00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.759+0000 7f12fd000640 1 -- 172.21.15.100:0/3569458403 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e0003530 con 0x7f12e00039f0 2024-04-23T14:34:57.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.759+0000 7f12fda00640 1 --2- 172.21.15.100:0/3569458403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12e0004d50 0x7f12e00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:57.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.759+0000 7f12fd000640 1 --2- 172.21.15.100:0/3569458403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e00039f0 0x7f12e0003e10 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009ae0 tx=0x7f12ec01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1869036e4b1ce422 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:57.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.759+0000 7f12f7400640 1 -- 172.21.15.100:0/3569458403 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec0096b0 con 0x7f12e00039f0 2024-04-23T14:34:57.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.759+0000 7f12f7400640 1 -- 172.21.15.100:0/3569458403 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12ec0a64a0 con 0x7f12e00039f0 2024-04-23T14:34:57.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.759+0000 7f12f7400640 1 -- 172.21.15.100:0/3569458403 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec0a6890 con 0x7f12e00039f0 2024-04-23T14:34:57.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12ffa00640 1 -- 172.21.15.100:0/3569458403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e00039f0 msgr2=0x7f12e0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12ffa00640 1 --2- 172.21.15.100:0/3569458403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e00039f0 0x7f12e0003e10 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009ae0 tx=0x7f12ec01c9b0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12fda00640 1 -- 172.21.15.100:0/3569458403 reap_dead start 2024-04-23T14:34:57.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12ffa00640 1 -- 172.21.15.100:0/3569458403 shutdown_connections 2024-04-23T14:34:57.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12ffa00640 1 -- 172.21.15.100:0/3569458403 >> 172.21.15.100:0/3569458403 conn(0x7f12f80b1e80 msgr2=0x7f12f80b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:57.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12ffa00640 1 -- 172.21.15.100:0/3569458403 shutdown_connections 2024-04-23T14:34:57.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12ffa00640 1 -- 172.21.15.100:0/3569458403 wait complete. 2024-04-23T14:34:57.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12ffa00640 1 Processor -- start 2024-04-23T14:34:57.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12ffa00640 1 -- start start 2024-04-23T14:34:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12ffa00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12e00039f0 0x7f12f8159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12ffa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12f815d2f0 0x7f12f815a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12ffa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12f81567c0 0x7f12f8156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12ffa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f12f8168540 con 0x7f12f81567c0 2024-04-23T14:34:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12ffa00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f12f815ca60 con 0x7f12f815d2f0 2024-04-23T14:34:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12ffa00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f12f81686c0 con 0x7f12e00039f0 2024-04-23T14:34:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12fda00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12f81567c0 0x7f12f8156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12fda00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12f81567c0 0x7f12f8156c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54154/0 (socket says 172.21.15.100:54154) 2024-04-23T14:34:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12fda00640 1 -- 172.21.15.100:0/1322294129 learned_addr learned my addr 172.21.15.100:0/1322294129 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12fd000640 1 --2- 172.21.15.100:0/1322294129 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12e00039f0 0x7f12f8159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12f7e00640 1 --2- 172.21.15.100:0/1322294129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12f815d2f0 0x7f12f815a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12fda00640 1 -- 172.21.15.100:0/1322294129 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12e00039f0 msgr2=0x7f12f8159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12fda00640 1 --2- 172.21.15.100:0/1322294129 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12e00039f0 0x7f12f8159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12fda00640 1 -- 172.21.15.100:0/1322294129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12f815d2f0 msgr2=0x7f12f815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12fda00640 1 --2- 172.21.15.100:0/1322294129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12f815d2f0 0x7f12f815a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12fda00640 1 -- 172.21.15.100:0/1322294129 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f8157190 con 0x7f12f81567c0 2024-04-23T14:34:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12fd000640 1 --2- 172.21.15.100:0/1322294129 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12e00039f0 0x7f12f8159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.763+0000 7f12f7e00640 1 --2- 172.21.15.100:0/1322294129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12f815d2f0 0x7f12f815a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.767+0000 7f12fda00640 1 --2- 172.21.15.100:0/1322294129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12f81567c0 0x7f12f8156c50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f12f0000f90 tx=0x7f12f000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.767+0000 7f12f4c00640 1 -- 172.21.15.100:0/1322294129 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f0006b60 con 0x7f12f81567c0 2024-04-23T14:34:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.767+0000 7f12f4c00640 1 -- 172.21.15.100:0/1322294129 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12f0006d00 con 0x7f12f81567c0 2024-04-23T14:34:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.767+0000 7f12f4c00640 1 -- 172.21.15.100:0/1322294129 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f0003980 con 0x7f12f81567c0 2024-04-23T14:34:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.767+0000 7f12ffa00640 1 -- 172.21.15.100:0/1322294129 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f80ce070 con 0x7f12f81567c0 2024-04-23T14:34:57.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.767+0000 7f12ffa00640 1 -- 172.21.15.100:0/1322294129 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f80ce630 con 0x7f12f81567c0 2024-04-23T14:34:57.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.767+0000 7f12ffa00640 1 -- 172.21.15.100:0/1322294129 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12f8068ad0 con 0x7f12f81567c0 2024-04-23T14:34:57.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.767+0000 7f12f4c00640 1 -- 172.21.15.100:0/1322294129 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f12f001a020 con 0x7f12f81567c0 2024-04-23T14:34:57.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.767+0000 7f12f4c00640 1 --2- 172.21.15.100:0/1322294129 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f12d8068380 0x7f12d806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.767+0000 7f12f4c00640 1 -- 172.21.15.100:0/1322294129 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f12f00860d0 con 0x7f12f81567c0 2024-04-23T14:34:57.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.767+0000 7f12fd000640 1 --2- 172.21.15.100:0/1322294129 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f12d8068380 0x7f12d806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.771+0000 7f12fd000640 1 --2- 172.21.15.100:0/1322294129 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f12d8068380 0x7f12d806a840 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0ad000 tx=0x7f12ec0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:57.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.771+0000 7f12f4c00640 1 -- 172.21.15.100:0/1322294129 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f12f8157190 con 0x7f12f81567c0 2024-04-23T14:34:57.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.931+0000 7f12ffa00640 1 -- 172.21.15.100:0/1322294129 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12f806fea0 con 0x7f12f81567c0 2024-04-23T14:34:57.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.935+0000 7f12f4c00640 1 -- 172.21.15.100:0/1322294129 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16950 (secure 0 0 0) 0x7f12f0056590 con 0x7f12f81567c0 2024-04-23T14:34:57.935 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:57.935 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":53,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:34:25.795515+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:34:22.491379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-23T14:34:23.120342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.935+0000 7f12ffa00640 1 -- 172.21.15.100:0/1322294129 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f12d8068380 msgr2=0x7f12d806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.935+0000 7f12ffa00640 1 --2- 172.21.15.100:0/1322294129 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f12d8068380 0x7f12d806a840 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0ad000 tx=0x7f12ec0046d0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.935+0000 7f12ffa00640 1 -- 172.21.15.100:0/1322294129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12f81567c0 msgr2=0x7f12f8156c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.935+0000 7f12ffa00640 1 --2- 172.21.15.100:0/1322294129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12f81567c0 0x7f12f8156c50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f12f0000f90 tx=0x7f12f000eb90 comp rx=0 tx=0).stop 2024-04-23T14:34:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.935+0000 7f12ffa00640 1 -- 172.21.15.100:0/1322294129 shutdown_connections 2024-04-23T14:34:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.935+0000 7f12ffa00640 1 --2- 172.21.15.100:0/1322294129 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f12d8068380 0x7f12d806a840 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.935+0000 7f12ffa00640 1 --2- 172.21.15.100:0/1322294129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12f81567c0 0x7f12f8156c50 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.935+0000 7f12ffa00640 1 --2- 172.21.15.100:0/1322294129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12f815d2f0 0x7f12f815a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.935+0000 7f12ffa00640 1 --2- 172.21.15.100:0/1322294129 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12e00039f0 0x7f12f8159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.935+0000 7f12ffa00640 1 -- 172.21.15.100:0/1322294129 >> 172.21.15.100:0/1322294129 conn(0x7f12f80b1e80 msgr2=0x7f12f80b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.935+0000 7f12ffa00640 1 -- 172.21.15.100:0/1322294129 shutdown_connections 2024-04-23T14:34:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:57.935+0000 7f12ffa00640 1 -- 172.21.15.100:0/1322294129 wait complete. 2024-04-23T14:34:57.957 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:34:22.491379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:34:23.120342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:34:57.957 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:34:58.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.095+0000 7ff83fe00640 1 Processor -- start 2024-04-23T14:34:58.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.095+0000 7ff83fe00640 1 -- start start 2024-04-23T14:34:58.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.095+0000 7ff83fe00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff83815e860 0x7ff8380b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.095+0000 7ff83fe00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff8380adb90 0x7ff8380adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.095+0000 7ff83fe00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff83806e5f0 con 0x7ff83806f950 2024-04-23T14:34:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.095+0000 7ff83fe00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff83806e8b0 con 0x7ff8380adb90 2024-04-23T14:34:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.095+0000 7ff83fe00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff83806eb90 con 0x7ff83815e860 2024-04-23T14:34:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.095+0000 7ff83ca00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff8380adb90 0x7ff8380adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.095+0000 7ff83ca00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff8380adb90 0x7ff8380adfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54156/0 (socket says 172.21.15.100:54156) 2024-04-23T14:34:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.095+0000 7ff83ca00640 1 -- 172.21.15.100:0/1017846114 learned_addr learned my addr 172.21.15.100:0/1017846114 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.095+0000 7ff83d400640 1 --2- 172.21.15.100:0/1017846114 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff83815e860 0x7ff8380b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.099+0000 7ff83ca00640 1 -- 172.21.15.100:0/1017846114 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff83815e860 msgr2=0x7ff8380b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.099+0000 7ff83ca00640 1 --2- 172.21.15.100:0/1017846114 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff83815e860 0x7ff8380b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.099+0000 7ff83ca00640 1 -- 172.21.15.100:0/1017846114 >> v1:172.21.15.151:6789/0 conn(0x7ff83806f950 legacy=0x7ff83806d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.099+0000 7ff83ca00640 1 -- 172.21.15.100:0/1017846114 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83806ee70 con 0x7ff8380adb90 2024-04-23T14:34:58.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.099+0000 7ff83ca00640 1 --2- 172.21.15.100:0/1017846114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8380adb90 0x7ff8380adfb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff824009870 tx=0x7ff82401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd38e23f6e6e4ff server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:58.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.099+0000 7ff837e00640 1 -- 172.21.15.100:0/1017846114 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8240a95f0 con 0x7ff8380adb90 2024-04-23T14:34:58.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.099+0000 7ff837e00640 1 -- 172.21.15.100:0/1017846114 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8240a9790 con 0x7ff8380adb90 2024-04-23T14:34:58.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.099+0000 7ff837e00640 1 -- 172.21.15.100:0/1017846114 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8240a9b60 con 0x7ff8380adb90 2024-04-23T14:34:58.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.099+0000 7ff83fe00640 1 -- 172.21.15.100:0/1017846114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8380adb90 msgr2=0x7ff8380adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.099+0000 7ff83fe00640 1 --2- 172.21.15.100:0/1017846114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8380adb90 0x7ff8380adfb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff824009870 tx=0x7ff82401c530 comp rx=0 tx=0).stop 2024-04-23T14:34:58.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83fe00640 1 -- 172.21.15.100:0/1017846114 shutdown_connections 2024-04-23T14:34:58.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83fe00640 1 --2- 172.21.15.100:0/1017846114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8380adb90 0x7ff8380adfb0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83fe00640 1 --2- 172.21.15.100:0/1017846114 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff83815e860 0x7ff8380b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83fe00640 1 -- 172.21.15.100:0/1017846114 >> 172.21.15.100:0/1017846114 conn(0x7ff838072590 msgr2=0x7ff8380729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:58.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83fe00640 1 -- 172.21.15.100:0/1017846114 shutdown_connections 2024-04-23T14:34:58.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83fe00640 1 -- 172.21.15.100:0/1017846114 wait complete. 2024-04-23T14:34:58.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83fe00640 1 Processor -- start 2024-04-23T14:34:58.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83fe00640 1 -- start start 2024-04-23T14:34:58.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff83806f950 0x7ff838142ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83fe00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff8380adb90 0x7ff83815edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff83815e860 0x7ff83815f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff83815ec40 con 0x7ff83806f950 2024-04-23T14:34:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83fe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff838073790 con 0x7ff83815e860 2024-04-23T14:34:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83fe00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff83806da60 con 0x7ff8380adb90 2024-04-23T14:34:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff83806f950 0x7ff838142ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff83806f950 0x7ff838142ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54172/0 (socket says 172.21.15.100:54172) 2024-04-23T14:34:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83d400640 1 -- 172.21.15.100:0/2867874251 learned_addr learned my addr 172.21.15.100:0/2867874251 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83ca00640 1 --2- 172.21.15.100:0/2867874251 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff8380adb90 0x7ff83815edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83de00640 1 --2- 172.21.15.100:0/2867874251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff83815e860 0x7ff83815f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83d400640 1 -- 172.21.15.100:0/2867874251 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff8380adb90 msgr2=0x7ff83815edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83d400640 1 --2- 172.21.15.100:0/2867874251 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff8380adb90 0x7ff83815edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83d400640 1 -- 172.21.15.100:0/2867874251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff83815e860 msgr2=0x7ff83815f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83d400640 1 --2- 172.21.15.100:0/2867874251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff83815e860 0x7ff83815f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83d400640 1 -- 172.21.15.100:0/2867874251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff838143840 con 0x7ff83806f950 2024-04-23T14:34:58.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.103+0000 7ff83d400640 1 --2- 172.21.15.100:0/2867874251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff83806f950 0x7ff838142ff0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00d5f0 tx=0x7ff82c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:58.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.107+0000 7ff835600640 1 -- 172.21.15.100:0/2867874251 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0125f0 con 0x7ff83806f950 2024-04-23T14:34:58.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.107+0000 7ff835600640 1 -- 172.21.15.100:0/2867874251 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff82c019070 con 0x7ff83806f950 2024-04-23T14:34:58.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.107+0000 7ff835600640 1 -- 172.21.15.100:0/2867874251 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0129d0 con 0x7ff83806f950 2024-04-23T14:34:58.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.107+0000 7ff83fe00640 1 -- 172.21.15.100:0/2867874251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff838143b20 con 0x7ff83806f950 2024-04-23T14:34:58.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.107+0000 7ff83fe00640 1 -- 172.21.15.100:0/2867874251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8381440e0 con 0x7ff83806f950 2024-04-23T14:34:58.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.107+0000 7ff83fe00640 1 -- 172.21.15.100:0/2867874251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff838068ad0 con 0x7ff83806f950 2024-04-23T14:34:58.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.107+0000 7ff835600640 1 -- 172.21.15.100:0/2867874251 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff82c012b70 con 0x7ff83806f950 2024-04-23T14:34:58.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.107+0000 7ff835600640 1 --2- 172.21.15.100:0/2867874251 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff818068380 0x7ff81806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.107+0000 7ff835600640 1 -- 172.21.15.100:0/2867874251 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7ff82c086f10 con 0x7ff83806f950 2024-04-23T14:34:58.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.107+0000 7ff83ca00640 1 --2- 172.21.15.100:0/2867874251 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff818068380 0x7ff81806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.107+0000 7ff83ca00640 1 --2- 172.21.15.100:0/2867874251 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff818068380 0x7ff81806a840 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff8240ae000 tx=0x7ff8240b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:58.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.111+0000 7ff835600640 1 -- 172.21.15.100:0/2867874251 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff82c057340 con 0x7ff83806f950 2024-04-23T14:34:58.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.267+0000 7ff83fe00640 1 -- 172.21.15.100:0/2867874251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff8381621e0 con 0x7ff83806f950 2024-04-23T14:34:58.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.267+0000 7ff835600640 1 -- 172.21.15.100:0/2867874251 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) ==== 93+0+10 (secure 0 0 0) 0x7ff82c056c40 con 0x7ff83806f950 2024-04-23T14:34:58.270 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:34:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.271+0000 7ff83fe00640 1 -- 172.21.15.100:0/2867874251 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff818068380 msgr2=0x7ff81806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.271+0000 7ff83fe00640 1 --2- 172.21.15.100:0/2867874251 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff818068380 0x7ff81806a840 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff8240ae000 tx=0x7ff8240b3040 comp rx=0 tx=0).stop 2024-04-23T14:34:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.271+0000 7ff83fe00640 1 -- 172.21.15.100:0/2867874251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff83806f950 msgr2=0x7ff838142ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.271+0000 7ff83fe00640 1 --2- 172.21.15.100:0/2867874251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff83806f950 0x7ff838142ff0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00d5f0 tx=0x7ff82c00dab0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.271+0000 7ff83fe00640 1 -- 172.21.15.100:0/2867874251 shutdown_connections 2024-04-23T14:34:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.271+0000 7ff83fe00640 1 --2- 172.21.15.100:0/2867874251 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff818068380 0x7ff81806a840 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.271+0000 7ff83fe00640 1 --2- 172.21.15.100:0/2867874251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff83815e860 0x7ff83815f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.271+0000 7ff83fe00640 1 --2- 172.21.15.100:0/2867874251 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff8380adb90 0x7ff83815edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.271+0000 7ff83fe00640 1 --2- 172.21.15.100:0/2867874251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff83806f950 0x7ff838142ff0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.271+0000 7ff83fe00640 1 -- 172.21.15.100:0/2867874251 >> 172.21.15.100:0/2867874251 conn(0x7ff838072590 msgr2=0x7ff838154b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.271+0000 7ff83fe00640 1 -- 172.21.15.100:0/2867874251 shutdown_connections 2024-04-23T14:34:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.271+0000 7ff83fe00640 1 -- 172.21.15.100:0/2867874251 wait complete. 2024-04-23T14:34:58.290 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:34:58.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.427+0000 7f7fb3a00640 1 Processor -- start 2024-04-23T14:34:58.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.427+0000 7f7fb3a00640 1 -- start start 2024-04-23T14:34:58.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.427+0000 7f7fb3a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7fac15ae60 0x7f7fac0cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.427+0000 7f7fb3a00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fac0cfc70 con 0x7f7fac06c9a0 2024-04-23T14:34:58.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.427+0000 7f7fb3a00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fac06bbe0 con 0x7f7fac0d0a70 2024-04-23T14:34:58.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.427+0000 7f7fb3a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7fac0caf40 con 0x7f7fac15ae60 2024-04-23T14:34:58.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.427+0000 7f7fb1a00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f7fac06c9a0 0x7f7fac06ce30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:56856/0 (socket says 172.21.15.100:56856) 2024-04-23T14:34:58.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.427+0000 7f7fb1a00640 1 -- 172.21.15.100:0/111581806 learned_addr learned my addr 172.21.15.100:0/111581806 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:58.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.427+0000 7f7fb1000640 1 --2- 172.21.15.100:0/111581806 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7fac15ae60 0x7f7fac0cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.427+0000 7f7fab400640 1 -- 172.21.15.100:0/111581806 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2210501031 0 0) 0x7f7fac0cfc70 con 0x7f7fac06c9a0 2024-04-23T14:34:58.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.427+0000 7f7fab400640 1 -- 172.21.15.100:0/111581806 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f90003450 con 0x7f7fac06c9a0 2024-04-23T14:34:58.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.427+0000 7f7fab400640 1 -- 172.21.15.100:0/111581806 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3595177821 0 0) 0x7f7fac06bbe0 con 0x7f7fac0d0a70 2024-04-23T14:34:58.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.427+0000 7f7fab400640 1 -- 172.21.15.100:0/111581806 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fac0cfc70 con 0x7f7fac0d0a70 2024-04-23T14:34:58.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.427+0000 7f7fb1000640 1 -- 172.21.15.100:0/111581806 >> v1:172.21.15.151:6789/0 conn(0x7f7fac0d0a70 legacy=0x7f7fac0ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.427+0000 7f7fb1000640 1 -- 172.21.15.100:0/111581806 >> v1:172.21.15.100:6789/0 conn(0x7f7fac06c9a0 legacy=0x7f7fac06ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.427+0000 7f7fb1000640 1 -- 172.21.15.100:0/111581806 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fac06bec0 con 0x7f7fac15ae60 2024-04-23T14:34:58.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.431+0000 7f7fb1000640 1 --2- 172.21.15.100:0/111581806 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7fac15ae60 0x7f7fac0cf730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0097f0 tx=0x7f7f9c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abb46fe3776c7f94 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:58.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.431+0000 7f7fab400640 1 -- 172.21.15.100:0/111581806 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c0b0070 con 0x7f7fac15ae60 2024-04-23T14:34:58.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.431+0000 7f7fab400640 1 -- 172.21.15.100:0/111581806 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f9c0ab600 con 0x7f7fac15ae60 2024-04-23T14:34:58.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.431+0000 7f7fab400640 1 -- 172.21.15.100:0/111581806 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c0a4e60 con 0x7f7fac15ae60 2024-04-23T14:34:58.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.431+0000 7f7fb3a00640 1 -- 172.21.15.100:0/111581806 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7fac15ae60 msgr2=0x7f7fac0cf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.431+0000 7f7fb3a00640 1 --2- 172.21.15.100:0/111581806 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7fac15ae60 0x7f7fac0cf730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0097f0 tx=0x7f7f9c01c450 comp rx=0 tx=0).stop 2024-04-23T14:34:58.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.431+0000 7f7fb3a00640 1 -- 172.21.15.100:0/111581806 shutdown_connections 2024-04-23T14:34:58.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.431+0000 7f7fb3a00640 1 --2- 172.21.15.100:0/111581806 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7fac15ae60 0x7f7fac0cf730 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.431+0000 7f7fb3a00640 1 -- 172.21.15.100:0/111581806 >> 172.21.15.100:0/111581806 conn(0x7f7fac0b5200 msgr2=0x7f7fac0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:58.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.431+0000 7f7fb3a00640 1 -- 172.21.15.100:0/111581806 shutdown_connections 2024-04-23T14:34:58.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.431+0000 7f7fb3a00640 1 -- 172.21.15.100:0/111581806 wait complete. 2024-04-23T14:34:58.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.435+0000 7f7fb3a00640 1 Processor -- start 2024-04-23T14:34:58.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.435+0000 7f7fb3a00640 1 -- start start 2024-04-23T14:34:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.435+0000 7f7fb3a00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7fac06c9a0 0x7f7fac1fa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.435+0000 7f7fb3a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fac0d0a70 0x7f7fac154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.435+0000 7f7fb3a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac15ae60 0x7f7fac0aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.435+0000 7f7fb3a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac0ccdf0 con 0x7f7fac15ae60 2024-04-23T14:34:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.435+0000 7f7fb3a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac0ccc70 con 0x7f7fac0d0a70 2024-04-23T14:34:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.435+0000 7f7fb3a00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7fac0ccf70 con 0x7f7fac06c9a0 2024-04-23T14:34:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.435+0000 7f7fb1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac15ae60 0x7f7fac0aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.435+0000 7f7fb1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac15ae60 0x7f7fac0aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54178/0 (socket says 172.21.15.100:54178) 2024-04-23T14:34:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.435+0000 7f7fb1a00640 1 -- 172.21.15.100:0/2018853160 learned_addr learned my addr 172.21.15.100:0/2018853160 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.435+0000 7f7fb1000640 1 --2- 172.21.15.100:0/2018853160 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7fac06c9a0 0x7f7fac1fa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.435+0000 7f7fabe00640 1 --2- 172.21.15.100:0/2018853160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fac0d0a70 0x7f7fac154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.435+0000 7f7fb1a00640 1 -- 172.21.15.100:0/2018853160 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7fac06c9a0 msgr2=0x7f7fac1fa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.435+0000 7f7fb1a00640 1 --2- 172.21.15.100:0/2018853160 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7fac06c9a0 0x7f7fac1fa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.435+0000 7f7fb1a00640 1 -- 172.21.15.100:0/2018853160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fac0d0a70 msgr2=0x7f7fac154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.435+0000 7f7fb1a00640 1 --2- 172.21.15.100:0/2018853160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fac0d0a70 0x7f7fac154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.435+0000 7f7fb1a00640 1 -- 172.21.15.100:0/2018853160 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fac154880 con 0x7f7fac15ae60 2024-04-23T14:34:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.435+0000 7f7fb1000640 1 --2- 172.21.15.100:0/2018853160 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7fac06c9a0 0x7f7fac1fa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:58.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.435+0000 7f7fb1a00640 1 --2- 172.21.15.100:0/2018853160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac15ae60 0x7f7fac0aee70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400cba0 tx=0x7f7fa400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.435+0000 7f7fa8c00640 1 -- 172.21.15.100:0/2018853160 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4017070 con 0x7f7fac15ae60 2024-04-23T14:34:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.439+0000 7f7fa8c00640 1 -- 172.21.15.100:0/2018853160 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fa400ace0 con 0x7f7fac15ae60 2024-04-23T14:34:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.439+0000 7f7fa8c00640 1 -- 172.21.15.100:0/2018853160 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4012620 con 0x7f7fac15ae60 2024-04-23T14:34:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.439+0000 7f7fb3a00640 1 -- 172.21.15.100:0/2018853160 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fac1f92e0 con 0x7f7fac15ae60 2024-04-23T14:34:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.439+0000 7f7fb3a00640 1 -- 172.21.15.100:0/2018853160 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fac1f98a0 con 0x7f7fac15ae60 2024-04-23T14:34:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.439+0000 7f7fb3a00640 1 -- 172.21.15.100:0/2018853160 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fac068ad0 con 0x7f7fac15ae60 2024-04-23T14:34:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.439+0000 7f7fa8c00640 1 -- 172.21.15.100:0/2018853160 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7fa401a070 con 0x7f7fac15ae60 2024-04-23T14:34:58.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.439+0000 7f7fa8c00640 1 --2- 172.21.15.100:0/2018853160 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7f8c0683b0 0x7f7f8c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.439+0000 7f7fa8c00640 1 -- 172.21.15.100:0/2018853160 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f7fa4086a40 con 0x7f7fac15ae60 2024-04-23T14:34:58.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.439+0000 7f7fb1000640 1 --2- 172.21.15.100:0/2018853160 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7f8c0683b0 0x7f7f8c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.439+0000 7f7fb1000640 1 --2- 172.21.15.100:0/2018853160 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7f8c0683b0 0x7f7f8c06a870 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0ad000 tx=0x7f7f9c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:58.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.443+0000 7f7fa8c00640 1 -- 172.21.15.100:0/2018853160 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7fa4056d50 con 0x7f7fac15ae60 2024-04-23T14:34:58.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.655+0000 7f7fb3a00640 1 -- 172.21.15.100:0/2018853160 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7fac0c7070 con 0x7f7fac15ae60 2024-04-23T14:34:58.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.655+0000 7f7fa8c00640 1 -- 172.21.15.100:0/2018853160 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) ==== 104+0+11 (secure 0 0 0) 0x7f7fa4056650 con 0x7f7fac15ae60 2024-04-23T14:34:58.657 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:34:58.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.659+0000 7f7fb3a00640 1 -- 172.21.15.100:0/2018853160 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7f8c0683b0 msgr2=0x7f7f8c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.659+0000 7f7fb3a00640 1 --2- 172.21.15.100:0/2018853160 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7f8c0683b0 0x7f7f8c06a870 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0ad000 tx=0x7f7f9c0a6000 comp rx=0 tx=0).stop 2024-04-23T14:34:58.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.659+0000 7f7fb3a00640 1 -- 172.21.15.100:0/2018853160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac15ae60 msgr2=0x7f7fac0aee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.659+0000 7f7fb3a00640 1 --2- 172.21.15.100:0/2018853160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac15ae60 0x7f7fac0aee70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400cba0 tx=0x7f7fa400a430 comp rx=0 tx=0).stop 2024-04-23T14:34:58.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.659+0000 7f7fb3a00640 1 -- 172.21.15.100:0/2018853160 shutdown_connections 2024-04-23T14:34:58.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.659+0000 7f7fb3a00640 1 --2- 172.21.15.100:0/2018853160 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f7f8c0683b0 0x7f7f8c06a870 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.659+0000 7f7fb3a00640 1 --2- 172.21.15.100:0/2018853160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac15ae60 0x7f7fac0aee70 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.659+0000 7f7fb3a00640 1 --2- 172.21.15.100:0/2018853160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fac0d0a70 0x7f7fac154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.659+0000 7f7fb3a00640 1 --2- 172.21.15.100:0/2018853160 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7fac06c9a0 0x7f7fac1fa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.659+0000 7f7fb3a00640 1 -- 172.21.15.100:0/2018853160 >> 172.21.15.100:0/2018853160 conn(0x7f7fac0b5200 msgr2=0x7f7fac0afd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:58.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.659+0000 7f7fb3a00640 1 -- 172.21.15.100:0/2018853160 shutdown_connections 2024-04-23T14:34:58.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.659+0000 7f7fb3a00640 1 -- 172.21.15.100:0/2018853160 wait complete. 2024-04-23T14:34:58.680 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:34:58.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.819+0000 7fba6fc00640 1 Processor -- start 2024-04-23T14:34:58.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.819+0000 7fba6fc00640 1 -- start start 2024-04-23T14:34:58.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.819+0000 7fba6fc00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba6815c7a0 0x7fba680af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.819+0000 7fba6fc00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fba6806bb20 0x7fba680ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.819+0000 7fba6fc00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba6806cbb0 con 0x7fba680cd130 2024-04-23T14:34:58.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.819+0000 7fba6fc00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fba680cd5a0 con 0x7fba6806bb20 2024-04-23T14:34:58.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.819+0000 7fba6fc00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fba6806d450 con 0x7fba6815c7a0 2024-04-23T14:34:58.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.819+0000 7fba6dc00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fba680cd130 0x7fba6806e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50464/0 (socket says 172.21.15.100:50464) 2024-04-23T14:34:58.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.819+0000 7fba6dc00640 1 -- 172.21.15.100:0/541001385 learned_addr learned my addr 172.21.15.100:0/541001385 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:58.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.819+0000 7fba67e00640 1 --2- 172.21.15.100:0/541001385 >> v2:172.21.15.100:3300/0 conn(0x7fba6806bb20 0x7fba680ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.819+0000 7fba6d200640 1 --2- 172.21.15.100:0/541001385 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba6815c7a0 0x7fba680af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.819+0000 7fba67e00640 1 -- 172.21.15.100:0/541001385 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba6815c7a0 msgr2=0x7fba680af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:58.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.819+0000 7fba67e00640 1 --2- 172.21.15.100:0/541001385 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba6815c7a0 0x7fba680af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.819+0000 7fba67e00640 1 -- 172.21.15.100:0/541001385 >> v1:172.21.15.100:6789/0 conn(0x7fba680cd130 legacy=0x7fba6806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.819+0000 7fba67e00640 1 -- 172.21.15.100:0/541001385 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba6806ce90 con 0x7fba6806bb20 2024-04-23T14:34:58.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.823+0000 7fba67e00640 1 --2- 172.21.15.100:0/541001385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba6806bb20 0x7fba680ccbf0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fba540097c0 tx=0x7fba5401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1de3d4e23ca7b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:58.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.823+0000 7fba67400640 1 -- 172.21.15.100:0/541001385 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba540b0070 con 0x7fba6806bb20 2024-04-23T14:34:58.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.823+0000 7fba67400640 1 -- 172.21.15.100:0/541001385 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba540ab600 con 0x7fba6806bb20 2024-04-23T14:34:58.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.823+0000 7fba67400640 1 -- 172.21.15.100:0/541001385 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba540ab9d0 con 0x7fba6806bb20 2024-04-23T14:34:58.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.823+0000 7fba6fc00640 1 -- 172.21.15.100:0/541001385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba6806bb20 msgr2=0x7fba680ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.823+0000 7fba6fc00640 1 --2- 172.21.15.100:0/541001385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba6806bb20 0x7fba680ccbf0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fba540097c0 tx=0x7fba5401c410 comp rx=0 tx=0).stop 2024-04-23T14:34:58.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.823+0000 7fba6fc00640 1 -- 172.21.15.100:0/541001385 shutdown_connections 2024-04-23T14:34:58.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.823+0000 7fba6fc00640 1 --2- 172.21.15.100:0/541001385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba6806bb20 0x7fba680ccbf0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.823+0000 7fba6fc00640 1 --2- 172.21.15.100:0/541001385 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba6815c7a0 0x7fba680af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.823+0000 7fba6fc00640 1 -- 172.21.15.100:0/541001385 >> 172.21.15.100:0/541001385 conn(0x7fba680b5200 msgr2=0x7fba680b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:58.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.823+0000 7fba6fc00640 1 -- 172.21.15.100:0/541001385 shutdown_connections 2024-04-23T14:34:58.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.823+0000 7fba6fc00640 1 -- 172.21.15.100:0/541001385 wait complete. 2024-04-23T14:34:58.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.823+0000 7fba6fc00640 1 Processor -- start 2024-04-23T14:34:58.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.823+0000 7fba6fc00640 1 -- start start 2024-04-23T14:34:58.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba6fc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba6806bb20 0x7fba68159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba6fc00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba680cd130 0x7fba6815a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba6fc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba6815c7a0 0x7fba681564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba6fc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fba68169590 con 0x7fba6815c7a0 2024-04-23T14:34:58.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba6fc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fba68169410 con 0x7fba6806bb20 2024-04-23T14:34:58.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba6fc00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fba68169710 con 0x7fba680cd130 2024-04-23T14:34:58.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba6dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba6815c7a0 0x7fba681564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba6dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba6815c7a0 0x7fba681564e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34284/0 (socket says 172.21.15.100:34284) 2024-04-23T14:34:58.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba6dc00640 1 -- 172.21.15.100:0/2716203883 learned_addr learned my addr 172.21.15.100:0/2716203883 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:58.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba67e00640 1 --2- 172.21.15.100:0/2716203883 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba680cd130 0x7fba6815a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba6d200640 1 --2- 172.21.15.100:0/2716203883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba6806bb20 0x7fba68159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba6dc00640 1 -- 172.21.15.100:0/2716203883 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba680cd130 msgr2=0x7fba6815a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba6dc00640 1 --2- 172.21.15.100:0/2716203883 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba680cd130 0x7fba6815a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba6dc00640 1 -- 172.21.15.100:0/2716203883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba6806bb20 msgr2=0x7fba68159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba6dc00640 1 --2- 172.21.15.100:0/2716203883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba6806bb20 0x7fba68159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba6dc00640 1 -- 172.21.15.100:0/2716203883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba680af500 con 0x7fba6815c7a0 2024-04-23T14:34:58.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba67e00640 1 --2- 172.21.15.100:0/2716203883 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba680cd130 0x7fba6815a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:58.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba6dc00640 1 --2- 172.21.15.100:0/2716203883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba6815c7a0 0x7fba681564e0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fba6000cbc0 tx=0x7fba6000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:58.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba64c00640 1 -- 172.21.15.100:0/2716203883 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60017070 con 0x7fba6815c7a0 2024-04-23T14:34:58.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba64c00640 1 -- 172.21.15.100:0/2716203883 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6000ad40 con 0x7fba6815c7a0 2024-04-23T14:34:58.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba6fc00640 1 -- 172.21.15.100:0/2716203883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba680cf0b0 con 0x7fba6815c7a0 2024-04-23T14:34:58.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba64c00640 1 -- 172.21.15.100:0/2716203883 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60012600 con 0x7fba6815c7a0 2024-04-23T14:34:58.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba6fc00640 1 -- 172.21.15.100:0/2716203883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba680cf670 con 0x7fba6815c7a0 2024-04-23T14:34:58.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.827+0000 7fba6fc00640 1 -- 172.21.15.100:0/2716203883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba68068ad0 con 0x7fba6815c7a0 2024-04-23T14:34:58.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.831+0000 7fba64c00640 1 -- 172.21.15.100:0/2716203883 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fba6001a070 con 0x7fba6815c7a0 2024-04-23T14:34:58.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.831+0000 7fba64c00640 1 --2- 172.21.15.100:0/2716203883 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fba48068380 0x7fba4806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.831+0000 7fba64c00640 1 -- 172.21.15.100:0/2716203883 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fba60086830 con 0x7fba6815c7a0 2024-04-23T14:34:58.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.831+0000 7fba6d200640 1 --2- 172.21.15.100:0/2716203883 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fba48068380 0x7fba4806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.831+0000 7fba6d200640 1 --2- 172.21.15.100:0/2716203883 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fba48068380 0x7fba4806a840 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009a50 tx=0x7fba5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:58.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.835+0000 7fba64c00640 1 -- 172.21.15.100:0/2716203883 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fba60056600 con 0x7fba6815c7a0 2024-04-23T14:34:58.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.991+0000 7fba6fc00640 1 -- 172.21.15.100:0/2716203883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fba68074f20 con 0x7fba6815c7a0 2024-04-23T14:34:58.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.991+0000 7fba64c00640 1 -- 172.21.15.100:0/2716203883 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) ==== 100+0+11 (secure 0 0 0) 0x7fba6005a900 con 0x7fba6815c7a0 2024-04-23T14:34:58.993 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:34:58.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.991+0000 7fba6fc00640 1 -- 172.21.15.100:0/2716203883 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fba48068380 msgr2=0x7fba4806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.991+0000 7fba6fc00640 1 --2- 172.21.15.100:0/2716203883 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fba48068380 0x7fba4806a840 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009a50 tx=0x7fba5c008040 comp rx=0 tx=0).stop 2024-04-23T14:34:58.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.995+0000 7fba6fc00640 1 -- 172.21.15.100:0/2716203883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba6815c7a0 msgr2=0x7fba681564e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.995+0000 7fba6fc00640 1 --2- 172.21.15.100:0/2716203883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba6815c7a0 0x7fba681564e0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fba6000cbc0 tx=0x7fba6000a430 comp rx=0 tx=0).stop 2024-04-23T14:34:58.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.995+0000 7fba6fc00640 1 -- 172.21.15.100:0/2716203883 shutdown_connections 2024-04-23T14:34:58.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.995+0000 7fba6fc00640 1 --2- 172.21.15.100:0/2716203883 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fba48068380 0x7fba4806a840 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.995+0000 7fba6fc00640 1 --2- 172.21.15.100:0/2716203883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba6815c7a0 0x7fba681564e0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.995+0000 7fba6fc00640 1 --2- 172.21.15.100:0/2716203883 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba680cd130 0x7fba6815a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.995+0000 7fba6fc00640 1 --2- 172.21.15.100:0/2716203883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba6806bb20 0x7fba68159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.995+0000 7fba6fc00640 1 -- 172.21.15.100:0/2716203883 >> 172.21.15.100:0/2716203883 conn(0x7fba680b5200 msgr2=0x7fba680cade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:58.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.995+0000 7fba6fc00640 1 -- 172.21.15.100:0/2716203883 shutdown_connections 2024-04-23T14:34:58.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:58.995+0000 7fba6fc00640 1 -- 172.21.15.100:0/2716203883 wait complete. 2024-04-23T14:34:59.015 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:34:59.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.151+0000 7f88e5e00640 1 Processor -- start 2024-04-23T14:34:59.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.151+0000 7f88e5e00640 1 -- start start 2024-04-23T14:34:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.151+0000 7f88e5e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f88e015d0c0 0x7f88e00cde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.151+0000 7f88e5e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f88e015d4a0 0x7f88e00cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.151+0000 7f88e5e00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88e00ce500 con 0x7f88e015d900 2024-04-23T14:34:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.151+0000 7f88e5e00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f88e00cff50 con 0x7f88e015d4a0 2024-04-23T14:34:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.151+0000 7f88e5e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f88e00ceda0 con 0x7f88e015d0c0 2024-04-23T14:34:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.151+0000 7f88d6000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f88e015d4a0 0x7f88e00cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.151+0000 7f88df400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f88e015d900 0x7f88e00cf160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50466/0 (socket says 172.21.15.100:50466) 2024-04-23T14:34:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.151+0000 7f88df400640 1 -- 172.21.15.100:0/2952483374 learned_addr learned my addr 172.21.15.100:0/2952483374 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:34:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.151+0000 7f88dea00640 1 --2- 172.21.15.100:0/2952483374 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f88e015d0c0 0x7f88e00cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.151+0000 7f88d6000640 1 -- 172.21.15.100:0/2952483374 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f88e015d0c0 msgr2=0x7f88e00cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.151+0000 7f88d6000640 1 --2- 172.21.15.100:0/2952483374 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f88e015d0c0 0x7f88e00cde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.151+0000 7f88d6000640 1 -- 172.21.15.100:0/2952483374 >> v1:172.21.15.100:6789/0 conn(0x7f88e015d900 legacy=0x7f88e00cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.151+0000 7f88d6000640 1 -- 172.21.15.100:0/2952483374 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e00ce7e0 con 0x7f88e015d4a0 2024-04-23T14:34:59.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.155+0000 7f88d6000640 1 --2- 172.21.15.100:0/2952483374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88e015d4a0 0x7f88e00cb9a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009d90 tx=0x7f88c801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b39258f1cbda78f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:59.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.155+0000 7f88de000640 1 -- 172.21.15.100:0/2952483374 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c80b2070 con 0x7f88e015d4a0 2024-04-23T14:34:59.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.155+0000 7f88de000640 1 -- 172.21.15.100:0/2952483374 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88c80a5970 con 0x7f88e015d4a0 2024-04-23T14:34:59.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.155+0000 7f88de000640 1 -- 172.21.15.100:0/2952483374 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c80a5d40 con 0x7f88e015d4a0 2024-04-23T14:34:59.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.155+0000 7f88e5e00640 1 -- 172.21.15.100:0/2952483374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88e015d4a0 msgr2=0x7f88e00cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:59.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.155+0000 7f88e5e00640 1 --2- 172.21.15.100:0/2952483374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88e015d4a0 0x7f88e00cb9a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009d90 tx=0x7f88c801c8b0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.155+0000 7f88e5e00640 1 -- 172.21.15.100:0/2952483374 shutdown_connections 2024-04-23T14:34:59.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.155+0000 7f88e5e00640 1 --2- 172.21.15.100:0/2952483374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88e015d4a0 0x7f88e00cb9a0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.155+0000 7f88e5e00640 1 --2- 172.21.15.100:0/2952483374 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f88e015d0c0 0x7f88e00cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.155+0000 7f88e5e00640 1 -- 172.21.15.100:0/2952483374 >> 172.21.15.100:0/2952483374 conn(0x7f88e00b5310 msgr2=0x7f88e00b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:59.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.155+0000 7f88e5e00640 1 -- 172.21.15.100:0/2952483374 shutdown_connections 2024-04-23T14:34:59.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.155+0000 7f88e5e00640 1 -- 172.21.15.100:0/2952483374 wait complete. 2024-04-23T14:34:59.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88e5e00640 1 Processor -- start 2024-04-23T14:34:59.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88e5e00640 1 -- start start 2024-04-23T14:34:59.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88e5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88e015d0c0 0x7f88e00c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:59.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88e5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88e015d4a0 0x7f88e014f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:59.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88e5e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f88e0167af0 0x7f88e014b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:59.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88e5e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f88e00cf9f0 con 0x7f88e015d0c0 2024-04-23T14:34:59.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88e5e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f88e00cf870 con 0x7f88e015d4a0 2024-04-23T14:34:59.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88e5e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f88e00cbee0 con 0x7f88e0167af0 2024-04-23T14:34:59.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88dea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88e015d0c0 0x7f88e00c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:59.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88dea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88e015d0c0 0x7f88e00c5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34300/0 (socket says 172.21.15.100:34300) 2024-04-23T14:34:59.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88dea00640 1 -- 172.21.15.100:0/1091510701 learned_addr learned my addr 172.21.15.100:0/1091510701 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:59.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88df400640 1 --2- 172.21.15.100:0/1091510701 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f88e0167af0 0x7f88e014b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:59.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88d6000640 1 --2- 172.21.15.100:0/1091510701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88e015d4a0 0x7f88e014f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:59.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88dea00640 1 -- 172.21.15.100:0/1091510701 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f88e0167af0 msgr2=0x7f88e014b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:59.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88dea00640 1 --2- 172.21.15.100:0/1091510701 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f88e0167af0 0x7f88e014b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88dea00640 1 -- 172.21.15.100:0/1091510701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88e015d4a0 msgr2=0x7f88e014f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:59.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88dea00640 1 --2- 172.21.15.100:0/1091510701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88e015d4a0 0x7f88e014f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88dea00640 1 -- 172.21.15.100:0/1091510701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e00c5220 con 0x7f88e015d0c0 2024-04-23T14:34:59.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88dea00640 1 --2- 172.21.15.100:0/1091510701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88e015d0c0 0x7f88e00c5fa0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f88d000d800 tx=0x7f88d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:59.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88d7e00640 1 -- 172.21.15.100:0/1091510701 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d000b6c0 con 0x7f88e015d0c0 2024-04-23T14:34:59.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88d7e00640 1 -- 172.21.15.100:0/1091510701 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88d0019070 con 0x7f88e015d0c0 2024-04-23T14:34:59.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88d7e00640 1 -- 172.21.15.100:0/1091510701 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d000ba60 con 0x7f88e015d0c0 2024-04-23T14:34:59.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88e5e00640 1 -- 172.21.15.100:0/1091510701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e00c54c0 con 0x7f88e015d0c0 2024-04-23T14:34:59.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88e5e00640 1 -- 172.21.15.100:0/1091510701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e00c5a80 con 0x7f88e015d0c0 2024-04-23T14:34:59.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.159+0000 7f88e5e00640 1 -- 172.21.15.100:0/1091510701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88e0068ad0 con 0x7f88e015d0c0 2024-04-23T14:34:59.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.163+0000 7f88d7e00640 1 -- 172.21.15.100:0/1091510701 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f88d000bc00 con 0x7f88e015d0c0 2024-04-23T14:34:59.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.163+0000 7f88d7e00640 1 --2- 172.21.15.100:0/1091510701 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f88bc068380 0x7f88bc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:59.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.163+0000 7f88d7e00640 1 -- 172.21.15.100:0/1091510701 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f88d0087040 con 0x7f88e015d0c0 2024-04-23T14:34:59.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.163+0000 7f88d6000640 1 --2- 172.21.15.100:0/1091510701 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f88bc068380 0x7f88bc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:59.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.163+0000 7f88d6000640 1 --2- 172.21.15.100:0/1091510701 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f88bc068380 0x7f88bc06a840 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f88c80af000 tx=0x7f88c80a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:59.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.167+0000 7f88d7e00640 1 -- 172.21.15.100:0/1091510701 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f88d0057470 con 0x7f88e015d0c0 2024-04-23T14:34:59.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.335+0000 7f88e5e00640 1 -- 172.21.15.100:0/1091510701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f88e006c730 con 0x7f88e015d0c0 2024-04-23T14:34:59.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.335+0000 7f88d7e00640 1 -- 172.21.15.100:0/1091510701 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) ==== 80+0+0 (secure 0 0 0) 0x7f88d0056d70 con 0x7f88e015d0c0 2024-04-23T14:34:59.335 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-04-23T14:34:59.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.335+0000 7f88e5e00640 1 -- 172.21.15.100:0/1091510701 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f88bc068380 msgr2=0x7f88bc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:59.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.335+0000 7f88e5e00640 1 --2- 172.21.15.100:0/1091510701 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f88bc068380 0x7f88bc06a840 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f88c80af000 tx=0x7f88c80a7070 comp rx=0 tx=0).stop 2024-04-23T14:34:59.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.335+0000 7f88e5e00640 1 -- 172.21.15.100:0/1091510701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88e015d0c0 msgr2=0x7f88e00c5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:59.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.335+0000 7f88e5e00640 1 --2- 172.21.15.100:0/1091510701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88e015d0c0 0x7f88e00c5fa0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f88d000d800 tx=0x7f88d000dcc0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.335+0000 7f88e5e00640 1 -- 172.21.15.100:0/1091510701 shutdown_connections 2024-04-23T14:34:59.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.335+0000 7f88e5e00640 1 --2- 172.21.15.100:0/1091510701 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f88bc068380 0x7f88bc06a840 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.335+0000 7f88e5e00640 1 --2- 172.21.15.100:0/1091510701 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f88e0167af0 0x7f88e014b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.335+0000 7f88e5e00640 1 --2- 172.21.15.100:0/1091510701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88e015d4a0 0x7f88e014f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.335+0000 7f88e5e00640 1 --2- 172.21.15.100:0/1091510701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88e015d0c0 0x7f88e00c5fa0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.335+0000 7f88e5e00640 1 -- 172.21.15.100:0/1091510701 >> 172.21.15.100:0/1091510701 conn(0x7f88e00b5310 msgr2=0x7f88e00cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:59.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.335+0000 7f88e5e00640 1 -- 172.21.15.100:0/1091510701 shutdown_connections 2024-04-23T14:34:59.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.335+0000 7f88e5e00640 1 -- 172.21.15.100:0/1091510701 wait complete. 2024-04-23T14:34:59.356 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi100.front.sepia.ceph.com... 2024-04-23T14:34:59.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:59.356 DEBUG:teuthology.orchestra.run.smithi100:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:59.374 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[42802]: fuse finished with error 0 and tester_r 0 2024-04-23T14:34:59.376 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:34:59.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:59.376 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:34:59.431 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:34:59.431 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:34:59.431 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:59.441 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3578 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:34:59.441 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-04-23T14:34:59.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:59.441 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-23T14:34:59.496 INFO:teuthology.orchestra.run.smithi151.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-23T14:34:59.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:34:59.497 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:34:59.497 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:34:59.497 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:59.505 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350140k,nr_inodes=4087535,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274960k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=820 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274956k,nr_inodes=818739,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274960k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:34:59.505 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:34:59.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:59.505 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-23T14:34:59.553 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-23T14:34:59.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:34:59.554 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:34:59.554 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:34:59.555 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:59.563 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3578 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:34:59.564 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-04-23T14:34:59.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:59.564 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-23T14:34:59.615 INFO:teuthology.orchestra.run.smithi151.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-23T14:34:59.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:34:59.616 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:34:59.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.751+0000 7f4f7a800640 1 Processor -- start 2024-04-23T14:34:59.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.751+0000 7f4f7a800640 1 -- start start 2024-04-23T14:34:59.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.751+0000 7f4f7a800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4f7415e5a0 0x7f4f741621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:59.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.751+0000 7f4f7a800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f4f740ced00 0x7f4f740cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:59.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.751+0000 7f4f7a800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f74162a20 con 0x7f4f740b3930 2024-04-23T14:34:59.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.751+0000 7f4f7a800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f4f740cf670 con 0x7f4f7415e5a0 2024-04-23T14:34:59.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.751+0000 7f4f7a800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f4f74162d00 con 0x7f4f740ced00 2024-04-23T14:34:59.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.751+0000 7f4f79400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4f7415e5a0 0x7f4f741621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:59.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.751+0000 7f4f79400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4f7415e5a0 0x7f4f741621d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34312/0 (socket says 172.21.15.100:34312) 2024-04-23T14:34:59.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.751+0000 7f4f79400640 1 -- 172.21.15.100:0/1667029802 learned_addr learned my addr 172.21.15.100:0/1667029802 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:59.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.751+0000 7f4f79e00640 1 --2- 172.21.15.100:0/1667029802 >> v2:172.21.15.151:3300/0 conn(0x7f4f740ced00 0x7f4f740cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:59.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.751+0000 7f4f79400640 1 -- 172.21.15.100:0/1667029802 >> v2:172.21.15.151:3300/0 conn(0x7f4f740ced00 msgr2=0x7f4f740cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:59.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.751+0000 7f4f79400640 1 --2- 172.21.15.100:0/1667029802 >> v2:172.21.15.151:3300/0 conn(0x7f4f740ced00 0x7f4f740cf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.751+0000 7f4f79400640 1 -- 172.21.15.100:0/1667029802 >> v1:172.21.15.100:6789/0 conn(0x7f4f740b3930 legacy=0x7f4f7406df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:59.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.751+0000 7f4f79400640 1 -- 172.21.15.100:0/1667029802 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f74162fe0 con 0x7f4f7415e5a0 2024-04-23T14:34:59.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.751+0000 7f4f73e00640 1 -- 172.21.15.100:0/1667029802 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3256342147 0 0) 0x7f4f74162a20 con 0x7f4f740b3930 2024-04-23T14:34:59.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.755+0000 7f4f79400640 1 --2- 172.21.15.100:0/1667029802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f7415e5a0 0x7f4f741621d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4f740b3dc0 tx=0x7f4f6801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6322ea0f85e258c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:59.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.755+0000 7f4f73e00640 1 -- 172.21.15.100:0/1667029802 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f680aa650 con 0x7f4f7415e5a0 2024-04-23T14:34:59.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.755+0000 7f4f73e00640 1 -- 172.21.15.100:0/1667029802 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f680af070 con 0x7f4f7415e5a0 2024-04-23T14:34:59.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.755+0000 7f4f73e00640 1 -- 172.21.15.100:0/1667029802 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f680aaa10 con 0x7f4f7415e5a0 2024-04-23T14:34:59.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.755+0000 7f4f7a800640 1 -- 172.21.15.100:0/1667029802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f7415e5a0 msgr2=0x7f4f741621d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:59.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.755+0000 7f4f7a800640 1 --2- 172.21.15.100:0/1667029802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f7415e5a0 0x7f4f741621d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4f740b3dc0 tx=0x7f4f6801c4f0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.755+0000 7f4f7a800640 1 -- 172.21.15.100:0/1667029802 shutdown_connections 2024-04-23T14:34:59.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.755+0000 7f4f7a800640 1 --2- 172.21.15.100:0/1667029802 >> v2:172.21.15.151:3300/0 conn(0x7f4f740ced00 0x7f4f740cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.755+0000 7f4f7a800640 1 --2- 172.21.15.100:0/1667029802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f7415e5a0 0x7f4f741621d0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.755+0000 7f4f7a800640 1 -- 172.21.15.100:0/1667029802 >> 172.21.15.100:0/1667029802 conn(0x7f4f740b3e60 msgr2=0x7f4f740b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:59.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.755+0000 7f4f7a800640 1 -- 172.21.15.100:0/1667029802 shutdown_connections 2024-04-23T14:34:59.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.755+0000 7f4f7a800640 1 -- 172.21.15.100:0/1667029802 wait complete. 2024-04-23T14:34:59.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.755+0000 7f4f7a800640 1 Processor -- start 2024-04-23T14:34:59.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.755+0000 7f4f7a800640 1 -- start start 2024-04-23T14:34:59.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f7a800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f740b3930 0x7f4f741591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:59.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f7a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f740ced00 0x7f4f7415ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:59.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f7a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7415e5a0 0x7f4f7415d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:59.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f7a800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7415e980 con 0x7f4f740ced00 2024-04-23T14:34:59.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f7a800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7406e690 con 0x7f4f7415e5a0 2024-04-23T14:34:59.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f7a800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4f7416b570 con 0x7f4f740b3930 2024-04-23T14:34:59.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f78a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f740ced00 0x7f4f7415ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:59.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f78a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f740ced00 0x7f4f7415ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34328/0 (socket says 172.21.15.100:34328) 2024-04-23T14:34:59.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f78a00640 1 -- 172.21.15.100:0/724290191 learned_addr learned my addr 172.21.15.100:0/724290191 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:34:59.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f79400640 1 --2- 172.21.15.100:0/724290191 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f740b3930 0x7f4f741591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:59.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f79e00640 1 --2- 172.21.15.100:0/724290191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7415e5a0 0x7f4f7415d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:59.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f78a00640 1 -- 172.21.15.100:0/724290191 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f740b3930 msgr2=0x7f4f741591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:59.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f78a00640 1 --2- 172.21.15.100:0/724290191 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f740b3930 0x7f4f741591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f78a00640 1 -- 172.21.15.100:0/724290191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7415e5a0 msgr2=0x7f4f7415d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:59.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f78a00640 1 --2- 172.21.15.100:0/724290191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7415e5a0 0x7f4f7415d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f78a00640 1 -- 172.21.15.100:0/724290191 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7415f7c0 con 0x7f4f740ced00 2024-04-23T14:34:59.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f78a00640 1 --2- 172.21.15.100:0/724290191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f740ced00 0x7f4f7415ce80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000c450 tx=0x7f4f6000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:59.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f71600640 1 -- 172.21.15.100:0/724290191 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f60017070 con 0x7f4f740ced00 2024-04-23T14:34:59.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f71600640 1 -- 172.21.15.100:0/724290191 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f60005480 con 0x7f4f740ced00 2024-04-23T14:34:59.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f71600640 1 -- 172.21.15.100:0/724290191 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f60003840 con 0x7f4f740ced00 2024-04-23T14:34:59.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f7a800640 1 -- 172.21.15.100:0/724290191 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f7415faa0 con 0x7f4f740ced00 2024-04-23T14:34:59.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f7a800640 1 -- 172.21.15.100:0/724290191 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f74160060 con 0x7f4f740ced00 2024-04-23T14:34:59.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.759+0000 7f4f7a800640 1 -- 172.21.15.100:0/724290191 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f74075c20 con 0x7f4f740ced00 2024-04-23T14:34:59.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.763+0000 7f4f71600640 1 -- 172.21.15.100:0/724290191 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4f6001a070 con 0x7f4f740ced00 2024-04-23T14:34:59.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.763+0000 7f4f71600640 1 --2- 172.21.15.100:0/724290191 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4f54068380 0x7f4f5406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:59.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.763+0000 7f4f71600640 1 -- 172.21.15.100:0/724290191 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f4f60085f10 con 0x7f4f740ced00 2024-04-23T14:34:59.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.763+0000 7f4f79400640 1 --2- 172.21.15.100:0/724290191 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4f54068380 0x7f4f5406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:59.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.763+0000 7f4f79400640 1 --2- 172.21.15.100:0/724290191 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4f54068380 0x7f4f5406a840 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4f680ae000 tx=0x7f4f680a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.767+0000 7f4f71600640 1 -- 172.21.15.100:0/724290191 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4f600562a0 con 0x7f4f740ced00 2024-04-23T14:34:59.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.959+0000 7f4f7a800640 1 -- 172.21.15.100:0/724290191 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f740b3710 con 0x7f4f740ced00 2024-04-23T14:34:59.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.959+0000 7f4f71600640 1 -- 172.21.15.100:0/724290191 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5190 (secure 0 0 0) 0x7f4f60055ba0 con 0x7f4f740ced00 2024-04-23T14:34:59.959 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:34:59.959 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5528,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2774610498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2774610498},{"type":"v1","addr":"172.21.15.151:6833","nonce":2774610498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5929,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1347883282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1347883282},{"type":"v1","addr":"172.21.15.151:6835","nonce":1347883282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":6177,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3319827520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3319827520},{"type":"v1","addr":"172.21.15.100:6836","nonce":3319827520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:34:24.162539+0000","modified":"2024-04-23T14:34:27.179639+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6180":{"gid":6180,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/608349550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":608349550},{"type":"v1","addr":"172.21.15.100:6837","nonce":608349550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6180,"qdb_cluster":[6180]},"id":3}]} 2024-04-23T14:34:59.960 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 28 2024-04-23T14:34:59.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.959+0000 7f4f7a800640 1 -- 172.21.15.100:0/724290191 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4f54068380 msgr2=0x7f4f5406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:59.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.959+0000 7f4f7a800640 1 --2- 172.21.15.100:0/724290191 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4f54068380 0x7f4f5406a840 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4f680ae000 tx=0x7f4f680a7000 comp rx=0 tx=0).stop 2024-04-23T14:34:59.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.959+0000 7f4f7a800640 1 -- 172.21.15.100:0/724290191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f740ced00 msgr2=0x7f4f7415ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:59.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.959+0000 7f4f7a800640 1 --2- 172.21.15.100:0/724290191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f740ced00 0x7f4f7415ce80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000c450 tx=0x7f4f6000c910 comp rx=0 tx=0).stop 2024-04-23T14:34:59.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.959+0000 7f4f7a800640 1 -- 172.21.15.100:0/724290191 shutdown_connections 2024-04-23T14:34:59.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.959+0000 7f4f7a800640 1 --2- 172.21.15.100:0/724290191 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4f54068380 0x7f4f5406a840 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.959+0000 7f4f7a800640 1 --2- 172.21.15.100:0/724290191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7415e5a0 0x7f4f7415d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.959+0000 7f4f7a800640 1 --2- 172.21.15.100:0/724290191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f740ced00 0x7f4f7415ce80 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.959+0000 7f4f7a800640 1 --2- 172.21.15.100:0/724290191 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4f740b3930 0x7f4f741591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.959+0000 7f4f7a800640 1 -- 172.21.15.100:0/724290191 >> 172.21.15.100:0/724290191 conn(0x7f4f740b3e60 msgr2=0x7f4f740b7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:59.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.959+0000 7f4f7a800640 1 -- 172.21.15.100:0/724290191 shutdown_connections 2024-04-23T14:34:59.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:34:59.959+0000 7f4f7a800640 1 -- 172.21.15.100:0/724290191 wait complete. 2024-04-23T14:34:59.980 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:35:00.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.119+0000 7f0605800640 1 Processor -- start 2024-04-23T14:35:00.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.119+0000 7f0605800640 1 -- start start 2024-04-23T14:35:00.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.119+0000 7f0605800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f06000ad870 0x7f06000b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.119+0000 7f0605800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0600160de0 0x7f060006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.119+0000 7f0605800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0600160320 con 0x7f060015c470 2024-04-23T14:35:00.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.119+0000 7f0605800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f06000cc7e0 con 0x7f0600160de0 2024-04-23T14:35:00.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.119+0000 7f0605800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f06001605e0 con 0x7f06000ad870 2024-04-23T14:35:00.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.119+0000 7f05ff400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0600160de0 0x7f060006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.119+0000 7f05fea00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f060015c470 0x7f06000ad160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50482/0 (socket says 172.21.15.100:50482) 2024-04-23T14:35:00.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.119+0000 7f05fea00640 1 -- 172.21.15.100:0/3982369262 learned_addr learned my addr 172.21.15.100:0/3982369262 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:00.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.119+0000 7f05fe000640 1 --2- 172.21.15.100:0/3982369262 >> v2:172.21.15.151:3300/0 conn(0x7f06000ad870 0x7f06000b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.119+0000 7f05ff400640 1 -- 172.21.15.100:0/3982369262 >> v2:172.21.15.151:3300/0 conn(0x7f06000ad870 msgr2=0x7f06000b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:00.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.119+0000 7f05ff400640 1 --2- 172.21.15.100:0/3982369262 >> v2:172.21.15.151:3300/0 conn(0x7f06000ad870 0x7f06000b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.119+0000 7f05ff400640 1 -- 172.21.15.100:0/3982369262 >> v1:172.21.15.100:6789/0 conn(0x7f060015c470 legacy=0x7f06000ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.119+0000 7f05ff400640 1 -- 172.21.15.100:0/3982369262 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06001608c0 con 0x7f0600160de0 2024-04-23T14:35:00.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.123+0000 7f05ff400640 1 --2- 172.21.15.100:0/3982369262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0600160de0 0x7f060006bab0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f060006d590 tx=0x7f05f801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654304b690ca705e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:00.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.123+0000 7f05fd600640 1 -- 172.21.15.100:0/3982369262 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f80b2070 con 0x7f0600160de0 2024-04-23T14:35:00.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.123+0000 7f05fd600640 1 -- 172.21.15.100:0/3982369262 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05f80a5870 con 0x7f0600160de0 2024-04-23T14:35:00.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.123+0000 7f05fd600640 1 -- 172.21.15.100:0/3982369262 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f80a5c40 con 0x7f0600160de0 2024-04-23T14:35:00.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.123+0000 7f0605800640 1 -- 172.21.15.100:0/3982369262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0600160de0 msgr2=0x7f060006bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.123+0000 7f0605800640 1 --2- 172.21.15.100:0/3982369262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0600160de0 0x7f060006bab0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f060006d590 tx=0x7f05f801c8a0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.123+0000 7f0605800640 1 -- 172.21.15.100:0/3982369262 shutdown_connections 2024-04-23T14:35:00.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.123+0000 7f0605800640 1 --2- 172.21.15.100:0/3982369262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0600160de0 0x7f060006bab0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.123+0000 7f0605800640 1 --2- 172.21.15.100:0/3982369262 >> v2:172.21.15.151:3300/0 conn(0x7f06000ad870 0x7f06000b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.123+0000 7f0605800640 1 -- 172.21.15.100:0/3982369262 >> 172.21.15.100:0/3982369262 conn(0x7f06000b1e80 msgr2=0x7f06000b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:00.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.123+0000 7f0605800640 1 -- 172.21.15.100:0/3982369262 shutdown_connections 2024-04-23T14:35:00.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.123+0000 7f0605800640 1 -- 172.21.15.100:0/3982369262 wait complete. 2024-04-23T14:35:00.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.123+0000 7f0605800640 1 Processor -- start 2024-04-23T14:35:00.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f0605800640 1 -- start start 2024-04-23T14:35:00.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f0605800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06000ad870 0x7f0600156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f0605800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f060015c470 0x7f06001660d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f0605800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0600160de0 0x7f06000730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f0605800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f060015ca60 con 0x7f06000ad870 2024-04-23T14:35:00.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f0605800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f06000cc580 con 0x7f060015c470 2024-04-23T14:35:00.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f0605800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0600169540 con 0x7f0600160de0 2024-04-23T14:35:00.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f05fea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06000ad870 0x7f0600156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f05fea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06000ad870 0x7f0600156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34356/0 (socket says 172.21.15.100:34356) 2024-04-23T14:35:00.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f05fea00640 1 -- 172.21.15.100:0/2102444179 learned_addr learned my addr 172.21.15.100:0/2102444179 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:00.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f05fe000640 1 --2- 172.21.15.100:0/2102444179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f060015c470 0x7f06001660d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f05ff400640 1 --2- 172.21.15.100:0/2102444179 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0600160de0 0x7f06000730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f05fea00640 1 -- 172.21.15.100:0/2102444179 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0600160de0 msgr2=0x7f06000730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f05fea00640 1 --2- 172.21.15.100:0/2102444179 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0600160de0 0x7f06000730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f05fea00640 1 -- 172.21.15.100:0/2102444179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f060015c470 msgr2=0x7f06001660d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f05fea00640 1 --2- 172.21.15.100:0/2102444179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f060015c470 0x7f06001660d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f05fea00640 1 -- 172.21.15.100:0/2102444179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0600166b70 con 0x7f06000ad870 2024-04-23T14:35:00.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f05fea00640 1 --2- 172.21.15.100:0/2102444179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06000ad870 0x7f0600156f90 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f05f000c450 tx=0x7f05f000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:00.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f05f6a00640 1 -- 172.21.15.100:0/2102444179 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f0017070 con 0x7f06000ad870 2024-04-23T14:35:00.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f05f6a00640 1 -- 172.21.15.100:0/2102444179 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05f0005cf0 con 0x7f06000ad870 2024-04-23T14:35:00.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f05f6a00640 1 -- 172.21.15.100:0/2102444179 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f0007290 con 0x7f06000ad870 2024-04-23T14:35:00.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f0605800640 1 -- 172.21.15.100:0/2102444179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0600166e50 con 0x7f06000ad870 2024-04-23T14:35:00.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f0605800640 1 -- 172.21.15.100:0/2102444179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06000ce6c0 con 0x7f06000ad870 2024-04-23T14:35:00.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.127+0000 7f0605800640 1 -- 172.21.15.100:0/2102444179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0600068ad0 con 0x7f06000ad870 2024-04-23T14:35:00.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.131+0000 7f05f6a00640 1 -- 172.21.15.100:0/2102444179 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f05f001a070 con 0x7f06000ad870 2024-04-23T14:35:00.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.131+0000 7f05f6a00640 1 --2- 172.21.15.100:0/2102444179 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f05e00683b0 0x7f05e006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.131+0000 7f05f6a00640 1 -- 172.21.15.100:0/2102444179 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f05f0085e70 con 0x7f06000ad870 2024-04-23T14:35:00.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.131+0000 7f05fe000640 1 --2- 172.21.15.100:0/2102444179 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f05e00683b0 0x7f05e006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.131+0000 7f05fe000640 1 --2- 172.21.15.100:0/2102444179 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f05e00683b0 0x7f05e006a870 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f06001450e0 tx=0x7f05ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:00.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.135+0000 7f05f6a00640 1 -- 172.21.15.100:0/2102444179 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f05f0056290 con 0x7f06000ad870 2024-04-23T14:35:00.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.295+0000 7f0605800640 1 -- 172.21.15.100:0/2102444179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0600167310 con 0x7f06000ad870 2024-04-23T14:35:00.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.295+0000 7f05f6a00640 1 -- 172.21.15.100:0/2102444179 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16950 (secure 0 0 0) 0x7f05f0055b90 con 0x7f06000ad870 2024-04-23T14:35:00.296 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:35:00.296 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":53,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:34:25.795515+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:34:22.491379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-23T14:34:23.120342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:35:00.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.295+0000 7f0605800640 1 -- 172.21.15.100:0/2102444179 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f05e00683b0 msgr2=0x7f05e006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.295+0000 7f0605800640 1 --2- 172.21.15.100:0/2102444179 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f05e00683b0 0x7f05e006a870 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f06001450e0 tx=0x7f05ec008040 comp rx=0 tx=0).stop 2024-04-23T14:35:00.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.299+0000 7f0605800640 1 -- 172.21.15.100:0/2102444179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06000ad870 msgr2=0x7f0600156f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.299+0000 7f0605800640 1 --2- 172.21.15.100:0/2102444179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06000ad870 0x7f0600156f90 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f05f000c450 tx=0x7f05f000c910 comp rx=0 tx=0).stop 2024-04-23T14:35:00.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.299+0000 7f0605800640 1 -- 172.21.15.100:0/2102444179 shutdown_connections 2024-04-23T14:35:00.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.299+0000 7f0605800640 1 --2- 172.21.15.100:0/2102444179 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f05e00683b0 0x7f05e006a870 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.299+0000 7f0605800640 1 --2- 172.21.15.100:0/2102444179 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0600160de0 0x7f06000730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.299+0000 7f0605800640 1 --2- 172.21.15.100:0/2102444179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f060015c470 0x7f06001660d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.299+0000 7f0605800640 1 --2- 172.21.15.100:0/2102444179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06000ad870 0x7f0600156f90 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.299+0000 7f0605800640 1 -- 172.21.15.100:0/2102444179 >> 172.21.15.100:0/2102444179 conn(0x7f06000b1e80 msgr2=0x7f060006f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:00.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.299+0000 7f0605800640 1 -- 172.21.15.100:0/2102444179 shutdown_connections 2024-04-23T14:35:00.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.299+0000 7f0605800640 1 -- 172.21.15.100:0/2102444179 wait complete. 2024-04-23T14:35:00.318 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:56.759842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:34:22.491379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:34:23.120342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:35:00.318 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:35:00.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.455+0000 7fd84ca00640 1 Processor -- start 2024-04-23T14:35:00.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.455+0000 7fd84ca00640 1 -- start start 2024-04-23T14:35:00.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.455+0000 7fd84ca00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd848159fb0 0x7fd84806fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.455+0000 7fd84ca00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd84806bab0 0x7fd84815d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.455+0000 7fd84ca00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd84815daf0 con 0x7fd84806c4c0 2024-04-23T14:35:00.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.455+0000 7fd84ca00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fd8480c7400 con 0x7fd84806bab0 2024-04-23T14:35:00.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.455+0000 7fd84ca00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd848070260 con 0x7fd848159fb0 2024-04-23T14:35:00.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd847e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fd84806c4c0 0x7fd8480c5940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50488/0 (socket says 172.21.15.100:50488) 2024-04-23T14:35:00.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd847e00640 1 -- 172.21.15.100:0/1427633487 learned_addr learned my addr 172.21.15.100:0/1427633487 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:00.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd83fe00640 1 --2- 172.21.15.100:0/1427633487 >> v2:172.21.15.151:3300/0 conn(0x7fd84806bab0 0x7fd84815d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd847400640 1 --2- 172.21.15.100:0/1427633487 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd848159fb0 0x7fd84806fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd846a00640 1 -- 172.21.15.100:0/1427633487 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2901101194 0 0) 0x7fd8400014a0 con 0x7fd84806c4c0 2024-04-23T14:35:00.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd846a00640 1 -- 172.21.15.100:0/1427633487 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd82c003530 con 0x7fd84806c4c0 2024-04-23T14:35:00.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd846a00640 1 -- 172.21.15.100:0/1427633487 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2083615645 0 0) 0x7fd82c003530 con 0x7fd84806c4c0 2024-04-23T14:35:00.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd846a00640 1 -- 172.21.15.100:0/1427633487 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd848159fb0 msgr2=0x7fd84806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:00.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd846a00640 1 --2- 172.21.15.100:0/1427633487 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd848159fb0 0x7fd84806fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd846a00640 1 -- 172.21.15.100:0/1427633487 >> v2:172.21.15.151:3300/0 conn(0x7fd84806bab0 msgr2=0x7fd84815d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:00.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd846a00640 1 --2- 172.21.15.100:0/1427633487 >> v2:172.21.15.151:3300/0 conn(0x7fd84806bab0 0x7fd84815d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd846a00640 1 -- 172.21.15.100:0/1427633487 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84806ec80 con 0x7fd84806c4c0 2024-04-23T14:35:00.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd846a00640 1 -- 172.21.15.100:0/1427633487 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7fd840001bd0 con 0x7fd84806c4c0 2024-04-23T14:35:00.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd846a00640 1 -- 172.21.15.100:0/1427633487 >> v1:172.21.15.100:6789/0 conn(0x7fd84806c4c0 legacy=0x7fd8480c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd846a00640 1 --2- 172.21.15.100:0/1427633487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd82c0039f0 0x7fd82c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd846a00640 1 --2- 172.21.15.100:0/1427633487 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd82c004810 0x7fd82c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd846a00640 1 --2- 172.21.15.100:0/1427633487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd82c0051d0 0x7fd82c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd846a00640 1 -- 172.21.15.100:0/1427633487 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd8480c7400 con 0x7fd82c0039f0 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd846a00640 1 -- 172.21.15.100:0/1427633487 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd848070260 con 0x7fd82c0051d0 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd846a00640 1 -- 172.21.15.100:0/1427633487 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd82c00bf10 con 0x7fd82c004810 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd847400640 1 --2- 172.21.15.100:0/1427633487 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd848159fb0 0x7fd84806fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd83fe00640 1 --2- 172.21.15.100:0/1427633487 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd82c004810 0x7fd82c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd847400640 1 --2- 172.21.15.100:0/1427633487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd82c0039f0 0x7fd82c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd847e00640 1 --2- 172.21.15.100:0/1427633487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd82c0051d0 0x7fd82c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd83fe00640 1 -- 172.21.15.100:0/1427633487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd82c0051d0 msgr2=0x7fd82c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd83fe00640 1 --2- 172.21.15.100:0/1427633487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd82c0051d0 0x7fd82c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd83fe00640 1 -- 172.21.15.100:0/1427633487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd82c0039f0 msgr2=0x7fd82c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd83fe00640 1 --2- 172.21.15.100:0/1427633487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd82c0039f0 0x7fd82c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.459+0000 7fd83fe00640 1 -- 172.21.15.100:0/1427633487 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd82c003530 con 0x7fd82c004810 2024-04-23T14:35:00.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.463+0000 7fd83fe00640 1 --2- 172.21.15.100:0/1427633487 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd82c004810 0x7fd82c004c90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd830009b10 tx=0x7fd83001ca80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=518e729cf7d4d2a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:00.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.463+0000 7fd846a00640 1 -- 172.21.15.100:0/1427633487 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8300b2070 con 0x7fd82c004810 2024-04-23T14:35:00.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.463+0000 7fd846a00640 1 -- 172.21.15.100:0/1427633487 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8300a5560 con 0x7fd82c004810 2024-04-23T14:35:00.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.463+0000 7fd846a00640 1 -- 172.21.15.100:0/1427633487 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8300096b0 con 0x7fd82c004810 2024-04-23T14:35:00.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.463+0000 7fd84ca00640 1 -- 172.21.15.100:0/1427633487 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd82c004810 msgr2=0x7fd82c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.463+0000 7fd84ca00640 1 --2- 172.21.15.100:0/1427633487 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd82c004810 0x7fd82c004c90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd830009b10 tx=0x7fd83001ca80 comp rx=0 tx=0).stop 2024-04-23T14:35:00.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.463+0000 7fd847e00640 1 -- 172.21.15.100:0/1427633487 reap_dead start 2024-04-23T14:35:00.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.463+0000 7fd84ca00640 1 -- 172.21.15.100:0/1427633487 shutdown_connections 2024-04-23T14:35:00.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.463+0000 7fd84ca00640 1 -- 172.21.15.100:0/1427633487 >> 172.21.15.100:0/1427633487 conn(0x7fd8480cd500 msgr2=0x7fd8480cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:00.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.463+0000 7fd84ca00640 1 -- 172.21.15.100:0/1427633487 shutdown_connections 2024-04-23T14:35:00.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.463+0000 7fd84ca00640 1 -- 172.21.15.100:0/1427633487 wait complete. 2024-04-23T14:35:00.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.463+0000 7fd84ca00640 1 Processor -- start 2024-04-23T14:35:00.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.463+0000 7fd84ca00640 1 -- start start 2024-04-23T14:35:00.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd84ca00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd82c004810 0x7fd8481564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd84ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd848158140 0x7fd8480cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd84ca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8480d0400 0x7fd8480b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd84ca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd8480c6460 con 0x7fd848158140 2024-04-23T14:35:00.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd84ca00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd84806f800 con 0x7fd8480d0400 2024-04-23T14:35:00.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd84ca00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd84815a390 con 0x7fd82c004810 2024-04-23T14:35:00.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd83fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd848158140 0x7fd8480cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd83fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd848158140 0x7fd8480cfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34368/0 (socket says 172.21.15.100:34368) 2024-04-23T14:35:00.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd83fe00640 1 -- 172.21.15.100:0/355297859 learned_addr learned my addr 172.21.15.100:0/355297859 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:00.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd847e00640 1 --2- 172.21.15.100:0/355297859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8480d0400 0x7fd8480b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd83fe00640 1 -- 172.21.15.100:0/355297859 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd82c004810 msgr2=0x7fd8481564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd847400640 1 --2- 172.21.15.100:0/355297859 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd82c004810 0x7fd8481564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd83fe00640 1 --2- 172.21.15.100:0/355297859 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd82c004810 0x7fd8481564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd83fe00640 1 -- 172.21.15.100:0/355297859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8480d0400 msgr2=0x7fd8480b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd83fe00640 1 --2- 172.21.15.100:0/355297859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8480d0400 0x7fd8480b13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd83fe00640 1 -- 172.21.15.100:0/355297859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8480adab0 con 0x7fd848158140 2024-04-23T14:35:00.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd83fe00640 1 --2- 172.21.15.100:0/355297859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd848158140 0x7fd8480cfec0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd830009b10 tx=0x7fd82c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:00.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd83f400640 1 -- 172.21.15.100:0/355297859 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8300b2040 con 0x7fd848158140 2024-04-23T14:35:00.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd83f400640 1 -- 172.21.15.100:0/355297859 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd83000f000 con 0x7fd848158140 2024-04-23T14:35:00.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd83f400640 1 -- 172.21.15.100:0/355297859 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830004020 con 0x7fd848158140 2024-04-23T14:35:00.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd84ca00640 1 -- 172.21.15.100:0/355297859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8480add90 con 0x7fd848158140 2024-04-23T14:35:00.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd84ca00640 1 -- 172.21.15.100:0/355297859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8480ae350 con 0x7fd848158140 2024-04-23T14:35:00.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd84ca00640 1 -- 172.21.15.100:0/355297859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8480691e0 con 0x7fd848158140 2024-04-23T14:35:00.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.467+0000 7fd83f400640 1 -- 172.21.15.100:0/355297859 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd8300c0020 con 0x7fd848158140 2024-04-23T14:35:00.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.471+0000 7fd83f400640 1 --2- 172.21.15.100:0/355297859 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd8240683b0 0x7fd82406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.471+0000 7fd83f400640 1 -- 172.21.15.100:0/355297859 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fd8301208b0 con 0x7fd848158140 2024-04-23T14:35:00.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.471+0000 7fd847400640 1 --2- 172.21.15.100:0/355297859 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd8240683b0 0x7fd82406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.471+0000 7fd847400640 1 --2- 172.21.15.100:0/355297859 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd8240683b0 0x7fd82406a870 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd8480b6c90 tx=0x7fd838008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:00.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.479+0000 7fd83f400640 1 -- 172.21.15.100:0/355297859 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd8300c7020 con 0x7fd848158140 2024-04-23T14:35:00.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.635+0000 7fd84ca00640 1 -- 172.21.15.100:0/355297859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd8480ce3f0 con 0x7fd848158140 2024-04-23T14:35:00.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.635+0000 7fd83f400640 1 -- 172.21.15.100:0/355297859 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) ==== 93+0+10 (secure 0 0 0) 0x7fd830016800 con 0x7fd848158140 2024-04-23T14:35:00.636 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-23T14:35:00.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.635+0000 7fd84ca00640 1 -- 172.21.15.100:0/355297859 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd8240683b0 msgr2=0x7fd82406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.635+0000 7fd84ca00640 1 --2- 172.21.15.100:0/355297859 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd8240683b0 0x7fd82406a870 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd8480b6c90 tx=0x7fd838008040 comp rx=0 tx=0).stop 2024-04-23T14:35:00.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.635+0000 7fd84ca00640 1 -- 172.21.15.100:0/355297859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd848158140 msgr2=0x7fd8480cfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.635+0000 7fd84ca00640 1 --2- 172.21.15.100:0/355297859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd848158140 0x7fd8480cfec0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd830009b10 tx=0x7fd82c005840 comp rx=0 tx=0).stop 2024-04-23T14:35:00.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.635+0000 7fd84ca00640 1 -- 172.21.15.100:0/355297859 shutdown_connections 2024-04-23T14:35:00.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.635+0000 7fd84ca00640 1 --2- 172.21.15.100:0/355297859 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd8240683b0 0x7fd82406a870 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.635+0000 7fd84ca00640 1 --2- 172.21.15.100:0/355297859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8480d0400 0x7fd8480b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.635+0000 7fd84ca00640 1 --2- 172.21.15.100:0/355297859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd848158140 0x7fd8480cfec0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.635+0000 7fd84ca00640 1 --2- 172.21.15.100:0/355297859 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd82c004810 0x7fd8481564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.635+0000 7fd84ca00640 1 -- 172.21.15.100:0/355297859 >> 172.21.15.100:0/355297859 conn(0x7fd8480cd500 msgr2=0x7fd8480cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:00.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.635+0000 7fd84ca00640 1 -- 172.21.15.100:0/355297859 shutdown_connections 2024-04-23T14:35:00.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.635+0000 7fd84ca00640 1 -- 172.21.15.100:0/355297859 wait complete. 2024-04-23T14:35:00.656 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:35:00.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.795+0000 7f9a0f400640 1 Processor -- start 2024-04-23T14:35:00.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.795+0000 7f9a0f400640 1 -- start start 2024-04-23T14:35:00.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.795+0000 7f9a0f400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9a0815e7a0 0x7f9a080aea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.795+0000 7f9a0f400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f9a080cee40 0x7f9a080cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.795+0000 7f9a0f400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a080ada30 con 0x7f9a080cf7f0 2024-04-23T14:35:00.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.795+0000 7f9a0f400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f9a080aefb0 con 0x7f9a0815e7a0 2024-04-23T14:35:00.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.795+0000 7f9a0f400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f9a080adcb0 con 0x7f9a080cee40 2024-04-23T14:35:00.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.795+0000 7f9a0ca00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9a0815e7a0 0x7f9a080aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.795+0000 7f9a0d400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f9a080cf7f0 0x7f9a0806e6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50494/0 (socket says 172.21.15.100:50494) 2024-04-23T14:35:00.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.795+0000 7f9a0ca00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9a0815e7a0 0x7f9a080aea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34378/0 (socket says 172.21.15.100:34378) 2024-04-23T14:35:00.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.795+0000 7f9a0d400640 1 -- 172.21.15.100:0/593676198 learned_addr learned my addr 172.21.15.100:0/593676198 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:00.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.795+0000 7f9a07e00640 1 --2- 172.21.15.100:0/593676198 >> v2:172.21.15.151:3300/0 conn(0x7f9a080cee40 0x7f9a080cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.795+0000 7f9a0ca00640 1 -- 172.21.15.100:0/593676198 >> v2:172.21.15.151:3300/0 conn(0x7f9a080cee40 msgr2=0x7f9a080cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:00.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.795+0000 7f9a0ca00640 1 --2- 172.21.15.100:0/593676198 >> v2:172.21.15.151:3300/0 conn(0x7f9a080cee40 0x7f9a080cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.795+0000 7f9a0ca00640 1 -- 172.21.15.100:0/593676198 >> v1:172.21.15.100:6789/0 conn(0x7f9a080cf7f0 legacy=0x7f9a0806e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.795+0000 7f9a0ca00640 1 -- 172.21.15.100:0/593676198 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a080adf90 con 0x7f9a0815e7a0 2024-04-23T14:35:00.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.795+0000 7f9a07400640 1 -- 172.21.15.100:0/593676198 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3030969576 0 0) 0x7f9a000014a0 con 0x7f9a080cf7f0 2024-04-23T14:35:00.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.799+0000 7f9a0ca00640 1 --2- 172.21.15.100:0/593676198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0815e7a0 0x7f9a080aea70 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9a080afc90 tx=0x7f99fc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e124c9f2eab8c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:00.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.799+0000 7f9a07400640 1 -- 172.21.15.100:0/593676198 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0b0070 con 0x7f9a0815e7a0 2024-04-23T14:35:00.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.799+0000 7f9a07400640 1 -- 172.21.15.100:0/593676198 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99fc0ab600 con 0x7f9a0815e7a0 2024-04-23T14:35:00.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.799+0000 7f9a07400640 1 -- 172.21.15.100:0/593676198 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0ab9d0 con 0x7f9a0815e7a0 2024-04-23T14:35:00.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.799+0000 7f9a0f400640 1 -- 172.21.15.100:0/593676198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0815e7a0 msgr2=0x7f9a080aea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.799+0000 7f9a0f400640 1 --2- 172.21.15.100:0/593676198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0815e7a0 0x7f9a080aea70 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9a080afc90 tx=0x7f99fc01c480 comp rx=0 tx=0).stop 2024-04-23T14:35:00.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.799+0000 7f9a0f400640 1 -- 172.21.15.100:0/593676198 shutdown_connections 2024-04-23T14:35:00.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.799+0000 7f9a0f400640 1 --2- 172.21.15.100:0/593676198 >> v2:172.21.15.151:3300/0 conn(0x7f9a080cee40 0x7f9a080cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.799+0000 7f9a0f400640 1 --2- 172.21.15.100:0/593676198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0815e7a0 0x7f9a080aea70 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.799+0000 7f9a0f400640 1 -- 172.21.15.100:0/593676198 >> 172.21.15.100:0/593676198 conn(0x7f9a080b5200 msgr2=0x7f9a080b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:00.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.799+0000 7f9a0f400640 1 -- 172.21.15.100:0/593676198 shutdown_connections 2024-04-23T14:35:00.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.799+0000 7f9a0f400640 1 -- 172.21.15.100:0/593676198 wait complete. 2024-04-23T14:35:00.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.799+0000 7f9a0f400640 1 Processor -- start 2024-04-23T14:35:00.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.799+0000 7f9a0f400640 1 -- start start 2024-04-23T14:35:00.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a0f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a080cee40 0x7f9a08098c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a0f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a080cf7f0 0x7f9a08097ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a0f400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9a0815e7a0 0x7f9a08157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a0f400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0815ed00 con 0x7f9a080cee40 2024-04-23T14:35:00.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a0f400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0815eb80 con 0x7f9a080cf7f0 2024-04-23T14:35:00.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a0f400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9a0816b400 con 0x7f9a0815e7a0 2024-04-23T14:35:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a0ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a080cee40 0x7f9a08098c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a0ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a080cee40 0x7f9a08098c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34394/0 (socket says 172.21.15.100:34394) 2024-04-23T14:35:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a0ca00640 1 -- 172.21.15.100:0/2820202493 learned_addr learned my addr 172.21.15.100:0/2820202493 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a07e00640 1 --2- 172.21.15.100:0/2820202493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a080cf7f0 0x7f9a08097ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a0d400640 1 --2- 172.21.15.100:0/2820202493 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9a0815e7a0 0x7f9a08157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a0ca00640 1 -- 172.21.15.100:0/2820202493 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9a0815e7a0 msgr2=0x7f9a08157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a0ca00640 1 --2- 172.21.15.100:0/2820202493 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9a0815e7a0 0x7f9a08157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a0ca00640 1 -- 172.21.15.100:0/2820202493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a080cf7f0 msgr2=0x7f9a08097ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a0ca00640 1 --2- 172.21.15.100:0/2820202493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a080cf7f0 0x7f9a08097ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a0ca00640 1 -- 172.21.15.100:0/2820202493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0815c9f0 con 0x7f9a080cee40 2024-04-23T14:35:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a0ca00640 1 --2- 172.21.15.100:0/2820202493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a080cee40 0x7f9a08098c50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0ae040 tx=0x7f99fc01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a04c00640 1 -- 172.21.15.100:0/2820202493 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0b0050 con 0x7f9a080cee40 2024-04-23T14:35:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a04c00640 1 -- 172.21.15.100:0/2820202493 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99fc0bc070 con 0x7f9a080cee40 2024-04-23T14:35:00.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a04c00640 1 -- 172.21.15.100:0/2820202493 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0b5830 con 0x7f9a080cee40 2024-04-23T14:35:00.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a0f400640 1 -- 172.21.15.100:0/2820202493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a08098530 con 0x7f9a080cee40 2024-04-23T14:35:00.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a0f400640 1 -- 172.21.15.100:0/2820202493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a080b1700 con 0x7f9a080cee40 2024-04-23T14:35:00.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a0f400640 1 -- 172.21.15.100:0/2820202493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a08068ad0 con 0x7f9a080cee40 2024-04-23T14:35:00.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.803+0000 7f9a04c00640 1 -- 172.21.15.100:0/2820202493 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f99fc0b59d0 con 0x7f9a080cee40 2024-04-23T14:35:00.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.807+0000 7f9a04c00640 1 --2- 172.21.15.100:0/2820202493 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f99e8068380 0x7f99e806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.807+0000 7f9a04c00640 1 -- 172.21.15.100:0/2820202493 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f99fc121490 con 0x7f9a080cee40 2024-04-23T14:35:00.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.807+0000 7f9a07e00640 1 --2- 172.21.15.100:0/2820202493 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f99e8068380 0x7f99e806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.807+0000 7f9a07e00640 1 --2- 172.21.15.100:0/2820202493 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f99e8068380 0x7f99e806a840 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9a080ad8e0 tx=0x7f99f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:00.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.811+0000 7f9a04c00640 1 -- 172.21.15.100:0/2820202493 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f99fc016070 con 0x7f9a080cee40 2024-04-23T14:35:00.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.967+0000 7f9a0f400640 1 -- 172.21.15.100:0/2820202493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9a0815d640 con 0x7f9a080cee40 2024-04-23T14:35:00.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.967+0000 7f9a04c00640 1 -- 172.21.15.100:0/2820202493 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) ==== 104+0+11 (secure 0 0 0) 0x7f99fc0f1820 con 0x7f9a080cee40 2024-04-23T14:35:00.969 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-23T14:35:00.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.967+0000 7f9a0f400640 1 -- 172.21.15.100:0/2820202493 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f99e8068380 msgr2=0x7f99e806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.967+0000 7f9a0f400640 1 --2- 172.21.15.100:0/2820202493 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f99e8068380 0x7f99e806a840 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9a080ad8e0 tx=0x7f99f4008040 comp rx=0 tx=0).stop 2024-04-23T14:35:00.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.967+0000 7f9a0f400640 1 -- 172.21.15.100:0/2820202493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a080cee40 msgr2=0x7f9a08098c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.967+0000 7f9a0f400640 1 --2- 172.21.15.100:0/2820202493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a080cee40 0x7f9a08098c50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0ae040 tx=0x7f99fc01acc0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.971+0000 7f9a0f400640 1 -- 172.21.15.100:0/2820202493 shutdown_connections 2024-04-23T14:35:00.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.971+0000 7f9a0f400640 1 --2- 172.21.15.100:0/2820202493 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f99e8068380 0x7f99e806a840 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.971+0000 7f9a0f400640 1 --2- 172.21.15.100:0/2820202493 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9a0815e7a0 0x7f9a08157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.971+0000 7f9a0f400640 1 --2- 172.21.15.100:0/2820202493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a080cf7f0 0x7f9a08097ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.971+0000 7f9a0f400640 1 --2- 172.21.15.100:0/2820202493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a080cee40 0x7f9a08098c50 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.971+0000 7f9a0f400640 1 -- 172.21.15.100:0/2820202493 >> 172.21.15.100:0/2820202493 conn(0x7f9a080b5200 msgr2=0x7f9a08073850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:00.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.971+0000 7f9a0f400640 1 -- 172.21.15.100:0/2820202493 shutdown_connections 2024-04-23T14:35:00.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:00.971+0000 7f9a0f400640 1 -- 172.21.15.100:0/2820202493 wait complete. 2024-04-23T14:35:00.990 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:35:01.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.131+0000 7fa943200640 1 Processor -- start 2024-04-23T14:35:01.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.131+0000 7fa943200640 1 -- start start 2024-04-23T14:35:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.131+0000 7fa943200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa93c15c7a0 0x7fa93c0af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.131+0000 7fa943200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fa93c0cd300 0x7fa93c0ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.131+0000 7fa943200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa93c06cbb0 con 0x7fa93c06bb20 2024-04-23T14:35:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.131+0000 7fa943200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fa93c06d450 con 0x7fa93c15c7a0 2024-04-23T14:35:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.131+0000 7fa943200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fa93c06ce90 con 0x7fa93c0cd300 2024-04-23T14:35:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.131+0000 7fa93be00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa93c15c7a0 0x7fa93c0af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.131+0000 7fa933400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fa93c06bb20 0x7fa93c0ccbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50496/0 (socket says 172.21.15.100:50496) 2024-04-23T14:35:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.131+0000 7fa93be00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa93c15c7a0 0x7fa93c0af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34396/0 (socket says 172.21.15.100:34396) 2024-04-23T14:35:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.131+0000 7fa933400640 1 -- 172.21.15.100:0/2122259368 learned_addr learned my addr 172.21.15.100:0/2122259368 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.131+0000 7fa941200640 1 --2- 172.21.15.100:0/2122259368 >> v2:172.21.15.151:3300/0 conn(0x7fa93c0cd300 0x7fa93c0ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.131+0000 7fa93be00640 1 -- 172.21.15.100:0/2122259368 >> v2:172.21.15.151:3300/0 conn(0x7fa93c0cd300 msgr2=0x7fa93c0ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.131+0000 7fa93be00640 1 --2- 172.21.15.100:0/2122259368 >> v2:172.21.15.151:3300/0 conn(0x7fa93c0cd300 0x7fa93c0ad430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.131+0000 7fa93be00640 1 -- 172.21.15.100:0/2122259368 >> v1:172.21.15.100:6789/0 conn(0x7fa93c06bb20 legacy=0x7fa93c0ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.131+0000 7fa93be00640 1 -- 172.21.15.100:0/2122259368 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93c06d170 con 0x7fa93c15c7a0 2024-04-23T14:35:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.135+0000 7fa93be00640 1 --2- 172.21.15.100:0/2122259368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa93c15c7a0 0x7fa93c0af190 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0cd8f0 tx=0x7fa92c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844efab329206400 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.135+0000 7fa93b400640 1 -- 172.21.15.100:0/2122259368 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c0b0070 con 0x7fa93c15c7a0 2024-04-23T14:35:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.135+0000 7fa93b400640 1 -- 172.21.15.100:0/2122259368 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa92c0ab600 con 0x7fa93c15c7a0 2024-04-23T14:35:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.135+0000 7fa93b400640 1 -- 172.21.15.100:0/2122259368 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c0ab9d0 con 0x7fa93c15c7a0 2024-04-23T14:35:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.135+0000 7fa943200640 1 -- 172.21.15.100:0/2122259368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa93c15c7a0 msgr2=0x7fa93c0af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.135+0000 7fa943200640 1 --2- 172.21.15.100:0/2122259368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa93c15c7a0 0x7fa93c0af190 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0cd8f0 tx=0x7fa92c01c3f0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.135+0000 7fa943200640 1 -- 172.21.15.100:0/2122259368 shutdown_connections 2024-04-23T14:35:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.135+0000 7fa943200640 1 --2- 172.21.15.100:0/2122259368 >> v2:172.21.15.151:3300/0 conn(0x7fa93c0cd300 0x7fa93c0ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.135+0000 7fa943200640 1 --2- 172.21.15.100:0/2122259368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa93c15c7a0 0x7fa93c0af190 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.135+0000 7fa943200640 1 -- 172.21.15.100:0/2122259368 >> 172.21.15.100:0/2122259368 conn(0x7fa93c0b5200 msgr2=0x7fa93c0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.135+0000 7fa943200640 1 -- 172.21.15.100:0/2122259368 shutdown_connections 2024-04-23T14:35:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.135+0000 7fa943200640 1 -- 172.21.15.100:0/2122259368 wait complete. 2024-04-23T14:35:01.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.135+0000 7fa943200640 1 Processor -- start 2024-04-23T14:35:01.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.135+0000 7fa943200640 1 -- start start 2024-04-23T14:35:01.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa943200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa93c06bb20 0x7fa93c158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:01.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa943200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa93c0cd300 0x7fa93c15d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:01.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa943200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa93c15c7a0 0x7fa93c15deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:01.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa943200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c0ae590 con 0x7fa93c15c7a0 2024-04-23T14:35:01.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa943200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c0ae410 con 0x7fa93c06bb20 2024-04-23T14:35:01.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa943200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa93c0ae710 con 0x7fa93c0cd300 2024-04-23T14:35:01.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa941200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa93c15c7a0 0x7fa93c15deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa941200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa93c15c7a0 0x7fa93c15deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34402/0 (socket says 172.21.15.100:34402) 2024-04-23T14:35:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa941200640 1 -- 172.21.15.100:0/2732139557 learned_addr learned my addr 172.21.15.100:0/2732139557 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa93be00640 1 --2- 172.21.15.100:0/2732139557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa93c06bb20 0x7fa93c158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa933400640 1 --2- 172.21.15.100:0/2732139557 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa93c0cd300 0x7fa93c15d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa941200640 1 -- 172.21.15.100:0/2732139557 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa93c0cd300 msgr2=0x7fa93c15d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa941200640 1 --2- 172.21.15.100:0/2732139557 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa93c0cd300 0x7fa93c15d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa941200640 1 -- 172.21.15.100:0/2732139557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa93c06bb20 msgr2=0x7fa93c158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa941200640 1 --2- 172.21.15.100:0/2732139557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa93c06bb20 0x7fa93c158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa941200640 1 -- 172.21.15.100:0/2732139557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93c15f120 con 0x7fa93c15c7a0 2024-04-23T14:35:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa933400640 1 --2- 172.21.15.100:0/2732139557 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa93c0cd300 0x7fa93c15d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:35:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa93be00640 1 --2- 172.21.15.100:0/2732139557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa93c06bb20 0x7fa93c158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:35:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa941200640 1 --2- 172.21.15.100:0/2732139557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa93c15c7a0 0x7fa93c15deb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa934007e60 tx=0x7fa93400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa938c00640 1 -- 172.21.15.100:0/2732139557 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934014420 con 0x7fa93c15c7a0 2024-04-23T14:35:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa938c00640 1 -- 172.21.15.100:0/2732139557 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa934019070 con 0x7fa93c15c7a0 2024-04-23T14:35:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa943200640 1 -- 172.21.15.100:0/2732139557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa93c15f400 con 0x7fa93c15c7a0 2024-04-23T14:35:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa938c00640 1 -- 172.21.15.100:0/2732139557 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934014820 con 0x7fa93c15c7a0 2024-04-23T14:35:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa943200640 1 -- 172.21.15.100:0/2732139557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa93c15f9c0 con 0x7fa93c15c7a0 2024-04-23T14:35:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa943200640 1 -- 172.21.15.100:0/2732139557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa93c068ad0 con 0x7fa93c15c7a0 2024-04-23T14:35:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.139+0000 7fa938c00640 1 -- 172.21.15.100:0/2732139557 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa9340149c0 con 0x7fa93c15c7a0 2024-04-23T14:35:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.143+0000 7fa938c00640 1 --2- 172.21.15.100:0/2732139557 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa9180683b0 0x7fa91806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.143+0000 7fa938c00640 1 -- 172.21.15.100:0/2732139557 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fa9340873f0 con 0x7fa93c15c7a0 2024-04-23T14:35:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.143+0000 7fa93be00640 1 --2- 172.21.15.100:0/2732139557 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa9180683b0 0x7fa91806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.143+0000 7fa93be00640 1 --2- 172.21.15.100:0/2732139557 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa9180683b0 0x7fa91806a870 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0ad000 tx=0x7fa92c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:01.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.147+0000 7fa938c00640 1 -- 172.21.15.100:0/2732139557 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa934057780 con 0x7fa93c15c7a0 2024-04-23T14:35:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.299+0000 7fa943200640 1 -- 172.21.15.100:0/2732139557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa93c1588a0 con 0x7fa93c15c7a0 2024-04-23T14:35:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.303+0000 7fa938c00640 1 -- 172.21.15.100:0/2732139557 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) ==== 100+0+11 (secure 0 0 0) 0x7fa934057080 con 0x7fa93c15c7a0 2024-04-23T14:35:01.303 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-23T14:35:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.303+0000 7fa943200640 1 -- 172.21.15.100:0/2732139557 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa9180683b0 msgr2=0x7fa91806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.303+0000 7fa943200640 1 --2- 172.21.15.100:0/2732139557 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa9180683b0 0x7fa91806a870 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0ad000 tx=0x7fa92c008040 comp rx=0 tx=0).stop 2024-04-23T14:35:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.303+0000 7fa943200640 1 -- 172.21.15.100:0/2732139557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa93c15c7a0 msgr2=0x7fa93c15deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.303+0000 7fa943200640 1 --2- 172.21.15.100:0/2732139557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa93c15c7a0 0x7fa93c15deb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa934007e60 tx=0x7fa93400e4b0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.303+0000 7fa943200640 1 -- 172.21.15.100:0/2732139557 shutdown_connections 2024-04-23T14:35:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.303+0000 7fa943200640 1 --2- 172.21.15.100:0/2732139557 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fa9180683b0 0x7fa91806a870 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.303+0000 7fa943200640 1 --2- 172.21.15.100:0/2732139557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa93c15c7a0 0x7fa93c15deb0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.303+0000 7fa943200640 1 --2- 172.21.15.100:0/2732139557 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa93c0cd300 0x7fa93c15d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.303+0000 7fa943200640 1 --2- 172.21.15.100:0/2732139557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa93c06bb20 0x7fa93c158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.303+0000 7fa943200640 1 -- 172.21.15.100:0/2732139557 >> 172.21.15.100:0/2732139557 conn(0x7fa93c0b5200 msgr2=0x7fa93c15b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.303+0000 7fa943200640 1 -- 172.21.15.100:0/2732139557 shutdown_connections 2024-04-23T14:35:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.303+0000 7fa943200640 1 -- 172.21.15.100:0/2732139557 wait complete. 2024-04-23T14:35:01.324 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:35:01.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.459+0000 7f699fa00640 1 Processor -- start 2024-04-23T14:35:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.459+0000 7f699fa00640 1 -- start start 2024-04-23T14:35:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.459+0000 7f699fa00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f699815b640 0x7f69980adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.459+0000 7f699fa00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f69980b2a10 0x7f6998072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.459+0000 7f699fa00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f69980cc750 0x7f6998073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.459+0000 7f699fa00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f69980af080 con 0x7f69980b2a10 2024-04-23T14:35:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.459+0000 7f699fa00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f69980afbc0 con 0x7f699815b640 2024-04-23T14:35:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.459+0000 7f699fa00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f699815ba20 con 0x7f69980cc750 2024-04-23T14:35:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.459+0000 7f698fe00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f69980b2a10 0x7f6998072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.459+0000 7f698fe00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f69980b2a10 0x7f6998072f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34408/0 (socket says 172.21.15.100:34408) 2024-04-23T14:35:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.459+0000 7f698fe00640 1 -- 172.21.15.100:0/3194826797 learned_addr learned my addr 172.21.15.100:0/3194826797 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.459+0000 7f699d000640 1 --2- 172.21.15.100:0/3194826797 >> v2:172.21.15.151:3300/0 conn(0x7f699815b640 0x7f69980adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.459+0000 7f699da00640 1 --2- 172.21.15.100:0/3194826797 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f69980cc750 0x7f6998073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.459+0000 7f698fe00640 1 -- 172.21.15.100:0/3194826797 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f69980cc750 msgr2=0x7f6998073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:01.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.459+0000 7f698fe00640 1 --2- 172.21.15.100:0/3194826797 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f69980cc750 0x7f6998073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.459+0000 7f698fe00640 1 -- 172.21.15.100:0/3194826797 >> v2:172.21.15.151:3300/0 conn(0x7f699815b640 msgr2=0x7f69980adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:01.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.459+0000 7f698fe00640 1 --2- 172.21.15.100:0/3194826797 >> v2:172.21.15.151:3300/0 conn(0x7f699815b640 0x7f69980adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.459+0000 7f698fe00640 1 -- 172.21.15.100:0/3194826797 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69980af320 con 0x7f69980b2a10 2024-04-23T14:35:01.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.459+0000 7f699d000640 1 --2- 172.21.15.100:0/3194826797 >> v2:172.21.15.151:3300/0 conn(0x7f699815b640 0x7f69980adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:35:01.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.463+0000 7f698fe00640 1 --2- 172.21.15.100:0/3194826797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69980b2a10 0x7f6998072f00 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f6998068100 tx=0x7f698001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b983d0f9940ce01a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:01.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.463+0000 7f6997e00640 1 -- 172.21.15.100:0/3194826797 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69800aa610 con 0x7f69980b2a10 2024-04-23T14:35:01.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.463+0000 7f6997e00640 1 -- 172.21.15.100:0/3194826797 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69800af070 con 0x7f69980b2a10 2024-04-23T14:35:01.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.463+0000 7f6997e00640 1 -- 172.21.15.100:0/3194826797 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69800aa9d0 con 0x7f69980b2a10 2024-04-23T14:35:01.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.463+0000 7f699fa00640 1 -- 172.21.15.100:0/3194826797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69980b2a10 msgr2=0x7f6998072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:01.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.463+0000 7f699fa00640 1 --2- 172.21.15.100:0/3194826797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69980b2a10 0x7f6998072f00 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f6998068100 tx=0x7f698001c4f0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.463+0000 7f699fa00640 1 -- 172.21.15.100:0/3194826797 shutdown_connections 2024-04-23T14:35:01.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.463+0000 7f699fa00640 1 --2- 172.21.15.100:0/3194826797 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f69980cc750 0x7f6998073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.463+0000 7f699fa00640 1 --2- 172.21.15.100:0/3194826797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69980b2a10 0x7f6998072f00 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.463+0000 7f699fa00640 1 --2- 172.21.15.100:0/3194826797 >> v2:172.21.15.151:3300/0 conn(0x7f699815b640 0x7f69980adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.463+0000 7f699fa00640 1 -- 172.21.15.100:0/3194826797 >> 172.21.15.100:0/3194826797 conn(0x7f69980b1e80 msgr2=0x7f69980b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:01.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.463+0000 7f699fa00640 1 -- 172.21.15.100:0/3194826797 shutdown_connections 2024-04-23T14:35:01.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.463+0000 7f699fa00640 1 -- 172.21.15.100:0/3194826797 wait complete. 2024-04-23T14:35:01.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f699fa00640 1 Processor -- start 2024-04-23T14:35:01.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f699fa00640 1 -- start start 2024-04-23T14:35:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f699fa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69980b2a10 0x7f699815bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f699fa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69980cc750 0x7f699815e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f699fa00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f699815b640 0x7f699814d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f699fa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f69980728c0 con 0x7f69980cc750 2024-04-23T14:35:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f699fa00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6998072740 con 0x7f69980b2a10 2024-04-23T14:35:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f699fa00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f6998072a40 con 0x7f699815b640 2024-04-23T14:35:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f698fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69980cc750 0x7f699815e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f698fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69980cc750 0x7f699815e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34414/0 (socket says 172.21.15.100:34414) 2024-04-23T14:35:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f698fe00640 1 -- 172.21.15.100:0/1655450236 learned_addr learned my addr 172.21.15.100:0/1655450236 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f699d000640 1 --2- 172.21.15.100:0/1655450236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69980b2a10 0x7f699815bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f699da00640 1 --2- 172.21.15.100:0/1655450236 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f699815b640 0x7f699814d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f698fe00640 1 -- 172.21.15.100:0/1655450236 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f699815b640 msgr2=0x7f699814d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f698fe00640 1 --2- 172.21.15.100:0/1655450236 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f699815b640 0x7f699814d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f698fe00640 1 -- 172.21.15.100:0/1655450236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69980b2a10 msgr2=0x7f699815bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f698fe00640 1 --2- 172.21.15.100:0/1655450236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69980b2a10 0x7f699815bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f698fe00640 1 -- 172.21.15.100:0/1655450236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699814c740 con 0x7f69980cc750 2024-04-23T14:35:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f699d000640 1 --2- 172.21.15.100:0/1655450236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69980b2a10 0x7f699815bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:35:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f698fe00640 1 --2- 172.21.15.100:0/1655450236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69980cc750 0x7f699815e620 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f69800ac040 tx=0x7f698001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f6995600640 1 -- 172.21.15.100:0/1655450236 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69800bd5f0 con 0x7f69980cc750 2024-04-23T14:35:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f6995600640 1 -- 172.21.15.100:0/1655450236 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69800bd790 con 0x7f69980cc750 2024-04-23T14:35:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f6995600640 1 -- 172.21.15.100:0/1655450236 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69800bdb40 con 0x7f69980cc750 2024-04-23T14:35:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f699fa00640 1 -- 172.21.15.100:0/1655450236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6998152f10 con 0x7f69980cc750 2024-04-23T14:35:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f699fa00640 1 -- 172.21.15.100:0/1655450236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6998152c30 con 0x7f69980cc750 2024-04-23T14:35:01.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.467+0000 7f699fa00640 1 -- 172.21.15.100:0/1655450236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6998068ad0 con 0x7f69980cc750 2024-04-23T14:35:01.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.471+0000 7f6995600640 1 -- 172.21.15.100:0/1655450236 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f69800b33f0 con 0x7f69980cc750 2024-04-23T14:35:01.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.471+0000 7f6995600640 1 --2- 172.21.15.100:0/1655450236 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6974068380 0x7f697406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:01.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.471+0000 7f6995600640 1 -- 172.21.15.100:0/1655450236 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f6980121b40 con 0x7f69980cc750 2024-04-23T14:35:01.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.471+0000 7f699d000640 1 --2- 172.21.15.100:0/1655450236 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6974068380 0x7f697406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:01.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.471+0000 7f699d000640 1 --2- 172.21.15.100:0/1655450236 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6974068380 0x7f697406a840 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f699806ffc0 tx=0x7f6988008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:01.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.475+0000 7f6995600640 1 -- 172.21.15.100:0/1655450236 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f699814c740 con 0x7f69980cc750 2024-04-23T14:35:01.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.663+0000 7f699fa00640 1 -- 172.21.15.100:0/1655450236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69981508b0 con 0x7f69980cc750 2024-04-23T14:35:01.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.667+0000 7f6995600640 1 -- 172.21.15.100:0/1655450236 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5190 (secure 0 0 0) 0x7f69800f6e20 con 0x7f69980cc750 2024-04-23T14:35:01.667 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:35:01.667 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5528,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2774610498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2774610498},{"type":"v1","addr":"172.21.15.151:6833","nonce":2774610498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5929,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1347883282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1347883282},{"type":"v1","addr":"172.21.15.151:6835","nonce":1347883282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":6177,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3319827520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3319827520},{"type":"v1","addr":"172.21.15.100:6836","nonce":3319827520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:34:24.162539+0000","modified":"2024-04-23T14:34:27.179639+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6180":{"gid":6180,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/608349550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":608349550},{"type":"v1","addr":"172.21.15.100:6837","nonce":608349550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6180,"qdb_cluster":[6180]},"id":3}]} 2024-04-23T14:35:01.668 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 28 2024-04-23T14:35:01.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.667+0000 7f699fa00640 1 -- 172.21.15.100:0/1655450236 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6974068380 msgr2=0x7f697406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:01.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.667+0000 7f699fa00640 1 --2- 172.21.15.100:0/1655450236 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6974068380 0x7f697406a840 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f699806ffc0 tx=0x7f6988008040 comp rx=0 tx=0).stop 2024-04-23T14:35:01.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.667+0000 7f699fa00640 1 -- 172.21.15.100:0/1655450236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69980cc750 msgr2=0x7f699815e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:01.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.667+0000 7f699fa00640 1 --2- 172.21.15.100:0/1655450236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69980cc750 0x7f699815e620 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f69800ac040 tx=0x7f698001ad30 comp rx=0 tx=0).stop 2024-04-23T14:35:01.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.667+0000 7f699fa00640 1 -- 172.21.15.100:0/1655450236 shutdown_connections 2024-04-23T14:35:01.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.667+0000 7f699fa00640 1 --2- 172.21.15.100:0/1655450236 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f6974068380 0x7f697406a840 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.667+0000 7f699fa00640 1 --2- 172.21.15.100:0/1655450236 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f699815b640 0x7f699814d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.667+0000 7f699fa00640 1 --2- 172.21.15.100:0/1655450236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69980cc750 0x7f699815e620 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.667+0000 7f699fa00640 1 --2- 172.21.15.100:0/1655450236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69980b2a10 0x7f699815bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.667+0000 7f699fa00640 1 -- 172.21.15.100:0/1655450236 >> 172.21.15.100:0/1655450236 conn(0x7f69980b1e80 msgr2=0x7f699806de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:01.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.667+0000 7f699fa00640 1 -- 172.21.15.100:0/1655450236 shutdown_connections 2024-04-23T14:35:01.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.667+0000 7f699fa00640 1 -- 172.21.15.100:0/1655450236 wait complete. 2024-04-23T14:35:01.687 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:35:01.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.823+0000 7eff82c00640 1 Processor -- start 2024-04-23T14:35:01.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.823+0000 7eff82c00640 1 -- start start 2024-04-23T14:35:01.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.823+0000 7eff82c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7eff7c0ad870 0x7eff7c0b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:01.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.823+0000 7eff82c00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff7c160320 con 0x7eff7c15c470 2024-04-23T14:35:01.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.823+0000 7eff82c00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff7c0cc7e0 con 0x7eff7c160de0 2024-04-23T14:35:01.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.823+0000 7eff82c00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7eff7c1605e0 con 0x7eff7c0ad870 2024-04-23T14:35:01.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.823+0000 7eff73400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7eff7c0ad870 0x7eff7c0b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:01.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.823+0000 7eff7be00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7eff7c15c470 0x7eff7c0ad160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50504/0 (socket says 172.21.15.100:50504) 2024-04-23T14:35:01.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.823+0000 7eff73400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7eff7c0ad870 0x7eff7c0b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34430/0 (socket says 172.21.15.100:34430) 2024-04-23T14:35:01.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.823+0000 7eff7be00640 1 -- 172.21.15.100:0/412405000 learned_addr learned my addr 172.21.15.100:0/412405000 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:01.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.823+0000 7eff73400640 1 -- 172.21.15.100:0/412405000 >> v1:172.21.15.151:6789/0 conn(0x7eff7c160de0 legacy=0x7eff7c06bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:01.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.823+0000 7eff73400640 1 -- 172.21.15.100:0/412405000 >> v1:172.21.15.100:6789/0 conn(0x7eff7c15c470 legacy=0x7eff7c0ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:01.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.823+0000 7eff73400640 1 -- 172.21.15.100:0/412405000 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7c1608c0 con 0x7eff7c0ad870 2024-04-23T14:35:01.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.827+0000 7eff73400640 1 --2- 172.21.15.100:0/412405000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7c0ad870 0x7eff7c0b29a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7eff640097c0 tx=0x7eff6401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252b4a93e5adb28f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:01.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.827+0000 7eff7b400640 1 -- 172.21.15.100:0/412405000 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff640b0070 con 0x7eff7c0ad870 2024-04-23T14:35:01.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.827+0000 7eff7b400640 1 -- 172.21.15.100:0/412405000 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff640ab600 con 0x7eff7c0ad870 2024-04-23T14:35:01.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.827+0000 7eff7b400640 1 -- 172.21.15.100:0/412405000 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff640ab9d0 con 0x7eff7c0ad870 2024-04-23T14:35:01.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.827+0000 7eff82c00640 1 -- 172.21.15.100:0/412405000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7c0ad870 msgr2=0x7eff7c0b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:01.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.827+0000 7eff82c00640 1 --2- 172.21.15.100:0/412405000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7c0ad870 0x7eff7c0b29a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7eff640097c0 tx=0x7eff6401c410 comp rx=0 tx=0).stop 2024-04-23T14:35:01.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.827+0000 7eff82c00640 1 -- 172.21.15.100:0/412405000 shutdown_connections 2024-04-23T14:35:01.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.827+0000 7eff82c00640 1 --2- 172.21.15.100:0/412405000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7c0ad870 0x7eff7c0b29a0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.827+0000 7eff82c00640 1 -- 172.21.15.100:0/412405000 >> 172.21.15.100:0/412405000 conn(0x7eff7c0b1e80 msgr2=0x7eff7c0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:01.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.827+0000 7eff82c00640 1 -- 172.21.15.100:0/412405000 shutdown_connections 2024-04-23T14:35:01.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.827+0000 7eff82c00640 1 -- 172.21.15.100:0/412405000 wait complete. 2024-04-23T14:35:01.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff82c00640 1 Processor -- start 2024-04-23T14:35:01.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff82c00640 1 -- start start 2024-04-23T14:35:01.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff82c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7c0ad870 0x7eff7c1517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:01.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff82c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff7c15c470 0x7eff7c144190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:01.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff82c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7eff7c160de0 0x7eff7c1446d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:01.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff82c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c15ca60 con 0x7eff7c0ad870 2024-04-23T14:35:01.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff82c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c0cc5c0 con 0x7eff7c15c470 2024-04-23T14:35:01.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff82c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7eff7c16a550 con 0x7eff7c160de0 2024-04-23T14:35:01.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff7be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7c0ad870 0x7eff7c1517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:01.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff7be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7c0ad870 0x7eff7c1517f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34444/0 (socket says 172.21.15.100:34444) 2024-04-23T14:35:01.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff7be00640 1 -- 172.21.15.100:0/309747372 learned_addr learned my addr 172.21.15.100:0/309747372 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:01.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff73400640 1 --2- 172.21.15.100:0/309747372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff7c15c470 0x7eff7c144190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:01.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff80c00640 1 --2- 172.21.15.100:0/309747372 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7eff7c160de0 0x7eff7c1446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:01.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff7be00640 1 -- 172.21.15.100:0/309747372 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7eff7c160de0 msgr2=0x7eff7c1446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:01.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff7be00640 1 --2- 172.21.15.100:0/309747372 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7eff7c160de0 0x7eff7c1446d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff7be00640 1 -- 172.21.15.100:0/309747372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff7c15c470 msgr2=0x7eff7c144190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:01.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff7be00640 1 --2- 172.21.15.100:0/309747372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff7c15c470 0x7eff7c144190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:01.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff7be00640 1 -- 172.21.15.100:0/309747372 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7c15f040 con 0x7eff7c0ad870 2024-04-23T14:35:01.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff73400640 1 --2- 172.21.15.100:0/309747372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff7c15c470 0x7eff7c144190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:35:01.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff7be00640 1 --2- 172.21.15.100:0/309747372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7c0ad870 0x7eff7c1517f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7eff6c00c500 tx=0x7eff6c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:01.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff78c00640 1 -- 172.21.15.100:0/309747372 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c017070 con 0x7eff7c0ad870 2024-04-23T14:35:01.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff78c00640 1 -- 172.21.15.100:0/309747372 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff6c005cf0 con 0x7eff7c0ad870 2024-04-23T14:35:01.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff82c00640 1 -- 172.21.15.100:0/309747372 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff7c15f2e0 con 0x7eff7c0ad870 2024-04-23T14:35:01.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff78c00640 1 -- 172.21.15.100:0/309747372 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c0072f0 con 0x7eff7c0ad870 2024-04-23T14:35:01.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff82c00640 1 -- 172.21.15.100:0/309747372 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff7c15f8a0 con 0x7eff7c0ad870 2024-04-23T14:35:01.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.831+0000 7eff82c00640 1 -- 172.21.15.100:0/309747372 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff7c068ad0 con 0x7eff7c0ad870 2024-04-23T14:35:01.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.835+0000 7eff78c00640 1 -- 172.21.15.100:0/309747372 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7eff6c01a070 con 0x7eff7c0ad870 2024-04-23T14:35:01.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.835+0000 7eff78c00640 1 --2- 172.21.15.100:0/309747372 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7eff58068380 0x7eff5806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:01.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.835+0000 7eff78c00640 1 -- 172.21.15.100:0/309747372 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7eff6c085e30 con 0x7eff7c0ad870 2024-04-23T14:35:01.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.835+0000 7eff73400640 1 --2- 172.21.15.100:0/309747372 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7eff58068380 0x7eff5806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:01.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.835+0000 7eff73400640 1 --2- 172.21.15.100:0/309747372 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7eff58068380 0x7eff5806a840 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7eff640ad000 tx=0x7eff64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:01.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.839+0000 7eff78c00640 1 -- 172.21.15.100:0/309747372 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7eff6c055c00 con 0x7eff7c0ad870 2024-04-23T14:35:01.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.995+0000 7eff82c00640 1 -- 172.21.15.100:0/309747372 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff7c15ae90 con 0x7eff7c0ad870 2024-04-23T14:35:01.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.995+0000 7eff78c00640 1 -- 172.21.15.100:0/309747372 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16950 (secure 0 0 0) 0x7eff6c059f00 con 0x7eff7c0ad870 2024-04-23T14:35:01.998 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:35:01.998 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":53,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:34:25.795515+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:34:22.491379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-23T14:34:23.120342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:35:02.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.999+0000 7eff82c00640 1 -- 172.21.15.100:0/309747372 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7eff58068380 msgr2=0x7eff5806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:02.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.999+0000 7eff82c00640 1 --2- 172.21.15.100:0/309747372 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7eff58068380 0x7eff5806a840 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7eff640ad000 tx=0x7eff64008040 comp rx=0 tx=0).stop 2024-04-23T14:35:02.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.999+0000 7eff82c00640 1 -- 172.21.15.100:0/309747372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7c0ad870 msgr2=0x7eff7c1517f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:02.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.999+0000 7eff82c00640 1 --2- 172.21.15.100:0/309747372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7c0ad870 0x7eff7c1517f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7eff6c00c500 tx=0x7eff6c00c9c0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.999+0000 7eff82c00640 1 -- 172.21.15.100:0/309747372 shutdown_connections 2024-04-23T14:35:02.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.999+0000 7eff82c00640 1 --2- 172.21.15.100:0/309747372 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7eff58068380 0x7eff5806a840 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.999+0000 7eff82c00640 1 --2- 172.21.15.100:0/309747372 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7eff7c160de0 0x7eff7c1446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.999+0000 7eff82c00640 1 --2- 172.21.15.100:0/309747372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff7c15c470 0x7eff7c144190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.999+0000 7eff82c00640 1 --2- 172.21.15.100:0/309747372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7c0ad870 0x7eff7c1517f0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.999+0000 7eff82c00640 1 -- 172.21.15.100:0/309747372 >> 172.21.15.100:0/309747372 conn(0x7eff7c0b1e80 msgr2=0x7eff7c159b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:02.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.999+0000 7eff82c00640 1 -- 172.21.15.100:0/309747372 shutdown_connections 2024-04-23T14:35:02.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:01.999+0000 7eff82c00640 1 -- 172.21.15.100:0/309747372 wait complete. 2024-04-23T14:35:02.020 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-23T14:35:02.021 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:35:02.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.155+0000 7fcea0600640 1 Processor -- start 2024-04-23T14:35:02.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.155+0000 7fcea0600640 1 -- start start 2024-04-23T14:35:02.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.155+0000 7fcea0600640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fce9815d600 0x7fce980d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:02.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.155+0000 7fcea0600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce980d04a0 con 0x7fce980cca50 2024-04-23T14:35:02.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.155+0000 7fcea0600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce980d0780 con 0x7fce9815c7e0 2024-04-23T14:35:02.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.155+0000 7fcea0600640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fce9815db00 con 0x7fce9815d600 2024-04-23T14:35:02.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.155+0000 7fce9dc00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fce980cca50 0x7fce980cce30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50518/0 (socket says 172.21.15.100:50518) 2024-04-23T14:35:02.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.155+0000 7fce9dc00640 1 -- 172.21.15.100:0/984928269 learned_addr learned my addr 172.21.15.100:0/984928269 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:02.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.155+0000 7fce9e600640 1 --2- 172.21.15.100:0/984928269 >> v2:172.21.15.151:3300/0 conn(0x7fce9815d600 0x7fce980d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:02.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.155+0000 7fce97e00640 1 -- 172.21.15.100:0/984928269 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3977509023 0 0) 0x7fce980d04a0 con 0x7fce980cca50 2024-04-23T14:35:02.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.155+0000 7fce97e00640 1 -- 172.21.15.100:0/984928269 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce80003530 con 0x7fce980cca50 2024-04-23T14:35:02.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.155+0000 7fce9e600640 1 -- 172.21.15.100:0/984928269 >> v1:172.21.15.151:6789/0 conn(0x7fce9815c7e0 legacy=0x7fce980d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:02.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.155+0000 7fce9e600640 1 -- 172.21.15.100:0/984928269 >> v1:172.21.15.100:6789/0 conn(0x7fce980cca50 legacy=0x7fce980cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:02.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.155+0000 7fce9e600640 1 -- 172.21.15.100:0/984928269 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce980d0a60 con 0x7fce9815d600 2024-04-23T14:35:02.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.159+0000 7fce9e600640 1 --2- 172.21.15.100:0/984928269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce9815d600 0x7fce980d1970 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fce90009d90 tx=0x7fce9001c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2c2d89f412fb5b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:02.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.159+0000 7fce97e00640 1 -- 172.21.15.100:0/984928269 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce900b2070 con 0x7fce9815d600 2024-04-23T14:35:02.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.159+0000 7fce97e00640 1 -- 172.21.15.100:0/984928269 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce900a5990 con 0x7fce9815d600 2024-04-23T14:35:02.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.159+0000 7fce97e00640 1 -- 172.21.15.100:0/984928269 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce900a5d40 con 0x7fce9815d600 2024-04-23T14:35:02.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.159+0000 7fcea0600640 1 -- 172.21.15.100:0/984928269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce9815d600 msgr2=0x7fce980d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:02.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.159+0000 7fcea0600640 1 --2- 172.21.15.100:0/984928269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce9815d600 0x7fce980d1970 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fce90009d90 tx=0x7fce9001c8b0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.159+0000 7fcea0600640 1 -- 172.21.15.100:0/984928269 shutdown_connections 2024-04-23T14:35:02.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.159+0000 7fcea0600640 1 --2- 172.21.15.100:0/984928269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce9815d600 0x7fce980d1970 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.159+0000 7fcea0600640 1 -- 172.21.15.100:0/984928269 >> 172.21.15.100:0/984928269 conn(0x7fce980cee80 msgr2=0x7fce980cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:02.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.159+0000 7fcea0600640 1 -- 172.21.15.100:0/984928269 shutdown_connections 2024-04-23T14:35:02.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.159+0000 7fcea0600640 1 -- 172.21.15.100:0/984928269 wait complete. 2024-04-23T14:35:02.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fcea0600640 1 Processor -- start 2024-04-23T14:35:02.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fcea0600640 1 -- start start 2024-04-23T14:35:02.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fcea0600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce980cca50 0x7fce9814bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:02.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fcea0600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce9815c7e0 0x7fce980c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:02.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fcea0600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce9815d600 0x7fce981669d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:02.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fcea0600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fce9816aae0 con 0x7fce9815d600 2024-04-23T14:35:02.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fcea0600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fce980b3e00 con 0x7fce9815c7e0 2024-04-23T14:35:02.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fcea0600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fce9816ac60 con 0x7fce980cca50 2024-04-23T14:35:02.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fce9e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce9815d600 0x7fce981669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:02.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fce9e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce9815d600 0x7fce981669d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34446/0 (socket says 172.21.15.100:34446) 2024-04-23T14:35:02.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fce9e600640 1 -- 172.21.15.100:0/4150594791 learned_addr learned my addr 172.21.15.100:0/4150594791 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:02.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fce9d200640 1 --2- 172.21.15.100:0/4150594791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce9815c7e0 0x7fce980c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:02.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fce9dc00640 1 --2- 172.21.15.100:0/4150594791 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce980cca50 0x7fce9814bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:02.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fce9e600640 1 -- 172.21.15.100:0/4150594791 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce980cca50 msgr2=0x7fce9814bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:02.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fce9e600640 1 --2- 172.21.15.100:0/4150594791 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce980cca50 0x7fce9814bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fce9e600640 1 -- 172.21.15.100:0/4150594791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce9815c7e0 msgr2=0x7fce980c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:02.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fce9e600640 1 --2- 172.21.15.100:0/4150594791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce9815c7e0 0x7fce980c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fce9e600640 1 -- 172.21.15.100:0/4150594791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9814c540 con 0x7fce9815d600 2024-04-23T14:35:02.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fce9dc00640 1 --2- 172.21.15.100:0/4150594791 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce980cca50 0x7fce9814bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:02.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fce9e600640 1 --2- 172.21.15.100:0/4150594791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce9815d600 0x7fce981669d0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fce900b0040 tx=0x7fce90019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:02.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fce95600640 1 -- 172.21.15.100:0/4150594791 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce900b2050 con 0x7fce9815d600 2024-04-23T14:35:02.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fce95600640 1 -- 172.21.15.100:0/4150594791 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce900be070 con 0x7fce9815d600 2024-04-23T14:35:02.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fce95600640 1 -- 172.21.15.100:0/4150594791 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90004040 con 0x7fce9815d600 2024-04-23T14:35:02.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fcea0600640 1 -- 172.21.15.100:0/4150594791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce981661d0 con 0x7fce9815d600 2024-04-23T14:35:02.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.163+0000 7fcea0600640 1 -- 172.21.15.100:0/4150594791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce9814cbd0 con 0x7fce9815d600 2024-04-23T14:35:02.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.167+0000 7fcea0600640 1 -- 172.21.15.100:0/4150594791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce98068ad0 con 0x7fce9815d600 2024-04-23T14:35:02.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.167+0000 7fce95600640 1 -- 172.21.15.100:0/4150594791 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fce900041e0 con 0x7fce9815d600 2024-04-23T14:35:02.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.167+0000 7fce95600640 1 --2- 172.21.15.100:0/4150594791 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fce78068380 0x7fce7806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:02.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.167+0000 7fce95600640 1 -- 172.21.15.100:0/4150594791 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fce90121cc0 con 0x7fce9815d600 2024-04-23T14:35:02.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.167+0000 7fce9dc00640 1 --2- 172.21.15.100:0/4150594791 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fce78068380 0x7fce7806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:02.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.167+0000 7fce9dc00640 1 --2- 172.21.15.100:0/4150594791 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fce78068380 0x7fce7806a840 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fce8c005890 tx=0x7fce8c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:02.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.171+0000 7fce95600640 1 -- 172.21.15.100:0/4150594791 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fce90128080 con 0x7fce9815d600 2024-04-23T14:35:02.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.359+0000 7fcea0600640 1 -- 172.21.15.100:0/4150594791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce98151540 con 0x7fce9815d600 2024-04-23T14:35:02.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.359+0000 7fce95600640 1 -- 172.21.15.100:0/4150594791 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5190 (secure 0 0 0) 0x7fce90016560 con 0x7fce9815d600 2024-04-23T14:35:02.362 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:35:02.363 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5528,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2774610498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2774610498},{"type":"v1","addr":"172.21.15.151:6833","nonce":2774610498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5929,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1347883282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1347883282},{"type":"v1","addr":"172.21.15.151:6835","nonce":1347883282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":6177,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3319827520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3319827520},{"type":"v1","addr":"172.21.15.100:6836","nonce":3319827520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:34:24.162539+0000","modified":"2024-04-23T14:34:27.179639+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6180":{"gid":6180,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/608349550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":608349550},{"type":"v1","addr":"172.21.15.100:6837","nonce":608349550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6180,"qdb_cluster":[6180]},"id":3}]} 2024-04-23T14:35:02.363 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 28 2024-04-23T14:35:02.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.363+0000 7fcea0600640 1 -- 172.21.15.100:0/4150594791 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fce78068380 msgr2=0x7fce7806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:02.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.363+0000 7fcea0600640 1 --2- 172.21.15.100:0/4150594791 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fce78068380 0x7fce7806a840 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fce8c005890 tx=0x7fce8c009040 comp rx=0 tx=0).stop 2024-04-23T14:35:02.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.363+0000 7fcea0600640 1 -- 172.21.15.100:0/4150594791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce9815d600 msgr2=0x7fce981669d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:02.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.363+0000 7fcea0600640 1 --2- 172.21.15.100:0/4150594791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce9815d600 0x7fce981669d0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fce900b0040 tx=0x7fce90019650 comp rx=0 tx=0).stop 2024-04-23T14:35:02.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.363+0000 7fcea0600640 1 -- 172.21.15.100:0/4150594791 shutdown_connections 2024-04-23T14:35:02.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.363+0000 7fcea0600640 1 --2- 172.21.15.100:0/4150594791 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fce78068380 0x7fce7806a840 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.363+0000 7fcea0600640 1 --2- 172.21.15.100:0/4150594791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce9815d600 0x7fce981669d0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.363+0000 7fcea0600640 1 --2- 172.21.15.100:0/4150594791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce9815c7e0 0x7fce980c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.363+0000 7fcea0600640 1 --2- 172.21.15.100:0/4150594791 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce980cca50 0x7fce9814bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.363+0000 7fcea0600640 1 -- 172.21.15.100:0/4150594791 >> 172.21.15.100:0/4150594791 conn(0x7fce980cee80 msgr2=0x7fce98152b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:02.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.363+0000 7fcea0600640 1 -- 172.21.15.100:0/4150594791 shutdown_connections 2024-04-23T14:35:02.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.363+0000 7fcea0600640 1 -- 172.21.15.100:0/4150594791 wait complete. 2024-04-23T14:35:02.384 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:35:02.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b1de00640 1 Processor -- start 2024-04-23T14:35:02.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b1de00640 1 -- start start 2024-04-23T14:35:02.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b1de00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f9b180cca50 0x7f9b180cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:02.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b1de00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9b1815c7e0 0x7f9b18160170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:02.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b1de00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b180cd370 con 0x7f9b1815d600 2024-04-23T14:35:02.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b1de00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f9b180d2270 con 0x7f9b180cca50 2024-04-23T14:35:02.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b1de00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9b1815db00 con 0x7f9b1815c7e0 2024-04-23T14:35:02.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b17400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f9b1815d600 0x7f9b181606b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50530/0 (socket says 172.21.15.100:50530) 2024-04-23T14:35:02.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b17400640 1 -- 172.21.15.100:0/2837045373 learned_addr learned my addr 172.21.15.100:0/2837045373 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:02.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b15600640 1 -- 172.21.15.100:0/2837045373 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 199769638 0 0) 0x7f9b180cd370 con 0x7f9b1815d600 2024-04-23T14:35:02.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b15600640 1 -- 172.21.15.100:0/2837045373 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9afc003530 con 0x7f9b1815d600 2024-04-23T14:35:02.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b16a00640 1 --2- 172.21.15.100:0/2837045373 >> v2:172.21.15.151:3300/0 conn(0x7f9b180cca50 0x7f9b180cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:02.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b16000640 1 --2- 172.21.15.100:0/2837045373 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9b1815c7e0 0x7f9b18160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b15600640 1 -- 172.21.15.100:0/2837045373 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1688151419 0 0) 0x7f9afc003530 con 0x7f9b1815d600 2024-04-23T14:35:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b15600640 1 -- 172.21.15.100:0/2837045373 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9b1815c7e0 msgr2=0x7f9b18160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b15600640 1 --2- 172.21.15.100:0/2837045373 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9b1815c7e0 0x7f9b18160170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b15600640 1 -- 172.21.15.100:0/2837045373 >> v2:172.21.15.151:3300/0 conn(0x7f9b180cca50 msgr2=0x7f9b180cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b15600640 1 --2- 172.21.15.100:0/2837045373 >> v2:172.21.15.151:3300/0 conn(0x7f9b180cca50 0x7f9b180cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b15600640 1 -- 172.21.15.100:0/2837045373 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b180d04a0 con 0x7f9b1815d600 2024-04-23T14:35:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b15600640 1 -- 172.21.15.100:0/2837045373 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f9b10001a00 con 0x7f9b1815d600 2024-04-23T14:35:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b15600640 1 -- 172.21.15.100:0/2837045373 >> v1:172.21.15.100:6789/0 conn(0x7f9b1815d600 legacy=0x7f9b181606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b15600640 1 --2- 172.21.15.100:0/2837045373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9afc0039c0 0x7f9afc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b15600640 1 --2- 172.21.15.100:0/2837045373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9afc004810 0x7f9afc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b15600640 1 --2- 172.21.15.100:0/2837045373 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9b1815c7e0 0x7f9afc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b15600640 1 -- 172.21.15.100:0/2837045373 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1815db00 con 0x7f9afc004810 2024-04-23T14:35:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b15600640 1 -- 172.21.15.100:0/2837045373 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc00b9d0 con 0x7f9afc0039c0 2024-04-23T14:35:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b15600640 1 -- 172.21.15.100:0/2837045373 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9afc00bb80 con 0x7f9b1815c7e0 2024-04-23T14:35:02.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b16000640 1 --2- 172.21.15.100:0/2837045373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9afc004810 0x7f9afc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:02.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b16a00640 1 --2- 172.21.15.100:0/2837045373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9afc0039c0 0x7f9afc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:02.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b17400640 1 --2- 172.21.15.100:0/2837045373 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9b1815c7e0 0x7f9afc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:02.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b16000640 1 -- 172.21.15.100:0/2837045373 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9b1815c7e0 msgr2=0x7f9afc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:02.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b16000640 1 --2- 172.21.15.100:0/2837045373 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9b1815c7e0 0x7f9afc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b16000640 1 -- 172.21.15.100:0/2837045373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9afc0039c0 msgr2=0x7f9afc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:02.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b16000640 1 --2- 172.21.15.100:0/2837045373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9afc0039c0 0x7f9afc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b16000640 1 -- 172.21.15.100:0/2837045373 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9afc003530 con 0x7f9afc004810 2024-04-23T14:35:02.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.519+0000 7f9b17400640 1 --2- 172.21.15.100:0/2837045373 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9b1815c7e0 0x7f9afc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:35:02.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.523+0000 7f9b16000640 1 --2- 172.21.15.100:0/2837045373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9afc004810 0x7f9afc004c90 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009ab0 tx=0x7f9b0401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2742b3e37b5615d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:02.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.523+0000 7f9b15600640 1 -- 172.21.15.100:0/2837045373 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b040b2070 con 0x7f9afc004810 2024-04-23T14:35:02.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.523+0000 7f9b15600640 1 -- 172.21.15.100:0/2837045373 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b040a5510 con 0x7f9afc004810 2024-04-23T14:35:02.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.523+0000 7f9b15600640 1 -- 172.21.15.100:0/2837045373 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b040096b0 con 0x7f9afc004810 2024-04-23T14:35:02.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.523+0000 7f9b1de00640 1 -- 172.21.15.100:0/2837045373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9afc004810 msgr2=0x7f9afc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:02.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.523+0000 7f9b1de00640 1 --2- 172.21.15.100:0/2837045373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9afc004810 0x7f9afc004c90 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009ab0 tx=0x7f9b0401c9c0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.523+0000 7f9b17400640 1 -- 172.21.15.100:0/2837045373 reap_dead start 2024-04-23T14:35:02.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.523+0000 7f9b1de00640 1 -- 172.21.15.100:0/2837045373 shutdown_connections 2024-04-23T14:35:02.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.523+0000 7f9b1de00640 1 -- 172.21.15.100:0/2837045373 >> 172.21.15.100:0/2837045373 conn(0x7f9b180cee80 msgr2=0x7f9b180cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:02.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.523+0000 7f9b1de00640 1 -- 172.21.15.100:0/2837045373 shutdown_connections 2024-04-23T14:35:02.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.523+0000 7f9b1de00640 1 -- 172.21.15.100:0/2837045373 wait complete. 2024-04-23T14:35:02.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b1de00640 1 Processor -- start 2024-04-23T14:35:02.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b1de00640 1 -- start start 2024-04-23T14:35:02.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b1de00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9afc004810 0x7f9b18150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:02.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b1de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b1806d450 0x7f9b1806d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:02.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b1de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b181526f0 0x7f9b18152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:02.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b1de00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9b180b5430 con 0x7f9b181526f0 2024-04-23T14:35:02.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b1de00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9b180b52b0 con 0x7f9b1806d450 2024-04-23T14:35:02.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b1de00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9b180b3bf0 con 0x7f9afc004810 2024-04-23T14:35:02.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b17400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b181526f0 0x7f9b18152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:02.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b17400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b181526f0 0x7f9b18152b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34464/0 (socket says 172.21.15.100:34464) 2024-04-23T14:35:02.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b17400640 1 -- 172.21.15.100:0/1891454764 learned_addr learned my addr 172.21.15.100:0/1891454764 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:02.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b16a00640 1 --2- 172.21.15.100:0/1891454764 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9afc004810 0x7f9b18150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:02.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b16000640 1 --2- 172.21.15.100:0/1891454764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b1806d450 0x7f9b1806d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:02.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b17400640 1 -- 172.21.15.100:0/1891454764 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9afc004810 msgr2=0x7f9b18150420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:02.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b17400640 1 --2- 172.21.15.100:0/1891454764 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9afc004810 0x7f9b18150420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b17400640 1 -- 172.21.15.100:0/1891454764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b1806d450 msgr2=0x7f9b1806d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:02.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b17400640 1 --2- 172.21.15.100:0/1891454764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b1806d450 0x7f9b1806d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b17400640 1 -- 172.21.15.100:0/1891454764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b18150960 con 0x7f9b181526f0 2024-04-23T14:35:02.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b16000640 1 --2- 172.21.15.100:0/1891454764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b1806d450 0x7f9b1806d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:35:02.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b17400640 1 --2- 172.21.15.100:0/1891454764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b181526f0 0x7f9b18152b70 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f9b10000f90 tx=0x7f9b1000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:02.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b0f400640 1 -- 172.21.15.100:0/1891454764 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1000eb90 con 0x7f9b181526f0 2024-04-23T14:35:02.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b0f400640 1 -- 172.21.15.100:0/1891454764 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b10018070 con 0x7f9b181526f0 2024-04-23T14:35:02.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b0f400640 1 -- 172.21.15.100:0/1891454764 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b10013420 con 0x7f9b181526f0 2024-04-23T14:35:02.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b1de00640 1 -- 172.21.15.100:0/1891454764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b1806ded0 con 0x7f9b181526f0 2024-04-23T14:35:02.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.527+0000 7f9b1de00640 1 -- 172.21.15.100:0/1891454764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b1806e490 con 0x7f9b181526f0 2024-04-23T14:35:02.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.531+0000 7f9b1de00640 1 -- 172.21.15.100:0/1891454764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b18068ad0 con 0x7f9b181526f0 2024-04-23T14:35:02.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.531+0000 7f9b0f400640 1 -- 172.21.15.100:0/1891454764 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9b100135c0 con 0x7f9b181526f0 2024-04-23T14:35:02.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.531+0000 7f9b0f400640 1 --2- 172.21.15.100:0/1891454764 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9af80683b0 0x7f9af806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:02.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.531+0000 7f9b0f400640 1 -- 172.21.15.100:0/1891454764 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f9b10086e30 con 0x7f9b181526f0 2024-04-23T14:35:02.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.531+0000 7f9b16a00640 1 --2- 172.21.15.100:0/1891454764 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9af80683b0 0x7f9af806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:02.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.531+0000 7f9b16a00640 1 --2- 172.21.15.100:0/1891454764 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9af80683b0 0x7f9af806a870 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009b60 tx=0x7f9b08008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:02.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.535+0000 7f9b0f400640 1 -- 172.21.15.100:0/1891454764 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9b10057240 con 0x7f9b181526f0 2024-04-23T14:35:02.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.723+0000 7f9b1de00640 1 -- 172.21.15.100:0/1891454764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b18157bf0 con 0x7f9b181526f0 2024-04-23T14:35:02.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.723+0000 7f9b0f400640 1 -- 172.21.15.100:0/1891454764 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5190 (secure 0 0 0) 0x7f9b10056b40 con 0x7f9b181526f0 2024-04-23T14:35:02.725 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:35:02.725 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5528,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2774610498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2774610498},{"type":"v1","addr":"172.21.15.151:6833","nonce":2774610498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5929,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1347883282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1347883282},{"type":"v1","addr":"172.21.15.151:6835","nonce":1347883282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":6177,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3319827520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3319827520},{"type":"v1","addr":"172.21.15.100:6836","nonce":3319827520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:34:24.162539+0000","modified":"2024-04-23T14:34:27.179639+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6180":{"gid":6180,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/608349550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":608349550},{"type":"v1","addr":"172.21.15.100:6837","nonce":608349550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6180,"qdb_cluster":[6180]},"id":3}]} 2024-04-23T14:35:02.725 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 28 2024-04-23T14:35:02.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.727+0000 7f9b1de00640 1 -- 172.21.15.100:0/1891454764 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9af80683b0 msgr2=0x7f9af806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:02.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.727+0000 7f9b1de00640 1 --2- 172.21.15.100:0/1891454764 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9af80683b0 0x7f9af806a870 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009b60 tx=0x7f9b08008040 comp rx=0 tx=0).stop 2024-04-23T14:35:02.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.727+0000 7f9b1de00640 1 -- 172.21.15.100:0/1891454764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b181526f0 msgr2=0x7f9b18152b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:02.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.727+0000 7f9b1de00640 1 --2- 172.21.15.100:0/1891454764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b181526f0 0x7f9b18152b70 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f9b10000f90 tx=0x7f9b1000e530 comp rx=0 tx=0).stop 2024-04-23T14:35:02.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.727+0000 7f9b1de00640 1 -- 172.21.15.100:0/1891454764 shutdown_connections 2024-04-23T14:35:02.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.727+0000 7f9b1de00640 1 --2- 172.21.15.100:0/1891454764 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9af80683b0 0x7f9af806a870 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.727+0000 7f9b1de00640 1 --2- 172.21.15.100:0/1891454764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b181526f0 0x7f9b18152b70 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.727+0000 7f9b1de00640 1 --2- 172.21.15.100:0/1891454764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b1806d450 0x7f9b1806d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.727+0000 7f9b1de00640 1 --2- 172.21.15.100:0/1891454764 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9afc004810 0x7f9b18150420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.727+0000 7f9b1de00640 1 -- 172.21.15.100:0/1891454764 >> 172.21.15.100:0/1891454764 conn(0x7f9b180cee80 msgr2=0x7f9b181652a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:02.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.727+0000 7f9b1de00640 1 -- 172.21.15.100:0/1891454764 shutdown_connections 2024-04-23T14:35:02.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.727+0000 7f9b1de00640 1 -- 172.21.15.100:0/1891454764 wait complete. 2024-04-23T14:35:02.747 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:35:02.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.883+0000 7f14f1800640 1 Processor -- start 2024-04-23T14:35:02.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.883+0000 7f14f1800640 1 -- start start 2024-04-23T14:35:02.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.883+0000 7f14f1800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f14ec15d100 0x7f14ec1630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:02.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.883+0000 7f14f1800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f14ec06bab0 0x7f14ec06d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:02.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.883+0000 7f14f1800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14ec06bf90 con 0x7f14ec071ce0 2024-04-23T14:35:02.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.883+0000 7f14f1800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f14ec06cba0 con 0x7f14ec15d100 2024-04-23T14:35:02.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.883+0000 7f14f1800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f14ec06c270 con 0x7f14ec06bab0 2024-04-23T14:35:02.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.883+0000 7f14eb400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f14ec071ce0 0x7f14ec06dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50542/0 (socket says 172.21.15.100:50542) 2024-04-23T14:35:02.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.883+0000 7f14eb400640 1 -- 172.21.15.100:0/2347795600 learned_addr learned my addr 172.21.15.100:0/2347795600 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:02.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.883+0000 7f14e3e00640 1 --2- 172.21.15.100:0/2347795600 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f14ec06bab0 0x7f14ec06d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:02.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.883+0000 7f14eaa00640 1 --2- 172.21.15.100:0/2347795600 >> v2:172.21.15.151:3300/0 conn(0x7f14ec15d100 0x7f14ec1630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:02.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.883+0000 7f14ea000640 1 -- 172.21.15.100:0/2347795600 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1585671833 0 0) 0x7f14ec06bf90 con 0x7f14ec071ce0 2024-04-23T14:35:02.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.883+0000 7f14ea000640 1 -- 172.21.15.100:0/2347795600 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14d0003530 con 0x7f14ec071ce0 2024-04-23T14:35:02.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.883+0000 7f14e3e00640 1 -- 172.21.15.100:0/2347795600 >> v2:172.21.15.151:3300/0 conn(0x7f14ec15d100 msgr2=0x7f14ec1630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:02.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.883+0000 7f14e3e00640 1 --2- 172.21.15.100:0/2347795600 >> v2:172.21.15.151:3300/0 conn(0x7f14ec15d100 0x7f14ec1630c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.883+0000 7f14e3e00640 1 -- 172.21.15.100:0/2347795600 >> v1:172.21.15.100:6789/0 conn(0x7f14ec071ce0 legacy=0x7f14ec06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:02.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.883+0000 7f14e3e00640 1 -- 172.21.15.100:0/2347795600 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ec06c550 con 0x7f14ec06bab0 2024-04-23T14:35:02.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.887+0000 7f14e3e00640 1 --2- 172.21.15.100:0/2347795600 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f14ec06bab0 0x7f14ec06d030 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f14d40097c0 tx=0x7f14d401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16ba4a35e5d36e93 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:02.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.887+0000 7f14ea000640 1 -- 172.21.15.100:0/2347795600 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d40ab600 con 0x7f14ec06bab0 2024-04-23T14:35:02.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.887+0000 7f14ea000640 1 -- 172.21.15.100:0/2347795600 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14d40b0070 con 0x7f14ec06bab0 2024-04-23T14:35:02.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.887+0000 7f14ea000640 1 -- 172.21.15.100:0/2347795600 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d40ab9a0 con 0x7f14ec06bab0 2024-04-23T14:35:02.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.887+0000 7f14f1800640 1 -- 172.21.15.100:0/2347795600 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f14ec06bab0 msgr2=0x7f14ec06d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:02.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.887+0000 7f14f1800640 1 --2- 172.21.15.100:0/2347795600 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f14ec06bab0 0x7f14ec06d030 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f14d40097c0 tx=0x7f14d401c470 comp rx=0 tx=0).stop 2024-04-23T14:35:02.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.887+0000 7f14f1800640 1 -- 172.21.15.100:0/2347795600 shutdown_connections 2024-04-23T14:35:02.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.887+0000 7f14f1800640 1 --2- 172.21.15.100:0/2347795600 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f14ec06bab0 0x7f14ec06d030 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.887+0000 7f14f1800640 1 --2- 172.21.15.100:0/2347795600 >> v2:172.21.15.151:3300/0 conn(0x7f14ec15d100 0x7f14ec1630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.887+0000 7f14f1800640 1 -- 172.21.15.100:0/2347795600 >> 172.21.15.100:0/2347795600 conn(0x7f14ec0b1e80 msgr2=0x7f14ec0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:02.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.887+0000 7f14f1800640 1 -- 172.21.15.100:0/2347795600 shutdown_connections 2024-04-23T14:35:02.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.887+0000 7f14f1800640 1 -- 172.21.15.100:0/2347795600 wait complete. 2024-04-23T14:35:02.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14f1800640 1 Processor -- start 2024-04-23T14:35:02.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14f1800640 1 -- start start 2024-04-23T14:35:02.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14f1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14ec06bab0 0x7f14ec1550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:02.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14f1800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14ec071ce0 0x7f14ec0cea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:02.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14f1800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f14ec15d100 0x7f14ec0b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:02.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14f1800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec072ae0 con 0x7f14ec06bab0 2024-04-23T14:35:02.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14f1800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec072960 con 0x7f14ec071ce0 2024-04-23T14:35:02.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14f1800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f14ec16fa50 con 0x7f14ec15d100 2024-04-23T14:35:02.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14eaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14ec06bab0 0x7f14ec1550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:02.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14eaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14ec06bab0 0x7f14ec1550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34468/0 (socket says 172.21.15.100:34468) 2024-04-23T14:35:02.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14eaa00640 1 -- 172.21.15.100:0/445832300 learned_addr learned my addr 172.21.15.100:0/445832300 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:02.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14e3e00640 1 --2- 172.21.15.100:0/445832300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14ec071ce0 0x7f14ec0cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:02.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14eb400640 1 --2- 172.21.15.100:0/445832300 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f14ec15d100 0x7f14ec0b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:02.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14eaa00640 1 -- 172.21.15.100:0/445832300 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f14ec15d100 msgr2=0x7f14ec0b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:02.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14eaa00640 1 --2- 172.21.15.100:0/445832300 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f14ec15d100 0x7f14ec0b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14eaa00640 1 -- 172.21.15.100:0/445832300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14ec071ce0 msgr2=0x7f14ec0cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:02.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14eaa00640 1 --2- 172.21.15.100:0/445832300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14ec071ce0 0x7f14ec0cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:02.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14eaa00640 1 -- 172.21.15.100:0/445832300 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ec15b4d0 con 0x7f14ec06bab0 2024-04-23T14:35:02.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14eaa00640 1 --2- 172.21.15.100:0/445832300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14ec06bab0 0x7f14ec1550b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00e930 tx=0x7f14dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:02.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14e2a00640 1 -- 172.21.15.100:0/445832300 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc017070 con 0x7f14ec06bab0 2024-04-23T14:35:02.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14e2a00640 1 -- 172.21.15.100:0/445832300 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14dc00ca00 con 0x7f14ec06bab0 2024-04-23T14:35:02.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14e2a00640 1 -- 172.21.15.100:0/445832300 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc00cdb0 con 0x7f14ec06bab0 2024-04-23T14:35:02.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14f1800640 1 -- 172.21.15.100:0/445832300 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ec0cf3f0 con 0x7f14ec06bab0 2024-04-23T14:35:02.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.891+0000 7f14f1800640 1 -- 172.21.15.100:0/445832300 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ec15a9f0 con 0x7f14ec06bab0 2024-04-23T14:35:02.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.895+0000 7f14f1800640 1 -- 172.21.15.100:0/445832300 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14ec068ad0 con 0x7f14ec06bab0 2024-04-23T14:35:02.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.895+0000 7f14e2a00640 1 -- 172.21.15.100:0/445832300 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f14dc01a070 con 0x7f14ec06bab0 2024-04-23T14:35:02.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.895+0000 7f14e2a00640 1 --2- 172.21.15.100:0/445832300 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f14c8068380 0x7f14c806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:02.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.895+0000 7f14e2a00640 1 -- 172.21.15.100:0/445832300 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f14dc086b90 con 0x7f14ec06bab0 2024-04-23T14:35:02.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.895+0000 7f14e3e00640 1 --2- 172.21.15.100:0/445832300 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f14c8068380 0x7f14c806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:02.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.895+0000 7f14e3e00640 1 --2- 172.21.15.100:0/445832300 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f14c8068380 0x7f14c806a840 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f14d40af000 tx=0x7f14d40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:02.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:02.899+0000 7f14e2a00640 1 -- 172.21.15.100:0/445832300 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f14dc057090 con 0x7f14ec06bab0 2024-04-23T14:35:03.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.059+0000 7f14f1800640 1 -- 172.21.15.100:0/445832300 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f14ec0ce800 con 0x7f14ec06bab0 2024-04-23T14:35:03.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.059+0000 7f14e2a00640 1 -- 172.21.15.100:0/445832300 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16950 (secure 0 0 0) 0x7f14dc056990 con 0x7f14ec06bab0 2024-04-23T14:35:03.060 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:35:03.061 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":53,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:34:25.795515+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:34:22.491379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-23T14:34:23.120342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:35:03.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.059+0000 7f14f1800640 1 -- 172.21.15.100:0/445832300 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f14c8068380 msgr2=0x7f14c806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.059+0000 7f14f1800640 1 --2- 172.21.15.100:0/445832300 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f14c8068380 0x7f14c806a840 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f14d40af000 tx=0x7f14d40a7000 comp rx=0 tx=0).stop 2024-04-23T14:35:03.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.059+0000 7f14f1800640 1 -- 172.21.15.100:0/445832300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14ec06bab0 msgr2=0x7f14ec1550b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.059+0000 7f14f1800640 1 --2- 172.21.15.100:0/445832300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14ec06bab0 0x7f14ec1550b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00e930 tx=0x7f14dc00edf0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.059+0000 7f14f1800640 1 -- 172.21.15.100:0/445832300 shutdown_connections 2024-04-23T14:35:03.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.059+0000 7f14f1800640 1 --2- 172.21.15.100:0/445832300 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f14c8068380 0x7f14c806a840 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.059+0000 7f14f1800640 1 --2- 172.21.15.100:0/445832300 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f14ec15d100 0x7f14ec0b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.059+0000 7f14f1800640 1 --2- 172.21.15.100:0/445832300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14ec071ce0 0x7f14ec0cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.059+0000 7f14f1800640 1 --2- 172.21.15.100:0/445832300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14ec06bab0 0x7f14ec1550b0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.059+0000 7f14f1800640 1 -- 172.21.15.100:0/445832300 >> 172.21.15.100:0/445832300 conn(0x7f14ec0b1e80 msgr2=0x7f14ec1633b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:03.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.063+0000 7f14f1800640 1 -- 172.21.15.100:0/445832300 shutdown_connections 2024-04-23T14:35:03.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.063+0000 7f14f1800640 1 -- 172.21.15.100:0/445832300 wait complete. 2024-04-23T14:35:03.082 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T14:35:03.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.215+0000 7f27ef800640 1 Processor -- start 2024-04-23T14:35:03.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.215+0000 7f27ef800640 1 -- start start 2024-04-23T14:35:03.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.215+0000 7f27ef800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f27e815b660 0x7f27e80c6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.215+0000 7f27ef800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27e80cee00 con 0x7f27e80d08a0 2024-04-23T14:35:03.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.215+0000 7f27ef800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27e806bc50 con 0x7f27e80c6d60 2024-04-23T14:35:03.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.215+0000 7f27ef800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f27e815ba40 con 0x7f27e815b660 2024-04-23T14:35:03.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.219+0000 7f27ece00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f27e815b660 0x7f27e80c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.219+0000 7f27ed800640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f27e80d08a0 0x7f27e80c5160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50554/0 (socket says 172.21.15.100:50554) 2024-04-23T14:35:03.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.219+0000 7f27ed800640 1 -- 172.21.15.100:0/1673705984 learned_addr learned my addr 172.21.15.100:0/1673705984 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:03.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.219+0000 7f27ece00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f27e815b660 0x7f27e80c6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34478/0 (socket says 172.21.15.100:34478) 2024-04-23T14:35:03.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.219+0000 7f27ece00640 1 -- 172.21.15.100:0/1673705984 >> v1:172.21.15.151:6789/0 conn(0x7f27e80c6d60 legacy=0x7f27e80d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:03.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.219+0000 7f27ece00640 1 -- 172.21.15.100:0/1673705984 >> v1:172.21.15.100:6789/0 conn(0x7f27e80d08a0 legacy=0x7f27e80c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.219+0000 7f27ece00640 1 -- 172.21.15.100:0/1673705984 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e806bf30 con 0x7f27e815b660 2024-04-23T14:35:03.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.219+0000 7f27ece00640 1 --2- 172.21.15.100:0/1673705984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e815b660 0x7f27e80c6820 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f27d80097c0 tx=0x7f27d801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494a6c914512e6b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:03.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.219+0000 7f27e7400640 1 -- 172.21.15.100:0/1673705984 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d80ab430 con 0x7f27e815b660 2024-04-23T14:35:03.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.219+0000 7f27e7400640 1 -- 172.21.15.100:0/1673705984 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27d80b0070 con 0x7f27e815b660 2024-04-23T14:35:03.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.219+0000 7f27e7400640 1 -- 172.21.15.100:0/1673705984 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d80ab7f0 con 0x7f27e815b660 2024-04-23T14:35:03.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ef800640 1 -- 172.21.15.100:0/1673705984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e815b660 msgr2=0x7f27e80c6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ef800640 1 --2- 172.21.15.100:0/1673705984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e815b660 0x7f27e80c6820 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f27d80097c0 tx=0x7f27d801c430 comp rx=0 tx=0).stop 2024-04-23T14:35:03.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ef800640 1 -- 172.21.15.100:0/1673705984 shutdown_connections 2024-04-23T14:35:03.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ef800640 1 --2- 172.21.15.100:0/1673705984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e815b660 0x7f27e80c6820 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ef800640 1 -- 172.21.15.100:0/1673705984 >> 172.21.15.100:0/1673705984 conn(0x7f27e80b5310 msgr2=0x7f27e80b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:03.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ef800640 1 -- 172.21.15.100:0/1673705984 shutdown_connections 2024-04-23T14:35:03.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ef800640 1 -- 172.21.15.100:0/1673705984 wait complete. 2024-04-23T14:35:03.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ef800640 1 Processor -- start 2024-04-23T14:35:03.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ef800640 1 -- start start 2024-04-23T14:35:03.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ef800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27e80c6d60 0x7f27e80ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ef800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f27e80d08a0 0x7f27e80cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ef800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e815b660 0x7f27e80afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ef800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f27e80c6200 con 0x7f27e815b660 2024-04-23T14:35:03.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ef800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f27e80c6080 con 0x7f27e80c6d60 2024-04-23T14:35:03.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ef800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f27e80c6380 con 0x7f27e80d08a0 2024-04-23T14:35:03.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ed800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e815b660 0x7f27e80afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ed800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e815b660 0x7f27e80afe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34490/0 (socket says 172.21.15.100:34490) 2024-04-23T14:35:03.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ed800640 1 -- 172.21.15.100:0/3841557234 learned_addr learned my addr 172.21.15.100:0/3841557234 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:03.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ece00640 1 --2- 172.21.15.100:0/3841557234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27e80c6d60 0x7f27e80ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27e7e00640 1 --2- 172.21.15.100:0/3841557234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f27e80d08a0 0x7f27e80cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ed800640 1 -- 172.21.15.100:0/3841557234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f27e80d08a0 msgr2=0x7f27e80cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ed800640 1 --2- 172.21.15.100:0/3841557234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f27e80d08a0 0x7f27e80cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ed800640 1 -- 172.21.15.100:0/3841557234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27e80c6d60 msgr2=0x7f27e80ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ed800640 1 --2- 172.21.15.100:0/3841557234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27e80c6d60 0x7f27e80ae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ed800640 1 -- 172.21.15.100:0/3841557234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e814ffe0 con 0x7f27e815b660 2024-04-23T14:35:03.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.223+0000 7f27ed800640 1 --2- 172.21.15.100:0/3841557234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e815b660 0x7f27e80afe70 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f27e000caa0 tx=0x7f27e000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:03.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.227+0000 7f27e4c00640 1 -- 172.21.15.100:0/3841557234 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0017070 con 0x7f27e815b660 2024-04-23T14:35:03.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.227+0000 7f27e4c00640 1 -- 172.21.15.100:0/3841557234 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27e000ac00 con 0x7f27e815b660 2024-04-23T14:35:03.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.227+0000 7f27e4c00640 1 -- 172.21.15.100:0/3841557234 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0012400 con 0x7f27e815b660 2024-04-23T14:35:03.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.227+0000 7f27ef800640 1 -- 172.21.15.100:0/3841557234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e81502c0 con 0x7f27e815b660 2024-04-23T14:35:03.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.227+0000 7f27ef800640 1 -- 172.21.15.100:0/3841557234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e8150b60 con 0x7f27e815b660 2024-04-23T14:35:03.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.227+0000 7f27ef800640 1 -- 172.21.15.100:0/3841557234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e8150490 con 0x7f27e815b660 2024-04-23T14:35:03.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.227+0000 7f27e4c00640 1 -- 172.21.15.100:0/3841557234 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f27e001a070 con 0x7f27e815b660 2024-04-23T14:35:03.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.227+0000 7f27e4c00640 1 --2- 172.21.15.100:0/3841557234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f27c8068380 0x7f27c806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.227+0000 7f27e4c00640 1 -- 172.21.15.100:0/3841557234 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f27e00867f0 con 0x7f27e815b660 2024-04-23T14:35:03.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.227+0000 7f27ece00640 1 --2- 172.21.15.100:0/3841557234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f27c8068380 0x7f27c806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.227+0000 7f27ece00640 1 --2- 172.21.15.100:0/3841557234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f27c8068380 0x7f27c806a840 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f27d80af000 tx=0x7f27d80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:03.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.231+0000 7f27e4c00640 1 -- 172.21.15.100:0/3841557234 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f27e80cee00 con 0x7f27e815b660 2024-04-23T14:35:03.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.419+0000 7f27ef800640 1 -- 172.21.15.100:0/3841557234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f27e8150d30 con 0x7f27e815b660 2024-04-23T14:35:03.422 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-04-23T14:35:03.419+0000 7f7a7e400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T14:35:03.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.423+0000 7f27e4c00640 1 -- 172.21.15.100:0/3841557234 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7f27e0056530 con 0x7f27e815b660 2024-04-23T14:35:03.424 INFO:teuthology.orchestra.run.smithi100.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T14:35:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.427+0000 7f27ef800640 1 -- 172.21.15.100:0/3841557234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f27c8068380 msgr2=0x7f27c806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.427+0000 7f27ef800640 1 --2- 172.21.15.100:0/3841557234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f27c8068380 0x7f27c806a840 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f27d80af000 tx=0x7f27d80a7000 comp rx=0 tx=0).stop 2024-04-23T14:35:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.427+0000 7f27ef800640 1 -- 172.21.15.100:0/3841557234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e815b660 msgr2=0x7f27e80afe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.427+0000 7f27ef800640 1 --2- 172.21.15.100:0/3841557234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e815b660 0x7f27e80afe70 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f27e000caa0 tx=0x7f27e000cf60 comp rx=0 tx=0).stop 2024-04-23T14:35:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.427+0000 7f27ef800640 1 -- 172.21.15.100:0/3841557234 shutdown_connections 2024-04-23T14:35:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.427+0000 7f27ef800640 1 --2- 172.21.15.100:0/3841557234 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f27c8068380 0x7f27c806a840 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.427+0000 7f27ef800640 1 --2- 172.21.15.100:0/3841557234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e815b660 0x7f27e80afe70 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.427+0000 7f27ef800640 1 --2- 172.21.15.100:0/3841557234 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f27e80d08a0 0x7f27e80cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.427+0000 7f27ef800640 1 --2- 172.21.15.100:0/3841557234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27e80c6d60 0x7f27e80ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.427+0000 7f27ef800640 1 -- 172.21.15.100:0/3841557234 >> 172.21.15.100:0/3841557234 conn(0x7f27e80b5310 msgr2=0x7f27e80cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:03.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.427+0000 7f27ef800640 1 -- 172.21.15.100:0/3841557234 shutdown_connections 2024-04-23T14:35:03.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.427+0000 7f27ef800640 1 -- 172.21.15.100:0/3841557234 wait complete. 2024-04-23T14:35:03.434 INFO:tasks.ceph.mds.a.smithi100.stderr: -3205> 2024-04-23T14:34:26.755+0000 7f5de8400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:35:03.448 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-23T14:35:03.480 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-04-23T14:35:03.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.587+0000 7f2974000640 1 Processor -- start 2024-04-23T14:35:03.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.587+0000 7f2974000640 1 -- start start 2024-04-23T14:35:03.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.587+0000 7f2974000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f296c06c6c0 0x7f296c0ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.587+0000 7f2974000640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f296c0ce620 0x7f296c0add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.587+0000 7f2974000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f296c06dd60 con 0x7f296c06f8c0 2024-04-23T14:35:03.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.587+0000 7f2974000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f296c0ad050 con 0x7f296c06c6c0 2024-04-23T14:35:03.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.587+0000 7f2974000640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f296c0ad8f0 con 0x7f296c0ce620 2024-04-23T14:35:03.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.591+0000 7f2970c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f296c06c6c0 0x7f296c0ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.591+0000 7f2970c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f296c06c6c0 0x7f296c0ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34530/0 (socket says 172.21.15.100:34530) 2024-04-23T14:35:03.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.591+0000 7f2970c00640 1 -- 172.21.15.100:0/4159660670 learned_addr learned my addr 172.21.15.100:0/4159660670 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:03.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.591+0000 7f2972000640 1 --2- 172.21.15.100:0/4159660670 >> v2:172.21.15.151:3300/0 conn(0x7f296c0ce620 0x7f296c0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.591+0000 7f2970c00640 1 -- 172.21.15.100:0/4159660670 >> v2:172.21.15.151:3300/0 conn(0x7f296c0ce620 msgr2=0x7f296c0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:03.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.591+0000 7f2970c00640 1 --2- 172.21.15.100:0/4159660670 >> v2:172.21.15.151:3300/0 conn(0x7f296c0ce620 0x7f296c0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.591+0000 7f2970c00640 1 -- 172.21.15.100:0/4159660670 >> v1:172.21.15.151:6789/0 conn(0x7f296c06f8c0 legacy=0x7f296c0d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.591+0000 7f2970c00640 1 -- 172.21.15.100:0/4159660670 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f296c0d09c0 con 0x7f296c06c6c0 2024-04-23T14:35:03.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.591+0000 7f2970c00640 1 --2- 172.21.15.100:0/4159660670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f296c06c6c0 0x7f296c0ce0e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f296c0b3120 tx=0x7f295801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9a615c3a7547dd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:03.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.591+0000 7f296be00640 1 -- 172.21.15.100:0/4159660670 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29580ab430 con 0x7f296c06c6c0 2024-04-23T14:35:03.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.591+0000 7f296be00640 1 -- 172.21.15.100:0/4159660670 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29580b0070 con 0x7f296c06c6c0 2024-04-23T14:35:03.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.591+0000 7f296be00640 1 -- 172.21.15.100:0/4159660670 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29580ab7f0 con 0x7f296c06c6c0 2024-04-23T14:35:03.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2974000640 1 -- 172.21.15.100:0/4159660670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f296c06c6c0 msgr2=0x7f296c0ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2974000640 1 --2- 172.21.15.100:0/4159660670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f296c06c6c0 0x7f296c0ce0e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f296c0b3120 tx=0x7f295801c520 comp rx=0 tx=0).stop 2024-04-23T14:35:03.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2974000640 1 -- 172.21.15.100:0/4159660670 shutdown_connections 2024-04-23T14:35:03.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2974000640 1 --2- 172.21.15.100:0/4159660670 >> v2:172.21.15.151:3300/0 conn(0x7f296c0ce620 0x7f296c0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2974000640 1 --2- 172.21.15.100:0/4159660670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f296c06c6c0 0x7f296c0ce0e0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2974000640 1 -- 172.21.15.100:0/4159660670 >> 172.21.15.100:0/4159660670 conn(0x7f296c0b5200 msgr2=0x7f296c0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:03.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2974000640 1 -- 172.21.15.100:0/4159660670 shutdown_connections 2024-04-23T14:35:03.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2974000640 1 -- 172.21.15.100:0/4159660670 wait complete. 2024-04-23T14:35:03.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2974000640 1 Processor -- start 2024-04-23T14:35:03.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2974000640 1 -- start start 2024-04-23T14:35:03.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2974000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f296c06c6c0 0x7f296c0cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2974000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f296c06f8c0 0x7f296c15dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2974000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f296c0ce620 0x7f296c0c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2974000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f296c0aee60 con 0x7f296c06f8c0 2024-04-23T14:35:03.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2974000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f296c0aece0 con 0x7f296c0ce620 2024-04-23T14:35:03.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2974000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f296c168400 con 0x7f296c06c6c0 2024-04-23T14:35:03.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2970c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f296c06f8c0 0x7f296c15dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2970c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f296c06f8c0 0x7f296c15dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34546/0 (socket says 172.21.15.100:34546) 2024-04-23T14:35:03.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2970c00640 1 -- 172.21.15.100:0/1621213789 learned_addr learned my addr 172.21.15.100:0/1621213789 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:03.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2972000640 1 --2- 172.21.15.100:0/1621213789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f296c0ce620 0x7f296c0c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2971600640 1 --2- 172.21.15.100:0/1621213789 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f296c06c6c0 0x7f296c0cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2970c00640 1 -- 172.21.15.100:0/1621213789 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f296c06c6c0 msgr2=0x7f296c0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2970c00640 1 --2- 172.21.15.100:0/1621213789 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f296c06c6c0 0x7f296c0cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2970c00640 1 -- 172.21.15.100:0/1621213789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f296c0ce620 msgr2=0x7f296c0c6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2970c00640 1 --2- 172.21.15.100:0/1621213789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f296c0ce620 0x7f296c0c6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2970c00640 1 -- 172.21.15.100:0/1621213789 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f296c0cc870 con 0x7f296c06f8c0 2024-04-23T14:35:03.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2971600640 1 --2- 172.21.15.100:0/1621213789 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f296c06c6c0 0x7f296c0cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:35:03.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.595+0000 7f2970c00640 1 --2- 172.21.15.100:0/1621213789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f296c06f8c0 0x7f296c15dd00 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f2958019130 tx=0x7f295801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:03.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.599+0000 7f2969600640 1 -- 172.21.15.100:0/1621213789 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29580be5f0 con 0x7f296c06f8c0 2024-04-23T14:35:03.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.599+0000 7f2969600640 1 -- 172.21.15.100:0/1621213789 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29580be790 con 0x7f296c06f8c0 2024-04-23T14:35:03.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.599+0000 7f2969600640 1 -- 172.21.15.100:0/1621213789 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29580beb80 con 0x7f296c06f8c0 2024-04-23T14:35:03.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.599+0000 7f2974000640 1 -- 172.21.15.100:0/1621213789 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f296c0c9440 con 0x7f296c06f8c0 2024-04-23T14:35:03.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.599+0000 7f2974000640 1 -- 172.21.15.100:0/1621213789 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f296c0c4890 con 0x7f296c06f8c0 2024-04-23T14:35:03.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.599+0000 7f2974000640 1 -- 172.21.15.100:0/1621213789 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f296c068ad0 con 0x7f296c06f8c0 2024-04-23T14:35:03.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.599+0000 7f2969600640 1 -- 172.21.15.100:0/1621213789 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f29580b43f0 con 0x7f296c06f8c0 2024-04-23T14:35:03.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.599+0000 7f2969600640 1 --2- 172.21.15.100:0/1621213789 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f294c068380 0x7f294c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.599+0000 7f2969600640 1 -- 172.21.15.100:0/1621213789 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6187+0+0 (secure 0 0 0) 0x7f29580a8030 con 0x7f296c06f8c0 2024-04-23T14:35:03.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.599+0000 7f2971600640 1 --2- 172.21.15.100:0/1621213789 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f294c068380 0x7f294c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.603+0000 7f2971600640 1 --2- 172.21.15.100:0/1621213789 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f294c068380 0x7f294c06a840 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f296c0cd930 tx=0x7f2960009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:03.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.603+0000 7f2969600640 1 -- 172.21.15.100:0/1621213789 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f296c0cc870 con 0x7f296c06f8c0 2024-04-23T14:35:03.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:03.799+0000 7f2974000640 1 -- 172.21.15.100:0/1621213789 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f296c06cc00 con 0x7f296c06f8c0 2024-04-23T14:35:04.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.423+0000 7f2969600640 1 -- 172.21.15.100:0/1621213789 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) ==== 104+0+0 (secure 0 0 0) 0x7f2958016110 con 0x7f296c06f8c0 2024-04-23T14:35:04.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.427+0000 7f2974000640 1 -- 172.21.15.100:0/1621213789 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f294c068380 msgr2=0x7f294c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.427+0000 7f2974000640 1 --2- 172.21.15.100:0/1621213789 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f294c068380 0x7f294c06a840 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f296c0cd930 tx=0x7f2960009040 comp rx=0 tx=0).stop 2024-04-23T14:35:04.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.427+0000 7f2974000640 1 -- 172.21.15.100:0/1621213789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f296c06f8c0 msgr2=0x7f296c15dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.427+0000 7f2974000640 1 --2- 172.21.15.100:0/1621213789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f296c06f8c0 0x7f296c15dd00 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f2958019130 tx=0x7f295801ad60 comp rx=0 tx=0).stop 2024-04-23T14:35:04.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.427+0000 7f2974000640 1 -- 172.21.15.100:0/1621213789 shutdown_connections 2024-04-23T14:35:04.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.427+0000 7f2974000640 1 --2- 172.21.15.100:0/1621213789 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f294c068380 0x7f294c06a840 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.427+0000 7f2974000640 1 --2- 172.21.15.100:0/1621213789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f296c0ce620 0x7f296c0c6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.427+0000 7f2974000640 1 --2- 172.21.15.100:0/1621213789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f296c06f8c0 0x7f296c15dd00 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.427+0000 7f2974000640 1 --2- 172.21.15.100:0/1621213789 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f296c06c6c0 0x7f296c0cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.427+0000 7f2974000640 1 -- 172.21.15.100:0/1621213789 >> 172.21.15.100:0/1621213789 conn(0x7f296c0b5200 msgr2=0x7f296c165200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:04.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.427+0000 7f2974000640 1 -- 172.21.15.100:0/1621213789 shutdown_connections 2024-04-23T14:35:04.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.427+0000 7f2974000640 1 -- 172.21.15.100:0/1621213789 wait complete. 2024-04-23T14:35:04.449 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-23T14:35:04.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.631+0000 7f98f3400640 1 Processor -- start 2024-04-23T14:35:04.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.631+0000 7f98f3400640 1 -- start start 2024-04-23T14:35:04.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.631+0000 7f98f3400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f98ec06d450 0x7f98ec06d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.631+0000 7f98f3400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98ec06dd70 con 0x7f98ec0cea60 2024-04-23T14:35:04.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.631+0000 7f98f3400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98ec0b1f90 con 0x7f98ec0cd460 2024-04-23T14:35:04.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.631+0000 7f98f3400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f98ec0b2270 con 0x7f98ec06d450 2024-04-23T14:35:04.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.631+0000 7f98f0a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f98ec06d450 0x7f98ec06d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.631+0000 7f98ebe00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f98ec0cea60 0x7f98ec0d0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50572/0 (socket says 172.21.15.100:50572) 2024-04-23T14:35:04.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.631+0000 7f98ebe00640 1 -- 172.21.15.100:0/3353410952 learned_addr learned my addr 172.21.15.100:0/3353410952 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:04.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.631+0000 7f98f0a00640 1 -- 172.21.15.100:0/3353410952 >> v1:172.21.15.151:6789/0 conn(0x7f98ec0cd460 legacy=0x7f98ec0b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.635+0000 7f98f0a00640 1 -- 172.21.15.100:0/3353410952 >> v1:172.21.15.100:6789/0 conn(0x7f98ec0cea60 legacy=0x7f98ec0d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.631+0000 7f98eb400640 1 -- 172.21.15.100:0/3353410952 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3305473404 0 0) 0x7f98ec06dd70 con 0x7f98ec0cea60 2024-04-23T14:35:04.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.635+0000 7f98f0a00640 1 -- 172.21.15.100:0/3353410952 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ec0b2550 con 0x7f98ec06d450 2024-04-23T14:35:04.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.635+0000 7f98f0a00640 1 --2- 172.21.15.100:0/3353410952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec06d450 0x7f98ec06d830 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f98e00097c0 tx=0x7f98e001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d394936a3076310 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:04.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.635+0000 7f98eb400640 1 -- 172.21.15.100:0/3353410952 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e00b0070 con 0x7f98ec06d450 2024-04-23T14:35:04.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.635+0000 7f98eb400640 1 -- 172.21.15.100:0/3353410952 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98e00ab600 con 0x7f98ec06d450 2024-04-23T14:35:04.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.635+0000 7f98eb400640 1 -- 172.21.15.100:0/3353410952 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e00a4e60 con 0x7f98ec06d450 2024-04-23T14:35:04.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.635+0000 7f98f3400640 1 -- 172.21.15.100:0/3353410952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec06d450 msgr2=0x7f98ec06d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.635+0000 7f98f3400640 1 --2- 172.21.15.100:0/3353410952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec06d450 0x7f98ec06d830 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f98e00097c0 tx=0x7f98e001c3d0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.635+0000 7f98f3400640 1 -- 172.21.15.100:0/3353410952 shutdown_connections 2024-04-23T14:35:04.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.635+0000 7f98f3400640 1 --2- 172.21.15.100:0/3353410952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec06d450 0x7f98ec06d830 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.635+0000 7f98f3400640 1 -- 172.21.15.100:0/3353410952 >> 172.21.15.100:0/3353410952 conn(0x7f98ec0cee80 msgr2=0x7f98ec0cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:04.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.635+0000 7f98f3400640 1 -- 172.21.15.100:0/3353410952 shutdown_connections 2024-04-23T14:35:04.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.635+0000 7f98f3400640 1 -- 172.21.15.100:0/3353410952 wait complete. 2024-04-23T14:35:04.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98f3400640 1 Processor -- start 2024-04-23T14:35:04.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98f3400640 1 -- start start 2024-04-23T14:35:04.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98f3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec06d450 0x7f98ec0cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98f3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98ec0cd460 0x7f98ec15b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98f3400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f98ec0cea60 0x7f98ec156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98f3400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec16cd50 con 0x7f98ec06d450 2024-04-23T14:35:04.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98f3400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec16cbd0 con 0x7f98ec0cd460 2024-04-23T14:35:04.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98f3400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f98ec16ced0 con 0x7f98ec0cea60 2024-04-23T14:35:04.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98f0a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec06d450 0x7f98ec0cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98f0a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec06d450 0x7f98ec0cbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34564/0 (socket says 172.21.15.100:34564) 2024-04-23T14:35:04.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98f0a00640 1 -- 172.21.15.100:0/1227280766 learned_addr learned my addr 172.21.15.100:0/1227280766 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:04.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98ebe00640 1 --2- 172.21.15.100:0/1227280766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98ec0cd460 0x7f98ec15b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98f1400640 1 --2- 172.21.15.100:0/1227280766 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f98ec0cea60 0x7f98ec156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98f0a00640 1 -- 172.21.15.100:0/1227280766 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f98ec0cea60 msgr2=0x7f98ec156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98f0a00640 1 --2- 172.21.15.100:0/1227280766 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f98ec0cea60 0x7f98ec156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98f0a00640 1 -- 172.21.15.100:0/1227280766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98ec0cd460 msgr2=0x7f98ec15b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98f0a00640 1 --2- 172.21.15.100:0/1227280766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98ec0cd460 0x7f98ec15b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98f0a00640 1 -- 172.21.15.100:0/1227280766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ec15b800 con 0x7f98ec06d450 2024-04-23T14:35:04.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98f0a00640 1 --2- 172.21.15.100:0/1227280766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec06d450 0x7f98ec0cbfd0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f98e0018fe0 tx=0x7f98e001ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:04.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98e8c00640 1 -- 172.21.15.100:0/1227280766 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e00b0050 con 0x7f98ec06d450 2024-04-23T14:35:04.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98e8c00640 1 -- 172.21.15.100:0/1227280766 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98e00bc070 con 0x7f98ec06d450 2024-04-23T14:35:04.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98e8c00640 1 -- 172.21.15.100:0/1227280766 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e00b5830 con 0x7f98ec06d450 2024-04-23T14:35:04.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98f3400640 1 -- 172.21.15.100:0/1227280766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98ec15bae0 con 0x7f98ec06d450 2024-04-23T14:35:04.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.639+0000 7f98f3400640 1 -- 172.21.15.100:0/1227280766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98ec15c0a0 con 0x7f98ec06d450 2024-04-23T14:35:04.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.643+0000 7f98f3400640 1 -- 172.21.15.100:0/1227280766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98ec068ad0 con 0x7f98ec06d450 2024-04-23T14:35:04.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.643+0000 7f98e8c00640 1 -- 172.21.15.100:0/1227280766 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f98e00b59d0 con 0x7f98ec06d450 2024-04-23T14:35:04.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.643+0000 7f98e8c00640 1 --2- 172.21.15.100:0/1227280766 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f98cc068380 0x7f98cc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.643+0000 7f98e8c00640 1 -- 172.21.15.100:0/1227280766 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6187+0+0 (secure 0 0 0) 0x7f98e0121300 con 0x7f98ec06d450 2024-04-23T14:35:04.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.643+0000 7f98ebe00640 1 --2- 172.21.15.100:0/1227280766 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f98cc068380 0x7f98cc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.643+0000 7f98ebe00640 1 --2- 172.21.15.100:0/1227280766 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f98cc068380 0x7f98cc06a840 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f98ec071fb0 tx=0x7f98d8007b60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:04.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.647+0000 7f98e8c00640 1 -- 172.21.15.100:0/1227280766 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f98e0016050 con 0x7f98ec06d450 2024-04-23T14:35:04.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:04.807+0000 7f98f3400640 1 -- 172.21.15.100:0/1227280766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f98ec0d0780 con 0x7f98ec06d450 2024-04-23T14:35:05.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.431+0000 7f98e8c00640 1 -- 172.21.15.100:0/1227280766 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v55) ==== 181+0+0 (secure 0 0 0) 0x7f98e00f16e0 con 0x7f98ec06d450 2024-04-23T14:35:05.432 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' removed 2024-04-23T14:35:05.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.435+0000 7f98f3400640 1 -- 172.21.15.100:0/1227280766 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f98cc068380 msgr2=0x7f98cc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:05.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.435+0000 7f98f3400640 1 --2- 172.21.15.100:0/1227280766 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f98cc068380 0x7f98cc06a840 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f98ec071fb0 tx=0x7f98d8007b60 comp rx=0 tx=0).stop 2024-04-23T14:35:05.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.435+0000 7f98f3400640 1 -- 172.21.15.100:0/1227280766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec06d450 msgr2=0x7f98ec0cbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:05.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.435+0000 7f98f3400640 1 --2- 172.21.15.100:0/1227280766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec06d450 0x7f98ec0cbfd0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f98e0018fe0 tx=0x7f98e001ac10 comp rx=0 tx=0).stop 2024-04-23T14:35:05.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.435+0000 7f98f3400640 1 -- 172.21.15.100:0/1227280766 shutdown_connections 2024-04-23T14:35:05.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.435+0000 7f98f3400640 1 --2- 172.21.15.100:0/1227280766 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f98cc068380 0x7f98cc06a840 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:05.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.435+0000 7f98f3400640 1 --2- 172.21.15.100:0/1227280766 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f98ec0cea60 0x7f98ec156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:05.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.435+0000 7f98f3400640 1 --2- 172.21.15.100:0/1227280766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98ec0cd460 0x7f98ec15b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:05.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.435+0000 7f98f3400640 1 --2- 172.21.15.100:0/1227280766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec06d450 0x7f98ec0cbfd0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:05.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.435+0000 7f98f3400640 1 -- 172.21.15.100:0/1227280766 >> 172.21.15.100:0/1227280766 conn(0x7f98ec0cee80 msgr2=0x7f98ec1613d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:05.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.435+0000 7f98f3400640 1 -- 172.21.15.100:0/1227280766 shutdown_connections 2024-04-23T14:35:05.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.435+0000 7f98f3400640 1 -- 172.21.15.100:0/1227280766 wait complete. 2024-04-23T14:35:05.460 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-23T14:35:05.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.599+0000 7f4137000640 1 Processor -- start 2024-04-23T14:35:05.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.599+0000 7f4137000640 1 -- start start 2024-04-23T14:35:05.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.599+0000 7f4137000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f41300cd300 0x7f41301671f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:05.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.599+0000 7f4137000640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f413006cbb0 con 0x7f413006bb20 2024-04-23T14:35:05.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.599+0000 7f4137000640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f413006ce90 con 0x7f413015c7a0 2024-04-23T14:35:05.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.599+0000 7f4137000640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f413006d450 con 0x7f41300cd300 2024-04-23T14:35:05.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.599+0000 7f4135000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f41300cd300 0x7f41301671f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:05.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.599+0000 7f4135000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f41300cd300 0x7f41301671f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34574/0 (socket says 172.21.15.100:34574) 2024-04-23T14:35:05.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.599+0000 7f4135000640 1 -- 172.21.15.100:0/2803045727 learned_addr learned my addr 172.21.15.100:0/2803045727 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:05.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.599+0000 7f4127400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f413006bb20 0x7f41300ccbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50586/0 (socket says 172.21.15.100:50586) 2024-04-23T14:35:05.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.599+0000 7f4135000640 1 -- 172.21.15.100:0/2803045727 >> v1:172.21.15.151:6789/0 conn(0x7f413015c7a0 legacy=0x7f41300af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:05.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.599+0000 7f4135000640 1 -- 172.21.15.100:0/2803045727 >> v1:172.21.15.100:6789/0 conn(0x7f413006bb20 legacy=0x7f41300ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:05.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.599+0000 7f4135000640 1 -- 172.21.15.100:0/2803045727 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413006d170 con 0x7f41300cd300 2024-04-23T14:35:05.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.603+0000 7f4135000640 1 --2- 172.21.15.100:0/2803045727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41300cd300 0x7f41301671f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f4128009e50 tx=0x7f412801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e119662b122dec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:05.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.603+0000 7f412f400640 1 -- 172.21.15.100:0/2803045727 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41280a5620 con 0x7f41300cd300 2024-04-23T14:35:05.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.603+0000 7f412f400640 1 -- 172.21.15.100:0/2803045727 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41280a57c0 con 0x7f41300cd300 2024-04-23T14:35:05.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.603+0000 7f412f400640 1 -- 172.21.15.100:0/2803045727 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41280a5b90 con 0x7f41300cd300 2024-04-23T14:35:05.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.603+0000 7f4137000640 1 -- 172.21.15.100:0/2803045727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41300cd300 msgr2=0x7f41301671f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:05.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.603+0000 7f4137000640 1 --2- 172.21.15.100:0/2803045727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41300cd300 0x7f41301671f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f4128009e50 tx=0x7f412801c960 comp rx=0 tx=0).stop 2024-04-23T14:35:05.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.603+0000 7f4137000640 1 -- 172.21.15.100:0/2803045727 shutdown_connections 2024-04-23T14:35:05.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.603+0000 7f4137000640 1 --2- 172.21.15.100:0/2803045727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41300cd300 0x7f41301671f0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:05.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.603+0000 7f4137000640 1 -- 172.21.15.100:0/2803045727 >> 172.21.15.100:0/2803045727 conn(0x7f41300b5200 msgr2=0x7f41300b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:05.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.603+0000 7f4137000640 1 -- 172.21.15.100:0/2803045727 shutdown_connections 2024-04-23T14:35:05.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.603+0000 7f4137000640 1 -- 172.21.15.100:0/2803045727 wait complete. 2024-04-23T14:35:05.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.607+0000 7f4137000640 1 Processor -- start 2024-04-23T14:35:05.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.607+0000 7f4137000640 1 -- start start 2024-04-23T14:35:05.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.607+0000 7f4137000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f413006bb20 0x7f41300c2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:05.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.607+0000 7f4137000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41300cd300 0x7f41300c3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:05.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.607+0000 7f4137000640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f413015c7a0 0x7f41300b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:05.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.607+0000 7f4137000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f413016a2e0 con 0x7f41300cd300 2024-04-23T14:35:05.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.607+0000 7f4137000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f413016a160 con 0x7f413006bb20 2024-04-23T14:35:05.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.607+0000 7f4137000640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f413016a460 con 0x7f413015c7a0 2024-04-23T14:35:05.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.607+0000 7f4127400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41300cd300 0x7f41300c3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:05.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.607+0000 7f4127400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41300cd300 0x7f41300c3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34584/0 (socket says 172.21.15.100:34584) 2024-04-23T14:35:05.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.607+0000 7f4127400640 1 -- 172.21.15.100:0/2932577051 learned_addr learned my addr 172.21.15.100:0/2932577051 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:05.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.607+0000 7f4127400640 1 -- 172.21.15.100:0/2932577051 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f413015c7a0 msgr2=0x7f41300b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:05.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.607+0000 7f4127400640 1 --2- 172.21.15.100:0/2932577051 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f413015c7a0 0x7f41300b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:05.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.607+0000 7f4127400640 1 -- 172.21.15.100:0/2932577051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f413006bb20 msgr2=0x7f41300c2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:05.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.607+0000 7f4127400640 1 --2- 172.21.15.100:0/2932577051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f413006bb20 0x7f41300c2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:05.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.607+0000 7f4127400640 1 -- 172.21.15.100:0/2932577051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41300d0c00 con 0x7f41300cd300 2024-04-23T14:35:05.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.607+0000 7f4127400640 1 --2- 172.21.15.100:0/2932577051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41300cd300 0x7f41300c3d10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f411800c370 tx=0x7f411800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:05.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.607+0000 7f412cc00640 1 -- 172.21.15.100:0/2932577051 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41180074e0 con 0x7f41300cd300 2024-04-23T14:35:05.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.607+0000 7f412cc00640 1 -- 172.21.15.100:0/2932577051 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4118007680 con 0x7f41300cd300 2024-04-23T14:35:05.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.607+0000 7f412cc00640 1 -- 172.21.15.100:0/2932577051 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118004a30 con 0x7f41300cd300 2024-04-23T14:35:05.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.607+0000 7f4137000640 1 -- 172.21.15.100:0/2932577051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41300c32d0 con 0x7f41300cd300 2024-04-23T14:35:05.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.607+0000 7f4137000640 1 -- 172.21.15.100:0/2932577051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41300977b0 con 0x7f41300cd300 2024-04-23T14:35:05.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.611+0000 7f4137000640 1 -- 172.21.15.100:0/2932577051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4130068ad0 con 0x7f41300cd300 2024-04-23T14:35:05.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.611+0000 7f412cc00640 1 -- 172.21.15.100:0/2932577051 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f411801a020 con 0x7f41300cd300 2024-04-23T14:35:05.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.611+0000 7f412cc00640 1 --2- 172.21.15.100:0/2932577051 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f410c068380 0x7f410c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:05.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.611+0000 7f412cc00640 1 -- 172.21.15.100:0/2932577051 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5749+0+0 (secure 0 0 0) 0x7f41180860f0 con 0x7f41300cd300 2024-04-23T14:35:05.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.611+0000 7f412fe00640 1 --2- 172.21.15.100:0/2932577051 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f410c068380 0x7f410c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:05.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.611+0000 7f412fe00640 1 --2- 172.21.15.100:0/2932577051 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f410c068380 0x7f410c06a840 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4120007fd0 tx=0x7f41200079a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:05.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.615+0000 7f412cc00640 1 -- 172.21.15.100:0/2932577051 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4118056710 con 0x7f41300cd300 2024-04-23T14:35:05.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:05.771+0000 7f4137000640 1 -- 172.21.15.100:0/2932577051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4130099940 con 0x7f41300cd300 2024-04-23T14:35:06.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.435+0000 7f412cc00640 1 -- 172.21.15.100:0/2932577051 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v56) ==== 169+0+0 (secure 0 0 0) 0x7f4118056010 con 0x7f41300cd300 2024-04-23T14:35:06.435 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' removed 2024-04-23T14:35:06.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.439+0000 7f4137000640 1 -- 172.21.15.100:0/2932577051 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f410c068380 msgr2=0x7f410c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:06.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.439+0000 7f4137000640 1 --2- 172.21.15.100:0/2932577051 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f410c068380 0x7f410c06a840 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4120007fd0 tx=0x7f41200079a0 comp rx=0 tx=0).stop 2024-04-23T14:35:06.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.439+0000 7f4137000640 1 -- 172.21.15.100:0/2932577051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41300cd300 msgr2=0x7f41300c3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:06.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.439+0000 7f4137000640 1 --2- 172.21.15.100:0/2932577051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41300cd300 0x7f41300c3d10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f411800c370 tx=0x7f411800c830 comp rx=0 tx=0).stop 2024-04-23T14:35:06.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.439+0000 7f4137000640 1 -- 172.21.15.100:0/2932577051 shutdown_connections 2024-04-23T14:35:06.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.439+0000 7f4137000640 1 --2- 172.21.15.100:0/2932577051 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f410c068380 0x7f410c06a840 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:06.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.439+0000 7f4137000640 1 --2- 172.21.15.100:0/2932577051 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f413015c7a0 0x7f41300b1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:06.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.439+0000 7f4137000640 1 --2- 172.21.15.100:0/2932577051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41300cd300 0x7f41300c3d10 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:06.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.439+0000 7f4137000640 1 --2- 172.21.15.100:0/2932577051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f413006bb20 0x7f41300c2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:06.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.439+0000 7f4137000640 1 -- 172.21.15.100:0/2932577051 >> 172.21.15.100:0/2932577051 conn(0x7f41300b5200 msgr2=0x7f41300cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:06.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.439+0000 7f4137000640 1 -- 172.21.15.100:0/2932577051 shutdown_connections 2024-04-23T14:35:06.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.439+0000 7f4137000640 1 -- 172.21.15.100:0/2932577051 wait complete. 2024-04-23T14:35:06.462 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-23T14:35:06.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.599+0000 7fc3cf200640 1 Processor -- start 2024-04-23T14:35:06.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.599+0000 7fc3cf200640 1 -- start start 2024-04-23T14:35:06.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.599+0000 7fc3cf200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc3c815b370 0x7fc3c815c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:06.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.599+0000 7fc3cf200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc3c8073560 0x7fc3c80ae020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:06.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.599+0000 7fc3cf200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c815efa0 con 0x7fc3c80cc3a0 2024-04-23T14:35:06.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.599+0000 7fc3cf200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc3c815b750 con 0x7fc3c815b370 2024-04-23T14:35:06.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.599+0000 7fc3cf200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c815f280 con 0x7fc3c8073560 2024-04-23T14:35:06.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.599+0000 7fc3cd400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fc3c80cc3a0 0x7fc3c8072e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:50594/0 (socket says 172.21.15.100:50594) 2024-04-23T14:35:06.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.599+0000 7fc3cde00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc3c815b370 0x7fc3c815c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:06.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.599+0000 7fc3cd400640 1 -- 172.21.15.100:0/3105522375 learned_addr learned my addr 172.21.15.100:0/3105522375 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:06.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.599+0000 7fc3ce800640 1 --2- 172.21.15.100:0/3105522375 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc3c8073560 0x7fc3c80ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:06.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.599+0000 7fc3cca00640 1 -- 172.21.15.100:0/3105522375 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4260296875 0 0) 0x7fc3c815efa0 con 0x7fc3c80cc3a0 2024-04-23T14:35:06.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.603+0000 7fc3cca00640 1 -- 172.21.15.100:0/3105522375 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3b0003530 con 0x7fc3c80cc3a0 2024-04-23T14:35:06.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.603+0000 7fc3cde00640 1 -- 172.21.15.100:0/3105522375 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc3c8073560 msgr2=0x7fc3c80ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:06.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.603+0000 7fc3cde00640 1 --2- 172.21.15.100:0/3105522375 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc3c8073560 0x7fc3c80ae020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:06.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.603+0000 7fc3cde00640 1 -- 172.21.15.100:0/3105522375 >> v1:172.21.15.100:6789/0 conn(0x7fc3c80cc3a0 legacy=0x7fc3c8072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:06.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.603+0000 7fc3cde00640 1 -- 172.21.15.100:0/3105522375 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c815f560 con 0x7fc3c815b370 2024-04-23T14:35:06.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.603+0000 7fc3cde00640 1 --2- 172.21.15.100:0/3105522375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3c815b370 0x7fc3c815c300 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc0097c0 tx=0x7fc3bc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405100d40ad9088 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:06.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.603+0000 7fc3cca00640 1 -- 172.21.15.100:0/3105522375 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc0b0070 con 0x7fc3c815b370 2024-04-23T14:35:06.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.603+0000 7fc3cca00640 1 -- 172.21.15.100:0/3105522375 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3bc0ab600 con 0x7fc3c815b370 2024-04-23T14:35:06.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.603+0000 7fc3cca00640 1 -- 172.21.15.100:0/3105522375 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc0ab9d0 con 0x7fc3c815b370 2024-04-23T14:35:06.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.603+0000 7fc3cf200640 1 -- 172.21.15.100:0/3105522375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3c815b370 msgr2=0x7fc3c815c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:06.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.603+0000 7fc3cf200640 1 --2- 172.21.15.100:0/3105522375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3c815b370 0x7fc3c815c300 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc0097c0 tx=0x7fc3bc01c410 comp rx=0 tx=0).stop 2024-04-23T14:35:06.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.603+0000 7fc3cf200640 1 -- 172.21.15.100:0/3105522375 shutdown_connections 2024-04-23T14:35:06.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.603+0000 7fc3cf200640 1 --2- 172.21.15.100:0/3105522375 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc3c8073560 0x7fc3c80ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:06.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.603+0000 7fc3cf200640 1 --2- 172.21.15.100:0/3105522375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3c815b370 0x7fc3c815c300 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:06.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.603+0000 7fc3cf200640 1 -- 172.21.15.100:0/3105522375 >> 172.21.15.100:0/3105522375 conn(0x7fc3c80b1d00 msgr2=0x7fc3c80b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:06.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.603+0000 7fc3cf200640 1 -- 172.21.15.100:0/3105522375 shutdown_connections 2024-04-23T14:35:06.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.607+0000 7fc3cf200640 1 -- 172.21.15.100:0/3105522375 wait complete. 2024-04-23T14:35:06.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.607+0000 7fc3cf200640 1 Processor -- start 2024-04-23T14:35:06.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.607+0000 7fc3cf200640 1 -- start start 2024-04-23T14:35:06.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.607+0000 7fc3cf200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3c8073560 0x7fc3c8099ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:06.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.607+0000 7fc3cf200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3c80cc3a0 0x7fc3c80cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:06.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.607+0000 7fc3cf200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc3c815b370 0x7fc3c80cf390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.607+0000 7fc3cf200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c80af100 con 0x7fc3c8073560 2024-04-23T14:35:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.607+0000 7fc3cf200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c80aef80 con 0x7fc3c80cc3a0 2024-04-23T14:35:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.607+0000 7fc3cf200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c80af280 con 0x7fc3c815b370 2024-04-23T14:35:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.607+0000 7fc3cde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3c8073560 0x7fc3c8099ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.607+0000 7fc3cde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3c8073560 0x7fc3c8099ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34602/0 (socket says 172.21.15.100:34602) 2024-04-23T14:35:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.607+0000 7fc3cde00640 1 -- 172.21.15.100:0/2411100675 learned_addr learned my addr 172.21.15.100:0/2411100675 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.607+0000 7fc3cde00640 1 -- 172.21.15.100:0/2411100675 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc3c815b370 msgr2=0x7fc3c80cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.607+0000 7fc3cde00640 1 --2- 172.21.15.100:0/2411100675 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc3c815b370 0x7fc3c80cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.607+0000 7fc3cde00640 1 -- 172.21.15.100:0/2411100675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3c80cc3a0 msgr2=0x7fc3c80cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.607+0000 7fc3cde00640 1 --2- 172.21.15.100:0/2411100675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3c80cc3a0 0x7fc3c80cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.607+0000 7fc3cde00640 1 -- 172.21.15.100:0/2411100675 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c80ce2f0 con 0x7fc3c8073560 2024-04-23T14:35:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.607+0000 7fc3cde00640 1 --2- 172.21.15.100:0/2411100675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3c8073560 0x7fc3c8099ac0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc01a040 tx=0x7fc3bc0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:06.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.607+0000 7fc3c2000640 1 -- 172.21.15.100:0/2411100675 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc0b8070 con 0x7fc3c8073560 2024-04-23T14:35:06.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.607+0000 7fc3c2000640 1 -- 172.21.15.100:0/2411100675 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3bc0b4410 con 0x7fc3c8073560 2024-04-23T14:35:06.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.607+0000 7fc3cf200640 1 -- 172.21.15.100:0/2411100675 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c80ce5d0 con 0x7fc3c8073560 2024-04-23T14:35:06.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.607+0000 7fc3c2000640 1 -- 172.21.15.100:0/2411100675 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc0b0040 con 0x7fc3c8073560 2024-04-23T14:35:06.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.611+0000 7fc3cf200640 1 -- 172.21.15.100:0/2411100675 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c80ceb90 con 0x7fc3c8073560 2024-04-23T14:35:06.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.611+0000 7fc3cf200640 1 -- 172.21.15.100:0/2411100675 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c8068ad0 con 0x7fc3c8073560 2024-04-23T14:35:06.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.611+0000 7fc3c2000640 1 -- 172.21.15.100:0/2411100675 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc3bc0bb070 con 0x7fc3c8073560 2024-04-23T14:35:06.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.611+0000 7fc3c2000640 1 --2- 172.21.15.100:0/2411100675 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc3ac068380 0x7fc3ac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:06.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.611+0000 7fc3c2000640 1 -- 172.21.15.100:0/2411100675 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7fc3bc120290 con 0x7fc3c8073560 2024-04-23T14:35:06.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.611+0000 7fc3cd400640 1 --2- 172.21.15.100:0/2411100675 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc3ac068380 0x7fc3ac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:06.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.611+0000 7fc3cd400640 1 --2- 172.21.15.100:0/2411100675 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc3ac068380 0x7fc3ac06a840 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8000f70 tx=0x7fc3b8007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:06.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.615+0000 7fc3c2000640 1 -- 172.21.15.100:0/2411100675 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc3bc016090 con 0x7fc3c8073560 2024-04-23T14:35:06.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.787+0000 7fc3cf200640 1 -- 172.21.15.100:0/2411100675 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fc3c8155ad0 con 0x7fc3c8073560 2024-04-23T14:35:06.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.791+0000 7fc3c2000640 1 -- 172.21.15.100:0/2411100675 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7fc3bc0f09c0 con 0x7fc3c8073560 2024-04-23T14:35:06.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.791+0000 7fc3cf200640 1 -- 172.21.15.100:0/2411100675 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc3ac068380 msgr2=0x7fc3ac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:06.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.791+0000 7fc3cf200640 1 --2- 172.21.15.100:0/2411100675 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc3ac068380 0x7fc3ac06a840 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8000f70 tx=0x7fc3b8007900 comp rx=0 tx=0).stop 2024-04-23T14:35:06.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.791+0000 7fc3cf200640 1 -- 172.21.15.100:0/2411100675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3c8073560 msgr2=0x7fc3c8099ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:06.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.791+0000 7fc3cf200640 1 --2- 172.21.15.100:0/2411100675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3c8073560 0x7fc3c8099ac0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc01a040 tx=0x7fc3bc0191b0 comp rx=0 tx=0).stop 2024-04-23T14:35:06.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.791+0000 7fc3cf200640 1 -- 172.21.15.100:0/2411100675 shutdown_connections 2024-04-23T14:35:06.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.791+0000 7fc3cf200640 1 --2- 172.21.15.100:0/2411100675 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc3ac068380 0x7fc3ac06a840 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:06.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.791+0000 7fc3cf200640 1 --2- 172.21.15.100:0/2411100675 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc3c815b370 0x7fc3c80cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:06.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.791+0000 7fc3cf200640 1 --2- 172.21.15.100:0/2411100675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3c80cc3a0 0x7fc3c80cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:06.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.791+0000 7fc3cf200640 1 --2- 172.21.15.100:0/2411100675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3c8073560 0x7fc3c8099ac0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:06.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.791+0000 7fc3cf200640 1 -- 172.21.15.100:0/2411100675 >> 172.21.15.100:0/2411100675 conn(0x7fc3c80b1d00 msgr2=0x7fc3c806ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:06.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.791+0000 7fc3cf200640 1 -- 172.21.15.100:0/2411100675 shutdown_connections 2024-04-23T14:35:06.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.791+0000 7fc3cf200640 1 -- 172.21.15.100:0/2411100675 wait complete. 2024-04-23T14:35:06.813 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2024-04-23T14:35:06.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.947+0000 7f9fcc600640 1 Processor -- start 2024-04-23T14:35:06.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.947+0000 7f9fcc600640 1 -- start start 2024-04-23T14:35:06.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.947+0000 7f9fcc600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9fc40cd030 0x7f9fc40b2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:06.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.947+0000 7f9fcc600640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f9fc415cb10 0x7f9fc415cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:06.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.947+0000 7f9fcc600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fc415d690 con 0x7f9fc4071980 2024-04-23T14:35:06.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.947+0000 7f9fcc600640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f9fc415d970 con 0x7f9fc415cb10 2024-04-23T14:35:06.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.947+0000 7f9fcc600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc40b2000 con 0x7f9fc40cd030 2024-04-23T14:35:06.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.947+0000 7f9fc9c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9fc40cd030 0x7f9fc40b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:06.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.947+0000 7f9fc9200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f9fc415cb10 0x7f9fc415cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:06.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.947+0000 7f9fca600640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f9fc4071980 0x7f9fc40d0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:34502/0 (socket says 172.21.15.100:34502) 2024-04-23T14:35:06.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.947+0000 7f9fca600640 1 -- 172.21.15.100:0/3996951430 learned_addr learned my addr 172.21.15.100:0/3996951430 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:06.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.947+0000 7f9fc9200640 1 -- 172.21.15.100:0/3996951430 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9fc40cd030 msgr2=0x7f9fc40b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:06.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.947+0000 7f9fc9200640 1 --2- 172.21.15.100:0/3996951430 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9fc40cd030 0x7f9fc40b2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:06.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.947+0000 7f9fc9200640 1 -- 172.21.15.100:0/3996951430 >> v1:172.21.15.151:6789/0 conn(0x7f9fc4071980 legacy=0x7f9fc40d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:06.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.947+0000 7f9fc9200640 1 -- 172.21.15.100:0/3996951430 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc40b22e0 con 0x7f9fc415cb10 2024-04-23T14:35:06.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.947+0000 7f9fc9c00640 1 --2- 172.21.15.100:0/3996951430 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9fc40cd030 0x7f9fc40b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:35:06.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.947+0000 7f9fc3e00640 1 -- 172.21.15.100:0/3996951430 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1785003350 0 0) 0x7f9fc415d690 con 0x7f9fc4071980 2024-04-23T14:35:06.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.951+0000 7f9fc9200640 1 --2- 172.21.15.100:0/3996951430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fc415cb10 0x7f9fc415cf90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00097c0 tx=0x7f9fb001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48567ae9a9e654cb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:06.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.951+0000 7f9fc3e00640 1 -- 172.21.15.100:0/3996951430 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00b1070 con 0x7f9fc415cb10 2024-04-23T14:35:06.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.951+0000 7f9fc3e00640 1 -- 172.21.15.100:0/3996951430 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb00ac410 con 0x7f9fc415cb10 2024-04-23T14:35:06.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.951+0000 7f9fc3e00640 1 -- 172.21.15.100:0/3996951430 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00ac7e0 con 0x7f9fc415cb10 2024-04-23T14:35:06.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.951+0000 7f9fcc600640 1 -- 172.21.15.100:0/3996951430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fc415cb10 msgr2=0x7f9fc415cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:06.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.951+0000 7f9fcc600640 1 --2- 172.21.15.100:0/3996951430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fc415cb10 0x7f9fc415cf90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00097c0 tx=0x7f9fb001c4a0 comp rx=0 tx=0).stop 2024-04-23T14:35:06.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.951+0000 7f9fcc600640 1 -- 172.21.15.100:0/3996951430 shutdown_connections 2024-04-23T14:35:06.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.951+0000 7f9fcc600640 1 --2- 172.21.15.100:0/3996951430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fc415cb10 0x7f9fc415cf90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:06.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.951+0000 7f9fcc600640 1 --2- 172.21.15.100:0/3996951430 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9fc40cd030 0x7f9fc40b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:06.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.951+0000 7f9fcc600640 1 -- 172.21.15.100:0/3996951430 >> 172.21.15.100:0/3996951430 conn(0x7f9fc40d24c0 msgr2=0x7f9fc40d28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:06.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.951+0000 7f9fcc600640 1 -- 172.21.15.100:0/3996951430 shutdown_connections 2024-04-23T14:35:06.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.951+0000 7f9fcc600640 1 -- 172.21.15.100:0/3996951430 wait complete. 2024-04-23T14:35:06.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fcc600640 1 Processor -- start 2024-04-23T14:35:06.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fcc600640 1 -- start start 2024-04-23T14:35:06.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fcc600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9fc4071980 0x7f9fc40c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:06.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fcc600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc40cd030 0x7f9fc40c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:06.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fcc600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fc415cb10 0x7f9fc406ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:06.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fcc600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc40721a0 con 0x7f9fc40cd030 2024-04-23T14:35:06.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fcc600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc40cd410 con 0x7f9fc415cb10 2024-04-23T14:35:06.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fcc600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc40c6470 con 0x7f9fc4071980 2024-04-23T14:35:06.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fc9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc40cd030 0x7f9fc40c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:06.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fc9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc40cd030 0x7f9fc40c82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34618/0 (socket says 172.21.15.100:34618) 2024-04-23T14:35:06.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fc9200640 1 -- 172.21.15.100:0/3809231288 learned_addr learned my addr 172.21.15.100:0/3809231288 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:06.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fca600640 1 --2- 172.21.15.100:0/3809231288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fc415cb10 0x7f9fc406ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:06.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fc9c00640 1 --2- 172.21.15.100:0/3809231288 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9fc4071980 0x7f9fc40c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:06.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fc9200640 1 -- 172.21.15.100:0/3809231288 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9fc4071980 msgr2=0x7f9fc40c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:06.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fc9200640 1 --2- 172.21.15.100:0/3809231288 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9fc4071980 0x7f9fc40c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:06.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fc9200640 1 -- 172.21.15.100:0/3809231288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fc415cb10 msgr2=0x7f9fc406ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:06.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fc9200640 1 --2- 172.21.15.100:0/3809231288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fc415cb10 0x7f9fc406ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:06.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fc9200640 1 -- 172.21.15.100:0/3809231288 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc40c6750 con 0x7f9fc40cd030 2024-04-23T14:35:06.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fca600640 1 --2- 172.21.15.100:0/3809231288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fc415cb10 0x7f9fc406ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:35:06.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fc9200640 1 --2- 172.21.15.100:0/3809231288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc40cd030 0x7f9fc40c82e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9fb001a0d0 tx=0x7f9fb0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:06.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fc1600640 1 -- 172.21.15.100:0/3809231288 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00b1040 con 0x7f9fc40cd030 2024-04-23T14:35:06.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fc1600640 1 -- 172.21.15.100:0/3809231288 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb00ba070 con 0x7f9fc40cd030 2024-04-23T14:35:06.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fc1600640 1 -- 172.21.15.100:0/3809231288 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00b5650 con 0x7f9fc40cd030 2024-04-23T14:35:06.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fcc600640 1 -- 172.21.15.100:0/3809231288 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc40c6a30 con 0x7f9fc40cd030 2024-04-23T14:35:06.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.955+0000 7f9fcc600640 1 -- 172.21.15.100:0/3809231288 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc40c6ff0 con 0x7f9fc40cd030 2024-04-23T14:35:06.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.959+0000 7f9fcc600640 1 -- 172.21.15.100:0/3809231288 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fc41f7f80 con 0x7f9fc40cd030 2024-04-23T14:35:06.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.959+0000 7f9fc1600640 1 -- 172.21.15.100:0/3809231288 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9fb00b57f0 con 0x7f9fc40cd030 2024-04-23T14:35:06.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.959+0000 7f9fc1600640 1 --2- 172.21.15.100:0/3809231288 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9fa40683b0 0x7f9fa406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:06.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.959+0000 7f9fc1600640 1 -- 172.21.15.100:0/3809231288 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f9fb0120340 con 0x7f9fc40cd030 2024-04-23T14:35:06.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.959+0000 7f9fc9c00640 1 --2- 172.21.15.100:0/3809231288 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9fa40683b0 0x7f9fa406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:06.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.959+0000 7f9fc9c00640 1 --2- 172.21.15.100:0/3809231288 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9fa40683b0 0x7f9fa406a870 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009580 tx=0x7f9fb8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:06.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:06.963+0000 7f9fc1600640 1 -- 172.21.15.100:0/3809231288 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9fb0016090 con 0x7f9fc40cd030 2024-04-23T14:35:07.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:07.175+0000 7f9fcc600640 1 -- 172.21.15.100:0/3809231288 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) -- 0x7f9fc415b900 con 0x7f9fc40cd030 2024-04-23T14:35:07.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:07.447+0000 7f9fc1600640 1 -- 172.21.15.100:0/3809231288 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v185) ==== 134+0+0 (secure 0 0 0) 0x7f9fc415b900 con 0x7f9fc40cd030 2024-04-23T14:35:07.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:07.451+0000 7f9fcc600640 1 -- 172.21.15.100:0/3809231288 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9fa40683b0 msgr2=0x7f9fa406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:07.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:07.451+0000 7f9fcc600640 1 --2- 172.21.15.100:0/3809231288 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9fa40683b0 0x7f9fa406a870 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009580 tx=0x7f9fb8009190 comp rx=0 tx=0).stop 2024-04-23T14:35:07.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:07.451+0000 7f9fcc600640 1 -- 172.21.15.100:0/3809231288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc40cd030 msgr2=0x7f9fc40c82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:07.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:07.451+0000 7f9fcc600640 1 --2- 172.21.15.100:0/3809231288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc40cd030 0x7f9fc40c82e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9fb001a0d0 tx=0x7f9fb0019240 comp rx=0 tx=0).stop 2024-04-23T14:35:07.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:07.451+0000 7f9fcc600640 1 -- 172.21.15.100:0/3809231288 shutdown_connections 2024-04-23T14:35:07.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:07.451+0000 7f9fcc600640 1 --2- 172.21.15.100:0/3809231288 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f9fa40683b0 0x7f9fa406a870 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:07.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:07.451+0000 7f9fcc600640 1 --2- 172.21.15.100:0/3809231288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fc415cb10 0x7f9fc406ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:07.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:07.451+0000 7f9fcc600640 1 --2- 172.21.15.100:0/3809231288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc40cd030 0x7f9fc40c82e0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:07.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:07.451+0000 7f9fcc600640 1 --2- 172.21.15.100:0/3809231288 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9fc4071980 0x7f9fc40c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:07.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:07.451+0000 7f9fcc600640 1 -- 172.21.15.100:0/3809231288 >> 172.21.15.100:0/3809231288 conn(0x7f9fc40d24c0 msgr2=0x7f9fc415f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:07.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:07.451+0000 7f9fcc600640 1 -- 172.21.15.100:0/3809231288 shutdown_connections 2024-04-23T14:35:07.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:07.451+0000 7f9fcc600640 1 -- 172.21.15.100:0/3809231288 wait complete. 2024-04-23T14:35:07.471 INFO:tasks.cephfs_test_runner:test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) ... ERROR 2024-04-23T14:35:07.472 INFO:tasks.cephfs_test_runner: 2024-04-23T14:35:07.472 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-23T14:35:07.472 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-04-23T14:35:07.472 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-23T14:35:07.472 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-23T14:35:07.472 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2024-04-23T14:35:07.472 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2024-04-23T14:35:07.472 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa/tasks/cephfs/mount.py", line 776, in run_python 2024-04-23T14:35:07.472 INFO:tasks.cephfs_test_runner: p.wait() 2024-04-23T14:35:07.472 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait 2024-04-23T14:35:07.472 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-04-23T14:35:07.472 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-04-23T14:35:07.472 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-04-23T14:35:07.472 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi100 with status 1: 'stdin-killer -- python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2024-04-23T14:35:07.472 INFO:tasks.cephfs_test_runner: 2024-04-23T14:35:07.473 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-23T14:35:07.473 INFO:tasks.cephfs_test_runner:Ran 2 tests in 124.489s 2024-04-23T14:35:07.473 INFO:tasks.cephfs_test_runner: 2024-04-23T14:35:07.473 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-04-23T14:35:07.473 INFO:tasks.cephfs_test_runner: 2024-04-23T14:35:07.473 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-23T14:35:07.473 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-04-23T14:35:07.473 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-23T14:35:07.473 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-23T14:35:07.473 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2024-04-23T14:35:07.473 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2024-04-23T14:35:07.473 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa/tasks/cephfs/mount.py", line 776, in run_python 2024-04-23T14:35:07.473 INFO:tasks.cephfs_test_runner: p.wait() 2024-04-23T14:35:07.473 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait 2024-04-23T14:35:07.473 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-04-23T14:35:07.473 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-04-23T14:35:07.473 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-04-23T14:35:07.473 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi100 with status 1: 'stdin-killer -- python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2024-04-23T14:35:07.474 INFO:tasks.cephfs_test_runner: 2024-04-23T14:35:07.474 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-04-23T14:35:07.693 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=6abe976484c14aa4b401bba1ebe056dc Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-04-23T14:35:07.695 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-23T14:35:07.703 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-23T14:35:07.740 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-23T14:35:07.740 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:35:07.740 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:35:07.744 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:35:07.755 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350140k,nr_inodes=4087535,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274960k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=820 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274956k,nr_inodes=818739,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274960k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:35:07.755 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-23T14:35:07.755 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:35:07.756 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:35:07.806 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350140k,nr_inodes=4087535,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274960k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=820 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274956k,nr_inodes=818739,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274960k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:35:07.806 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:35:07.806 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:35:07.859 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3578 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:35:07.859 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-23T14:35:07.859 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:35:07.912 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3578 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:35:07.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:07.912 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-23T14:35:07.921 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:35:07.921 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:35:07.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:07.922 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:35:07.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:07.982 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:35:08.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:08.050 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-04-23T14:35:08.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:08.209 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-04-23T14:35:08.218 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-23T14:35:08.219 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-23T14:35:08.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:08.219 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:35:08.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:08.285 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-23T14:35:08.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:08.345 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link delete ceph-brx 2024-04-23T14:35:08.502 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-23T14:35:08.510 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-23T14:35:08.510 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:35:08.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.643+0000 7fc28ea00640 1 Processor -- start 2024-04-23T14:35:08.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.643+0000 7fc28ea00640 1 -- start start 2024-04-23T14:35:08.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.647+0000 7fc28ea00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc28815b640 0x7fc2880adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:08.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.647+0000 7fc28ea00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc2880b2a10 0x7fc288072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:08.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.647+0000 7fc28ea00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2880af080 con 0x7fc2880cc750 2024-04-23T14:35:08.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.647+0000 7fc28ea00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc2880afbc0 con 0x7fc28815b640 2024-04-23T14:35:08.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.647+0000 7fc28ea00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fc28815ba20 con 0x7fc2880b2a10 2024-04-23T14:35:08.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.647+0000 7fc287e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc28815b640 0x7fc2880adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:08.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.647+0000 7fc287e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc28815b640 0x7fc2880adfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40902/0 (socket says 172.21.15.100:40902) 2024-04-23T14:35:08.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.647+0000 7fc287e00640 1 -- 172.21.15.100:0/894186563 learned_addr learned my addr 172.21.15.100:0/894186563 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:08.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.647+0000 7fc287400640 1 --2- 172.21.15.100:0/894186563 >> v2:172.21.15.151:3300/0 conn(0x7fc2880b2a10 0x7fc288072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:08.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.647+0000 7fc287e00640 1 -- 172.21.15.100:0/894186563 >> v2:172.21.15.151:3300/0 conn(0x7fc2880b2a10 msgr2=0x7fc288072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:08.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.647+0000 7fc287e00640 1 --2- 172.21.15.100:0/894186563 >> v2:172.21.15.151:3300/0 conn(0x7fc2880b2a10 0x7fc288072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:08.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.647+0000 7fc287e00640 1 -- 172.21.15.100:0/894186563 >> v1:172.21.15.151:6789/0 conn(0x7fc2880cc750 legacy=0x7fc288071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:08.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.647+0000 7fc287e00640 1 -- 172.21.15.100:0/894186563 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2880af320 con 0x7fc28815b640 2024-04-23T14:35:08.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.651+0000 7fc287e00640 1 --2- 172.21.15.100:0/894186563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28815b640 0x7fc2880adfc0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc2880ada50 tx=0x7fc27801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca54644fabbd0d82 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:08.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.651+0000 7fc286a00640 1 -- 172.21.15.100:0/894186563 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2780a57f0 con 0x7fc28815b640 2024-04-23T14:35:08.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.651+0000 7fc286a00640 1 -- 172.21.15.100:0/894186563 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2780a5990 con 0x7fc28815b640 2024-04-23T14:35:08.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.651+0000 7fc286a00640 1 -- 172.21.15.100:0/894186563 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2780a5d60 con 0x7fc28815b640 2024-04-23T14:35:08.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.651+0000 7fc28ea00640 1 -- 172.21.15.100:0/894186563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28815b640 msgr2=0x7fc2880adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:08.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.651+0000 7fc28ea00640 1 --2- 172.21.15.100:0/894186563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28815b640 0x7fc2880adfc0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc2880ada50 tx=0x7fc27801c930 comp rx=0 tx=0).stop 2024-04-23T14:35:08.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.651+0000 7fc28ea00640 1 -- 172.21.15.100:0/894186563 shutdown_connections 2024-04-23T14:35:08.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.651+0000 7fc28ea00640 1 --2- 172.21.15.100:0/894186563 >> v2:172.21.15.151:3300/0 conn(0x7fc2880b2a10 0x7fc288072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:08.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.651+0000 7fc28ea00640 1 --2- 172.21.15.100:0/894186563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28815b640 0x7fc2880adfc0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:08.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.651+0000 7fc28ea00640 1 -- 172.21.15.100:0/894186563 >> 172.21.15.100:0/894186563 conn(0x7fc2880b1e80 msgr2=0x7fc2880b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:08.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.651+0000 7fc28ea00640 1 -- 172.21.15.100:0/894186563 shutdown_connections 2024-04-23T14:35:08.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.651+0000 7fc28ea00640 1 -- 172.21.15.100:0/894186563 wait complete. 2024-04-23T14:35:08.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.651+0000 7fc28ea00640 1 Processor -- start 2024-04-23T14:35:08.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.651+0000 7fc28ea00640 1 -- start start 2024-04-23T14:35:08.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc28ea00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc2880b2a10 0x7fc28809c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:08.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc28ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2880cc750 0x7fc28814d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:08.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc28ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc28815b640 0x7fc28814d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:08.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc28ea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc2881683c0 con 0x7fc2880cc750 2024-04-23T14:35:08.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc28ea00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc288073850 con 0x7fc28815b640 2024-04-23T14:35:08.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc28ea00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc288168540 con 0x7fc2880b2a10 2024-04-23T14:35:08.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc287400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2880cc750 0x7fc28814d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:08.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc287400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2880cc750 0x7fc28814d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40912/0 (socket says 172.21.15.100:40912) 2024-04-23T14:35:08.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc287400640 1 -- 172.21.15.100:0/2705953449 learned_addr learned my addr 172.21.15.100:0/2705953449 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:08.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc287e00640 1 --2- 172.21.15.100:0/2705953449 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc2880b2a10 0x7fc28809c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:08.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc28ca00640 1 --2- 172.21.15.100:0/2705953449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc28815b640 0x7fc28814d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:08.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc287400640 1 -- 172.21.15.100:0/2705953449 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc2880b2a10 msgr2=0x7fc28809c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:08.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc287400640 1 --2- 172.21.15.100:0/2705953449 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc2880b2a10 0x7fc28809c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:08.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc287400640 1 -- 172.21.15.100:0/2705953449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc28815b640 msgr2=0x7fc28814d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:08.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc287400640 1 --2- 172.21.15.100:0/2705953449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc28815b640 0x7fc28814d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:08.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc287400640 1 -- 172.21.15.100:0/2705953449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28809ba30 con 0x7fc2880cc750 2024-04-23T14:35:08.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc28ca00640 1 --2- 172.21.15.100:0/2705953449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc28815b640 0x7fc28814d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:35:08.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc287400640 1 --2- 172.21.15.100:0/2705953449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2880cc750 0x7fc28814d430 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc27400e8d0 tx=0x7fc27400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:08.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc27fe00640 1 -- 172.21.15.100:0/2705953449 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274017070 con 0x7fc2880cc750 2024-04-23T14:35:08.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc27fe00640 1 -- 172.21.15.100:0/2705953449 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc27400c940 con 0x7fc2880cc750 2024-04-23T14:35:08.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc28ea00640 1 -- 172.21.15.100:0/2705953449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc288150e50 con 0x7fc2880cc750 2024-04-23T14:35:08.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc27fe00640 1 -- 172.21.15.100:0/2705953449 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27400cd10 con 0x7fc2880cc750 2024-04-23T14:35:08.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc28ea00640 1 -- 172.21.15.100:0/2705953449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc288150890 con 0x7fc2880cc750 2024-04-23T14:35:08.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc28ea00640 1 -- 172.21.15.100:0/2705953449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc288068ad0 con 0x7fc2880cc750 2024-04-23T14:35:08.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.655+0000 7fc27fe00640 1 -- 172.21.15.100:0/2705953449 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc27401a070 con 0x7fc2880cc750 2024-04-23T14:35:08.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.659+0000 7fc27fe00640 1 --2- 172.21.15.100:0/2705953449 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc268068380 0x7fc26806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:08.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.659+0000 7fc27fe00640 1 -- 172.21.15.100:0/2705953449 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7fc274086880 con 0x7fc2880cc750 2024-04-23T14:35:08.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.659+0000 7fc287e00640 1 --2- 172.21.15.100:0/2705953449 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc268068380 0x7fc26806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:08.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.659+0000 7fc287e00640 1 --2- 172.21.15.100:0/2705953449 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc268068380 0x7fc26806a840 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc2780ae000 tx=0x7fc2780a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:08.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.663+0000 7fc27fe00640 1 -- 172.21.15.100:0/2705953449 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc2740568c0 con 0x7fc2880cc750 2024-04-23T14:35:08.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.819+0000 7fc28ea00640 1 -- 172.21.15.100:0/2705953449 --> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc288074f20 con 0x7fc268068380 2024-04-23T14:35:08.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.819+0000 7fc27fe00640 1 -- 172.21.15.100:0/2705953449 <== mgr.4101 v2:172.21.15.100:6832/2859853957 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13489 (secure 0 0 0) 0x7fc288074f20 con 0x7fc268068380 2024-04-23T14:35:08.820 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:35:08.821 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-04-23T14:35:08.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.823+0000 7fc28ea00640 1 -- 172.21.15.100:0/2705953449 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc268068380 msgr2=0x7fc26806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:08.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.823+0000 7fc28ea00640 1 --2- 172.21.15.100:0/2705953449 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc268068380 0x7fc26806a840 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc2780ae000 tx=0x7fc2780a7070 comp rx=0 tx=0).stop 2024-04-23T14:35:08.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.823+0000 7fc28ea00640 1 -- 172.21.15.100:0/2705953449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2880cc750 msgr2=0x7fc28814d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:08.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.823+0000 7fc28ea00640 1 --2- 172.21.15.100:0/2705953449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2880cc750 0x7fc28814d430 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc27400e8d0 tx=0x7fc27400ed90 comp rx=0 tx=0).stop 2024-04-23T14:35:08.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.823+0000 7fc28ea00640 1 -- 172.21.15.100:0/2705953449 shutdown_connections 2024-04-23T14:35:08.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.823+0000 7fc28ea00640 1 --2- 172.21.15.100:0/2705953449 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc268068380 0x7fc26806a840 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:08.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.823+0000 7fc28ea00640 1 --2- 172.21.15.100:0/2705953449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc28815b640 0x7fc28814d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:08.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.823+0000 7fc28ea00640 1 --2- 172.21.15.100:0/2705953449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2880cc750 0x7fc28814d430 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:08.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.823+0000 7fc28ea00640 1 --2- 172.21.15.100:0/2705953449 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc2880b2a10 0x7fc28809c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:08.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.823+0000 7fc28ea00640 1 -- 172.21.15.100:0/2705953449 >> 172.21.15.100:0/2705953449 conn(0x7fc2880b1e80 msgr2=0x7fc28806c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:08.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.823+0000 7fc28ea00640 1 -- 172.21.15.100:0/2705953449 shutdown_connections 2024-04-23T14:35:08.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.823+0000 7fc28ea00640 1 -- 172.21.15.100:0/2705953449 wait complete. 2024-04-23T14:35:08.841 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-04-23T14:35:08.772263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":125,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":226708,"kb_used_data":6360,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748012,"statfs":{"total":773094113280,"available":772861964288,"internally_reserved":0,"allocated":6512640,"data_stored":3576463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17096,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-71,"num_write_kb":-42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001073"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":237,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-23T14:35:06.634831+0000","last_change":"2024-04-23T14:31:58.314235+0000","last_active":"2024-04-23T14:35:06.634831+0000","last_peered":"2024-04-23T14:35:06.634831+0000","last_clean":"2024-04-23T14:35:06.634831+0000","last_became_active":"2024-04-23T14:31:58.313165+0000","last_became_peered":"2024-04-23T14:31:58.313165+0000","last_unstale":"2024-04-23T14:35:06.634831+0000","last_undegraded":"2024-04-23T14:35:06.634831+0000","last_fullsized":"2024-04-23T14:35:06.634831+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_clean_scrub_stamp":"2024-04-23T14:31:57.140712+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:44.170436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28252,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343588,"statfs":{"total":96636764160,"available":96607834112,"internally_reserved":0,"allocated":1257472,"data_stored":889144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27652,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":647168,"data_stored":292381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27688,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":679936,"data_stored":304733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27672,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":663552,"data_stored":298776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27672,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":663552,"data_stored":298776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574891,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27672,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":663552,"data_stored":298776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27672,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":663552,"data_stored":298776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1273856,"data_stored":895101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:35:08.842 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:35:08.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.979+0000 7fb222c00640 1 Processor -- start 2024-04-23T14:35:08.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.979+0000 7fb222c00640 1 -- start start 2024-04-23T14:35:08.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.979+0000 7fb222c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb21c0cfd30 0x7fb21c162ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:08.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.979+0000 7fb222c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb21c1479e0 0x7fb21c06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:08.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.979+0000 7fb222c00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb21c06bc50 con 0x7fb21c071ce0 2024-04-23T14:35:08.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.979+0000 7fb222c00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fb21c06c580 con 0x7fb21c1479e0 2024-04-23T14:35:08.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.979+0000 7fb222c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb21c06bf30 con 0x7fb21c0cfd30 2024-04-23T14:35:08.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.979+0000 7fb21b400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb21c1479e0 0x7fb21c06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:08.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.979+0000 7fb220c00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fb21c071ce0 0x7fb21c06ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42154/0 (socket says 172.21.15.100:42154) 2024-04-23T14:35:08.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.979+0000 7fb220c00640 1 -- 172.21.15.100:0/1199991446 learned_addr learned my addr 172.21.15.100:0/1199991446 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:08.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.979+0000 7fb21be00640 1 --2- 172.21.15.100:0/1199991446 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb21c0cfd30 0x7fb21c162ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:08.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.979+0000 7fb21b400640 1 -- 172.21.15.100:0/1199991446 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb21c0cfd30 msgr2=0x7fb21c162ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:08.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.979+0000 7fb21b400640 1 --2- 172.21.15.100:0/1199991446 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb21c0cfd30 0x7fb21c162ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:08.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.979+0000 7fb21b400640 1 -- 172.21.15.100:0/1199991446 >> v1:172.21.15.100:6789/0 conn(0x7fb21c071ce0 legacy=0x7fb21c06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:08.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.979+0000 7fb21b400640 1 -- 172.21.15.100:0/1199991446 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb21c06c210 con 0x7fb21c1479e0 2024-04-23T14:35:08.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.983+0000 7fb21b400640 1 --2- 172.21.15.100:0/1199991446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb21c1479e0 0x7fb21c06dc90 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb2080097c0 tx=0x7fb20801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9939c4f6dcef25 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:08.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.983+0000 7fb21aa00640 1 -- 172.21.15.100:0/1199991446 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2080b0070 con 0x7fb21c1479e0 2024-04-23T14:35:08.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.983+0000 7fb21aa00640 1 -- 172.21.15.100:0/1199991446 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2080ab600 con 0x7fb21c1479e0 2024-04-23T14:35:08.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.983+0000 7fb21aa00640 1 -- 172.21.15.100:0/1199991446 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2080ab9d0 con 0x7fb21c1479e0 2024-04-23T14:35:08.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.983+0000 7fb222c00640 1 -- 172.21.15.100:0/1199991446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb21c1479e0 msgr2=0x7fb21c06dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:08.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.983+0000 7fb222c00640 1 --2- 172.21.15.100:0/1199991446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb21c1479e0 0x7fb21c06dc90 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb2080097c0 tx=0x7fb20801c4a0 comp rx=0 tx=0).stop 2024-04-23T14:35:08.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.983+0000 7fb222c00640 1 -- 172.21.15.100:0/1199991446 shutdown_connections 2024-04-23T14:35:08.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.983+0000 7fb222c00640 1 --2- 172.21.15.100:0/1199991446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb21c1479e0 0x7fb21c06dc90 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:08.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.983+0000 7fb222c00640 1 --2- 172.21.15.100:0/1199991446 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb21c0cfd30 0x7fb21c162ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:08.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.983+0000 7fb222c00640 1 -- 172.21.15.100:0/1199991446 >> 172.21.15.100:0/1199991446 conn(0x7fb21c0b1e80 msgr2=0x7fb21c0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:08.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.983+0000 7fb222c00640 1 -- 172.21.15.100:0/1199991446 shutdown_connections 2024-04-23T14:35:08.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.983+0000 7fb222c00640 1 -- 172.21.15.100:0/1199991446 wait complete. 2024-04-23T14:35:08.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb222c00640 1 Processor -- start 2024-04-23T14:35:08.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb222c00640 1 -- start start 2024-04-23T14:35:08.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb222c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb21c071ce0 0x7fb21c099ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:08.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb222c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb21c0cfd30 0x7fb21c09de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:08.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb222c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb21c1479e0 0x7fb21c160db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:08.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb222c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c16be80 con 0x7fb21c071ce0 2024-04-23T14:35:08.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb222c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c06e3e0 con 0x7fb21c0cfd30 2024-04-23T14:35:08.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb222c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb21c16c000 con 0x7fb21c1479e0 2024-04-23T14:35:08.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb21be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb21c071ce0 0x7fb21c099ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:08.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb21be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb21c071ce0 0x7fb21c099ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40928/0 (socket says 172.21.15.100:40928) 2024-04-23T14:35:08.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb21be00640 1 -- 172.21.15.100:0/3452379133 learned_addr learned my addr 172.21.15.100:0/3452379133 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:08.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb220c00640 1 --2- 172.21.15.100:0/3452379133 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb21c1479e0 0x7fb21c160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:08.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb21b400640 1 --2- 172.21.15.100:0/3452379133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb21c0cfd30 0x7fb21c09de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:08.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb21be00640 1 -- 172.21.15.100:0/3452379133 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb21c1479e0 msgr2=0x7fb21c160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:08.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb21be00640 1 --2- 172.21.15.100:0/3452379133 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb21c1479e0 0x7fb21c160db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:08.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb21be00640 1 -- 172.21.15.100:0/3452379133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb21c0cfd30 msgr2=0x7fb21c09de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:08.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb21be00640 1 --2- 172.21.15.100:0/3452379133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb21c0cfd30 0x7fb21c09de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:08.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb21be00640 1 -- 172.21.15.100:0/3452379133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb21c157050 con 0x7fb21c071ce0 2024-04-23T14:35:08.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb21b400640 1 --2- 172.21.15.100:0/3452379133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb21c0cfd30 0x7fb21c09de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:35:08.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb21be00640 1 --2- 172.21.15.100:0/3452379133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb21c071ce0 0x7fb21c099ac0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00d830 tx=0x7fb20c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:08.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb213e00640 1 -- 172.21.15.100:0/3452379133 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c00b6c0 con 0x7fb21c071ce0 2024-04-23T14:35:08.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb213e00640 1 -- 172.21.15.100:0/3452379133 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb20c019070 con 0x7fb21c071ce0 2024-04-23T14:35:08.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb222c00640 1 -- 172.21.15.100:0/3452379133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb21c15f4c0 con 0x7fb21c071ce0 2024-04-23T14:35:08.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb213e00640 1 -- 172.21.15.100:0/3452379133 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c00ba60 con 0x7fb21c071ce0 2024-04-23T14:35:08.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.987+0000 7fb222c00640 1 -- 172.21.15.100:0/3452379133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb21c1463f0 con 0x7fb21c071ce0 2024-04-23T14:35:08.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.991+0000 7fb222c00640 1 -- 172.21.15.100:0/3452379133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb21c068ad0 con 0x7fb21c071ce0 2024-04-23T14:35:08.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.991+0000 7fb213e00640 1 -- 172.21.15.100:0/3452379133 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb20c01e460 con 0x7fb21c071ce0 2024-04-23T14:35:08.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.991+0000 7fb213e00640 1 --2- 172.21.15.100:0/3452379133 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb1fc068380 0x7fb1fc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:08.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.991+0000 7fb213e00640 1 -- 172.21.15.100:0/3452379133 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7fb20c086fe0 con 0x7fb21c071ce0 2024-04-23T14:35:08.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.991+0000 7fb21b400640 1 --2- 172.21.15.100:0/3452379133 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb1fc068380 0x7fb1fc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:08.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.991+0000 7fb21b400640 1 --2- 172.21.15.100:0/3452379133 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb1fc068380 0x7fb1fc06a840 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb2080ad000 tx=0x7fb2080a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:08.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:08.995+0000 7fb213e00640 1 -- 172.21.15.100:0/3452379133 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb20c057620 con 0x7fb21c071ce0 2024-04-23T14:35:09.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.147+0000 7fb222c00640 1 -- 172.21.15.100:0/3452379133 --> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb21c162b90 con 0x7fb1fc068380 2024-04-23T14:35:09.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.151+0000 7fb213e00640 1 -- 172.21.15.100:0/3452379133 <== mgr.4101 v2:172.21.15.100:6832/2859853957 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13489 (secure 0 0 0) 0x7fb21c162b90 con 0x7fb1fc068380 2024-04-23T14:35:09.151 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:35:09.151 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-04-23T14:35:09.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.151+0000 7fb222c00640 1 -- 172.21.15.100:0/3452379133 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb1fc068380 msgr2=0x7fb1fc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:09.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.151+0000 7fb222c00640 1 --2- 172.21.15.100:0/3452379133 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb1fc068380 0x7fb1fc06a840 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb2080ad000 tx=0x7fb2080a6000 comp rx=0 tx=0).stop 2024-04-23T14:35:09.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.151+0000 7fb222c00640 1 -- 172.21.15.100:0/3452379133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb21c071ce0 msgr2=0x7fb21c099ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:09.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.151+0000 7fb222c00640 1 --2- 172.21.15.100:0/3452379133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb21c071ce0 0x7fb21c099ac0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00d830 tx=0x7fb20c00dcf0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.151+0000 7fb222c00640 1 -- 172.21.15.100:0/3452379133 shutdown_connections 2024-04-23T14:35:09.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.151+0000 7fb222c00640 1 --2- 172.21.15.100:0/3452379133 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb1fc068380 0x7fb1fc06a840 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.151+0000 7fb222c00640 1 --2- 172.21.15.100:0/3452379133 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb21c1479e0 0x7fb21c160db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.151+0000 7fb222c00640 1 --2- 172.21.15.100:0/3452379133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb21c0cfd30 0x7fb21c09de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.151+0000 7fb222c00640 1 --2- 172.21.15.100:0/3452379133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb21c071ce0 0x7fb21c099ac0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.151+0000 7fb222c00640 1 -- 172.21.15.100:0/3452379133 >> 172.21.15.100:0/3452379133 conn(0x7fb21c0b1e80 msgr2=0x7fb21c06e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:09.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.151+0000 7fb222c00640 1 -- 172.21.15.100:0/3452379133 shutdown_connections 2024-04-23T14:35:09.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.151+0000 7fb222c00640 1 -- 172.21.15.100:0/3452379133 wait complete. 2024-04-23T14:35:09.172 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-04-23T14:35:08.772263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":125,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":226708,"kb_used_data":6360,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748012,"statfs":{"total":773094113280,"available":772861964288,"internally_reserved":0,"allocated":6512640,"data_stored":3576463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17096,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-71,"num_write_kb":-42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001073"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":237,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-23T14:35:06.634831+0000","last_change":"2024-04-23T14:31:58.314235+0000","last_active":"2024-04-23T14:35:06.634831+0000","last_peered":"2024-04-23T14:35:06.634831+0000","last_clean":"2024-04-23T14:35:06.634831+0000","last_became_active":"2024-04-23T14:31:58.313165+0000","last_became_peered":"2024-04-23T14:31:58.313165+0000","last_unstale":"2024-04-23T14:35:06.634831+0000","last_undegraded":"2024-04-23T14:35:06.634831+0000","last_fullsized":"2024-04-23T14:35:06.634831+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_clean_scrub_stamp":"2024-04-23T14:31:57.140712+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:44.170436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28252,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343588,"statfs":{"total":96636764160,"available":96607834112,"internally_reserved":0,"allocated":1257472,"data_stored":889144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27652,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":647168,"data_stored":292381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27688,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":679936,"data_stored":304733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27672,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":663552,"data_stored":298776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27672,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":663552,"data_stored":298776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574891,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27672,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":663552,"data_stored":298776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27672,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":663552,"data_stored":298776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1273856,"data_stored":895101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:35:09.173 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-23T14:35:09.173 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:35:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.307+0000 7f0b12c00640 1 Processor -- start 2024-04-23T14:35:09.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.307+0000 7f0b12c00640 1 -- start start 2024-04-23T14:35:09.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.307+0000 7f0b12c00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f0b0c0cea60 0x7f0b0c0d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:09.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.307+0000 7f0b12c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0b0c0cd460 0x7f0b0c0b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:09.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.307+0000 7f0b12c00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b0c0b1f90 con 0x7f0b0c06d450 2024-04-23T14:35:09.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.307+0000 7f0b12c00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f0b0c0b2270 con 0x7f0b0c0cea60 2024-04-23T14:35:09.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.307+0000 7f0b12c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0b0c0b2550 con 0x7f0b0c0cd460 2024-04-23T14:35:09.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.307+0000 7f0b0be00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f0b0c06d450 0x7f0b0c06d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42166/0 (socket says 172.21.15.100:42166) 2024-04-23T14:35:09.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.307+0000 7f0b0be00640 1 -- 172.21.15.100:0/1688988772 learned_addr learned my addr 172.21.15.100:0/1688988772 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:09.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.307+0000 7f0b0b400640 1 --2- 172.21.15.100:0/1688988772 >> v2:172.21.15.151:3300/0 conn(0x7f0b0c0cea60 0x7f0b0c0d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:09.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.307+0000 7f0b10c00640 1 --2- 172.21.15.100:0/1688988772 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0b0c0cd460 0x7f0b0c0b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:09.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.307+0000 7f0b0aa00640 1 -- 172.21.15.100:0/1688988772 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3755635611 0 0) 0x7f0b0c0b1f90 con 0x7f0b0c06d450 2024-04-23T14:35:09.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.307+0000 7f0b0aa00640 1 -- 172.21.15.100:0/1688988772 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0af0003530 con 0x7f0b0c06d450 2024-04-23T14:35:09.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.307+0000 7f0b0b400640 1 -- 172.21.15.100:0/1688988772 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0b0c0cd460 msgr2=0x7f0b0c0b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:09.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.307+0000 7f0b0b400640 1 --2- 172.21.15.100:0/1688988772 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0b0c0cd460 0x7f0b0c0b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.307+0000 7f0b0b400640 1 -- 172.21.15.100:0/1688988772 >> v1:172.21.15.100:6789/0 conn(0x7f0b0c06d450 legacy=0x7f0b0c06d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:09.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.307+0000 7f0b0b400640 1 -- 172.21.15.100:0/1688988772 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0c0b2830 con 0x7f0b0c0cea60 2024-04-23T14:35:09.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.311+0000 7f0b0b400640 1 --2- 172.21.15.100:0/1688988772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b0c0cea60 0x7f0b0c0d0300 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0af80097c0 tx=0x7f0af801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66f30e292282c4b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:09.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.311+0000 7f0b0aa00640 1 -- 172.21.15.100:0/1688988772 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af80b0070 con 0x7f0b0c0cea60 2024-04-23T14:35:09.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.311+0000 7f0b0aa00640 1 -- 172.21.15.100:0/1688988772 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0af80ab600 con 0x7f0b0c0cea60 2024-04-23T14:35:09.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.311+0000 7f0b0aa00640 1 -- 172.21.15.100:0/1688988772 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af80ab9d0 con 0x7f0b0c0cea60 2024-04-23T14:35:09.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.311+0000 7f0b12c00640 1 -- 172.21.15.100:0/1688988772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b0c0cea60 msgr2=0x7f0b0c0d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:09.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.311+0000 7f0b12c00640 1 --2- 172.21.15.100:0/1688988772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b0c0cea60 0x7f0b0c0d0300 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0af80097c0 tx=0x7f0af801c410 comp rx=0 tx=0).stop 2024-04-23T14:35:09.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.311+0000 7f0b12c00640 1 -- 172.21.15.100:0/1688988772 shutdown_connections 2024-04-23T14:35:09.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.311+0000 7f0b12c00640 1 --2- 172.21.15.100:0/1688988772 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0b0c0cd460 0x7f0b0c0b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.311+0000 7f0b12c00640 1 --2- 172.21.15.100:0/1688988772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b0c0cea60 0x7f0b0c0d0300 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.311+0000 7f0b12c00640 1 -- 172.21.15.100:0/1688988772 >> 172.21.15.100:0/1688988772 conn(0x7f0b0c0cee80 msgr2=0x7f0b0c0cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:09.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.311+0000 7f0b12c00640 1 -- 172.21.15.100:0/1688988772 shutdown_connections 2024-04-23T14:35:09.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.311+0000 7f0b12c00640 1 -- 172.21.15.100:0/1688988772 wait complete. 2024-04-23T14:35:09.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b12c00640 1 Processor -- start 2024-04-23T14:35:09.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b12c00640 1 -- start start 2024-04-23T14:35:09.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b12c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0b0c06d450 0x7f0b0c141b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:09.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b12c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b0c0cd460 0x7f0b0c0c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:09.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b12c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b0c0cea60 0x7f0b0c15b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:09.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b12c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c16b740 con 0x7f0b0c0cea60 2024-04-23T14:35:09.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b12c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c16b5c0 con 0x7f0b0c0cd460 2024-04-23T14:35:09.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b12c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0b0c16b8c0 con 0x7f0b0c06d450 2024-04-23T14:35:09.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b10c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b0c0cea60 0x7f0b0c15b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:09.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b0be00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0b0c06d450 0x7f0b0c141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:09.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b0b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b0c0cd460 0x7f0b0c0c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:09.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b10c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b0c0cea60 0x7f0b0c15b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40930/0 (socket says 172.21.15.100:40930) 2024-04-23T14:35:09.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b10c00640 1 -- 172.21.15.100:0/4260688202 learned_addr learned my addr 172.21.15.100:0/4260688202 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:09.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b10c00640 1 -- 172.21.15.100:0/4260688202 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0b0c06d450 msgr2=0x7f0b0c141b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:09.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b10c00640 1 --2- 172.21.15.100:0/4260688202 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0b0c06d450 0x7f0b0c141b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b10c00640 1 -- 172.21.15.100:0/4260688202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b0c0cd460 msgr2=0x7f0b0c0c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:09.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b10c00640 1 --2- 172.21.15.100:0/4260688202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b0c0cd460 0x7f0b0c0c7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b10c00640 1 -- 172.21.15.100:0/4260688202 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0c142090 con 0x7f0b0c0cea60 2024-04-23T14:35:09.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b10c00640 1 --2- 172.21.15.100:0/4260688202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b0c0cea60 0x7f0b0c15b1e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400e7f0 tx=0x7f0b0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:09.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b03e00640 1 -- 172.21.15.100:0/4260688202 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04017070 con 0x7f0b0c0cea60 2024-04-23T14:35:09.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b03e00640 1 -- 172.21.15.100:0/4260688202 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b0400c940 con 0x7f0b0c0cea60 2024-04-23T14:35:09.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b12c00640 1 -- 172.21.15.100:0/4260688202 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0c0c94d0 con 0x7f0b0c0cea60 2024-04-23T14:35:09.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b03e00640 1 -- 172.21.15.100:0/4260688202 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0400eea0 con 0x7f0b0c0cea60 2024-04-23T14:35:09.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.315+0000 7f0b12c00640 1 -- 172.21.15.100:0/4260688202 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b0c0cb180 con 0x7f0b0c0cea60 2024-04-23T14:35:09.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.319+0000 7f0b12c00640 1 -- 172.21.15.100:0/4260688202 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b0c068ad0 con 0x7f0b0c0cea60 2024-04-23T14:35:09.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.319+0000 7f0b03e00640 1 -- 172.21.15.100:0/4260688202 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0b0401a070 con 0x7f0b0c0cea60 2024-04-23T14:35:09.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.319+0000 7f0b03e00640 1 --2- 172.21.15.100:0/4260688202 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0aec068380 0x7f0aec06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:09.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.319+0000 7f0b03e00640 1 -- 172.21.15.100:0/4260688202 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f0b04086690 con 0x7f0b0c0cea60 2024-04-23T14:35:09.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.319+0000 7f0b0be00640 1 --2- 172.21.15.100:0/4260688202 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0aec068380 0x7f0aec06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:09.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.319+0000 7f0b0be00640 1 --2- 172.21.15.100:0/4260688202 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0aec068380 0x7f0aec06a840 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0afc005c10 tx=0x7f0afc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:09.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.323+0000 7f0b03e00640 1 -- 172.21.15.100:0/4260688202 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0b0c142090 con 0x7f0b0c0cea60 2024-04-23T14:35:09.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.479+0000 7f0b12c00640 1 -- 172.21.15.100:0/4260688202 --> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0b0c0caf50 con 0x7f0aec068380 2024-04-23T14:35:09.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.479+0000 7f0b03e00640 1 -- 172.21.15.100:0/4260688202 <== mgr.4101 v2:172.21.15.100:6832/2859853957 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13489 (secure 0 0 0) 0x7f0b0c0caf50 con 0x7f0aec068380 2024-04-23T14:35:09.480 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:35:09.480 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-04-23T14:35:09.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.479+0000 7f0b12c00640 1 -- 172.21.15.100:0/4260688202 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0aec068380 msgr2=0x7f0aec06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:09.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.479+0000 7f0b12c00640 1 --2- 172.21.15.100:0/4260688202 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0aec068380 0x7f0aec06a840 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0afc005c10 tx=0x7f0afc009040 comp rx=0 tx=0).stop 2024-04-23T14:35:09.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.479+0000 7f0b12c00640 1 -- 172.21.15.100:0/4260688202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b0c0cea60 msgr2=0x7f0b0c15b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:09.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.479+0000 7f0b12c00640 1 --2- 172.21.15.100:0/4260688202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b0c0cea60 0x7f0b0c15b1e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400e7f0 tx=0x7f0b0400ecb0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.479+0000 7f0b12c00640 1 -- 172.21.15.100:0/4260688202 shutdown_connections 2024-04-23T14:35:09.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.479+0000 7f0b12c00640 1 --2- 172.21.15.100:0/4260688202 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f0aec068380 0x7f0aec06a840 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.479+0000 7f0b12c00640 1 --2- 172.21.15.100:0/4260688202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b0c0cea60 0x7f0b0c15b1e0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.479+0000 7f0b12c00640 1 --2- 172.21.15.100:0/4260688202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b0c0cd460 0x7f0b0c0c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.479+0000 7f0b12c00640 1 --2- 172.21.15.100:0/4260688202 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0b0c06d450 0x7f0b0c141b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.479+0000 7f0b12c00640 1 -- 172.21.15.100:0/4260688202 >> 172.21.15.100:0/4260688202 conn(0x7f0b0c0cee80 msgr2=0x7f0b0c06f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:09.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.483+0000 7f0b12c00640 1 -- 172.21.15.100:0/4260688202 shutdown_connections 2024-04-23T14:35:09.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.483+0000 7f0b12c00640 1 -- 172.21.15.100:0/4260688202 wait complete. 2024-04-23T14:35:09.500 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-04-23T14:35:08.772263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":125,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":226708,"kb_used_data":6360,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748012,"statfs":{"total":773094113280,"available":772861964288,"internally_reserved":0,"allocated":6512640,"data_stored":3576463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17096,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-71,"num_write_kb":-42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001073"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":237,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-23T14:35:06.634831+0000","last_change":"2024-04-23T14:31:58.314235+0000","last_active":"2024-04-23T14:35:06.634831+0000","last_peered":"2024-04-23T14:35:06.634831+0000","last_clean":"2024-04-23T14:35:06.634831+0000","last_became_active":"2024-04-23T14:31:58.313165+0000","last_became_peered":"2024-04-23T14:31:58.313165+0000","last_unstale":"2024-04-23T14:35:06.634831+0000","last_undegraded":"2024-04-23T14:35:06.634831+0000","last_fullsized":"2024-04-23T14:35:06.634831+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_clean_scrub_stamp":"2024-04-23T14:31:57.140712+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:44.170436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28252,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343588,"statfs":{"total":96636764160,"available":96607834112,"internally_reserved":0,"allocated":1257472,"data_stored":889144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27652,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":647168,"data_stored":292381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27688,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":679936,"data_stored":304733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27672,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":663552,"data_stored":298776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27672,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":663552,"data_stored":298776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574891,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27672,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":663552,"data_stored":298776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27672,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":663552,"data_stored":298776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1273856,"data_stored":895101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:35:09.501 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:35:09.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.635+0000 7fd4fb200640 1 Processor -- start 2024-04-23T14:35:09.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.635+0000 7fd4fb200640 1 -- start start 2024-04-23T14:35:09.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.635+0000 7fd4fb200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd4f40b2a10 0x7fd4f40adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:09.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.635+0000 7fd4fb200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd4f4073610 0x7fd4f406df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:09.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.635+0000 7fd4fb200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4f40af080 con 0x7fd4f415b640 2024-04-23T14:35:09.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.635+0000 7fd4fb200640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fd4f40afbc0 con 0x7fd4f4073610 2024-04-23T14:35:09.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.635+0000 7fd4fb200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd4f415ba20 con 0x7fd4f40b2a10 2024-04-23T14:35:09.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.639+0000 7fd4f9200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd4f4073610 0x7fd4f406df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:09.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.639+0000 7fd4eb400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd4f40b2a10 0x7fd4f40adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:09.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.639+0000 7fd4f9200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd4f4073610 0x7fd4f406df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:48732/0 (socket says 172.21.15.100:48732) 2024-04-23T14:35:09.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.639+0000 7fd4f9200640 1 -- 172.21.15.100:0/2288169730 learned_addr learned my addr 172.21.15.100:0/2288169730 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:09.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.639+0000 7fd4f3e00640 1 --1- 172.21.15.100:0/2288169730 >> v1:172.21.15.151:6789/0 conn(0x7fd4f415b640 0x7fd4f4072f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:44830/0 (socket says 172.21.15.100:44830) 2024-04-23T14:35:09.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.639+0000 7fd4f9200640 1 -- 172.21.15.100:0/2288169730 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd4f40b2a10 msgr2=0x7fd4f40adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:09.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.639+0000 7fd4f9200640 1 --2- 172.21.15.100:0/2288169730 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd4f40b2a10 0x7fd4f40adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.639+0000 7fd4f9200640 1 -- 172.21.15.100:0/2288169730 >> v1:172.21.15.151:6789/0 conn(0x7fd4f415b640 legacy=0x7fd4f4072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:09.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.639+0000 7fd4f9200640 1 -- 172.21.15.100:0/2288169730 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f40af320 con 0x7fd4f4073610 2024-04-23T14:35:09.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.639+0000 7fd4eb400640 1 --2- 172.21.15.100:0/2288169730 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd4f40b2a10 0x7fd4f40adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:35:09.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.639+0000 7fd4f9200640 1 --2- 172.21.15.100:0/2288169730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4f4073610 0x7fd4f406df70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009870 tx=0x7fd4ec01e740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8b0953cdf6cb8fb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:09.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4f3400640 1 -- 172.21.15.100:0/2288169730 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec0a8710 con 0x7fd4f4073610 2024-04-23T14:35:09.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4f3400640 1 -- 172.21.15.100:0/2288169730 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4ec0a88b0 con 0x7fd4f4073610 2024-04-23T14:35:09.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4f3400640 1 -- 172.21.15.100:0/2288169730 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec0a8c80 con 0x7fd4f4073610 2024-04-23T14:35:09.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4fb200640 1 -- 172.21.15.100:0/2288169730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4f4073610 msgr2=0x7fd4f406df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:09.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4fb200640 1 --2- 172.21.15.100:0/2288169730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4f4073610 0x7fd4f406df70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009870 tx=0x7fd4ec01e740 comp rx=0 tx=0).stop 2024-04-23T14:35:09.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4fb200640 1 -- 172.21.15.100:0/2288169730 shutdown_connections 2024-04-23T14:35:09.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4fb200640 1 --2- 172.21.15.100:0/2288169730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4f4073610 0x7fd4f406df70 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4fb200640 1 --2- 172.21.15.100:0/2288169730 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd4f40b2a10 0x7fd4f40adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4fb200640 1 -- 172.21.15.100:0/2288169730 >> 172.21.15.100:0/2288169730 conn(0x7fd4f40b1e80 msgr2=0x7fd4f40b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:09.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4fb200640 1 -- 172.21.15.100:0/2288169730 shutdown_connections 2024-04-23T14:35:09.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4fb200640 1 -- 172.21.15.100:0/2288169730 wait complete. 2024-04-23T14:35:09.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4fb200640 1 Processor -- start 2024-04-23T14:35:09.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4fb200640 1 -- start start 2024-04-23T14:35:09.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4fb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f4073610 0x7fd4f4147d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:09.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4fb200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd4f40b2a10 0x7fd4f415ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:09.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4fb200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4f415b640 0x7fd4f4157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:09.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4fb200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f406f090 con 0x7fd4f4073610 2024-04-23T14:35:09.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4fb200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f406ef10 con 0x7fd4f415b640 2024-04-23T14:35:09.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4fb200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd4f406f210 con 0x7fd4f40b2a10 2024-04-23T14:35:09.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4f3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f4073610 0x7fd4f4147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:09.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4f3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f4073610 0x7fd4f4147d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40936/0 (socket says 172.21.15.100:40936) 2024-04-23T14:35:09.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4f3e00640 1 -- 172.21.15.100:0/3306996523 learned_addr learned my addr 172.21.15.100:0/3306996523 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:09.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4f9200640 1 --2- 172.21.15.100:0/3306996523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4f415b640 0x7fd4f4157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:09.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.643+0000 7fd4eb400640 1 --2- 172.21.15.100:0/3306996523 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd4f40b2a10 0x7fd4f415ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:09.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.647+0000 7fd4f3e00640 1 -- 172.21.15.100:0/3306996523 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd4f40b2a10 msgr2=0x7fd4f415ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:09.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.647+0000 7fd4f3e00640 1 --2- 172.21.15.100:0/3306996523 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd4f40b2a10 0x7fd4f415ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.647+0000 7fd4f3e00640 1 -- 172.21.15.100:0/3306996523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4f415b640 msgr2=0x7fd4f4157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:09.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.647+0000 7fd4f3e00640 1 --2- 172.21.15.100:0/3306996523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4f415b640 0x7fd4f4157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.647+0000 7fd4f3e00640 1 -- 172.21.15.100:0/3306996523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f4153230 con 0x7fd4f4073610 2024-04-23T14:35:09.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.647+0000 7fd4f3e00640 1 --2- 172.21.15.100:0/3306996523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f4073610 0x7fd4f4147d00 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400c730 tx=0x7fd4e400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:09.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.647+0000 7fd4f0c00640 1 -- 172.21.15.100:0/3306996523 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4017070 con 0x7fd4f4073610 2024-04-23T14:35:09.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.647+0000 7fd4f0c00640 1 -- 172.21.15.100:0/3306996523 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4e400ce40 con 0x7fd4f4073610 2024-04-23T14:35:09.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.647+0000 7fd4fb200640 1 -- 172.21.15.100:0/3306996523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f4148ae0 con 0x7fd4f4073610 2024-04-23T14:35:09.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.647+0000 7fd4f0c00640 1 -- 172.21.15.100:0/3306996523 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e400a860 con 0x7fd4f4073610 2024-04-23T14:35:09.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.647+0000 7fd4fb200640 1 -- 172.21.15.100:0/3306996523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f4071e20 con 0x7fd4f4073610 2024-04-23T14:35:09.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.647+0000 7fd4fb200640 1 -- 172.21.15.100:0/3306996523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f4068ad0 con 0x7fd4f4073610 2024-04-23T14:35:09.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.647+0000 7fd4f0c00640 1 -- 172.21.15.100:0/3306996523 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd4e401a070 con 0x7fd4f4073610 2024-04-23T14:35:09.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.647+0000 7fd4f0c00640 1 --2- 172.21.15.100:0/3306996523 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd4d00683b0 0x7fd4d006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:09.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.647+0000 7fd4f0c00640 1 -- 172.21.15.100:0/3306996523 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7fd4e4085d80 con 0x7fd4f4073610 2024-04-23T14:35:09.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.647+0000 7fd4eb400640 1 --2- 172.21.15.100:0/3306996523 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd4d00683b0 0x7fd4d006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:09.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.651+0000 7fd4eb400640 1 --2- 172.21.15.100:0/3306996523 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd4d00683b0 0x7fd4d006a870 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009580 tx=0x7fd4dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:09.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.651+0000 7fd4f0c00640 1 -- 172.21.15.100:0/3306996523 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd4e4055e50 con 0x7fd4f4073610 2024-04-23T14:35:09.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.811+0000 7fd4fb200640 1 -- 172.21.15.100:0/3306996523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4f414b740 con 0x7fd4f4073610 2024-04-23T14:35:09.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.811+0000 7fd4f0c00640 1 -- 172.21.15.100:0/3306996523 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+13305 (secure 0 0 0) 0x7fd4e405a150 con 0x7fd4f4073610 2024-04-23T14:35:09.813 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:35:09.813 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":56,"fsid":"167d2af1-01c4-4f87-8bce-0de22791f823","created":"2024-04-23T14:31:43.678435+0000","modified":"2024-04-23T14:35:06.433457+0000","last_up_change":"2024-04-23T14:31:56.138373+0000","last_in_change":"2024-04-23T14:31:46.195181+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:56.759842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff0dca4f-928b-498f-aae5-8909e2f05df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6801","nonce":43918908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6803","nonce":43918908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6807","nonce":43918908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":43918908},{"type":"v1","addr":"172.21.15.100:6805","nonce":43918908}]},"public_addr":"172.21.15.100:6801/43918908","cluster_addr":"172.21.15.100:6803/43918908","heartbeat_back_addr":"172.21.15.100:6807/43918908","heartbeat_front_addr":"172.21.15.100:6805/43918908","state":["exists","up"]},{"osd":1,"uuid":"c313da00-b031-49e3-822b-727da06be7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6809","nonce":3988727915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6811","nonce":3988727915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6815","nonce":3988727915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3988727915},{"type":"v1","addr":"172.21.15.100:6813","nonce":3988727915}]},"public_addr":"172.21.15.100:6809/3988727915","cluster_addr":"172.21.15.100:6811/3988727915","heartbeat_back_addr":"172.21.15.100:6815/3988727915","heartbeat_front_addr":"172.21.15.100:6813/3988727915","state":["exists","up"]},{"osd":2,"uuid":"ec06612c-20c6-4c09-9f2b-2380186182f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6817","nonce":1257826180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6819","nonce":1257826180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6823","nonce":1257826180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1257826180},{"type":"v1","addr":"172.21.15.100:6821","nonce":1257826180}]},"public_addr":"172.21.15.100:6817/1257826180","cluster_addr":"172.21.15.100:6819/1257826180","heartbeat_back_addr":"172.21.15.100:6823/1257826180","heartbeat_front_addr":"172.21.15.100:6821/1257826180","state":["exists","up"]},{"osd":3,"uuid":"6066e9dc-0909-4b36-8a4c-d03c2c2e08e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6825","nonce":1142032886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6827","nonce":1142032886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6831","nonce":1142032886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1142032886},{"type":"v1","addr":"172.21.15.100:6829","nonce":1142032886}]},"public_addr":"172.21.15.100:6825/1142032886","cluster_addr":"172.21.15.100:6827/1142032886","heartbeat_back_addr":"172.21.15.100:6831/1142032886","heartbeat_front_addr":"172.21.15.100:6829/1142032886","state":["exists","up"]},{"osd":4,"uuid":"0eb3f089-4f39-449f-8598-c48b0dee081b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6801","nonce":4191386446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6803","nonce":4191386446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6807","nonce":4191386446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":4191386446},{"type":"v1","addr":"172.21.15.151:6805","nonce":4191386446}]},"public_addr":"172.21.15.151:6801/4191386446","cluster_addr":"172.21.15.151:6803/4191386446","heartbeat_back_addr":"172.21.15.151:6807/4191386446","heartbeat_front_addr":"172.21.15.151:6805/4191386446","state":["exists","up"]},{"osd":5,"uuid":"8fd37274-87b3-464b-b6d2-d45a98d50b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6809","nonce":4283546311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6811","nonce":4283546311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6815","nonce":4283546311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4283546311},{"type":"v1","addr":"172.21.15.151:6813","nonce":4283546311}]},"public_addr":"172.21.15.151:6809/4283546311","cluster_addr":"172.21.15.151:6811/4283546311","heartbeat_back_addr":"172.21.15.151:6815/4283546311","heartbeat_front_addr":"172.21.15.151:6813/4283546311","state":["exists","up"]},{"osd":6,"uuid":"b05d77f3-e8aa-499c-b8a7-e1e53efb8ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6825","nonce":547449762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6827","nonce":547449762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6831","nonce":547449762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":547449762},{"type":"v1","addr":"172.21.15.151:6829","nonce":547449762}]},"public_addr":"172.21.15.151:6825/547449762","cluster_addr":"172.21.15.151:6827/547449762","heartbeat_back_addr":"172.21.15.151:6831/547449762","heartbeat_front_addr":"172.21.15.151:6829/547449762","state":["exists","up"]},{"osd":7,"uuid":"015597f2-4f97-492d-b821-56111164ed6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6817","nonce":389311850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6819","nonce":389311850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6823","nonce":389311850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":389311850},{"type":"v1","addr":"172.21.15.151:6821","nonce":389311850}]},"public_addr":"172.21.15.151:6817/389311850","cluster_addr":"172.21.15.151:6819/389311850","heartbeat_back_addr":"172.21.15.151:6823/389311850","heartbeat_front_addr":"172.21.15.151:6821/389311850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.798520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.799203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.054539+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:52.084663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.789931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.940038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:51.908520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6837/608349550":"2024-04-24T14:35:03.423331+0000","172.21.15.100:6835/608349550":"2024-04-24T14:35:03.423331+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:35:09.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.815+0000 7fd4fb200640 1 -- 172.21.15.100:0/3306996523 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd4d00683b0 msgr2=0x7fd4d006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:09.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.815+0000 7fd4fb200640 1 --2- 172.21.15.100:0/3306996523 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd4d00683b0 0x7fd4d006a870 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009580 tx=0x7fd4dc008040 comp rx=0 tx=0).stop 2024-04-23T14:35:09.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.815+0000 7fd4fb200640 1 -- 172.21.15.100:0/3306996523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f4073610 msgr2=0x7fd4f4147d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:09.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.815+0000 7fd4fb200640 1 --2- 172.21.15.100:0/3306996523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f4073610 0x7fd4f4147d00 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400c730 tx=0x7fd4e400cbf0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.815+0000 7fd4fb200640 1 -- 172.21.15.100:0/3306996523 shutdown_connections 2024-04-23T14:35:09.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.815+0000 7fd4fb200640 1 --2- 172.21.15.100:0/3306996523 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fd4d00683b0 0x7fd4d006a870 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.815+0000 7fd4fb200640 1 --2- 172.21.15.100:0/3306996523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4f415b640 0x7fd4f4157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.815+0000 7fd4fb200640 1 --2- 172.21.15.100:0/3306996523 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd4f40b2a10 0x7fd4f415ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.815+0000 7fd4fb200640 1 --2- 172.21.15.100:0/3306996523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f4073610 0x7fd4f4147d00 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:09.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.815+0000 7fd4fb200640 1 -- 172.21.15.100:0/3306996523 >> 172.21.15.100:0/3306996523 conn(0x7fd4f40b1e80 msgr2=0x7fd4f40b1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:09.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.815+0000 7fd4fb200640 1 -- 172.21.15.100:0/3306996523 shutdown_connections 2024-04-23T14:35:09.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:09.815+0000 7fd4fb200640 1 -- 172.21.15.100:0/3306996523 wait complete. 2024-04-23T14:35:10.836 INFO:tasks.ceph:Scrubbing osd.0 2024-04-23T14:35:10.836 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-23T14:35:10.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.979+0000 7ff6bda00640 1 Processor -- start 2024-04-23T14:35:10.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.979+0000 7ff6bda00640 1 -- start start 2024-04-23T14:35:10.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.983+0000 7ff6bda00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff6b806c6c0 0x7ff6b80ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:10.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.983+0000 7ff6bda00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6b806dd60 con 0x7ff6b80ce620 2024-04-23T14:35:10.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.983+0000 7ff6bda00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6b80d09c0 con 0x7ff6b806f8c0 2024-04-23T14:35:10.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.983+0000 7ff6bda00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff6b80ad050 con 0x7ff6b806c6c0 2024-04-23T14:35:10.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.983+0000 7ff6ae000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff6b806c6c0 0x7ff6b80ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:10.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.983+0000 7ff6b7400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff6b80ce620 0x7ff6b80add80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42170/0 (socket says 172.21.15.100:42170) 2024-04-23T14:35:10.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.983+0000 7ff6b7400640 1 -- 172.21.15.100:0/2860697505 learned_addr learned my addr 172.21.15.100:0/2860697505 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:10.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.983+0000 7ff6ae000640 1 -- 172.21.15.100:0/2860697505 >> v1:172.21.15.151:6789/0 conn(0x7ff6b806f8c0 legacy=0x7ff6b80d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:10.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.983+0000 7ff6ae000640 1 -- 172.21.15.100:0/2860697505 >> v1:172.21.15.100:6789/0 conn(0x7ff6b80ce620 legacy=0x7ff6b80add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:10.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.983+0000 7ff6ae000640 1 -- 172.21.15.100:0/2860697505 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b80ad2a0 con 0x7ff6b806c6c0 2024-04-23T14:35:10.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.983+0000 7ff6b6000640 1 -- 172.21.15.100:0/2860697505 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2106132495 0 0) 0x7ff6b00014a0 con 0x7ff6b80ce620 2024-04-23T14:35:10.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.987+0000 7ff6ae000640 1 --2- 172.21.15.100:0/2860697505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6b806c6c0 0x7ff6b80ce0e0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff6a00097c0 tx=0x7ff6a001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed5b0c7633ffdf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:10.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.987+0000 7ff6b6000640 1 -- 172.21.15.100:0/2860697505 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a00b0070 con 0x7ff6b806c6c0 2024-04-23T14:35:10.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.987+0000 7ff6b6000640 1 -- 172.21.15.100:0/2860697505 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6a00ab600 con 0x7ff6b806c6c0 2024-04-23T14:35:10.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.987+0000 7ff6b6000640 1 -- 172.21.15.100:0/2860697505 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a00ab9d0 con 0x7ff6b806c6c0 2024-04-23T14:35:10.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.987+0000 7ff6bda00640 1 -- 172.21.15.100:0/2860697505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6b806c6c0 msgr2=0x7ff6b80ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:10.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.987+0000 7ff6bda00640 1 --2- 172.21.15.100:0/2860697505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6b806c6c0 0x7ff6b80ce0e0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff6a00097c0 tx=0x7ff6a001c4a0 comp rx=0 tx=0).stop 2024-04-23T14:35:10.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.987+0000 7ff6bda00640 1 -- 172.21.15.100:0/2860697505 shutdown_connections 2024-04-23T14:35:10.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.987+0000 7ff6bda00640 1 --2- 172.21.15.100:0/2860697505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6b806c6c0 0x7ff6b80ce0e0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:10.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.987+0000 7ff6bda00640 1 -- 172.21.15.100:0/2860697505 >> 172.21.15.100:0/2860697505 conn(0x7ff6b80b5200 msgr2=0x7ff6b80b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:10.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.987+0000 7ff6bda00640 1 -- 172.21.15.100:0/2860697505 shutdown_connections 2024-04-23T14:35:10.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.987+0000 7ff6bda00640 1 -- 172.21.15.100:0/2860697505 wait complete. 2024-04-23T14:35:10.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.987+0000 7ff6bda00640 1 Processor -- start 2024-04-23T14:35:10.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.987+0000 7ff6bda00640 1 -- start start 2024-04-23T14:35:10.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.987+0000 7ff6bda00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6b806c6c0 0x7ff6b80c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:10.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.987+0000 7ff6bda00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6b806f8c0 0x7ff6b815bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:10.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.987+0000 7ff6bda00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6b80ce620 0x7ff6b80cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:10.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.987+0000 7ff6bda00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b8169e30 con 0x7ff6b806c6c0 2024-04-23T14:35:10.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.987+0000 7ff6bda00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b80ad8f0 con 0x7ff6b80ce620 2024-04-23T14:35:10.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.987+0000 7ff6bda00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff6b8169fb0 con 0x7ff6b806f8c0 2024-04-23T14:35:10.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.991+0000 7ff6b6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6b806c6c0 0x7ff6b80c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:10.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.991+0000 7ff6b6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6b806c6c0 0x7ff6b80c58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40946/0 (socket says 172.21.15.100:40946) 2024-04-23T14:35:10.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.991+0000 7ff6b6a00640 1 -- 172.21.15.100:0/3557079316 learned_addr learned my addr 172.21.15.100:0/3557079316 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:10.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.991+0000 7ff6b7400640 1 --2- 172.21.15.100:0/3557079316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6b80ce620 0x7ff6b80cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:10.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.991+0000 7ff6ae000640 1 --2- 172.21.15.100:0/3557079316 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6b806f8c0 0x7ff6b815bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:10.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.991+0000 7ff6b6a00640 1 -- 172.21.15.100:0/3557079316 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6b806f8c0 msgr2=0x7ff6b815bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:10.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.991+0000 7ff6b6a00640 1 --2- 172.21.15.100:0/3557079316 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6b806f8c0 0x7ff6b815bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:10.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.991+0000 7ff6b6a00640 1 -- 172.21.15.100:0/3557079316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6b80ce620 msgr2=0x7ff6b80cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:10.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.991+0000 7ff6b6a00640 1 --2- 172.21.15.100:0/3557079316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6b80ce620 0x7ff6b80cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:10.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.991+0000 7ff6b6a00640 1 -- 172.21.15.100:0/3557079316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b8073820 con 0x7ff6b806c6c0 2024-04-23T14:35:10.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.991+0000 7ff6b6a00640 1 --2- 172.21.15.100:0/3557079316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6b806c6c0 0x7ff6b80c58e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800c3f0 tx=0x7ff6a800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:10.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.991+0000 7ff6af400640 1 -- 172.21.15.100:0/3557079316 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8017070 con 0x7ff6b806c6c0 2024-04-23T14:35:10.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.991+0000 7ff6af400640 1 -- 172.21.15.100:0/3557079316 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6a80052d0 con 0x7ff6b806c6c0 2024-04-23T14:35:10.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.991+0000 7ff6af400640 1 -- 172.21.15.100:0/3557079316 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8003840 con 0x7ff6b806c6c0 2024-04-23T14:35:10.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.991+0000 7ff6bda00640 1 -- 172.21.15.100:0/3557079316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6b8073b00 con 0x7ff6b806c6c0 2024-04-23T14:35:10.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.991+0000 7ff6bda00640 1 -- 172.21.15.100:0/3557079316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6b815c5b0 con 0x7ff6b806c6c0 2024-04-23T14:35:10.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.991+0000 7ff6bda00640 1 -- 172.21.15.100:0/3557079316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff6b8068ad0 con 0x7ff6b806c6c0 2024-04-23T14:35:10.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.991+0000 7ff6af400640 1 -- 172.21.15.100:0/3557079316 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff6a801a070 con 0x7ff6b806c6c0 2024-04-23T14:35:10.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.991+0000 7ff6af400640 1 --2- 172.21.15.100:0/3557079316 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff694068380 0x7ff69406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:10.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.991+0000 7ff6af400640 1 -- 172.21.15.100:0/3557079316 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7ff6a8085c00 con 0x7ff6b806c6c0 2024-04-23T14:35:10.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.995+0000 7ff6ae000640 1 --2- 172.21.15.100:0/3557079316 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff694068380 0x7ff69406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:10.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.995+0000 7ff6af400640 1 --2- 172.21.15.100:0/3557079316 >> [v2:172.21.15.100:6800/43918908,v1:172.21.15.100:6801/43918908] conn(0x7ff69406e820 0x7ff694070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:10.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.995+0000 7ff6af400640 1 -- 172.21.15.100:0/3557079316 --> [v2:172.21.15.100:6800/43918908,v1:172.21.15.100:6801/43918908] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff6a8016ec0 con 0x7ff69406e820 2024-04-23T14:35:10.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.995+0000 7ff6af400640 1 -- 172.21.15.100:0/3557079316 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7ff6a808c050 con 0x7ff6b806c6c0 2024-04-23T14:35:10.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.995+0000 7ff6b7400640 1 --2- 172.21.15.100:0/3557079316 >> [v2:172.21.15.100:6800/43918908,v1:172.21.15.100:6801/43918908] conn(0x7ff69406e820 0x7ff694070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:10.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.995+0000 7ff6ae000640 1 --2- 172.21.15.100:0/3557079316 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff694068380 0x7ff69406a840 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff6a00ad000 tx=0x7ff6a00a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:10.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.995+0000 7ff6b7400640 1 --2- 172.21.15.100:0/3557079316 >> [v2:172.21.15.100:6800/43918908,v1:172.21.15.100:6801/43918908] conn(0x7ff69406e820 0x7ff694070c60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:10.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:10.995+0000 7ff6af400640 1 -- 172.21.15.100:0/3557079316 <== osd.0 v2:172.21.15.100:6800/43918908 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff6a8016ec0 con 0x7ff69406e820 2024-04-23T14:35:11.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.007+0000 7ff6bda00640 1 -- 172.21.15.100:0/3557079316 --> [v2:172.21.15.100:6800/43918908,v1:172.21.15.100:6801/43918908] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff6b806cc00 con 0x7ff69406e820 2024-04-23T14:35:11.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.007+0000 7ff6af400640 1 -- 172.21.15.100:0/3557079316 <== osd.0 v2:172.21.15.100:6800/43918908 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff6b806cc00 con 0x7ff69406e820 2024-04-23T14:35:11.009 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:35:11.010 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T14:35:11.010 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:35:11.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.007+0000 7ff6bda00640 1 -- 172.21.15.100:0/3557079316 >> [v2:172.21.15.100:6800/43918908,v1:172.21.15.100:6801/43918908] conn(0x7ff69406e820 msgr2=0x7ff694070c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.007+0000 7ff6bda00640 1 --2- 172.21.15.100:0/3557079316 >> [v2:172.21.15.100:6800/43918908,v1:172.21.15.100:6801/43918908] conn(0x7ff69406e820 0x7ff694070c60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.007+0000 7ff6bda00640 1 -- 172.21.15.100:0/3557079316 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff694068380 msgr2=0x7ff69406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.007+0000 7ff6bda00640 1 --2- 172.21.15.100:0/3557079316 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff694068380 0x7ff69406a840 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff6a00ad000 tx=0x7ff6a00a6000 comp rx=0 tx=0).stop 2024-04-23T14:35:11.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.007+0000 7ff6bda00640 1 -- 172.21.15.100:0/3557079316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6b806c6c0 msgr2=0x7ff6b80c58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.007+0000 7ff6bda00640 1 --2- 172.21.15.100:0/3557079316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6b806c6c0 0x7ff6b80c58e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800c3f0 tx=0x7ff6a800c8b0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.007+0000 7ff6b7400640 1 -- 172.21.15.100:0/3557079316 reap_dead start 2024-04-23T14:35:11.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.007+0000 7ff6bda00640 1 -- 172.21.15.100:0/3557079316 shutdown_connections 2024-04-23T14:35:11.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.007+0000 7ff6bda00640 1 -- 172.21.15.100:0/3557079316 >> 172.21.15.100:0/3557079316 conn(0x7ff6b80b5200 msgr2=0x7ff6b815f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:11.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.007+0000 7ff6bda00640 1 -- 172.21.15.100:0/3557079316 shutdown_connections 2024-04-23T14:35:11.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.007+0000 7ff6bda00640 1 -- 172.21.15.100:0/3557079316 wait complete. 2024-04-23T14:35:11.027 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-23T14:35:11.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.163+0000 7f322f200640 1 Processor -- start 2024-04-23T14:35:11.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.163+0000 7f322f200640 1 -- start start 2024-04-23T14:35:11.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.163+0000 7f322f200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f322815b660 0x7f32280c6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:11.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.163+0000 7f322f200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f32280c6d60 0x7f32280d0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:11.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.163+0000 7f322f200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32280cee00 con 0x7f32280d06d0 2024-04-23T14:35:11.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.163+0000 7f322f200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f32280d0b60 con 0x7f32280c6d60 2024-04-23T14:35:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.163+0000 7f322f200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f322815ba40 con 0x7f322815b660 2024-04-23T14:35:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.163+0000 7f3227400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f32280c6d60 0x7f32280d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.163+0000 7f322d200640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f32280d06d0 0x7f322815f0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42186/0 (socket says 172.21.15.100:42186) 2024-04-23T14:35:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.163+0000 7f322d200640 1 -- 172.21.15.100:0/443454914 learned_addr learned my addr 172.21.15.100:0/443454914 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.163+0000 7f3227e00640 1 --2- 172.21.15.100:0/443454914 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f322815b660 0x7f32280c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.167+0000 7f3227400640 1 -- 172.21.15.100:0/443454914 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f322815b660 msgr2=0x7f32280c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.167+0000 7f3227400640 1 --2- 172.21.15.100:0/443454914 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f322815b660 0x7f32280c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.167+0000 7f3227400640 1 -- 172.21.15.100:0/443454914 >> v1:172.21.15.100:6789/0 conn(0x7f32280d06d0 legacy=0x7f322815f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.167+0000 7f3227400640 1 -- 172.21.15.100:0/443454914 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322806bc50 con 0x7f32280c6d60 2024-04-23T14:35:11.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.167+0000 7f3227400640 1 --2- 172.21.15.100:0/443454914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32280c6d60 0x7f32280d0190 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f321400ba60 tx=0x7f321401eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2b3df5253abd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:11.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.167+0000 7f3226a00640 1 -- 172.21.15.100:0/443454914 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32140b4070 con 0x7f32280c6d60 2024-04-23T14:35:11.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.167+0000 7f3226a00640 1 -- 172.21.15.100:0/443454914 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32140a7c10 con 0x7f32280c6d60 2024-04-23T14:35:11.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.167+0000 7f3226a00640 1 -- 172.21.15.100:0/443454914 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32140af440 con 0x7f32280c6d60 2024-04-23T14:35:11.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.167+0000 7f322f200640 1 -- 172.21.15.100:0/443454914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32280c6d60 msgr2=0x7f32280d0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.167+0000 7f322f200640 1 --2- 172.21.15.100:0/443454914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32280c6d60 0x7f32280d0190 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f321400ba60 tx=0x7f321401eb40 comp rx=0 tx=0).stop 2024-04-23T14:35:11.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.167+0000 7f322f200640 1 -- 172.21.15.100:0/443454914 shutdown_connections 2024-04-23T14:35:11.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.167+0000 7f322f200640 1 --2- 172.21.15.100:0/443454914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32280c6d60 0x7f32280d0190 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.167+0000 7f322f200640 1 --2- 172.21.15.100:0/443454914 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f322815b660 0x7f32280c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.167+0000 7f322f200640 1 -- 172.21.15.100:0/443454914 >> 172.21.15.100:0/443454914 conn(0x7f32280b5310 msgr2=0x7f32280b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:11.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.167+0000 7f322f200640 1 -- 172.21.15.100:0/443454914 shutdown_connections 2024-04-23T14:35:11.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f322f200640 1 -- 172.21.15.100:0/443454914 wait complete. 2024-04-23T14:35:11.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f322f200640 1 Processor -- start 2024-04-23T14:35:11.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f322f200640 1 -- start start 2024-04-23T14:35:11.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f322f200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32280c6d60 0x7f3228152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:11.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f322f200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f32280d06d0 0x7f3228165c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:11.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f322f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f322815b660 0x7f3228142690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:11.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f322f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f32280cccd0 con 0x7f322815b660 2024-04-23T14:35:11.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f322f200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f322806c7d0 con 0x7f32280c6d60 2024-04-23T14:35:11.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f322f200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f32280cce50 con 0x7f32280d06d0 2024-04-23T14:35:11.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f322d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f322815b660 0x7f3228142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:11.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f322d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f322815b660 0x7f3228142690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40974/0 (socket says 172.21.15.100:40974) 2024-04-23T14:35:11.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f322d200640 1 -- 172.21.15.100:0/894073532 learned_addr learned my addr 172.21.15.100:0/894073532 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:11.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f322d200640 1 -- 172.21.15.100:0/894073532 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f32280d06d0 msgr2=0x7f3228165c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f322d200640 1 --2- 172.21.15.100:0/894073532 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f32280d06d0 0x7f3228165c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f322d200640 1 -- 172.21.15.100:0/894073532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32280c6d60 msgr2=0x7f3228152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f3227e00640 1 --2- 172.21.15.100:0/894073532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32280c6d60 0x7f3228152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:11.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f322d200640 1 --2- 172.21.15.100:0/894073532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32280c6d60 0x7f3228152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f322d200640 1 -- 172.21.15.100:0/894073532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3228152a90 con 0x7f322815b660 2024-04-23T14:35:11.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f3227e00640 1 --2- 172.21.15.100:0/894073532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32280c6d60 0x7f3228152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:11.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f322d200640 1 --2- 172.21.15.100:0/894073532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f322815b660 0x7f3228142690 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f322000c400 tx=0x7f322000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:11.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f321fe00640 1 -- 172.21.15.100:0/894073532 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220017070 con 0x7f322815b660 2024-04-23T14:35:11.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f321fe00640 1 -- 172.21.15.100:0/894073532 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3220005cf0 con 0x7f322815b660 2024-04-23T14:35:11.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f322f200640 1 -- 172.21.15.100:0/894073532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32281406d0 con 0x7f322815b660 2024-04-23T14:35:11.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f321fe00640 1 -- 172.21.15.100:0/894073532 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32200072d0 con 0x7f322815b660 2024-04-23T14:35:11.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.171+0000 7f322f200640 1 -- 172.21.15.100:0/894073532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3228158ca0 con 0x7f322815b660 2024-04-23T14:35:11.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.175+0000 7f322f200640 1 -- 172.21.15.100:0/894073532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3228068ad0 con 0x7f322815b660 2024-04-23T14:35:11.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.175+0000 7f321fe00640 1 -- 172.21.15.100:0/894073532 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f322001a070 con 0x7f322815b660 2024-04-23T14:35:11.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.175+0000 7f321fe00640 1 --2- 172.21.15.100:0/894073532 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3208068380 0x7f320806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:11.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.175+0000 7f321fe00640 1 -- 172.21.15.100:0/894073532 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f3220085b60 con 0x7f322815b660 2024-04-23T14:35:11.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.175+0000 7f3227e00640 1 --2- 172.21.15.100:0/894073532 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3208068380 0x7f320806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:11.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.175+0000 7f3227e00640 1 --2- 172.21.15.100:0/894073532 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3208068380 0x7f320806a840 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3228157e00 tx=0x7f321800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:11.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.179+0000 7f321fe00640 1 -- 172.21.15.100:0/894073532 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3228152a90 con 0x7f322815b660 2024-04-23T14:35:11.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.335+0000 7f322f200640 1 -- 172.21.15.100:0/894073532 --> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f3228071dc0 con 0x7f3208068380 2024-04-23T14:35:11.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.339+0000 7f321fe00640 1 -- 172.21.15.100:0/894073532 <== mgr.4101 v2:172.21.15.100:6832/2859853957 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3228071dc0 con 0x7f3208068380 2024-04-23T14:35:11.338 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 0 to deep-scrub 2024-04-23T14:35:11.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.339+0000 7f322f200640 1 -- 172.21.15.100:0/894073532 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3208068380 msgr2=0x7f320806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.339+0000 7f322f200640 1 --2- 172.21.15.100:0/894073532 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3208068380 0x7f320806a840 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3228157e00 tx=0x7f321800b040 comp rx=0 tx=0).stop 2024-04-23T14:35:11.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.339+0000 7f322f200640 1 -- 172.21.15.100:0/894073532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f322815b660 msgr2=0x7f3228142690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.339+0000 7f322f200640 1 --2- 172.21.15.100:0/894073532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f322815b660 0x7f3228142690 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f322000c400 tx=0x7f322000c8c0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.339+0000 7f322f200640 1 -- 172.21.15.100:0/894073532 shutdown_connections 2024-04-23T14:35:11.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.339+0000 7f322f200640 1 --2- 172.21.15.100:0/894073532 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3208068380 0x7f320806a840 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.339+0000 7f322f200640 1 --2- 172.21.15.100:0/894073532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f322815b660 0x7f3228142690 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.339+0000 7f322f200640 1 --2- 172.21.15.100:0/894073532 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f32280d06d0 0x7f3228165c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.339+0000 7f322f200640 1 --2- 172.21.15.100:0/894073532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32280c6d60 0x7f3228152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.339+0000 7f322f200640 1 -- 172.21.15.100:0/894073532 >> 172.21.15.100:0/894073532 conn(0x7f32280b5310 msgr2=0x7f32280725a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:11.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.339+0000 7f322f200640 1 -- 172.21.15.100:0/894073532 shutdown_connections 2024-04-23T14:35:11.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.339+0000 7f322f200640 1 -- 172.21.15.100:0/894073532 wait complete. 2024-04-23T14:35:11.359 INFO:tasks.ceph:Scrubbing osd.1 2024-04-23T14:35:11.360 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-23T14:35:11.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.499+0000 7f3abba00640 1 Processor -- start 2024-04-23T14:35:11.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.499+0000 7f3abba00640 1 -- start start 2024-04-23T14:35:11.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.499+0000 7f3abba00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f3ab406d8b0 0x7f3ab40d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:11.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.499+0000 7f3abba00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ab40cecc0 con 0x7f3ab415e950 2024-04-23T14:35:11.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.499+0000 7f3abba00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ab40cef60 con 0x7f3ab406d450 2024-04-23T14:35:11.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.499+0000 7f3abba00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f3ab40d0300 con 0x7f3ab406d8b0 2024-04-23T14:35:11.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.499+0000 7f3ab9000640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f3ab415e950 0x7f3ab40cd460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42202/0 (socket says 172.21.15.100:42202) 2024-04-23T14:35:11.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.499+0000 7f3ab9000640 1 -- 172.21.15.100:0/484593466 learned_addr learned my addr 172.21.15.100:0/484593466 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:11.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.499+0000 7f3ab9a00640 1 --2- 172.21.15.100:0/484593466 >> v2:172.21.15.151:3300/0 conn(0x7f3ab406d8b0 0x7f3ab40d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:11.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.499+0000 7f3ab3400640 1 -- 172.21.15.100:0/484593466 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1887587174 0 0) 0x7f3ab40cecc0 con 0x7f3ab415e950 2024-04-23T14:35:11.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.499+0000 7f3ab3400640 1 -- 172.21.15.100:0/484593466 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a98003530 con 0x7f3ab415e950 2024-04-23T14:35:11.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.499+0000 7f3ab9a00640 1 -- 172.21.15.100:0/484593466 >> v1:172.21.15.151:6789/0 conn(0x7f3ab406d450 legacy=0x7f3ab40d0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.499+0000 7f3ab3400640 1 -- 172.21.15.100:0/484593466 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4272676786 0 0) 0x7f3ab40cef60 con 0x7f3ab406d450 2024-04-23T14:35:11.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.499+0000 7f3ab9a00640 1 -- 172.21.15.100:0/484593466 >> v1:172.21.15.100:6789/0 conn(0x7f3ab415e950 legacy=0x7f3ab40cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.499+0000 7f3ab9a00640 1 -- 172.21.15.100:0/484593466 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab40cf240 con 0x7f3ab406d8b0 2024-04-23T14:35:11.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.503+0000 7f3ab9a00640 1 --2- 172.21.15.100:0/484593466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ab406d8b0 0x7f3ab40d0ea0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009d90 tx=0x7f3aac01c720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a41b1ccf4fa0b50 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:11.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.503+0000 7f3ab3400640 1 -- 172.21.15.100:0/484593466 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4001680 con 0x7f3ab406d8b0 2024-04-23T14:35:11.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.503+0000 7f3ab3400640 1 -- 172.21.15.100:0/484593466 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aac0a64c0 con 0x7f3ab406d8b0 2024-04-23T14:35:11.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.503+0000 7f3ab3400640 1 -- 172.21.15.100:0/484593466 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac0a6870 con 0x7f3ab406d8b0 2024-04-23T14:35:11.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.503+0000 7f3abba00640 1 -- 172.21.15.100:0/484593466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ab406d8b0 msgr2=0x7f3ab40d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.503+0000 7f3abba00640 1 --2- 172.21.15.100:0/484593466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ab406d8b0 0x7f3ab40d0ea0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009d90 tx=0x7f3aac01c720 comp rx=0 tx=0).stop 2024-04-23T14:35:11.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3abba00640 1 -- 172.21.15.100:0/484593466 shutdown_connections 2024-04-23T14:35:11.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3abba00640 1 --2- 172.21.15.100:0/484593466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ab406d8b0 0x7f3ab40d0ea0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3abba00640 1 -- 172.21.15.100:0/484593466 >> 172.21.15.100:0/484593466 conn(0x7f3ab40719d0 msgr2=0x7f3ab4071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:11.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3abba00640 1 -- 172.21.15.100:0/484593466 shutdown_connections 2024-04-23T14:35:11.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3abba00640 1 -- 172.21.15.100:0/484593466 wait complete. 2024-04-23T14:35:11.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3abba00640 1 Processor -- start 2024-04-23T14:35:11.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3abba00640 1 -- start start 2024-04-23T14:35:11.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3abba00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3ab406d450 0x7f3ab40c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:11.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3abba00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab406d8b0 0x7f3ab4150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:11.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3abba00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ab415e950 0x7f3ab4150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:11.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3abba00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab416c790 con 0x7f3ab406d8b0 2024-04-23T14:35:11.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3abba00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab40cdb70 con 0x7f3ab415e950 2024-04-23T14:35:11.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3abba00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f3ab416c910 con 0x7f3ab406d450 2024-04-23T14:35:11.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3ab3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab406d8b0 0x7f3ab4150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:11.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3ab3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab406d8b0 0x7f3ab4150530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40988/0 (socket says 172.21.15.100:40988) 2024-04-23T14:35:11.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3ab3e00640 1 -- 172.21.15.100:0/927849030 learned_addr learned my addr 172.21.15.100:0/927849030 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:11.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3ab3e00640 1 -- 172.21.15.100:0/927849030 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3ab406d450 msgr2=0x7f3ab40c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3ab3e00640 1 --2- 172.21.15.100:0/927849030 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3ab406d450 0x7f3ab40c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3ab3e00640 1 -- 172.21.15.100:0/927849030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ab415e950 msgr2=0x7f3ab4150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3ab3e00640 1 --2- 172.21.15.100:0/927849030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ab415e950 0x7f3ab4150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3ab3e00640 1 -- 172.21.15.100:0/927849030 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab4151660 con 0x7f3ab406d8b0 2024-04-23T14:35:11.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3ab3e00640 1 --2- 172.21.15.100:0/927849030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab406d8b0 0x7f3ab4150530 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000c4d0 tx=0x7f3aa000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:11.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3ab0c00640 1 -- 172.21.15.100:0/927849030 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0017070 con 0x7f3ab406d8b0 2024-04-23T14:35:11.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3ab0c00640 1 -- 172.21.15.100:0/927849030 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aa0005480 con 0x7f3ab406d8b0 2024-04-23T14:35:11.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3ab0c00640 1 -- 172.21.15.100:0/927849030 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0003840 con 0x7f3ab406d8b0 2024-04-23T14:35:11.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3abba00640 1 -- 172.21.15.100:0/927849030 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab414e2b0 con 0x7f3ab406d8b0 2024-04-23T14:35:11.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.507+0000 7f3abba00640 1 -- 172.21.15.100:0/927849030 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab4151380 con 0x7f3ab406d8b0 2024-04-23T14:35:11.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.511+0000 7f3abba00640 1 -- 172.21.15.100:0/927849030 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3ab4068ad0 con 0x7f3ab406d8b0 2024-04-23T14:35:11.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.511+0000 7f3ab0c00640 1 -- 172.21.15.100:0/927849030 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3aa001a070 con 0x7f3ab406d8b0 2024-04-23T14:35:11.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.511+0000 7f3ab0c00640 1 --2- 172.21.15.100:0/927849030 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3a94068380 0x7f3a9406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:11.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.511+0000 7f3ab0c00640 1 -- 172.21.15.100:0/927849030 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f3aa0085be0 con 0x7f3ab406d8b0 2024-04-23T14:35:11.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.511+0000 7f3ab9000640 1 --2- 172.21.15.100:0/927849030 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3a94068380 0x7f3a9406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:11.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.511+0000 7f3ab0c00640 1 --2- 172.21.15.100:0/927849030 >> [v2:172.21.15.100:6808/3988727915,v1:172.21.15.100:6809/3988727915] conn(0x7f3a9406e820 0x7f3a94070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:11.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.511+0000 7f3ab0c00640 1 -- 172.21.15.100:0/927849030 --> [v2:172.21.15.100:6808/3988727915,v1:172.21.15.100:6809/3988727915] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3aa0013e50 con 0x7f3a9406e820 2024-04-23T14:35:11.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.511+0000 7f3ab0c00640 1 -- 172.21.15.100:0/927849030 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f3aa00882e0 con 0x7f3ab406d8b0 2024-04-23T14:35:11.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.511+0000 7f3ab9a00640 1 --2- 172.21.15.100:0/927849030 >> [v2:172.21.15.100:6808/3988727915,v1:172.21.15.100:6809/3988727915] conn(0x7f3a9406e820 0x7f3a94070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:11.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.511+0000 7f3ab9000640 1 --2- 172.21.15.100:0/927849030 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3a94068380 0x7f3a9406a840 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40017f0 tx=0x7f3aa4001560 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:11.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.511+0000 7f3ab9a00640 1 --2- 172.21.15.100:0/927849030 >> [v2:172.21.15.100:6808/3988727915,v1:172.21.15.100:6809/3988727915] conn(0x7f3a9406e820 0x7f3a94070c60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:11.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.515+0000 7f3ab0c00640 1 -- 172.21.15.100:0/927849030 <== osd.1 v2:172.21.15.100:6808/3988727915 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3aa0013e50 con 0x7f3a9406e820 2024-04-23T14:35:11.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.527+0000 7f3abba00640 1 -- 172.21.15.100:0/927849030 --> [v2:172.21.15.100:6808/3988727915,v1:172.21.15.100:6809/3988727915] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3ab40b36f0 con 0x7f3a9406e820 2024-04-23T14:35:11.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.527+0000 7f3ab0c00640 1 -- 172.21.15.100:0/927849030 <== osd.1 v2:172.21.15.100:6808/3988727915 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3ab40b36f0 con 0x7f3a9406e820 2024-04-23T14:35:11.528 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:35:11.528 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T14:35:11.528 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:35:11.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.527+0000 7f3abba00640 1 -- 172.21.15.100:0/927849030 >> [v2:172.21.15.100:6808/3988727915,v1:172.21.15.100:6809/3988727915] conn(0x7f3a9406e820 msgr2=0x7f3a94070c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.527+0000 7f3abba00640 1 --2- 172.21.15.100:0/927849030 >> [v2:172.21.15.100:6808/3988727915,v1:172.21.15.100:6809/3988727915] conn(0x7f3a9406e820 0x7f3a94070c60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.527+0000 7f3abba00640 1 -- 172.21.15.100:0/927849030 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3a94068380 msgr2=0x7f3a9406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.527+0000 7f3abba00640 1 --2- 172.21.15.100:0/927849030 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f3a94068380 0x7f3a9406a840 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40017f0 tx=0x7f3aa4001560 comp rx=0 tx=0).stop 2024-04-23T14:35:11.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.527+0000 7f3abba00640 1 -- 172.21.15.100:0/927849030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab406d8b0 msgr2=0x7f3ab4150530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.527+0000 7f3abba00640 1 --2- 172.21.15.100:0/927849030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab406d8b0 0x7f3ab4150530 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000c4d0 tx=0x7f3aa000c990 comp rx=0 tx=0).stop 2024-04-23T14:35:11.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.527+0000 7f3ab9a00640 1 -- 172.21.15.100:0/927849030 reap_dead start 2024-04-23T14:35:11.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.527+0000 7f3abba00640 1 -- 172.21.15.100:0/927849030 shutdown_connections 2024-04-23T14:35:11.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.527+0000 7f3abba00640 1 -- 172.21.15.100:0/927849030 >> 172.21.15.100:0/927849030 conn(0x7f3ab40719d0 msgr2=0x7f3ab4070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:11.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.527+0000 7f3abba00640 1 -- 172.21.15.100:0/927849030 shutdown_connections 2024-04-23T14:35:11.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.527+0000 7f3abba00640 1 -- 172.21.15.100:0/927849030 wait complete. 2024-04-23T14:35:11.546 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-23T14:35:11.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.683+0000 7f2d20e00640 1 Processor -- start 2024-04-23T14:35:11.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.683+0000 7f2d20e00640 1 -- start start 2024-04-23T14:35:11.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.683+0000 7f2d20e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f2d1c0cf160 0x7f2d1c0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:11.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.683+0000 7f2d20e00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d1c0cfab0 con 0x7f2d1c15d0c0 2024-04-23T14:35:11.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.683+0000 7f2d20e00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d1c0ce500 con 0x7f2d1c0cde20 2024-04-23T14:35:11.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.683+0000 7f2d20e00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f2d1c0cff50 con 0x7f2d1c0cf160 2024-04-23T14:35:11.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.683+0000 7f2d19e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f2d1c15d0c0 0x7f2d1c15d4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42204/0 (socket says 172.21.15.100:42204) 2024-04-23T14:35:11.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.683+0000 7f2d19e00640 1 -- 172.21.15.100:0/2895892962 learned_addr learned my addr 172.21.15.100:0/2895892962 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:11.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.683+0000 7f2d1a800640 1 --2- 172.21.15.100:0/2895892962 >> v2:172.21.15.151:3300/0 conn(0x7f2d1c0cf160 0x7f2d1c0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:11.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.683+0000 7f2d19400640 1 -- 172.21.15.100:0/2895892962 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3302533504 0 0) 0x7f2d0c0014a0 con 0x7f2d1c15d0c0 2024-04-23T14:35:11.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.683+0000 7f2d19400640 1 -- 172.21.15.100:0/2895892962 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d00003530 con 0x7f2d1c15d0c0 2024-04-23T14:35:11.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.683+0000 7f2d19400640 1 -- 172.21.15.100:0/2895892962 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2760533798 0 0) 0x7f2d1c0ce500 con 0x7f2d1c0cde20 2024-04-23T14:35:11.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.683+0000 7f2d19400640 1 -- 172.21.15.100:0/2895892962 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d0c0014a0 con 0x7f2d1c0cde20 2024-04-23T14:35:11.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.683+0000 7f2d1a800640 1 -- 172.21.15.100:0/2895892962 >> v1:172.21.15.151:6789/0 conn(0x7f2d1c0cde20 legacy=0x7f2d1c0cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.683+0000 7f2d1a800640 1 -- 172.21.15.100:0/2895892962 >> v1:172.21.15.100:6789/0 conn(0x7f2d1c15d0c0 legacy=0x7f2d1c15d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.683+0000 7f2d1a800640 1 -- 172.21.15.100:0/2895892962 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1c0ce7e0 con 0x7f2d1c0cf160 2024-04-23T14:35:11.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.687+0000 7f2d1a800640 1 --2- 172.21.15.100:0/2895892962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1c0cf160 0x7f2d1c0cf570 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2d140097c0 tx=0x7f2d1401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=840f39e7f70ac811 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:11.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.687+0000 7f2d19400640 1 -- 172.21.15.100:0/2895892962 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d140b0070 con 0x7f2d1c0cf160 2024-04-23T14:35:11.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.687+0000 7f2d19400640 1 -- 172.21.15.100:0/2895892962 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d140ab600 con 0x7f2d1c0cf160 2024-04-23T14:35:11.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.687+0000 7f2d19400640 1 -- 172.21.15.100:0/2895892962 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d140ab9d0 con 0x7f2d1c0cf160 2024-04-23T14:35:11.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.687+0000 7f2d20e00640 1 -- 172.21.15.100:0/2895892962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1c0cf160 msgr2=0x7f2d1c0cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.687+0000 7f2d20e00640 1 --2- 172.21.15.100:0/2895892962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1c0cf160 0x7f2d1c0cf570 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2d140097c0 tx=0x7f2d1401c410 comp rx=0 tx=0).stop 2024-04-23T14:35:11.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.687+0000 7f2d20e00640 1 -- 172.21.15.100:0/2895892962 shutdown_connections 2024-04-23T14:35:11.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.687+0000 7f2d20e00640 1 --2- 172.21.15.100:0/2895892962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1c0cf160 0x7f2d1c0cf570 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.687+0000 7f2d20e00640 1 -- 172.21.15.100:0/2895892962 >> 172.21.15.100:0/2895892962 conn(0x7f2d1c0b5310 msgr2=0x7f2d1c0b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:11.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.687+0000 7f2d20e00640 1 -- 172.21.15.100:0/2895892962 shutdown_connections 2024-04-23T14:35:11.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.687+0000 7f2d20e00640 1 -- 172.21.15.100:0/2895892962 wait complete. 2024-04-23T14:35:11.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d20e00640 1 Processor -- start 2024-04-23T14:35:11.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d20e00640 1 -- start start 2024-04-23T14:35:11.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d20e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1c0cde20 0x7f2d1c14e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:11.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d20e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2d1c0cf160 0x7f2d1c155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:11.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d20e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d1c15d0c0 0x7f2d1c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:11.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d20e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c0cc230 con 0x7f2d1c15d0c0 2024-04-23T14:35:11.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d20e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c0cc0b0 con 0x7f2d1c0cde20 2024-04-23T14:35:11.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d20e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f2d1c152e30 con 0x7f2d1c0cf160 2024-04-23T14:35:11.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d1a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d1c15d0c0 0x7f2d1c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d1a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d1c15d0c0 0x7f2d1c06ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41002/0 (socket says 172.21.15.100:41002) 2024-04-23T14:35:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d1a800640 1 -- 172.21.15.100:0/2706386212 learned_addr learned my addr 172.21.15.100:0/2706386212 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d11400640 1 --2- 172.21.15.100:0/2706386212 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2d1c0cf160 0x7f2d1c155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d19e00640 1 --2- 172.21.15.100:0/2706386212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1c0cde20 0x7f2d1c14e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d1a800640 1 -- 172.21.15.100:0/2706386212 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2d1c0cf160 msgr2=0x7f2d1c155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d1a800640 1 --2- 172.21.15.100:0/2706386212 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2d1c0cf160 0x7f2d1c155ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d1a800640 1 -- 172.21.15.100:0/2706386212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1c0cde20 msgr2=0x7f2d1c14e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d1a800640 1 --2- 172.21.15.100:0/2706386212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1c0cde20 0x7f2d1c14e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d1a800640 1 -- 172.21.15.100:0/2706386212 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1c0ae520 con 0x7f2d1c15d0c0 2024-04-23T14:35:11.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d19e00640 1 --2- 172.21.15.100:0/2706386212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1c0cde20 0x7f2d1c14e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:11.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d1a800640 1 --2- 172.21.15.100:0/2706386212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d1c15d0c0 0x7f2d1c06ef60 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f2d14019020 tx=0x7f2d140191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:11.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d12a00640 1 -- 172.21.15.100:0/2706386212 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d140b0040 con 0x7f2d1c15d0c0 2024-04-23T14:35:11.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d12a00640 1 -- 172.21.15.100:0/2706386212 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d140b9070 con 0x7f2d1c15d0c0 2024-04-23T14:35:11.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d20e00640 1 -- 172.21.15.100:0/2706386212 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1c0ae800 con 0x7f2d1c15d0c0 2024-04-23T14:35:11.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d12a00640 1 -- 172.21.15.100:0/2706386212 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d140b4630 con 0x7f2d1c15d0c0 2024-04-23T14:35:11.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d20e00640 1 -- 172.21.15.100:0/2706386212 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d1c0aedc0 con 0x7f2d1c15d0c0 2024-04-23T14:35:11.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.691+0000 7f2d0b400640 1 -- 172.21.15.100:0/2706386212 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d1c068ad0 con 0x7f2d1c15d0c0 2024-04-23T14:35:11.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.695+0000 7f2d12a00640 1 -- 172.21.15.100:0/2706386212 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2d140b47d0 con 0x7f2d1c15d0c0 2024-04-23T14:35:11.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.695+0000 7f2d12a00640 1 --2- 172.21.15.100:0/2706386212 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2cf80683b0 0x7f2cf806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:11.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.695+0000 7f2d12a00640 1 -- 172.21.15.100:0/2706386212 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f2d141205a0 con 0x7f2d1c15d0c0 2024-04-23T14:35:11.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.695+0000 7f2d19e00640 1 --2- 172.21.15.100:0/2706386212 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2cf80683b0 0x7f2cf806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:11.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.695+0000 7f2d19e00640 1 --2- 172.21.15.100:0/2706386212 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2cf80683b0 0x7f2cf806a870 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0012f0 tx=0x7f2d0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:11.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.699+0000 7f2d12a00640 1 -- 172.21.15.100:0/2706386212 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2d14016090 con 0x7f2d1c15d0c0 2024-04-23T14:35:11.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.855+0000 7f2d0b400640 1 -- 172.21.15.100:0/2706386212 --> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f2d1c156000 con 0x7f2cf80683b0 2024-04-23T14:35:11.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.859+0000 7f2d12a00640 1 -- 172.21.15.100:0/2706386212 <== mgr.4101 v2:172.21.15.100:6832/2859853957 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2d0c001f70 con 0x7f2cf80683b0 2024-04-23T14:35:11.859 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 1 to deep-scrub 2024-04-23T14:35:11.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.859+0000 7f2d0b400640 1 -- 172.21.15.100:0/2706386212 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2cf80683b0 msgr2=0x7f2cf806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.859+0000 7f2d0b400640 1 --2- 172.21.15.100:0/2706386212 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2cf80683b0 0x7f2cf806a870 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0012f0 tx=0x7f2d0c009040 comp rx=0 tx=0).stop 2024-04-23T14:35:11.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.859+0000 7f2d0b400640 1 -- 172.21.15.100:0/2706386212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d1c15d0c0 msgr2=0x7f2d1c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:11.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.859+0000 7f2d0b400640 1 --2- 172.21.15.100:0/2706386212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d1c15d0c0 0x7f2d1c06ef60 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f2d14019020 tx=0x7f2d140191b0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.859+0000 7f2d0b400640 1 -- 172.21.15.100:0/2706386212 shutdown_connections 2024-04-23T14:35:11.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.859+0000 7f2d0b400640 1 --2- 172.21.15.100:0/2706386212 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f2cf80683b0 0x7f2cf806a870 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.859+0000 7f2d0b400640 1 --2- 172.21.15.100:0/2706386212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d1c15d0c0 0x7f2d1c06ef60 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.859+0000 7f2d0b400640 1 --2- 172.21.15.100:0/2706386212 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2d1c0cf160 0x7f2d1c155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.859+0000 7f2d0b400640 1 --2- 172.21.15.100:0/2706386212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1c0cde20 0x7f2d1c14e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:11.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.859+0000 7f2d0b400640 1 -- 172.21.15.100:0/2706386212 >> 172.21.15.100:0/2706386212 conn(0x7f2d1c0b5310 msgr2=0x7f2d1c15c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:11.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.859+0000 7f2d0b400640 1 -- 172.21.15.100:0/2706386212 shutdown_connections 2024-04-23T14:35:11.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:11.859+0000 7f2d0b400640 1 -- 172.21.15.100:0/2706386212 wait complete. 2024-04-23T14:35:11.879 INFO:tasks.ceph:Scrubbing osd.2 2024-04-23T14:35:11.879 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-23T14:35:12.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.019+0000 7fc695600640 1 Processor -- start 2024-04-23T14:35:12.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.019+0000 7fc695600640 1 -- start start 2024-04-23T14:35:12.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.019+0000 7fc695600640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc69015e830 0x7fc69015ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.019+0000 7fc695600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc6900cdd60 0x7fc6900cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.019+0000 7fc695600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc69015f320 con 0x7fc6900cfee0 2024-04-23T14:35:12.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.019+0000 7fc695600640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc6900d0330 con 0x7fc6900cdd60 2024-04-23T14:35:12.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.019+0000 7fc695600640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fc69015f5e0 con 0x7fc69015e830 2024-04-23T14:35:12.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.019+0000 7fc68e000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc6900cdd60 0x7fc6900cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.019+0000 7fc68e000640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc6900cdd60 0x7fc6900cf9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41008/0 (socket says 172.21.15.100:41008) 2024-04-23T14:35:12.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.019+0000 7fc68e000640 1 -- 172.21.15.100:0/2191743628 learned_addr learned my addr 172.21.15.100:0/2191743628 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:12.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.019+0000 7fc68ea00640 1 --2- 172.21.15.100:0/2191743628 >> v2:172.21.15.151:3300/0 conn(0x7fc69015e830 0x7fc69015ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.019+0000 7fc68e000640 1 -- 172.21.15.100:0/2191743628 >> v2:172.21.15.151:3300/0 conn(0x7fc69015e830 msgr2=0x7fc69015ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:12.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.019+0000 7fc68e000640 1 --2- 172.21.15.100:0/2191743628 >> v2:172.21.15.151:3300/0 conn(0x7fc69015e830 0x7fc69015ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.019+0000 7fc68e000640 1 -- 172.21.15.100:0/2191743628 >> v1:172.21.15.151:6789/0 conn(0x7fc6900cfee0 legacy=0x7fc690162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.019+0000 7fc68e000640 1 -- 172.21.15.100:0/2191743628 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69015f8c0 con 0x7fc6900cdd60 2024-04-23T14:35:12.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.019+0000 7fc68ea00640 1 --2- 172.21.15.100:0/2191743628 >> v2:172.21.15.151:3300/0 conn(0x7fc69015e830 0x7fc69015ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:12.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.023+0000 7fc68e000640 1 --2- 172.21.15.100:0/2191743628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6900cdd60 0x7fc6900cf9a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc690074300 tx=0x7fc67c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ea9551223b574f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:12.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.023+0000 7fc68d600640 1 -- 172.21.15.100:0/2191743628 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c0b1070 con 0x7fc6900cdd60 2024-04-23T14:35:12.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.023+0000 7fc68d600640 1 -- 172.21.15.100:0/2191743628 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc67c0ac600 con 0x7fc6900cdd60 2024-04-23T14:35:12.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.023+0000 7fc68d600640 1 -- 172.21.15.100:0/2191743628 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c0ac9d0 con 0x7fc6900cdd60 2024-04-23T14:35:12.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.023+0000 7fc695600640 1 -- 172.21.15.100:0/2191743628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6900cdd60 msgr2=0x7fc6900cf9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.023+0000 7fc695600640 1 --2- 172.21.15.100:0/2191743628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6900cdd60 0x7fc6900cf9a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc690074300 tx=0x7fc67c01c520 comp rx=0 tx=0).stop 2024-04-23T14:35:12.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.023+0000 7fc695600640 1 -- 172.21.15.100:0/2191743628 shutdown_connections 2024-04-23T14:35:12.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.023+0000 7fc695600640 1 --2- 172.21.15.100:0/2191743628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6900cdd60 0x7fc6900cf9a0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.023+0000 7fc695600640 1 --2- 172.21.15.100:0/2191743628 >> v2:172.21.15.151:3300/0 conn(0x7fc69015e830 0x7fc69015ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.023+0000 7fc695600640 1 -- 172.21.15.100:0/2191743628 >> 172.21.15.100:0/2191743628 conn(0x7fc6900704d0 msgr2=0x7fc6900708e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:12.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.023+0000 7fc695600640 1 -- 172.21.15.100:0/2191743628 shutdown_connections 2024-04-23T14:35:12.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.023+0000 7fc695600640 1 -- 172.21.15.100:0/2191743628 wait complete. 2024-04-23T14:35:12.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc695600640 1 Processor -- start 2024-04-23T14:35:12.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc695600640 1 -- start start 2024-04-23T14:35:12.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc695600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc6900cdd60 0x7fc690153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc695600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6900cfee0 0x7fc6901596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc695600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc69015e830 0x7fc690152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc695600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc69016b610 con 0x7fc6900cfee0 2024-04-23T14:35:12.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc695600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc69016b490 con 0x7fc69015e830 2024-04-23T14:35:12.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc695600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc69016b790 con 0x7fc6900cdd60 2024-04-23T14:35:12.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc68e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6900cfee0 0x7fc6901596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc68e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6900cfee0 0x7fc6901596a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41024/0 (socket says 172.21.15.100:41024) 2024-04-23T14:35:12.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc68e000640 1 -- 172.21.15.100:0/1391135253 learned_addr learned my addr 172.21.15.100:0/1391135253 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:12.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc68ea00640 1 --2- 172.21.15.100:0/1391135253 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc6900cdd60 0x7fc690153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc68f400640 1 --2- 172.21.15.100:0/1391135253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc69015e830 0x7fc690152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc68e000640 1 -- 172.21.15.100:0/1391135253 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc6900cdd60 msgr2=0x7fc690153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc68e000640 1 --2- 172.21.15.100:0/1391135253 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc6900cdd60 0x7fc690153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc68e000640 1 -- 172.21.15.100:0/1391135253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc69015e830 msgr2=0x7fc690152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc68e000640 1 --2- 172.21.15.100:0/1391135253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc69015e830 0x7fc690152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc68e000640 1 -- 172.21.15.100:0/1391135253 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc690159ef0 con 0x7fc6900cfee0 2024-04-23T14:35:12.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc68ea00640 1 --2- 172.21.15.100:0/1391135253 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc6900cdd60 0x7fc690153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:35:12.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc68f400640 1 --2- 172.21.15.100:0/1391135253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc69015e830 0x7fc690152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:35:12.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc68e000640 1 --2- 172.21.15.100:0/1391135253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6900cfee0 0x7fc6901596a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc67c01a150 tx=0x7fc67c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:12.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc686a00640 1 -- 172.21.15.100:0/1391135253 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c0b1040 con 0x7fc6900cfee0 2024-04-23T14:35:12.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc686a00640 1 -- 172.21.15.100:0/1391135253 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc67c0b7450 con 0x7fc6900cfee0 2024-04-23T14:35:12.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc686a00640 1 -- 172.21.15.100:0/1391135253 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c0b7840 con 0x7fc6900cfee0 2024-04-23T14:35:12.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc695600640 1 -- 172.21.15.100:0/1391135253 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69015a1d0 con 0x7fc6900cfee0 2024-04-23T14:35:12.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.027+0000 7fc695600640 1 -- 172.21.15.100:0/1391135253 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc69015ce00 con 0x7fc6900cfee0 2024-04-23T14:35:12.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.031+0000 7fc695600640 1 -- 172.21.15.100:0/1391135253 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc69015d4f0 con 0x7fc6900cfee0 2024-04-23T14:35:12.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.031+0000 7fc686a00640 1 -- 172.21.15.100:0/1391135253 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc67c0bf3c0 con 0x7fc6900cfee0 2024-04-23T14:35:12.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.031+0000 7fc686a00640 1 --2- 172.21.15.100:0/1391135253 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc670068380 0x7fc67006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.031+0000 7fc686a00640 1 -- 172.21.15.100:0/1391135253 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7fc67c120740 con 0x7fc6900cfee0 2024-04-23T14:35:12.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.031+0000 7fc68ea00640 1 --2- 172.21.15.100:0/1391135253 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc670068380 0x7fc67006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.031+0000 7fc686a00640 1 --2- 172.21.15.100:0/1391135253 >> [v2:172.21.15.100:6816/1257826180,v1:172.21.15.100:6817/1257826180] conn(0x7fc67006e820 0x7fc670070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.031+0000 7fc686a00640 1 -- 172.21.15.100:0/1391135253 --> [v2:172.21.15.100:6816/1257826180,v1:172.21.15.100:6817/1257826180] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc670071330 con 0x7fc67006e820 2024-04-23T14:35:12.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.031+0000 7fc686a00640 1 -- 172.21.15.100:0/1391135253 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fc67c120b30 con 0x7fc6900cfee0 2024-04-23T14:35:12.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.031+0000 7fc68f400640 1 --2- 172.21.15.100:0/1391135253 >> [v2:172.21.15.100:6816/1257826180,v1:172.21.15.100:6817/1257826180] conn(0x7fc67006e820 0x7fc670070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.031+0000 7fc68ea00640 1 --2- 172.21.15.100:0/1391135253 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc670068380 0x7fc67006a840 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc6901faee0 tx=0x7fc680008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:12.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.031+0000 7fc68f400640 1 --2- 172.21.15.100:0/1391135253 >> [v2:172.21.15.100:6816/1257826180,v1:172.21.15.100:6817/1257826180] conn(0x7fc67006e820 0x7fc670070c60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:12.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.035+0000 7fc686a00640 1 -- 172.21.15.100:0/1391135253 <== osd.2 v2:172.21.15.100:6816/1257826180 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc670071330 con 0x7fc67006e820 2024-04-23T14:35:12.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.047+0000 7fc695600640 1 -- 172.21.15.100:0/1391135253 --> [v2:172.21.15.100:6816/1257826180,v1:172.21.15.100:6817/1257826180] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc690068ad0 con 0x7fc67006e820 2024-04-23T14:35:12.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.047+0000 7fc686a00640 1 -- 172.21.15.100:0/1391135253 <== osd.2 v2:172.21.15.100:6816/1257826180 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc690068ad0 con 0x7fc67006e820 2024-04-23T14:35:12.047 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:35:12.048 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T14:35:12.048 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:35:12.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.047+0000 7fc695600640 1 -- 172.21.15.100:0/1391135253 >> [v2:172.21.15.100:6816/1257826180,v1:172.21.15.100:6817/1257826180] conn(0x7fc67006e820 msgr2=0x7fc670070c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.047+0000 7fc695600640 1 --2- 172.21.15.100:0/1391135253 >> [v2:172.21.15.100:6816/1257826180,v1:172.21.15.100:6817/1257826180] conn(0x7fc67006e820 0x7fc670070c60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.047+0000 7fc695600640 1 -- 172.21.15.100:0/1391135253 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc670068380 msgr2=0x7fc67006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.047+0000 7fc695600640 1 --2- 172.21.15.100:0/1391135253 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc670068380 0x7fc67006a840 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc6901faee0 tx=0x7fc680008040 comp rx=0 tx=0).stop 2024-04-23T14:35:12.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.047+0000 7fc695600640 1 -- 172.21.15.100:0/1391135253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6900cfee0 msgr2=0x7fc6901596a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.047+0000 7fc695600640 1 --2- 172.21.15.100:0/1391135253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6900cfee0 0x7fc6901596a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc67c01a150 tx=0x7fc67c01ad60 comp rx=0 tx=0).stop 2024-04-23T14:35:12.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.047+0000 7fc68f400640 1 -- 172.21.15.100:0/1391135253 reap_dead start 2024-04-23T14:35:12.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.047+0000 7fc695600640 1 -- 172.21.15.100:0/1391135253 shutdown_connections 2024-04-23T14:35:12.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.047+0000 7fc695600640 1 -- 172.21.15.100:0/1391135253 >> 172.21.15.100:0/1391135253 conn(0x7fc6900704d0 msgr2=0x7fc6900ae0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:12.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.047+0000 7fc695600640 1 -- 172.21.15.100:0/1391135253 shutdown_connections 2024-04-23T14:35:12.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.047+0000 7fc695600640 1 -- 172.21.15.100:0/1391135253 wait complete. 2024-04-23T14:35:12.065 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-23T14:35:12.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.199+0000 7f1cac400640 1 Processor -- start 2024-04-23T14:35:12.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.199+0000 7f1cac400640 1 -- start start 2024-04-23T14:35:12.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1cac400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f1ca40c6520 0x7f1ca4071d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1cac400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca40cdfc0 con 0x7f1ca40c6d20 2024-04-23T14:35:12.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1cac400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca40cfdf0 con 0x7f1ca415ae60 2024-04-23T14:35:12.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1cac400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f1ca40d00d0 con 0x7f1ca40c6520 2024-04-23T14:35:12.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1caa400640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1ca40c6d20 0x7f1ca40ccc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42210/0 (socket says 172.21.15.100:42210) 2024-04-23T14:35:12.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1caa400640 1 -- 172.21.15.100:0/1941571858 learned_addr learned my addr 172.21.15.100:0/1941571858 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:12.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1ca9000640 1 --2- 172.21.15.100:0/1941571858 >> v2:172.21.15.151:3300/0 conn(0x7f1ca40c6520 0x7f1ca4071d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1ca3e00640 1 -- 172.21.15.100:0/1941571858 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3538175610 0 0) 0x7f1ca40cdfc0 con 0x7f1ca40c6d20 2024-04-23T14:35:12.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1ca3e00640 1 -- 172.21.15.100:0/1941571858 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c8c003450 con 0x7f1ca40c6d20 2024-04-23T14:35:12.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1ca3e00640 1 -- 172.21.15.100:0/1941571858 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3271214315 0 0) 0x7f1ca40cfdf0 con 0x7f1ca415ae60 2024-04-23T14:35:12.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1ca3e00640 1 -- 172.21.15.100:0/1941571858 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ca40cdfc0 con 0x7f1ca415ae60 2024-04-23T14:35:12.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1ca3e00640 1 -- 172.21.15.100:0/1941571858 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2925044791 0 0) 0x7f1c8c003450 con 0x7f1ca40c6d20 2024-04-23T14:35:12.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1ca3e00640 1 -- 172.21.15.100:0/1941571858 >> v2:172.21.15.151:3300/0 conn(0x7f1ca40c6520 msgr2=0x7f1ca4071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:12.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1ca3e00640 1 --2- 172.21.15.100:0/1941571858 >> v2:172.21.15.151:3300/0 conn(0x7f1ca40c6520 0x7f1ca4071d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1ca3e00640 1 -- 172.21.15.100:0/1941571858 >> v1:172.21.15.151:6789/0 conn(0x7f1ca415ae60 legacy=0x7f1ca406c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1ca3e00640 1 -- 172.21.15.100:0/1941571858 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca40ce260 con 0x7f1ca40c6d20 2024-04-23T14:35:12.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1ca3e00640 1 -- 172.21.15.100:0/1941571858 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f1c9c001680 con 0x7f1ca40c6d20 2024-04-23T14:35:12.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1ca9000640 1 --2- 172.21.15.100:0/1941571858 >> v2:172.21.15.151:3300/0 conn(0x7f1ca40c6520 0x7f1ca4071d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:35:12.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1ca3e00640 1 -- 172.21.15.100:0/1941571858 >> v1:172.21.15.100:6789/0 conn(0x7f1ca40c6d20 legacy=0x7f1ca40ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1ca3e00640 1 --2- 172.21.15.100:0/1941571858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c8c003c40 0x7f1c8c008860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1ca3e00640 1 --2- 172.21.15.100:0/1941571858 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1c8c004170 0x7f1c8c00ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1ca3e00640 1 --2- 172.21.15.100:0/1941571858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c8c00b4c0 0x7f1c8c00d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1ca3e00640 1 -- 172.21.15.100:0/1941571858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca40d00d0 con 0x7f1c8c00b4c0 2024-04-23T14:35:12.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1ca3e00640 1 -- 172.21.15.100:0/1941571858 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c00e260 con 0x7f1c8c003c40 2024-04-23T14:35:12.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1ca3e00640 1 -- 172.21.15.100:0/1941571858 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1c8c00e460 con 0x7f1c8c004170 2024-04-23T14:35:12.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1caa400640 1 --2- 172.21.15.100:0/1941571858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c8c00b4c0 0x7f1c8c00d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1ca9a00640 1 --2- 172.21.15.100:0/1941571858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c8c003c40 0x7f1c8c008860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1ca9000640 1 --2- 172.21.15.100:0/1941571858 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1c8c004170 0x7f1c8c00ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1caa400640 1 -- 172.21.15.100:0/1941571858 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1c8c004170 msgr2=0x7f1c8c00ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:12.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1caa400640 1 --2- 172.21.15.100:0/1941571858 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1c8c004170 0x7f1c8c00ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1caa400640 1 -- 172.21.15.100:0/1941571858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c8c003c40 msgr2=0x7f1c8c008860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:12.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1caa400640 1 --2- 172.21.15.100:0/1941571858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c8c003c40 0x7f1c8c008860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.203+0000 7f1caa400640 1 -- 172.21.15.100:0/1941571858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8c003450 con 0x7f1c8c00b4c0 2024-04-23T14:35:12.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.207+0000 7f1caa400640 1 -- 172.21.15.100:0/1941571858 reap_dead start 2024-04-23T14:35:12.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.207+0000 7f1caa400640 1 --2- 172.21.15.100:0/1941571858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c8c00b4c0 0x7f1c8c00d980 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00c160 tx=0x7f1c9c01f350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d98a2e32979e046 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:12.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.207+0000 7f1ca3e00640 1 -- 172.21.15.100:0/1941571858 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c007cb0 con 0x7f1c8c00b4c0 2024-04-23T14:35:12.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.207+0000 7f1ca3e00640 1 -- 172.21.15.100:0/1941571858 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c9c005b00 con 0x7f1c8c00b4c0 2024-04-23T14:35:12.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.207+0000 7f1ca3e00640 1 -- 172.21.15.100:0/1941571858 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c007070 con 0x7f1c8c00b4c0 2024-04-23T14:35:12.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.207+0000 7f1cac400640 1 -- 172.21.15.100:0/1941571858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c8c00b4c0 msgr2=0x7f1c8c00d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.207+0000 7f1cac400640 1 --2- 172.21.15.100:0/1941571858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c8c00b4c0 0x7f1c8c00d980 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00c160 tx=0x7f1c9c01f350 comp rx=0 tx=0).stop 2024-04-23T14:35:12.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.207+0000 7f1cac400640 1 -- 172.21.15.100:0/1941571858 shutdown_connections 2024-04-23T14:35:12.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.207+0000 7f1cac400640 1 --2- 172.21.15.100:0/1941571858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c8c00b4c0 0x7f1c8c00d980 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.207+0000 7f1cac400640 1 -- 172.21.15.100:0/1941571858 >> 172.21.15.100:0/1941571858 conn(0x7f1ca40b5310 msgr2=0x7f1ca40b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:12.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.207+0000 7f1cac400640 1 -- 172.21.15.100:0/1941571858 shutdown_connections 2024-04-23T14:35:12.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.207+0000 7f1cac400640 1 -- 172.21.15.100:0/1941571858 wait complete. 2024-04-23T14:35:12.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.207+0000 7f1cac400640 1 Processor -- start 2024-04-23T14:35:12.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1cac400640 1 -- start start 2024-04-23T14:35:12.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1cac400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c8c00b4c0 0x7f1ca40b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1cac400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1ca415cb20 0x7f1ca40b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1cac400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ca415cf60 0x7f1ca414f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1cac400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca40ce540 con 0x7f1ca415cf60 2024-04-23T14:35:12.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1cac400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca40cfb90 con 0x7f1c8c00b4c0 2024-04-23T14:35:12.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1cac400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca40d0250 con 0x7f1ca415cb20 2024-04-23T14:35:12.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1caa400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ca415cf60 0x7f1ca414f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1caa400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ca415cf60 0x7f1ca414f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41044/0 (socket says 172.21.15.100:41044) 2024-04-23T14:35:12.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1caa400640 1 -- 172.21.15.100:0/1755973029 learned_addr learned my addr 172.21.15.100:0/1755973029 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:12.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1ca9a00640 1 --2- 172.21.15.100:0/1755973029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c8c00b4c0 0x7f1ca40b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1ca9000640 1 --2- 172.21.15.100:0/1755973029 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1ca415cb20 0x7f1ca40b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1caa400640 1 -- 172.21.15.100:0/1755973029 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1ca415cb20 msgr2=0x7f1ca40b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1caa400640 1 --2- 172.21.15.100:0/1755973029 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1ca415cb20 0x7f1ca40b0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1caa400640 1 -- 172.21.15.100:0/1755973029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c8c00b4c0 msgr2=0x7f1ca40b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1caa400640 1 --2- 172.21.15.100:0/1755973029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c8c00b4c0 0x7f1ca40b12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1caa400640 1 -- 172.21.15.100:0/1755973029 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca4159210 con 0x7f1ca415cf60 2024-04-23T14:35:12.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1caa400640 1 --2- 172.21.15.100:0/1755973029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ca415cf60 0x7f1ca414f670 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c01f180 tx=0x7f1c8c003740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:12.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1ca1600640 1 -- 172.21.15.100:0/1755973029 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c007070 con 0x7f1ca415cf60 2024-04-23T14:35:12.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1ca1600640 1 -- 172.21.15.100:0/1755973029 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c9c007210 con 0x7f1ca415cf60 2024-04-23T14:35:12.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1cac400640 1 -- 172.21.15.100:0/1755973029 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca4157230 con 0x7f1ca415cf60 2024-04-23T14:35:12.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1ca1600640 1 -- 172.21.15.100:0/1755973029 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c0075c0 con 0x7f1ca415cf60 2024-04-23T14:35:12.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1cac400640 1 -- 172.21.15.100:0/1755973029 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca41577f0 con 0x7f1ca415cf60 2024-04-23T14:35:12.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.211+0000 7f1cac400640 1 -- 172.21.15.100:0/1755973029 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca41593e0 con 0x7f1ca415cf60 2024-04-23T14:35:12.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.215+0000 7f1ca1600640 1 -- 172.21.15.100:0/1755973029 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1c9c0b93f0 con 0x7f1ca415cf60 2024-04-23T14:35:12.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.215+0000 7f1ca1600640 1 --2- 172.21.15.100:0/1755973029 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1c840683b0 0x7f1c8406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.215+0000 7f1ca1600640 1 -- 172.21.15.100:0/1755973029 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f1c9c123040 con 0x7f1ca415cf60 2024-04-23T14:35:12.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.215+0000 7f1ca9a00640 1 --2- 172.21.15.100:0/1755973029 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1c840683b0 0x7f1c8406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.215+0000 7f1ca9a00640 1 --2- 172.21.15.100:0/1755973029 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1c840683b0 0x7f1c8406a870 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1c98000f90 tx=0x7f1c98009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:12.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.219+0000 7f1ca1600640 1 -- 172.21.15.100:0/1755973029 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1c9c0b5070 con 0x7f1ca415cf60 2024-04-23T14:35:12.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.375+0000 7f1cac400640 1 -- 172.21.15.100:0/1755973029 --> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f1ca4068ad0 con 0x7f1c840683b0 2024-04-23T14:35:12.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.379+0000 7f1ca1600640 1 -- 172.21.15.100:0/1755973029 <== mgr.4101 v2:172.21.15.100:6832/2859853957 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1ca4068ad0 con 0x7f1c840683b0 2024-04-23T14:35:12.379 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 2 to deep-scrub 2024-04-23T14:35:12.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.379+0000 7f1cac400640 1 -- 172.21.15.100:0/1755973029 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1c840683b0 msgr2=0x7f1c8406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.379+0000 7f1cac400640 1 --2- 172.21.15.100:0/1755973029 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1c840683b0 0x7f1c8406a870 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1c98000f90 tx=0x7f1c98009040 comp rx=0 tx=0).stop 2024-04-23T14:35:12.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.379+0000 7f1cac400640 1 -- 172.21.15.100:0/1755973029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ca415cf60 msgr2=0x7f1ca414f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.379+0000 7f1cac400640 1 --2- 172.21.15.100:0/1755973029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ca415cf60 0x7f1ca414f670 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c01f180 tx=0x7f1c8c003740 comp rx=0 tx=0).stop 2024-04-23T14:35:12.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.379+0000 7f1cac400640 1 -- 172.21.15.100:0/1755973029 shutdown_connections 2024-04-23T14:35:12.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.379+0000 7f1cac400640 1 --2- 172.21.15.100:0/1755973029 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f1c840683b0 0x7f1c8406a870 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.379+0000 7f1cac400640 1 --2- 172.21.15.100:0/1755973029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ca415cf60 0x7f1ca414f670 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.379+0000 7f1cac400640 1 --2- 172.21.15.100:0/1755973029 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1ca415cb20 0x7f1ca40b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.379+0000 7f1cac400640 1 --2- 172.21.15.100:0/1755973029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c8c00b4c0 0x7f1ca40b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.379+0000 7f1cac400640 1 -- 172.21.15.100:0/1755973029 >> 172.21.15.100:0/1755973029 conn(0x7f1ca40b5310 msgr2=0x7f1ca406ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:12.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.379+0000 7f1cac400640 1 -- 172.21.15.100:0/1755973029 shutdown_connections 2024-04-23T14:35:12.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.379+0000 7f1cac400640 1 -- 172.21.15.100:0/1755973029 wait complete. 2024-04-23T14:35:12.399 INFO:tasks.ceph:Scrubbing osd.3 2024-04-23T14:35:12.399 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-23T14:35:12.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.539+0000 7f70a1e00640 1 Processor -- start 2024-04-23T14:35:12.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.539+0000 7f70a1e00640 1 -- start start 2024-04-23T14:35:12.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.539+0000 7f70a1e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f709c15e7a0 0x7f709c0aea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.539+0000 7f70a1e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f709c0cee40 0x7f709c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.539+0000 7f70a1e00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f709c0ada30 con 0x7f709c0cf7c0 2024-04-23T14:35:12.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.539+0000 7f70a1e00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f709c0aefb0 con 0x7f709c15e7a0 2024-04-23T14:35:12.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.539+0000 7f70a1e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f709c0adcb0 con 0x7f709c0cee40 2024-04-23T14:35:12.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.539+0000 7f709aa00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f709c15e7a0 0x7f709c0aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.539+0000 7f709aa00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f709c15e7a0 0x7f709c0aea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41058/0 (socket says 172.21.15.100:41058) 2024-04-23T14:35:12.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.539+0000 7f709aa00640 1 -- 172.21.15.100:0/1023222016 learned_addr learned my addr 172.21.15.100:0/1023222016 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:12.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.539+0000 7f7092000640 1 --2- 172.21.15.100:0/1023222016 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f709c0cee40 0x7f709c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.539+0000 7f709aa00640 1 -- 172.21.15.100:0/1023222016 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f709c0cee40 msgr2=0x7f709c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:12.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.539+0000 7f709aa00640 1 --2- 172.21.15.100:0/1023222016 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f709c0cee40 0x7f709c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.539+0000 7f709aa00640 1 -- 172.21.15.100:0/1023222016 >> v1:172.21.15.100:6789/0 conn(0x7f709c0cf7c0 legacy=0x7f709c06e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.539+0000 7f709aa00640 1 -- 172.21.15.100:0/1023222016 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f709c0adf90 con 0x7f709c15e7a0 2024-04-23T14:35:12.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.539+0000 7f7092000640 1 --2- 172.21.15.100:0/1023222016 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f709c0cee40 0x7f709c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:35:12.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.543+0000 7f709aa00640 1 --2- 172.21.15.100:0/1023222016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f709c15e7a0 0x7f709c0aea70 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f708c009e50 tx=0x7f708c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cdda35c42e31d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:12.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.543+0000 7f709a000640 1 -- 172.21.15.100:0/1023222016 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c0a5900 con 0x7f709c15e7a0 2024-04-23T14:35:12.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.543+0000 7f709a000640 1 -- 172.21.15.100:0/1023222016 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f708c0a5aa0 con 0x7f709c15e7a0 2024-04-23T14:35:12.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.543+0000 7f709a000640 1 -- 172.21.15.100:0/1023222016 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c0ad3f0 con 0x7f709c15e7a0 2024-04-23T14:35:12.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.543+0000 7f70a1e00640 1 -- 172.21.15.100:0/1023222016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f709c15e7a0 msgr2=0x7f709c0aea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.543+0000 7f70a1e00640 1 --2- 172.21.15.100:0/1023222016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f709c15e7a0 0x7f709c0aea70 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f708c009e50 tx=0x7f708c01ca30 comp rx=0 tx=0).stop 2024-04-23T14:35:12.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.543+0000 7f70a1e00640 1 -- 172.21.15.100:0/1023222016 shutdown_connections 2024-04-23T14:35:12.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.543+0000 7f70a1e00640 1 --2- 172.21.15.100:0/1023222016 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f709c0cee40 0x7f709c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.543+0000 7f70a1e00640 1 --2- 172.21.15.100:0/1023222016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f709c15e7a0 0x7f709c0aea70 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.543+0000 7f70a1e00640 1 -- 172.21.15.100:0/1023222016 >> 172.21.15.100:0/1023222016 conn(0x7f709c0b5200 msgr2=0x7f709c0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:12.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.543+0000 7f70a1e00640 1 -- 172.21.15.100:0/1023222016 shutdown_connections 2024-04-23T14:35:12.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.543+0000 7f70a1e00640 1 -- 172.21.15.100:0/1023222016 wait complete. 2024-04-23T14:35:12.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.543+0000 7f70a1e00640 1 Processor -- start 2024-04-23T14:35:12.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.543+0000 7f70a1e00640 1 -- start start 2024-04-23T14:35:12.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f70a1e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f709c0cee40 0x7f709c159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f70a1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f709c0cf7c0 0x7f709c0cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f70a1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f709c15e7a0 0x7f709c0b5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f70a1e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f709c15ed00 con 0x7f709c0cf7c0 2024-04-23T14:35:12.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f70a1e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f709c15eb80 con 0x7f709c15e7a0 2024-04-23T14:35:12.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f70a1e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f709c16b400 con 0x7f709c0cee40 2024-04-23T14:35:12.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f7092000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f709c0cf7c0 0x7f709c0cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f7092000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f709c0cf7c0 0x7f709c0cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41062/0 (socket says 172.21.15.100:41062) 2024-04-23T14:35:12.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f7092000640 1 -- 172.21.15.100:0/3313780980 learned_addr learned my addr 172.21.15.100:0/3313780980 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:12.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f709aa00640 1 --2- 172.21.15.100:0/3313780980 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f709c0cee40 0x7f709c159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f709b400640 1 --2- 172.21.15.100:0/3313780980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f709c15e7a0 0x7f709c0b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f7092000640 1 -- 172.21.15.100:0/3313780980 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f709c0cee40 msgr2=0x7f709c159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f7092000640 1 --2- 172.21.15.100:0/3313780980 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f709c0cee40 0x7f709c159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f7092000640 1 -- 172.21.15.100:0/3313780980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f709c15e7a0 msgr2=0x7f709c0b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f7092000640 1 --2- 172.21.15.100:0/3313780980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f709c15e7a0 0x7f709c0b5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f7092000640 1 -- 172.21.15.100:0/3313780980 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f709c0c4c50 con 0x7f709c0cf7c0 2024-04-23T14:35:12.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f7092000640 1 --2- 172.21.15.100:0/3313780980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f709c0cf7c0 0x7f709c0cad40 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f708400d830 tx=0x7f708400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:12.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f7093e00640 1 -- 172.21.15.100:0/3313780980 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708400b6c0 con 0x7f709c0cf7c0 2024-04-23T14:35:12.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f7093e00640 1 -- 172.21.15.100:0/3313780980 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7084019070 con 0x7f709c0cf7c0 2024-04-23T14:35:12.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f7093e00640 1 -- 172.21.15.100:0/3313780980 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708400baa0 con 0x7f709c0cf7c0 2024-04-23T14:35:12.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f70a1e00640 1 -- 172.21.15.100:0/3313780980 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f709c0cb4b0 con 0x7f709c0cf7c0 2024-04-23T14:35:12.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f70a1e00640 1 -- 172.21.15.100:0/3313780980 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f709c0c51d0 con 0x7f709c0cf7c0 2024-04-23T14:35:12.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f70a1e00640 1 -- 172.21.15.100:0/3313780980 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f709c068ad0 con 0x7f709c0cf7c0 2024-04-23T14:35:12.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.547+0000 7f7093e00640 1 -- 172.21.15.100:0/3313780980 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f708401e460 con 0x7f709c0cf7c0 2024-04-23T14:35:12.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.551+0000 7f7093e00640 1 --2- 172.21.15.100:0/3313780980 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f70780683b0 0x7f707806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.551+0000 7f7093e00640 1 -- 172.21.15.100:0/3313780980 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f7084086eb0 con 0x7f709c0cf7c0 2024-04-23T14:35:12.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.551+0000 7f709aa00640 1 --2- 172.21.15.100:0/3313780980 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f70780683b0 0x7f707806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.551+0000 7f7093e00640 1 --2- 172.21.15.100:0/3313780980 >> [v2:172.21.15.100:6824/1142032886,v1:172.21.15.100:6825/1142032886] conn(0x7f707806e850 0x7f7078070c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.551+0000 7f7093e00640 1 -- 172.21.15.100:0/3313780980 --> [v2:172.21.15.100:6824/1142032886,v1:172.21.15.100:6825/1142032886] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f70840044a0 con 0x7f707806e850 2024-04-23T14:35:12.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.551+0000 7f7093e00640 1 -- 172.21.15.100:0/3313780980 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f70840894d0 con 0x7f709c0cf7c0 2024-04-23T14:35:12.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.551+0000 7f709b400640 1 --2- 172.21.15.100:0/3313780980 >> [v2:172.21.15.100:6824/1142032886,v1:172.21.15.100:6825/1142032886] conn(0x7f707806e850 0x7f7078070c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.551+0000 7f709aa00640 1 --2- 172.21.15.100:0/3313780980 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f70780683b0 0x7f707806a870 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f708c0ae000 tx=0x7f708c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:12.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.551+0000 7f709b400640 1 --2- 172.21.15.100:0/3313780980 >> [v2:172.21.15.100:6824/1142032886,v1:172.21.15.100:6825/1142032886] conn(0x7f707806e850 0x7f7078070c90 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:12.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.551+0000 7f7093e00640 1 -- 172.21.15.100:0/3313780980 <== osd.3 v2:172.21.15.100:6824/1142032886 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f70840044a0 con 0x7f707806e850 2024-04-23T14:35:12.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.563+0000 7f70a1e00640 1 -- 172.21.15.100:0/3313780980 --> [v2:172.21.15.100:6824/1142032886,v1:172.21.15.100:6825/1142032886] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f709c1550f0 con 0x7f707806e850 2024-04-23T14:35:12.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.563+0000 7f7093e00640 1 -- 172.21.15.100:0/3313780980 <== osd.3 v2:172.21.15.100:6824/1142032886 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f709c1550f0 con 0x7f707806e850 2024-04-23T14:35:12.566 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:35:12.566 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T14:35:12.566 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:35:12.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.563+0000 7f70a1e00640 1 -- 172.21.15.100:0/3313780980 >> [v2:172.21.15.100:6824/1142032886,v1:172.21.15.100:6825/1142032886] conn(0x7f707806e850 msgr2=0x7f7078070c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.563+0000 7f70a1e00640 1 --2- 172.21.15.100:0/3313780980 >> [v2:172.21.15.100:6824/1142032886,v1:172.21.15.100:6825/1142032886] conn(0x7f707806e850 0x7f7078070c90 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.563+0000 7f70a1e00640 1 -- 172.21.15.100:0/3313780980 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f70780683b0 msgr2=0x7f707806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.563+0000 7f70a1e00640 1 --2- 172.21.15.100:0/3313780980 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f70780683b0 0x7f707806a870 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f708c0ae000 tx=0x7f708c0a7070 comp rx=0 tx=0).stop 2024-04-23T14:35:12.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.563+0000 7f70a1e00640 1 -- 172.21.15.100:0/3313780980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f709c0cf7c0 msgr2=0x7f709c0cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.563+0000 7f70a1e00640 1 --2- 172.21.15.100:0/3313780980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f709c0cf7c0 0x7f709c0cad40 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f708400d830 tx=0x7f708400dcf0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.563+0000 7f709b400640 1 -- 172.21.15.100:0/3313780980 reap_dead start 2024-04-23T14:35:12.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.567+0000 7f70a1e00640 1 -- 172.21.15.100:0/3313780980 shutdown_connections 2024-04-23T14:35:12.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.567+0000 7f70a1e00640 1 -- 172.21.15.100:0/3313780980 >> 172.21.15.100:0/3313780980 conn(0x7f709c0b5200 msgr2=0x7f709c0990e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:12.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.567+0000 7f70a1e00640 1 -- 172.21.15.100:0/3313780980 shutdown_connections 2024-04-23T14:35:12.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.567+0000 7f70a1e00640 1 -- 172.21.15.100:0/3313780980 wait complete. 2024-04-23T14:35:12.583 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-23T14:35:12.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.723+0000 7f40bb200640 1 Processor -- start 2024-04-23T14:35:12.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.723+0000 7f40bb200640 1 -- start start 2024-04-23T14:35:12.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.723+0000 7f40bb200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f40bc0cf270 0x7f40bc169480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.723+0000 7f40bb200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40bc15eca0 con 0x7f40bc15e720 2024-04-23T14:35:12.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.723+0000 7f40bb200640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40bc15ef80 con 0x7f40bc0c77e0 2024-04-23T14:35:12.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.723+0000 7f40bb200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f40bc06d450 con 0x7f40bc0cf270 2024-04-23T14:35:12.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.723+0000 7f40ba800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f40bc0cf270 0x7f40bc169480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.723+0000 7f40b9e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f40bc15e720 0x7f40bc0c70d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42220/0 (socket says 172.21.15.100:42220) 2024-04-23T14:35:12.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.723+0000 7f40b9e00640 1 -- 172.21.15.100:0/1325755452 learned_addr learned my addr 172.21.15.100:0/1325755452 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:12.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.723+0000 7f40ba800640 1 -- 172.21.15.100:0/1325755452 >> v1:172.21.15.151:6789/0 conn(0x7f40bc0c77e0 legacy=0x7f40bc0ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:12.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.723+0000 7f40b9400640 1 -- 172.21.15.100:0/1325755452 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2384147151 0 0) 0x7f40bc15eca0 con 0x7f40bc15e720 2024-04-23T14:35:12.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.723+0000 7f40ba800640 1 -- 172.21.15.100:0/1325755452 >> v1:172.21.15.100:6789/0 conn(0x7f40bc15e720 legacy=0x7f40bc0c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.723+0000 7f40ba800640 1 -- 172.21.15.100:0/1325755452 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40bc15f260 con 0x7f40bc0cf270 2024-04-23T14:35:12.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.727+0000 7f40ba800640 1 --2- 172.21.15.100:0/1325755452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40bc0cf270 0x7f40bc169480 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f40b40097c0 tx=0x7f40b401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1927c4589ef8a82f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:12.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.727+0000 7f40b9400640 1 -- 172.21.15.100:0/1325755452 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b40b0070 con 0x7f40bc0cf270 2024-04-23T14:35:12.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.727+0000 7f40b9400640 1 -- 172.21.15.100:0/1325755452 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40b40ab600 con 0x7f40bc0cf270 2024-04-23T14:35:12.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.727+0000 7f40b9400640 1 -- 172.21.15.100:0/1325755452 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b40a4e60 con 0x7f40bc0cf270 2024-04-23T14:35:12.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.727+0000 7f40bb200640 1 -- 172.21.15.100:0/1325755452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40bc0cf270 msgr2=0x7f40bc169480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.727+0000 7f40bb200640 1 --2- 172.21.15.100:0/1325755452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40bc0cf270 0x7f40bc169480 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f40b40097c0 tx=0x7f40b401c3d0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.727+0000 7f40bb200640 1 -- 172.21.15.100:0/1325755452 shutdown_connections 2024-04-23T14:35:12.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.727+0000 7f40bb200640 1 --2- 172.21.15.100:0/1325755452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40bc0cf270 0x7f40bc169480 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.727+0000 7f40bb200640 1 -- 172.21.15.100:0/1325755452 >> 172.21.15.100:0/1325755452 conn(0x7f40bc0b3e60 msgr2=0x7f40bc0b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:12.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.727+0000 7f40bb200640 1 -- 172.21.15.100:0/1325755452 shutdown_connections 2024-04-23T14:35:12.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.727+0000 7f40bb200640 1 -- 172.21.15.100:0/1325755452 wait complete. 2024-04-23T14:35:12.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40bb200640 1 Processor -- start 2024-04-23T14:35:12.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40bb200640 1 -- start start 2024-04-23T14:35:12.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40bb200640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f40bc0c77e0 0x7f40bc1538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40bb200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40bc0cf270 0x7f40bc0d23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40bb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40bc15e720 0x7f40bc151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40bb200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc16c980 con 0x7f40bc15e720 2024-04-23T14:35:12.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40bb200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc16c800 con 0x7f40bc0cf270 2024-04-23T14:35:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40bb200640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f40bc16cb00 con 0x7f40bc0c77e0 2024-04-23T14:35:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40ba800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40bc15e720 0x7f40bc151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40ba800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40bc15e720 0x7f40bc151260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41084/0 (socket says 172.21.15.100:41084) 2024-04-23T14:35:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40ba800640 1 -- 172.21.15.100:0/2514799254 learned_addr learned my addr 172.21.15.100:0/2514799254 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40b1400640 1 --2- 172.21.15.100:0/2514799254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40bc0cf270 0x7f40bc0d23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40b9e00640 1 --2- 172.21.15.100:0/2514799254 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f40bc0c77e0 0x7f40bc1538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40ba800640 1 -- 172.21.15.100:0/2514799254 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f40bc0c77e0 msgr2=0x7f40bc1538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40ba800640 1 --2- 172.21.15.100:0/2514799254 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f40bc0c77e0 0x7f40bc1538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40ba800640 1 -- 172.21.15.100:0/2514799254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40bc0cf270 msgr2=0x7f40bc0d23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40ba800640 1 --2- 172.21.15.100:0/2514799254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40bc0cf270 0x7f40bc0d23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40ba800640 1 -- 172.21.15.100:0/2514799254 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40bc1541f0 con 0x7f40bc15e720 2024-04-23T14:35:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40b9e00640 1 --2- 172.21.15.100:0/2514799254 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f40bc0c77e0 0x7f40bc1538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:35:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40ba800640 1 --2- 172.21.15.100:0/2514799254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40bc15e720 0x7f40bc151260 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f40b40097c0 tx=0x7f40b401ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40b2a00640 1 -- 172.21.15.100:0/2514799254 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b40b0050 con 0x7f40bc15e720 2024-04-23T14:35:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40b2a00640 1 -- 172.21.15.100:0/2514799254 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40b40bc070 con 0x7f40bc15e720 2024-04-23T14:35:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40bb200640 1 -- 172.21.15.100:0/2514799254 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40bc1544d0 con 0x7f40bc15e720 2024-04-23T14:35:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40b2a00640 1 -- 172.21.15.100:0/2514799254 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b40b5830 con 0x7f40bc15e720 2024-04-23T14:35:12.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40bb200640 1 -- 172.21.15.100:0/2514799254 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40bc154a90 con 0x7f40bc15e720 2024-04-23T14:35:12.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.731+0000 7f40bb200640 1 -- 172.21.15.100:0/2514799254 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40bc075c20 con 0x7f40bc15e720 2024-04-23T14:35:12.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.735+0000 7f40b2a00640 1 -- 172.21.15.100:0/2514799254 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f40b40b59d0 con 0x7f40bc15e720 2024-04-23T14:35:12.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.735+0000 7f40b2a00640 1 --2- 172.21.15.100:0/2514799254 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f40980683b0 0x7f409806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:12.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.735+0000 7f40b2a00640 1 -- 172.21.15.100:0/2514799254 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f40b4120ff0 con 0x7f40bc15e720 2024-04-23T14:35:12.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.735+0000 7f40b9e00640 1 --2- 172.21.15.100:0/2514799254 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f40980683b0 0x7f409806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:12.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.735+0000 7f40b9e00640 1 --2- 172.21.15.100:0/2514799254 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f40980683b0 0x7f409806a870 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f40ac005fd0 tx=0x7f40ac005b80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:12.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.739+0000 7f40b2a00640 1 -- 172.21.15.100:0/2514799254 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f40b4016050 con 0x7f40bc15e720 2024-04-23T14:35:12.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.899+0000 7f40bb200640 1 -- 172.21.15.100:0/2514799254 --> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f40bc0cef00 con 0x7f40980683b0 2024-04-23T14:35:12.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.899+0000 7f40b2a00640 1 -- 172.21.15.100:0/2514799254 <== mgr.4101 v2:172.21.15.100:6832/2859853957 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f40bc0cef00 con 0x7f40980683b0 2024-04-23T14:35:12.899 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 3 to deep-scrub 2024-04-23T14:35:12.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.899+0000 7f40bb200640 1 -- 172.21.15.100:0/2514799254 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f40980683b0 msgr2=0x7f409806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.899+0000 7f40bb200640 1 --2- 172.21.15.100:0/2514799254 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f40980683b0 0x7f409806a870 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f40ac005fd0 tx=0x7f40ac005b80 comp rx=0 tx=0).stop 2024-04-23T14:35:12.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.899+0000 7f40bb200640 1 -- 172.21.15.100:0/2514799254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40bc15e720 msgr2=0x7f40bc151260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:12.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.899+0000 7f40bb200640 1 --2- 172.21.15.100:0/2514799254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40bc15e720 0x7f40bc151260 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f40b40097c0 tx=0x7f40b401ac10 comp rx=0 tx=0).stop 2024-04-23T14:35:12.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.899+0000 7f40bb200640 1 -- 172.21.15.100:0/2514799254 shutdown_connections 2024-04-23T14:35:12.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.899+0000 7f40bb200640 1 --2- 172.21.15.100:0/2514799254 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f40980683b0 0x7f409806a870 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.899+0000 7f40bb200640 1 --2- 172.21.15.100:0/2514799254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40bc15e720 0x7f40bc151260 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.899+0000 7f40bb200640 1 --2- 172.21.15.100:0/2514799254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40bc0cf270 0x7f40bc0d23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.899+0000 7f40bb200640 1 --2- 172.21.15.100:0/2514799254 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f40bc0c77e0 0x7f40bc1538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:12.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.899+0000 7f40bb200640 1 -- 172.21.15.100:0/2514799254 >> 172.21.15.100:0/2514799254 conn(0x7f40bc0b3e60 msgr2=0x7f40bc0ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:12.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.899+0000 7f40bb200640 1 -- 172.21.15.100:0/2514799254 shutdown_connections 2024-04-23T14:35:12.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:12.899+0000 7f40bb200640 1 -- 172.21.15.100:0/2514799254 wait complete. 2024-04-23T14:35:12.921 INFO:tasks.ceph:Scrubbing osd.4 2024-04-23T14:35:12.921 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-23T14:35:13.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.059+0000 7ff512600640 1 Processor -- start 2024-04-23T14:35:13.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.059+0000 7ff512600640 1 -- start start 2024-04-23T14:35:13.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.059+0000 7ff512600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff50c15e7a0 0x7ff50c0aea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.059+0000 7ff512600640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff50c06bc70 0x7ff50c06c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.059+0000 7ff512600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff50c0ada30 con 0x7ff50c0cee40 2024-04-23T14:35:13.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.059+0000 7ff512600640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7ff50c06f6e0 con 0x7ff50c06bc70 2024-04-23T14:35:13.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.059+0000 7ff512600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff50c0aefb0 con 0x7ff50c15e7a0 2024-04-23T14:35:13.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.059+0000 7ff50b400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff50c15e7a0 0x7ff50c0aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.059+0000 7ff50be00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff50c06bc70 0x7ff50c06c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.059+0000 7ff50b400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff50c15e7a0 0x7ff50c0aea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:34350/0 (socket says 172.21.15.100:34350) 2024-04-23T14:35:13.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.059+0000 7ff50b400640 1 -- 172.21.15.100:0/1508150105 learned_addr learned my addr 172.21.15.100:0/1508150105 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:13.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.059+0000 7ff50be00640 1 -- 172.21.15.100:0/1508150105 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff50c15e7a0 msgr2=0x7ff50c0aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:13.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.059+0000 7ff50be00640 1 --2- 172.21.15.100:0/1508150105 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff50c15e7a0 0x7ff50c0aea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.059+0000 7ff50be00640 1 -- 172.21.15.100:0/1508150105 >> v1:172.21.15.151:6789/0 conn(0x7ff50c0cee40 legacy=0x7ff50c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.059+0000 7ff50be00640 1 -- 172.21.15.100:0/1508150105 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50c0adcb0 con 0x7ff50c06bc70 2024-04-23T14:35:13.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.059+0000 7ff50b400640 1 --2- 172.21.15.100:0/1508150105 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff50c15e7a0 0x7ff50c0aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:35:13.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.063+0000 7ff50be00640 1 --2- 172.21.15.100:0/1508150105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff50c06bc70 0x7ff50c06c0e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff504009d90 tx=0x7ff50401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa9329d91cf6dfe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:13.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.063+0000 7ff50a000640 1 -- 172.21.15.100:0/1508150105 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5040b2070 con 0x7ff50c06bc70 2024-04-23T14:35:13.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.063+0000 7ff50a000640 1 -- 172.21.15.100:0/1508150105 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5040a5a00 con 0x7ff50c06bc70 2024-04-23T14:35:13.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.063+0000 7ff50a000640 1 -- 172.21.15.100:0/1508150105 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5040a5dd0 con 0x7ff50c06bc70 2024-04-23T14:35:13.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.063+0000 7ff512600640 1 -- 172.21.15.100:0/1508150105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff50c06bc70 msgr2=0x7ff50c06c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.063+0000 7ff512600640 1 --2- 172.21.15.100:0/1508150105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff50c06bc70 0x7ff50c06c0e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff504009d90 tx=0x7ff50401c940 comp rx=0 tx=0).stop 2024-04-23T14:35:13.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.063+0000 7ff512600640 1 -- 172.21.15.100:0/1508150105 shutdown_connections 2024-04-23T14:35:13.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.063+0000 7ff512600640 1 --2- 172.21.15.100:0/1508150105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff50c06bc70 0x7ff50c06c0e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.063+0000 7ff512600640 1 --2- 172.21.15.100:0/1508150105 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff50c15e7a0 0x7ff50c0aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.063+0000 7ff512600640 1 -- 172.21.15.100:0/1508150105 >> 172.21.15.100:0/1508150105 conn(0x7ff50c0b5200 msgr2=0x7ff50c0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:13.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.063+0000 7ff512600640 1 -- 172.21.15.100:0/1508150105 shutdown_connections 2024-04-23T14:35:13.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.063+0000 7ff512600640 1 -- 172.21.15.100:0/1508150105 wait complete. 2024-04-23T14:35:13.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff512600640 1 Processor -- start 2024-04-23T14:35:13.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff512600640 1 -- start start 2024-04-23T14:35:13.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff512600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff50c06bc70 0x7ff50c1562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff512600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff50c0cee40 0x7ff50c0c6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff512600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff50c15e7a0 0x7ff50c0992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff512600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c15ed00 con 0x7ff50c0cee40 2024-04-23T14:35:13.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff512600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c15eb80 con 0x7ff50c06bc70 2024-04-23T14:35:13.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff512600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff50c0cf990 con 0x7ff50c15e7a0 2024-04-23T14:35:13.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff50aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff50c0cee40 0x7ff50c0c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff50aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff50c0cee40 0x7ff50c0c6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41100/0 (socket says 172.21.15.100:41100) 2024-04-23T14:35:13.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff50aa00640 1 -- 172.21.15.100:0/2687345731 learned_addr learned my addr 172.21.15.100:0/2687345731 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:13.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff50b400640 1 --2- 172.21.15.100:0/2687345731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff50c06bc70 0x7ff50c1562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff50be00640 1 --2- 172.21.15.100:0/2687345731 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff50c15e7a0 0x7ff50c0992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff50aa00640 1 -- 172.21.15.100:0/2687345731 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff50c15e7a0 msgr2=0x7ff50c0992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff50aa00640 1 --2- 172.21.15.100:0/2687345731 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff50c15e7a0 0x7ff50c0992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff50aa00640 1 -- 172.21.15.100:0/2687345731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff50c06bc70 msgr2=0x7ff50c1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff50aa00640 1 --2- 172.21.15.100:0/2687345731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff50c06bc70 0x7ff50c1562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff50aa00640 1 -- 172.21.15.100:0/2687345731 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50c0c6d70 con 0x7ff50c0cee40 2024-04-23T14:35:13.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff50aa00640 1 --2- 172.21.15.100:0/2687345731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff50c0cee40 0x7ff50c0c6830 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800c450 tx=0x7ff4f800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:13.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff503400640 1 -- 172.21.15.100:0/2687345731 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8016020 con 0x7ff50c0cee40 2024-04-23T14:35:13.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff503400640 1 -- 172.21.15.100:0/2687345731 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4f80052d0 con 0x7ff50c0cee40 2024-04-23T14:35:13.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff512600640 1 -- 172.21.15.100:0/2687345731 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50c1568a0 con 0x7ff50c0cee40 2024-04-23T14:35:13.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.067+0000 7ff503400640 1 -- 172.21.15.100:0/2687345731 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8007260 con 0x7ff50c0cee40 2024-04-23T14:35:13.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.071+0000 7ff512600640 1 -- 172.21.15.100:0/2687345731 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff50c156e60 con 0x7ff50c0cee40 2024-04-23T14:35:13.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.071+0000 7ff512600640 1 -- 172.21.15.100:0/2687345731 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff50c068ad0 con 0x7ff50c0cee40 2024-04-23T14:35:13.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.071+0000 7ff503400640 1 -- 172.21.15.100:0/2687345731 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff4f8007400 con 0x7ff50c0cee40 2024-04-23T14:35:13.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.071+0000 7ff503400640 1 --2- 172.21.15.100:0/2687345731 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff4ec068380 0x7ff4ec06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.071+0000 7ff503400640 1 -- 172.21.15.100:0/2687345731 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7ff4f8086110 con 0x7ff50c0cee40 2024-04-23T14:35:13.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.071+0000 7ff503400640 1 --2- 172.21.15.100:0/2687345731 >> [v2:172.21.15.151:6800/4191386446,v1:172.21.15.151:6801/4191386446] conn(0x7ff4ec06e820 0x7ff4ec070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.071+0000 7ff503400640 1 -- 172.21.15.100:0/2687345731 --> [v2:172.21.15.151:6800/4191386446,v1:172.21.15.151:6801/4191386446] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff4f801eec0 con 0x7ff4ec06e820 2024-04-23T14:35:13.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.071+0000 7ff503400640 1 -- 172.21.15.100:0/2687345731 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7ff4f808d050 con 0x7ff50c0cee40 2024-04-23T14:35:13.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.071+0000 7ff50b400640 1 --2- 172.21.15.100:0/2687345731 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff4ec068380 0x7ff4ec06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.071+0000 7ff50be00640 1 --2- 172.21.15.100:0/2687345731 >> [v2:172.21.15.151:6800/4191386446,v1:172.21.15.151:6801/4191386446] conn(0x7ff4ec06e820 0x7ff4ec070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.071+0000 7ff50b400640 1 --2- 172.21.15.100:0/2687345731 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff4ec068380 0x7ff4ec06a840 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009630 tx=0x7ff4fc009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:13.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.075+0000 7ff50be00640 1 --2- 172.21.15.100:0/2687345731 >> [v2:172.21.15.151:6800/4191386446,v1:172.21.15.151:6801/4191386446] conn(0x7ff4ec06e820 0x7ff4ec070c60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:13.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.075+0000 7ff503400640 1 -- 172.21.15.100:0/2687345731 <== osd.4 v2:172.21.15.151:6800/4191386446 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff4f801eec0 con 0x7ff4ec06e820 2024-04-23T14:35:13.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.087+0000 7ff512600640 1 -- 172.21.15.100:0/2687345731 --> [v2:172.21.15.151:6800/4191386446,v1:172.21.15.151:6801/4191386446] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff50c0cf770 con 0x7ff4ec06e820 2024-04-23T14:35:13.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.087+0000 7ff503400640 1 -- 172.21.15.100:0/2687345731 <== osd.4 v2:172.21.15.151:6800/4191386446 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff50c0cf770 con 0x7ff4ec06e820 2024-04-23T14:35:13.089 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:35:13.089 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T14:35:13.089 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:35:13.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.087+0000 7ff512600640 1 -- 172.21.15.100:0/2687345731 >> [v2:172.21.15.151:6800/4191386446,v1:172.21.15.151:6801/4191386446] conn(0x7ff4ec06e820 msgr2=0x7ff4ec070c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.087+0000 7ff512600640 1 --2- 172.21.15.100:0/2687345731 >> [v2:172.21.15.151:6800/4191386446,v1:172.21.15.151:6801/4191386446] conn(0x7ff4ec06e820 0x7ff4ec070c60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.087+0000 7ff512600640 1 -- 172.21.15.100:0/2687345731 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff4ec068380 msgr2=0x7ff4ec06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.087+0000 7ff512600640 1 --2- 172.21.15.100:0/2687345731 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff4ec068380 0x7ff4ec06a840 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009630 tx=0x7ff4fc009240 comp rx=0 tx=0).stop 2024-04-23T14:35:13.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.087+0000 7ff512600640 1 -- 172.21.15.100:0/2687345731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff50c0cee40 msgr2=0x7ff50c0c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.087+0000 7ff512600640 1 --2- 172.21.15.100:0/2687345731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff50c0cee40 0x7ff50c0c6830 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800c450 tx=0x7ff4f800c910 comp rx=0 tx=0).stop 2024-04-23T14:35:13.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.087+0000 7ff50be00640 1 -- 172.21.15.100:0/2687345731 reap_dead start 2024-04-23T14:35:13.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.087+0000 7ff512600640 1 -- 172.21.15.100:0/2687345731 shutdown_connections 2024-04-23T14:35:13.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.087+0000 7ff512600640 1 -- 172.21.15.100:0/2687345731 >> 172.21.15.100:0/2687345731 conn(0x7ff50c0b5200 msgr2=0x7ff50c157da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:13.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.087+0000 7ff512600640 1 -- 172.21.15.100:0/2687345731 shutdown_connections 2024-04-23T14:35:13.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.087+0000 7ff512600640 1 -- 172.21.15.100:0/2687345731 wait complete. 2024-04-23T14:35:13.108 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-23T14:35:13.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.243+0000 7fc130e00640 1 Processor -- start 2024-04-23T14:35:13.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.243+0000 7fc130e00640 1 -- start start 2024-04-23T14:35:13.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.243+0000 7fc130e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc12c15e750 0x7fc12c0cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.243+0000 7fc130e00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc12c099ba0 con 0x7fc12c0c6310 2024-04-23T14:35:13.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.243+0000 7fc130e00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc12c0ccad0 con 0x7fc12c15eb30 2024-04-23T14:35:13.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.243+0000 7fc130e00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc12c099e80 con 0x7fc12c15e750 2024-04-23T14:35:13.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.243+0000 7fc129e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc12c15e750 0x7fc12c0cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.243+0000 7fc129e00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc12c15e750 0x7fc12c0cd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41116/0 (socket says 172.21.15.100:41116) 2024-04-23T14:35:13.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.243+0000 7fc129e00640 1 -- 172.21.15.100:0/3550676824 learned_addr learned my addr 172.21.15.100:0/3550676824 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:13.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.243+0000 7fc12a800640 1 --1- 172.21.15.100:0/3550676824 >> v1:172.21.15.100:6789/0 conn(0x7fc12c0c6310 0x7fc12c06c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42232/0 (socket says 172.21.15.100:42232) 2024-04-23T14:35:13.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.243+0000 7fc129e00640 1 -- 172.21.15.100:0/3550676824 >> v1:172.21.15.151:6789/0 conn(0x7fc12c15eb30 legacy=0x7fc12c162f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.243+0000 7fc129e00640 1 -- 172.21.15.100:0/3550676824 >> v1:172.21.15.100:6789/0 conn(0x7fc12c0c6310 legacy=0x7fc12c06c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.247+0000 7fc129e00640 1 -- 172.21.15.100:0/3550676824 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12c0ccdb0 con 0x7fc12c15e750 2024-04-23T14:35:13.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.247+0000 7fc129400640 1 -- 172.21.15.100:0/3550676824 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1724024966 0 0) 0x7fc12c099ba0 con 0x7fc12c0c6310 2024-04-23T14:35:13.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.247+0000 7fc129e00640 1 --2- 172.21.15.100:0/3550676824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c15e750 0x7fc12c0cd5b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fc11c009870 tx=0x7fc11c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ca2adf3e623531 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:13.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.247+0000 7fc129400640 1 -- 172.21.15.100:0/3550676824 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c0b0070 con 0x7fc12c15e750 2024-04-23T14:35:13.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.247+0000 7fc129400640 1 -- 172.21.15.100:0/3550676824 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc11c0ab600 con 0x7fc12c15e750 2024-04-23T14:35:13.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.247+0000 7fc129400640 1 -- 172.21.15.100:0/3550676824 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c0ab9d0 con 0x7fc12c15e750 2024-04-23T14:35:13.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.247+0000 7fc130e00640 1 -- 172.21.15.100:0/3550676824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c15e750 msgr2=0x7fc12c0cd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.247+0000 7fc130e00640 1 --2- 172.21.15.100:0/3550676824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c15e750 0x7fc12c0cd5b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fc11c009870 tx=0x7fc11c01c440 comp rx=0 tx=0).stop 2024-04-23T14:35:13.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.247+0000 7fc130e00640 1 -- 172.21.15.100:0/3550676824 shutdown_connections 2024-04-23T14:35:13.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.247+0000 7fc130e00640 1 --2- 172.21.15.100:0/3550676824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c15e750 0x7fc12c0cd5b0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.247+0000 7fc130e00640 1 -- 172.21.15.100:0/3550676824 >> 172.21.15.100:0/3550676824 conn(0x7fc12c0b5310 msgr2=0x7fc12c0b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:13.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.247+0000 7fc130e00640 1 -- 172.21.15.100:0/3550676824 shutdown_connections 2024-04-23T14:35:13.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.247+0000 7fc130e00640 1 -- 172.21.15.100:0/3550676824 wait complete. 2024-04-23T14:35:13.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc130e00640 1 Processor -- start 2024-04-23T14:35:13.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc130e00640 1 -- start start 2024-04-23T14:35:13.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc130e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc12c0c6310 0x7fc12c0ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc130e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c15e750 0x7fc12c15d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc130e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc12c15eb30 0x7fc12c072c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc130e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c06d0e0 con 0x7fc12c15e750 2024-04-23T14:35:13.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc130e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c06cf60 con 0x7fc12c0c6310 2024-04-23T14:35:13.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc130e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc12c16d0c0 con 0x7fc12c15eb30 2024-04-23T14:35:13.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc121400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c15e750 0x7fc12c15d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc121400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c15e750 0x7fc12c15d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41130/0 (socket says 172.21.15.100:41130) 2024-04-23T14:35:13.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc121400640 1 -- 172.21.15.100:0/1115128416 learned_addr learned my addr 172.21.15.100:0/1115128416 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:13.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc129e00640 1 --2- 172.21.15.100:0/1115128416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc12c0c6310 0x7fc12c0ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc12a800640 1 --2- 172.21.15.100:0/1115128416 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc12c15eb30 0x7fc12c072c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc121400640 1 -- 172.21.15.100:0/1115128416 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc12c15eb30 msgr2=0x7fc12c072c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc121400640 1 --2- 172.21.15.100:0/1115128416 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc12c15eb30 0x7fc12c072c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc121400640 1 -- 172.21.15.100:0/1115128416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc12c0c6310 msgr2=0x7fc12c0ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc121400640 1 --2- 172.21.15.100:0/1115128416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc12c0c6310 0x7fc12c0ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc121400640 1 -- 172.21.15.100:0/1115128416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12c15c720 con 0x7fc12c15e750 2024-04-23T14:35:13.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc12a800640 1 --2- 172.21.15.100:0/1115128416 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc12c15eb30 0x7fc12c072c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:35:13.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc121400640 1 --2- 172.21.15.100:0/1115128416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c15e750 0x7fc12c15d1c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc11400c3f0 tx=0x7fc11400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:13.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc122a00640 1 -- 172.21.15.100:0/1115128416 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114017070 con 0x7fc12c15e750 2024-04-23T14:35:13.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc122a00640 1 -- 172.21.15.100:0/1115128416 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1140052d0 con 0x7fc12c15e750 2024-04-23T14:35:13.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc130e00640 1 -- 172.21.15.100:0/1115128416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc12c0aee70 con 0x7fc12c15e750 2024-04-23T14:35:13.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc122a00640 1 -- 172.21.15.100:0/1115128416 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114003840 con 0x7fc12c15e750 2024-04-23T14:35:13.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.251+0000 7fc130e00640 1 -- 172.21.15.100:0/1115128416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc12c15c9c0 con 0x7fc12c15e750 2024-04-23T14:35:13.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.255+0000 7fc130e00640 1 -- 172.21.15.100:0/1115128416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc12c0af390 con 0x7fc12c15e750 2024-04-23T14:35:13.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.255+0000 7fc122a00640 1 -- 172.21.15.100:0/1115128416 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc11401a070 con 0x7fc12c15e750 2024-04-23T14:35:13.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.255+0000 7fc122a00640 1 --2- 172.21.15.100:0/1115128416 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc108068380 0x7fc10806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.255+0000 7fc122a00640 1 -- 172.21.15.100:0/1115128416 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7fc114085bd0 con 0x7fc12c15e750 2024-04-23T14:35:13.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.255+0000 7fc129e00640 1 --2- 172.21.15.100:0/1115128416 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc108068380 0x7fc10806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.255+0000 7fc129e00640 1 --2- 172.21.15.100:0/1115128416 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc108068380 0x7fc10806a840 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0ad000 tx=0x7fc11c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:13.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.259+0000 7fc122a00640 1 -- 172.21.15.100:0/1115128416 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc114005480 con 0x7fc12c15e750 2024-04-23T14:35:13.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.415+0000 7fc130e00640 1 -- 172.21.15.100:0/1115128416 --> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fc12c0994b0 con 0x7fc108068380 2024-04-23T14:35:13.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.415+0000 7fc122a00640 1 -- 172.21.15.100:0/1115128416 <== mgr.4101 v2:172.21.15.100:6832/2859853957 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc12c0994b0 con 0x7fc108068380 2024-04-23T14:35:13.417 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 4 to deep-scrub 2024-04-23T14:35:13.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.419+0000 7fc130e00640 1 -- 172.21.15.100:0/1115128416 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc108068380 msgr2=0x7fc10806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.419+0000 7fc130e00640 1 --2- 172.21.15.100:0/1115128416 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc108068380 0x7fc10806a840 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0ad000 tx=0x7fc11c0a6000 comp rx=0 tx=0).stop 2024-04-23T14:35:13.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.419+0000 7fc130e00640 1 -- 172.21.15.100:0/1115128416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c15e750 msgr2=0x7fc12c15d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.419+0000 7fc130e00640 1 --2- 172.21.15.100:0/1115128416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c15e750 0x7fc12c15d1c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc11400c3f0 tx=0x7fc11400c8b0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.419+0000 7fc130e00640 1 -- 172.21.15.100:0/1115128416 shutdown_connections 2024-04-23T14:35:13.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.419+0000 7fc130e00640 1 --2- 172.21.15.100:0/1115128416 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fc108068380 0x7fc10806a840 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.419+0000 7fc130e00640 1 --2- 172.21.15.100:0/1115128416 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc12c15eb30 0x7fc12c072c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.419+0000 7fc130e00640 1 --2- 172.21.15.100:0/1115128416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c15e750 0x7fc12c15d1c0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.419+0000 7fc130e00640 1 --2- 172.21.15.100:0/1115128416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc12c0c6310 0x7fc12c0ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.419+0000 7fc130e00640 1 -- 172.21.15.100:0/1115128416 >> 172.21.15.100:0/1115128416 conn(0x7fc12c0b5310 msgr2=0x7fc12c06e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:13.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.419+0000 7fc130e00640 1 -- 172.21.15.100:0/1115128416 shutdown_connections 2024-04-23T14:35:13.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.419+0000 7fc130e00640 1 -- 172.21.15.100:0/1115128416 wait complete. 2024-04-23T14:35:13.441 INFO:tasks.ceph:Scrubbing osd.5 2024-04-23T14:35:13.441 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-23T14:35:13.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.579+0000 7fea8fe00640 1 Processor -- start 2024-04-23T14:35:13.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.579+0000 7fea8fe00640 1 -- start start 2024-04-23T14:35:13.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.583+0000 7fea8fe00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fea900d1d60 0x7fea9015c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.583+0000 7fea8fe00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea900cc940 0x7fea900ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.583+0000 7fea8fe00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea90153840 con 0x7fea90153460 2024-04-23T14:35:13.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.583+0000 7fea8fe00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fea900cd2e0 con 0x7fea900d1d60 2024-04-23T14:35:13.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.583+0000 7fea8fe00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fea9015cfe0 con 0x7fea900cc940 2024-04-23T14:35:13.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.583+0000 7fea87e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fea900d1d60 0x7fea9015c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.583+0000 7fea8f400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea900cc940 0x7fea900ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.583+0000 7fea87e00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fea900d1d60 0x7fea9015c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:48886/0 (socket says 172.21.15.100:48886) 2024-04-23T14:35:13.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.583+0000 7fea87e00640 1 -- 172.21.15.100:0/2128997723 learned_addr learned my addr 172.21.15.100:0/2128997723 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:13.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.583+0000 7fea87e00640 1 -- 172.21.15.100:0/2128997723 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea900cc940 msgr2=0x7fea900ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:13.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.583+0000 7fea87e00640 1 --2- 172.21.15.100:0/2128997723 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea900cc940 0x7fea900ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.583+0000 7fea87e00640 1 -- 172.21.15.100:0/2128997723 >> v1:172.21.15.151:6789/0 conn(0x7fea90153460 legacy=0x7fea900d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.583+0000 7fea87e00640 1 -- 172.21.15.100:0/2128997723 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9015d2c0 con 0x7fea900d1d60 2024-04-23T14:35:13.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.587+0000 7fea87e00640 1 --2- 172.21.15.100:0/2128997723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea900d1d60 0x7fea9015c940 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fea78009870 tx=0x7fea7801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cd097c8f1fa1716 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:13.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.587+0000 7fea8e000640 1 -- 172.21.15.100:0/2128997723 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea780aa600 con 0x7fea900d1d60 2024-04-23T14:35:13.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.587+0000 7fea8e000640 1 -- 172.21.15.100:0/2128997723 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea780af070 con 0x7fea900d1d60 2024-04-23T14:35:13.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.587+0000 7fea8e000640 1 -- 172.21.15.100:0/2128997723 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea780aa9c0 con 0x7fea900d1d60 2024-04-23T14:35:13.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.587+0000 7fea8fe00640 1 -- 172.21.15.100:0/2128997723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea900d1d60 msgr2=0x7fea9015c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.587+0000 7fea8fe00640 1 --2- 172.21.15.100:0/2128997723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea900d1d60 0x7fea9015c940 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fea78009870 tx=0x7fea7801c4a0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.587+0000 7fea8fe00640 1 -- 172.21.15.100:0/2128997723 shutdown_connections 2024-04-23T14:35:13.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.587+0000 7fea8fe00640 1 --2- 172.21.15.100:0/2128997723 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea900cc940 0x7fea900ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.587+0000 7fea8fe00640 1 --2- 172.21.15.100:0/2128997723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea900d1d60 0x7fea9015c940 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.587+0000 7fea8fe00640 1 -- 172.21.15.100:0/2128997723 >> 172.21.15.100:0/2128997723 conn(0x7fea900b3e60 msgr2=0x7fea900b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:13.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.587+0000 7fea8fe00640 1 -- 172.21.15.100:0/2128997723 shutdown_connections 2024-04-23T14:35:13.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.587+0000 7fea8fe00640 1 -- 172.21.15.100:0/2128997723 wait complete. 2024-04-23T14:35:13.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.587+0000 7fea8fe00640 1 Processor -- start 2024-04-23T14:35:13.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.587+0000 7fea8fe00640 1 -- start start 2024-04-23T14:35:13.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.587+0000 7fea8fe00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea900cc940 0x7fea900c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.587+0000 7fea8fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea900d1d60 0x7fea90152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.587+0000 7fea8fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea90153460 0x7fea90152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.587+0000 7fea8fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fea900ce850 con 0x7fea900d1d60 2024-04-23T14:35:13.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.587+0000 7fea8fe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fea900b3790 con 0x7fea90153460 2024-04-23T14:35:13.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.587+0000 7fea8fe00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fea900ce9d0 con 0x7fea900cc940 2024-04-23T14:35:13.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.591+0000 7fea87e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea900d1d60 0x7fea90152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.591+0000 7fea87e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea900d1d60 0x7fea90152150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41144/0 (socket says 172.21.15.100:41144) 2024-04-23T14:35:13.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.591+0000 7fea87e00640 1 -- 172.21.15.100:0/275411688 learned_addr learned my addr 172.21.15.100:0/275411688 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:13.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.591+0000 7fea8f400640 1 --2- 172.21.15.100:0/275411688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea90153460 0x7fea90152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.591+0000 7fea8ea00640 1 --2- 172.21.15.100:0/275411688 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea900cc940 0x7fea900c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.591+0000 7fea87e00640 1 -- 172.21.15.100:0/275411688 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea900cc940 msgr2=0x7fea900c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.591+0000 7fea87e00640 1 --2- 172.21.15.100:0/275411688 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea900cc940 0x7fea900c5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.591+0000 7fea87e00640 1 -- 172.21.15.100:0/275411688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea90153460 msgr2=0x7fea90152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.591+0000 7fea87e00640 1 --2- 172.21.15.100:0/275411688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea90153460 0x7fea90152690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.591+0000 7fea87e00640 1 -- 172.21.15.100:0/275411688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea90165e80 con 0x7fea900d1d60 2024-04-23T14:35:13.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.591+0000 7fea8f400640 1 --2- 172.21.15.100:0/275411688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea90153460 0x7fea90152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:35:13.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.591+0000 7fea87e00640 1 --2- 172.21.15.100:0/275411688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea900d1d60 0x7fea90152150 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fea7801ab50 tx=0x7fea7801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:13.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.591+0000 7fea86a00640 1 -- 172.21.15.100:0/275411688 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea780bd5f0 con 0x7fea900d1d60 2024-04-23T14:35:13.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.591+0000 7fea86a00640 1 -- 172.21.15.100:0/275411688 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea780bd790 con 0x7fea900d1d60 2024-04-23T14:35:13.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.591+0000 7fea86a00640 1 -- 172.21.15.100:0/275411688 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea780bdb80 con 0x7fea900d1d60 2024-04-23T14:35:13.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.591+0000 7fea8fe00640 1 -- 172.21.15.100:0/275411688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea9006ec00 con 0x7fea900d1d60 2024-04-23T14:35:13.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.591+0000 7fea8fe00640 1 -- 172.21.15.100:0/275411688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea9006f1c0 con 0x7fea900d1d60 2024-04-23T14:35:13.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.591+0000 7fea8fe00640 1 -- 172.21.15.100:0/275411688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea9006f400 con 0x7fea900d1d60 2024-04-23T14:35:13.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.591+0000 7fea86a00640 1 -- 172.21.15.100:0/275411688 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fea780a9070 con 0x7fea900d1d60 2024-04-23T14:35:13.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.595+0000 7fea86a00640 1 --2- 172.21.15.100:0/275411688 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fea6c0683b0 0x7fea6c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.595+0000 7fea86a00640 1 -- 172.21.15.100:0/275411688 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7fea78121280 con 0x7fea900d1d60 2024-04-23T14:35:13.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.595+0000 7fea8ea00640 1 --2- 172.21.15.100:0/275411688 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fea6c0683b0 0x7fea6c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.595+0000 7fea86a00640 1 --2- 172.21.15.100:0/275411688 >> [v2:172.21.15.151:6808/4283546311,v1:172.21.15.151:6809/4283546311] conn(0x7fea6c06e850 0x7fea6c070c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.595+0000 7fea86a00640 1 -- 172.21.15.100:0/275411688 --> [v2:172.21.15.151:6808/4283546311,v1:172.21.15.151:6809/4283546311] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea780bfe50 con 0x7fea6c06e850 2024-04-23T14:35:13.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.595+0000 7fea86a00640 1 -- 172.21.15.100:0/275411688 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fea78121630 con 0x7fea900d1d60 2024-04-23T14:35:13.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.595+0000 7fea8f400640 1 --2- 172.21.15.100:0/275411688 >> [v2:172.21.15.151:6808/4283546311,v1:172.21.15.151:6809/4283546311] conn(0x7fea6c06e850 0x7fea6c070c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.595+0000 7fea8ea00640 1 --2- 172.21.15.100:0/275411688 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fea6c0683b0 0x7fea6c06a870 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fea80007fd0 tx=0x7fea80009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:13.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.595+0000 7fea8f400640 1 --2- 172.21.15.100:0/275411688 >> [v2:172.21.15.151:6808/4283546311,v1:172.21.15.151:6809/4283546311] conn(0x7fea6c06e850 0x7fea6c070c90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:13.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.595+0000 7fea86a00640 1 -- 172.21.15.100:0/275411688 <== osd.5 v2:172.21.15.151:6808/4283546311 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fea780bfe50 con 0x7fea6c06e850 2024-04-23T14:35:13.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.607+0000 7fea8fe00640 1 -- 172.21.15.100:0/275411688 --> [v2:172.21.15.151:6808/4283546311,v1:172.21.15.151:6809/4283546311] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fea90075c20 con 0x7fea6c06e850 2024-04-23T14:35:13.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.607+0000 7fea86a00640 1 -- 172.21.15.100:0/275411688 <== osd.5 v2:172.21.15.151:6808/4283546311 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fea90075c20 con 0x7fea6c06e850 2024-04-23T14:35:13.610 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:35:13.610 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T14:35:13.610 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:35:13.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.607+0000 7fea8fe00640 1 -- 172.21.15.100:0/275411688 >> [v2:172.21.15.151:6808/4283546311,v1:172.21.15.151:6809/4283546311] conn(0x7fea6c06e850 msgr2=0x7fea6c070c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.611+0000 7fea8fe00640 1 --2- 172.21.15.100:0/275411688 >> [v2:172.21.15.151:6808/4283546311,v1:172.21.15.151:6809/4283546311] conn(0x7fea6c06e850 0x7fea6c070c90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.611+0000 7fea8fe00640 1 -- 172.21.15.100:0/275411688 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fea6c0683b0 msgr2=0x7fea6c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.611+0000 7fea8fe00640 1 --2- 172.21.15.100:0/275411688 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fea6c0683b0 0x7fea6c06a870 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fea80007fd0 tx=0x7fea80009040 comp rx=0 tx=0).stop 2024-04-23T14:35:13.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.611+0000 7fea8fe00640 1 -- 172.21.15.100:0/275411688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea900d1d60 msgr2=0x7fea90152150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.611+0000 7fea8fe00640 1 --2- 172.21.15.100:0/275411688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea900d1d60 0x7fea90152150 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fea7801ab50 tx=0x7fea7801ace0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.611+0000 7fea8f400640 1 -- 172.21.15.100:0/275411688 reap_dead start 2024-04-23T14:35:13.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.611+0000 7fea8fe00640 1 -- 172.21.15.100:0/275411688 shutdown_connections 2024-04-23T14:35:13.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.611+0000 7fea8fe00640 1 -- 172.21.15.100:0/275411688 >> 172.21.15.100:0/275411688 conn(0x7fea900b3e60 msgr2=0x7fea9015dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:13.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.611+0000 7fea8fe00640 1 -- 172.21.15.100:0/275411688 shutdown_connections 2024-04-23T14:35:13.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.611+0000 7fea8fe00640 1 -- 172.21.15.100:0/275411688 wait complete. 2024-04-23T14:35:13.629 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-23T14:35:13.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.763+0000 7f40f0800640 1 Processor -- start 2024-04-23T14:35:13.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.763+0000 7f40f0800640 1 -- start start 2024-04-23T14:35:13.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.767+0000 7f40f0800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f40e80d0620 0x7f40e80c5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.767+0000 7f40f0800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40e80cee00 con 0x7f40e815b660 2024-04-23T14:35:13.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.767+0000 7f40f0800640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40e80d0ab0 con 0x7f40e80d0190 2024-04-23T14:35:13.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.767+0000 7f40f0800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f40e815ba40 con 0x7f40e80d0620 2024-04-23T14:35:13.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.767+0000 7f40ee800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f40e80d0620 0x7f40e80c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.767+0000 7f40ede00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f40e815b660 0x7f40e80c6820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42244/0 (socket says 172.21.15.100:42244) 2024-04-23T14:35:13.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.767+0000 7f40ede00640 1 -- 172.21.15.100:0/1659478721 learned_addr learned my addr 172.21.15.100:0/1659478721 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:13.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.767+0000 7f40ee800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f40e80d0620 0x7f40e80c5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41152/0 (socket says 172.21.15.100:41152) 2024-04-23T14:35:13.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.767+0000 7f40ee800640 1 -- 172.21.15.100:0/1659478721 >> v1:172.21.15.151:6789/0 conn(0x7f40e80d0190 legacy=0x7f40e80c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.767+0000 7f40ee800640 1 -- 172.21.15.100:0/1659478721 >> v1:172.21.15.100:6789/0 conn(0x7f40e815b660 legacy=0x7f40e80c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.767+0000 7f40ee800640 1 -- 172.21.15.100:0/1659478721 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e806bc50 con 0x7f40e80d0620 2024-04-23T14:35:13.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.771+0000 7f40ee800640 1 --2- 172.21.15.100:0/1659478721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e80d0620 0x7f40e80c5870 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f40e400bfa0 tx=0x7f40e401eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6f6064628816fa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:13.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.771+0000 7f40eca00640 1 -- 172.21.15.100:0/1659478721 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e400b2d0 con 0x7f40e80d0620 2024-04-23T14:35:13.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.771+0000 7f40eca00640 1 -- 172.21.15.100:0/1659478721 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e40a7a10 con 0x7f40e80d0620 2024-04-23T14:35:13.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.771+0000 7f40eca00640 1 -- 172.21.15.100:0/1659478721 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e40a7de0 con 0x7f40e80d0620 2024-04-23T14:35:13.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.771+0000 7f40f0800640 1 -- 172.21.15.100:0/1659478721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e80d0620 msgr2=0x7f40e80c5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.771+0000 7f40f0800640 1 --2- 172.21.15.100:0/1659478721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e80d0620 0x7f40e80c5870 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f40e400bfa0 tx=0x7f40e401eb40 comp rx=0 tx=0).stop 2024-04-23T14:35:13.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.771+0000 7f40f0800640 1 -- 172.21.15.100:0/1659478721 shutdown_connections 2024-04-23T14:35:13.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.771+0000 7f40f0800640 1 --2- 172.21.15.100:0/1659478721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e80d0620 0x7f40e80c5870 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.771+0000 7f40f0800640 1 -- 172.21.15.100:0/1659478721 >> 172.21.15.100:0/1659478721 conn(0x7f40e80b5310 msgr2=0x7f40e80b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:13.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.771+0000 7f40f0800640 1 -- 172.21.15.100:0/1659478721 shutdown_connections 2024-04-23T14:35:13.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.771+0000 7f40f0800640 1 -- 172.21.15.100:0/1659478721 wait complete. 2024-04-23T14:35:13.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.771+0000 7f40f0800640 1 Processor -- start 2024-04-23T14:35:13.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.771+0000 7f40f0800640 1 -- start start 2024-04-23T14:35:13.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.771+0000 7f40f0800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f40e80d0190 0x7f40e8147a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.771+0000 7f40f0800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40e80d0620 0x7f40e806eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.771+0000 7f40f0800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e815b660 0x7f40e814ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.771+0000 7f40f0800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f40e80c6f30 con 0x7f40e815b660 2024-04-23T14:35:13.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.771+0000 7f40f0800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f40e806c7d0 con 0x7f40e80d0620 2024-04-23T14:35:13.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.771+0000 7f40f0800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f40e80c70b0 con 0x7f40e80d0190 2024-04-23T14:35:13.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.771+0000 7f40ee800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e815b660 0x7f40e814ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.771+0000 7f40ee800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e815b660 0x7f40e814ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41156/0 (socket says 172.21.15.100:41156) 2024-04-23T14:35:13.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.771+0000 7f40ee800640 1 -- 172.21.15.100:0/2961477235 learned_addr learned my addr 172.21.15.100:0/2961477235 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:13.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.775+0000 7f40ede00640 1 --2- 172.21.15.100:0/2961477235 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f40e80d0190 0x7f40e8147a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.775+0000 7f40ed400640 1 --2- 172.21.15.100:0/2961477235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40e80d0620 0x7f40e806eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.775+0000 7f40ee800640 1 -- 172.21.15.100:0/2961477235 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f40e80d0190 msgr2=0x7f40e8147a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.775+0000 7f40ee800640 1 --2- 172.21.15.100:0/2961477235 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f40e80d0190 0x7f40e8147a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.775+0000 7f40ee800640 1 -- 172.21.15.100:0/2961477235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40e80d0620 msgr2=0x7f40e806eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.775+0000 7f40ee800640 1 --2- 172.21.15.100:0/2961477235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40e80d0620 0x7f40e806eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.775+0000 7f40ee800640 1 -- 172.21.15.100:0/2961477235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e8147fb0 con 0x7f40e815b660 2024-04-23T14:35:13.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.775+0000 7f40ee800640 1 --2- 172.21.15.100:0/2961477235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e815b660 0x7f40e814ce40 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f40e40b1040 tx=0x7f40e401b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:13.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.775+0000 7f40e2000640 1 -- 172.21.15.100:0/2961477235 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4009950 con 0x7f40e815b660 2024-04-23T14:35:13.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.775+0000 7f40e2000640 1 -- 172.21.15.100:0/2961477235 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e40be070 con 0x7f40e815b660 2024-04-23T14:35:13.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.775+0000 7f40e2000640 1 -- 172.21.15.100:0/2961477235 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4009d70 con 0x7f40e815b660 2024-04-23T14:35:13.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.775+0000 7f40f0800640 1 -- 172.21.15.100:0/2961477235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e81484b0 con 0x7f40e815b660 2024-04-23T14:35:13.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.775+0000 7f40f0800640 1 -- 172.21.15.100:0/2961477235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e80cc6c0 con 0x7f40e815b660 2024-04-23T14:35:13.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.775+0000 7f40f0800640 1 -- 172.21.15.100:0/2961477235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40e8068ad0 con 0x7f40e815b660 2024-04-23T14:35:13.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.775+0000 7f40e2000640 1 -- 172.21.15.100:0/2961477235 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f40e40b6070 con 0x7f40e815b660 2024-04-23T14:35:13.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.775+0000 7f40e2000640 1 --2- 172.21.15.100:0/2961477235 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f40cc068380 0x7f40cc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:13.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.775+0000 7f40e2000640 1 -- 172.21.15.100:0/2961477235 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f40e41219d0 con 0x7f40e815b660 2024-04-23T14:35:13.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.775+0000 7f40ede00640 1 --2- 172.21.15.100:0/2961477235 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f40cc068380 0x7f40cc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:13.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.779+0000 7f40ede00640 1 --2- 172.21.15.100:0/2961477235 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f40cc068380 0x7f40cc06a840 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f40dc005890 tx=0x7f40dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:13.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.783+0000 7f40e2000640 1 -- 172.21.15.100:0/2961477235 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f40e4018760 con 0x7f40e815b660 2024-04-23T14:35:13.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.943+0000 7f40f0800640 1 -- 172.21.15.100:0/2961477235 --> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f40e8156090 con 0x7f40cc068380 2024-04-23T14:35:13.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.943+0000 7f40e2000640 1 -- 172.21.15.100:0/2961477235 <== mgr.4101 v2:172.21.15.100:6832/2859853957 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f40e8156090 con 0x7f40cc068380 2024-04-23T14:35:13.943 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 5 to deep-scrub 2024-04-23T14:35:13.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.943+0000 7f40f0800640 1 -- 172.21.15.100:0/2961477235 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f40cc068380 msgr2=0x7f40cc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.943+0000 7f40f0800640 1 --2- 172.21.15.100:0/2961477235 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f40cc068380 0x7f40cc06a840 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f40dc005890 tx=0x7f40dc009040 comp rx=0 tx=0).stop 2024-04-23T14:35:13.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.943+0000 7f40f0800640 1 -- 172.21.15.100:0/2961477235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e815b660 msgr2=0x7f40e814ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:13.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.943+0000 7f40f0800640 1 --2- 172.21.15.100:0/2961477235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e815b660 0x7f40e814ce40 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f40e40b1040 tx=0x7f40e401b8e0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.943+0000 7f40f0800640 1 -- 172.21.15.100:0/2961477235 shutdown_connections 2024-04-23T14:35:13.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.943+0000 7f40f0800640 1 --2- 172.21.15.100:0/2961477235 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f40cc068380 0x7f40cc06a840 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.943+0000 7f40f0800640 1 --2- 172.21.15.100:0/2961477235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e815b660 0x7f40e814ce40 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.943+0000 7f40f0800640 1 --2- 172.21.15.100:0/2961477235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40e80d0620 0x7f40e806eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.943+0000 7f40f0800640 1 --2- 172.21.15.100:0/2961477235 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f40e80d0190 0x7f40e8147a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:13.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.943+0000 7f40f0800640 1 -- 172.21.15.100:0/2961477235 >> 172.21.15.100:0/2961477235 conn(0x7f40e80b5310 msgr2=0x7f40e80afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:13.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.943+0000 7f40f0800640 1 -- 172.21.15.100:0/2961477235 shutdown_connections 2024-04-23T14:35:13.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:13.943+0000 7f40f0800640 1 -- 172.21.15.100:0/2961477235 wait complete. 2024-04-23T14:35:13.964 INFO:tasks.ceph:Scrubbing osd.6 2024-04-23T14:35:13.964 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-23T14:35:14.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.103+0000 7fb6aca00640 1 Processor -- start 2024-04-23T14:35:14.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.103+0000 7fb6aca00640 1 -- start start 2024-04-23T14:35:14.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.103+0000 7fb6aca00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6a806f8c0 0x7fb6a806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.103+0000 7fb6aca00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fb6a80d0170 0x7fb6a80ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.103+0000 7fb6aca00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6a806dd60 con 0x7fb6a80ce620 2024-04-23T14:35:14.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.103+0000 7fb6aca00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fb6a80ad050 con 0x7fb6a80d0170 2024-04-23T14:35:14.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.103+0000 7fb6aca00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb6a80ad8f0 con 0x7fb6a806f8c0 2024-04-23T14:35:14.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.103+0000 7fb69d400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fb6a80d0170 0x7fb6a80ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.103+0000 7fb69d400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fb6a80d0170 0x7fb6a80ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:48920/0 (socket says 172.21.15.100:48920) 2024-04-23T14:35:14.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.103+0000 7fb69d400640 1 -- 172.21.15.100:0/3593145167 learned_addr learned my addr 172.21.15.100:0/3593145167 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:14.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.103+0000 7fb6a5e00640 1 --2- 172.21.15.100:0/3593145167 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6a806f8c0 0x7fb6a806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.103+0000 7fb6a5e00640 1 -- 172.21.15.100:0/3593145167 >> v2:172.21.15.151:3300/0 conn(0x7fb6a80d0170 msgr2=0x7fb6a80ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:14.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.103+0000 7fb6a5e00640 1 --2- 172.21.15.100:0/3593145167 >> v2:172.21.15.151:3300/0 conn(0x7fb6a80d0170 0x7fb6a80ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.103+0000 7fb6a5e00640 1 -- 172.21.15.100:0/3593145167 >> v1:172.21.15.151:6789/0 conn(0x7fb6a80ce620 legacy=0x7fb6a815f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.103+0000 7fb6a5e00640 1 -- 172.21.15.100:0/3593145167 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a80d09c0 con 0x7fb6a806f8c0 2024-04-23T14:35:14.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.103+0000 7fb69d400640 1 --2- 172.21.15.100:0/3593145167 >> v2:172.21.15.151:3300/0 conn(0x7fb6a80d0170 0x7fb6a80ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:35:14.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.107+0000 7fb6a5e00640 1 --2- 172.21.15.100:0/3593145167 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6a806f8c0 0x7fb6a806c6c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb6980097c0 tx=0x7fb69801c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e13b0854d0575f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:14.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.107+0000 7fb6a5400640 1 -- 172.21.15.100:0/3593145167 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6980b0070 con 0x7fb6a806f8c0 2024-04-23T14:35:14.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.107+0000 7fb6a5400640 1 -- 172.21.15.100:0/3593145167 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6980ab600 con 0x7fb6a806f8c0 2024-04-23T14:35:14.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.107+0000 7fb6a5400640 1 -- 172.21.15.100:0/3593145167 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6980ab9d0 con 0x7fb6a806f8c0 2024-04-23T14:35:14.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.107+0000 7fb6aca00640 1 -- 172.21.15.100:0/3593145167 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6a806f8c0 msgr2=0x7fb6a806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.107+0000 7fb6aca00640 1 --2- 172.21.15.100:0/3593145167 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6a806f8c0 0x7fb6a806c6c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb6980097c0 tx=0x7fb69801c330 comp rx=0 tx=0).stop 2024-04-23T14:35:14.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.107+0000 7fb6aca00640 1 -- 172.21.15.100:0/3593145167 shutdown_connections 2024-04-23T14:35:14.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.107+0000 7fb6aca00640 1 --2- 172.21.15.100:0/3593145167 >> v2:172.21.15.151:3300/0 conn(0x7fb6a80d0170 0x7fb6a80ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.107+0000 7fb6aca00640 1 --2- 172.21.15.100:0/3593145167 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6a806f8c0 0x7fb6a806c6c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.107+0000 7fb6aca00640 1 -- 172.21.15.100:0/3593145167 >> 172.21.15.100:0/3593145167 conn(0x7fb6a80b5200 msgr2=0x7fb6a80b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:14.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.107+0000 7fb6aca00640 1 -- 172.21.15.100:0/3593145167 shutdown_connections 2024-04-23T14:35:14.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.107+0000 7fb6aca00640 1 -- 172.21.15.100:0/3593145167 wait complete. 2024-04-23T14:35:14.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb6aca00640 1 Processor -- start 2024-04-23T14:35:14.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb6aca00640 1 -- start start 2024-04-23T14:35:14.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb6aca00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6a806f8c0 0x7fb6a81f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb6aca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6a80ce620 0x7fb6a815a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb6aca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6a80d0170 0x7fb6a815e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb6aca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a80aed90 con 0x7fb6a80ce620 2024-04-23T14:35:14.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb6aca00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a80ad580 con 0x7fb6a80d0170 2024-04-23T14:35:14.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb6aca00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb6a80d05b0 con 0x7fb6a806f8c0 2024-04-23T14:35:14.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb69d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6a80ce620 0x7fb6a815a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb69d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6a80ce620 0x7fb6a815a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41172/0 (socket says 172.21.15.100:41172) 2024-04-23T14:35:14.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb69d400640 1 -- 172.21.15.100:0/1201987182 learned_addr learned my addr 172.21.15.100:0/1201987182 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:14.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb6a5e00640 1 --2- 172.21.15.100:0/1201987182 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6a806f8c0 0x7fb6a81f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb6a6800640 1 --2- 172.21.15.100:0/1201987182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6a80d0170 0x7fb6a815e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb69d400640 1 -- 172.21.15.100:0/1201987182 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6a806f8c0 msgr2=0x7fb6a81f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb69d400640 1 --2- 172.21.15.100:0/1201987182 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6a806f8c0 0x7fb6a81f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb69d400640 1 -- 172.21.15.100:0/1201987182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6a80d0170 msgr2=0x7fb6a815e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb69d400640 1 --2- 172.21.15.100:0/1201987182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6a80d0170 0x7fb6a815e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb69d400640 1 -- 172.21.15.100:0/1201987182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a8072570 con 0x7fb6a80ce620 2024-04-23T14:35:14.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb69d400640 1 --2- 172.21.15.100:0/1201987182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6a80ce620 0x7fb6a815a6c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fb69000e930 tx=0x7fb69000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:14.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb69ea00640 1 -- 172.21.15.100:0/1201987182 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb690017070 con 0x7fb6a80ce620 2024-04-23T14:35:14.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb69ea00640 1 -- 172.21.15.100:0/1201987182 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb69000ca00 con 0x7fb6a80ce620 2024-04-23T14:35:14.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb6aca00640 1 -- 172.21.15.100:0/1201987182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a8072850 con 0x7fb6a80ce620 2024-04-23T14:35:14.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb69ea00640 1 -- 172.21.15.100:0/1201987182 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69000cdd0 con 0x7fb6a80ce620 2024-04-23T14:35:14.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.111+0000 7fb6aca00640 1 -- 172.21.15.100:0/1201987182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a815ec30 con 0x7fb6a80ce620 2024-04-23T14:35:14.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.115+0000 7fb6aca00640 1 -- 172.21.15.100:0/1201987182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb6a80c8780 con 0x7fb6a80ce620 2024-04-23T14:35:14.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.115+0000 7fb69ea00640 1 -- 172.21.15.100:0/1201987182 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb69001a070 con 0x7fb6a80ce620 2024-04-23T14:35:14.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.115+0000 7fb69ea00640 1 --2- 172.21.15.100:0/1201987182 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb684068380 0x7fb68406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.115+0000 7fb69ea00640 1 -- 172.21.15.100:0/1201987182 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7fb690086970 con 0x7fb6a80ce620 2024-04-23T14:35:14.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.115+0000 7fb6a5e00640 1 --2- 172.21.15.100:0/1201987182 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb684068380 0x7fb68406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.115+0000 7fb69ea00640 1 --2- 172.21.15.100:0/1201987182 >> [v2:172.21.15.151:6824/547449762,v1:172.21.15.151:6825/547449762] conn(0x7fb68406e820 0x7fb684070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.115+0000 7fb69ea00640 1 -- 172.21.15.100:0/1201987182 --> [v2:172.21.15.151:6824/547449762,v1:172.21.15.151:6825/547449762] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb690015e50 con 0x7fb68406e820 2024-04-23T14:35:14.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.115+0000 7fb69ea00640 1 -- 172.21.15.100:0/1201987182 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fb69008b2c0 con 0x7fb6a80ce620 2024-04-23T14:35:14.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.115+0000 7fb6a6800640 1 --2- 172.21.15.100:0/1201987182 >> [v2:172.21.15.151:6824/547449762,v1:172.21.15.151:6825/547449762] conn(0x7fb68406e820 0x7fb684070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.115+0000 7fb6a5e00640 1 --2- 172.21.15.100:0/1201987182 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb684068380 0x7fb68406a840 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb6980ad000 tx=0x7fb698008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:14.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.115+0000 7fb6a6800640 1 --2- 172.21.15.100:0/1201987182 >> [v2:172.21.15.151:6824/547449762,v1:172.21.15.151:6825/547449762] conn(0x7fb68406e820 0x7fb684070c60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:14.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.119+0000 7fb69ea00640 1 -- 172.21.15.100:0/1201987182 <== osd.6 v2:172.21.15.151:6824/547449762 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb690015e50 con 0x7fb68406e820 2024-04-23T14:35:14.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.131+0000 7fb6aca00640 1 -- 172.21.15.100:0/1201987182 --> [v2:172.21.15.151:6824/547449762,v1:172.21.15.151:6825/547449762] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb6a80cb6e0 con 0x7fb68406e820 2024-04-23T14:35:14.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.131+0000 7fb69ea00640 1 -- 172.21.15.100:0/1201987182 <== osd.6 v2:172.21.15.151:6824/547449762 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb6a80cb6e0 con 0x7fb68406e820 2024-04-23T14:35:14.132 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:35:14.133 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T14:35:14.133 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:35:14.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.131+0000 7fb6aca00640 1 -- 172.21.15.100:0/1201987182 >> [v2:172.21.15.151:6824/547449762,v1:172.21.15.151:6825/547449762] conn(0x7fb68406e820 msgr2=0x7fb684070c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.131+0000 7fb6aca00640 1 --2- 172.21.15.100:0/1201987182 >> [v2:172.21.15.151:6824/547449762,v1:172.21.15.151:6825/547449762] conn(0x7fb68406e820 0x7fb684070c60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.131+0000 7fb6aca00640 1 -- 172.21.15.100:0/1201987182 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb684068380 msgr2=0x7fb68406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.131+0000 7fb6aca00640 1 --2- 172.21.15.100:0/1201987182 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fb684068380 0x7fb68406a840 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb6980ad000 tx=0x7fb698008040 comp rx=0 tx=0).stop 2024-04-23T14:35:14.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.131+0000 7fb6aca00640 1 -- 172.21.15.100:0/1201987182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6a80ce620 msgr2=0x7fb6a815a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.131+0000 7fb6aca00640 1 --2- 172.21.15.100:0/1201987182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6a80ce620 0x7fb6a815a6c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fb69000e930 tx=0x7fb69000edf0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.131+0000 7fb6a6800640 1 -- 172.21.15.100:0/1201987182 reap_dead start 2024-04-23T14:35:14.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.131+0000 7fb6aca00640 1 -- 172.21.15.100:0/1201987182 shutdown_connections 2024-04-23T14:35:14.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.131+0000 7fb6aca00640 1 -- 172.21.15.100:0/1201987182 >> 172.21.15.100:0/1201987182 conn(0x7fb6a80b5200 msgr2=0x7fb6a806edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:14.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.131+0000 7fb6aca00640 1 -- 172.21.15.100:0/1201987182 shutdown_connections 2024-04-23T14:35:14.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.131+0000 7fb6aca00640 1 -- 172.21.15.100:0/1201987182 wait complete. 2024-04-23T14:35:14.150 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-23T14:35:14.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.283+0000 7fcfb2600640 1 Processor -- start 2024-04-23T14:35:14.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.283+0000 7fcfb2600640 1 -- start start 2024-04-23T14:35:14.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.287+0000 7fcfb2600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcfac15b660 0x7fcfac0c6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.287+0000 7fcfb2600640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fcfac0c6d60 0x7fcfac0d0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.287+0000 7fcfb2600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfac0cee00 con 0x7fcfac0d06d0 2024-04-23T14:35:14.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.287+0000 7fcfb2600640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fcfac0d0b60 con 0x7fcfac15b660 2024-04-23T14:35:14.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.287+0000 7fcfb2600640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fcfac15ba40 con 0x7fcfac0c6d60 2024-04-23T14:35:14.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.287+0000 7fcfab400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcfac15b660 0x7fcfac0c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.287+0000 7fcfab400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcfac15b660 0x7fcfac0c6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41186/0 (socket says 172.21.15.100:41186) 2024-04-23T14:35:14.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.287+0000 7fcfab400640 1 -- 172.21.15.100:0/3037923645 learned_addr learned my addr 172.21.15.100:0/3037923645 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:14.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.287+0000 7fcfabe00640 1 --1- 172.21.15.100:0/3037923645 >> v1:172.21.15.100:6789/0 conn(0x7fcfac0d06d0 0x7fcfac15f0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42252/0 (socket says 172.21.15.100:42252) 2024-04-23T14:35:14.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.287+0000 7fcfa2a00640 1 --2- 172.21.15.100:0/3037923645 >> v2:172.21.15.151:3300/0 conn(0x7fcfac0c6d60 0x7fcfac0d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.287+0000 7fcfab400640 1 -- 172.21.15.100:0/3037923645 >> v2:172.21.15.151:3300/0 conn(0x7fcfac0c6d60 msgr2=0x7fcfac0d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:14.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.287+0000 7fcfab400640 1 --2- 172.21.15.100:0/3037923645 >> v2:172.21.15.151:3300/0 conn(0x7fcfac0c6d60 0x7fcfac0d0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.287+0000 7fcfab400640 1 -- 172.21.15.100:0/3037923645 >> v1:172.21.15.100:6789/0 conn(0x7fcfac0d06d0 legacy=0x7fcfac15f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.287+0000 7fcfab400640 1 -- 172.21.15.100:0/3037923645 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfac06bc50 con 0x7fcfac15b660 2024-04-23T14:35:14.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.287+0000 7fcfaaa00640 1 -- 172.21.15.100:0/3037923645 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 308511308 0 0) 0x7fcfa40014a0 con 0x7fcfac0d06d0 2024-04-23T14:35:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.287+0000 7fcfab400640 1 --2- 172.21.15.100:0/3037923645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfac15b660 0x7fcfac0c6820 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0b63a0 tx=0x7fcf9c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc8b72da5c24206 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.291+0000 7fcfaaa00640 1 -- 172.21.15.100:0/3037923645 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c0aa650 con 0x7fcfac15b660 2024-04-23T14:35:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.291+0000 7fcfaaa00640 1 -- 172.21.15.100:0/3037923645 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf9c0af070 con 0x7fcfac15b660 2024-04-23T14:35:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.291+0000 7fcfaaa00640 1 -- 172.21.15.100:0/3037923645 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c0aaa10 con 0x7fcfac15b660 2024-04-23T14:35:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.291+0000 7fcfb2600640 1 -- 172.21.15.100:0/3037923645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfac15b660 msgr2=0x7fcfac0c6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.291+0000 7fcfb2600640 1 --2- 172.21.15.100:0/3037923645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfac15b660 0x7fcfac0c6820 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0b63a0 tx=0x7fcf9c01c4f0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.291+0000 7fcfb2600640 1 -- 172.21.15.100:0/3037923645 shutdown_connections 2024-04-23T14:35:14.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.291+0000 7fcfb2600640 1 --2- 172.21.15.100:0/3037923645 >> v2:172.21.15.151:3300/0 conn(0x7fcfac0c6d60 0x7fcfac0d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.291+0000 7fcfb2600640 1 --2- 172.21.15.100:0/3037923645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfac15b660 0x7fcfac0c6820 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.291+0000 7fcfb2600640 1 -- 172.21.15.100:0/3037923645 >> 172.21.15.100:0/3037923645 conn(0x7fcfac0b5310 msgr2=0x7fcfac0b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:14.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.291+0000 7fcfb2600640 1 -- 172.21.15.100:0/3037923645 shutdown_connections 2024-04-23T14:35:14.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.291+0000 7fcfb2600640 1 -- 172.21.15.100:0/3037923645 wait complete. 2024-04-23T14:35:14.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.291+0000 7fcfb2600640 1 Processor -- start 2024-04-23T14:35:14.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.291+0000 7fcfb2600640 1 -- start start 2024-04-23T14:35:14.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.291+0000 7fcfb2600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfac0c6d60 0x7fcfac153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.291+0000 7fcfb2600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcfac0d06d0 0x7fcfac0ad160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.291+0000 7fcfb2600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfac15b660 0x7fcfac0ad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.291+0000 7fcfb2600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac0cccd0 con 0x7fcfac0c6d60 2024-04-23T14:35:14.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.291+0000 7fcfb2600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac06c7d0 con 0x7fcfac15b660 2024-04-23T14:35:14.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.291+0000 7fcfb2600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fcfac0cce50 con 0x7fcfac0d06d0 2024-04-23T14:35:14.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.291+0000 7fcfab400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfac0c6d60 0x7fcfac153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.291+0000 7fcfab400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfac0c6d60 0x7fcfac153cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41192/0 (socket says 172.21.15.100:41192) 2024-04-23T14:35:14.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.291+0000 7fcfab400640 1 -- 172.21.15.100:0/2921726976 learned_addr learned my addr 172.21.15.100:0/2921726976 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:14.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.295+0000 7fcfab400640 1 -- 172.21.15.100:0/2921726976 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcfac0d06d0 msgr2=0x7fcfac0ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.295+0000 7fcfab400640 1 --2- 172.21.15.100:0/2921726976 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcfac0d06d0 0x7fcfac0ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.295+0000 7fcfab400640 1 -- 172.21.15.100:0/2921726976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfac15b660 msgr2=0x7fcfac0ad6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.295+0000 7fcfab400640 1 --2- 172.21.15.100:0/2921726976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfac15b660 0x7fcfac0ad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.295+0000 7fcfab400640 1 -- 172.21.15.100:0/2921726976 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfac0b09c0 con 0x7fcfac0c6d60 2024-04-23T14:35:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.295+0000 7fcfab400640 1 --2- 172.21.15.100:0/2921726976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfac0c6d60 0x7fcfac153cd0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c01aba0 tx=0x7fcf9c01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.295+0000 7fcfa3e00640 1 -- 172.21.15.100:0/2921726976 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c0b9070 con 0x7fcfac0c6d60 2024-04-23T14:35:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.295+0000 7fcfa3e00640 1 -- 172.21.15.100:0/2921726976 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf9c0a6040 con 0x7fcfac0c6d60 2024-04-23T14:35:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.295+0000 7fcfb2600640 1 -- 172.21.15.100:0/2921726976 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfac0adcd0 con 0x7fcfac0c6d60 2024-04-23T14:35:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.295+0000 7fcfa3e00640 1 -- 172.21.15.100:0/2921726976 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c004420 con 0x7fcfac0c6d60 2024-04-23T14:35:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.295+0000 7fcfb2600640 1 -- 172.21.15.100:0/2921726976 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfac158670 con 0x7fcfac0c6d60 2024-04-23T14:35:14.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.295+0000 7fcfb2600640 1 -- 172.21.15.100:0/2921726976 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfac068ad0 con 0x7fcfac0c6d60 2024-04-23T14:35:14.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.295+0000 7fcfa3e00640 1 -- 172.21.15.100:0/2921726976 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcf9c0aa8a0 con 0x7fcfac0c6d60 2024-04-23T14:35:14.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.295+0000 7fcfa3e00640 1 --2- 172.21.15.100:0/2921726976 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcf88068380 0x7fcf8806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.295+0000 7fcfa3e00640 1 -- 172.21.15.100:0/2921726976 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7fcf9c1209f0 con 0x7fcfac0c6d60 2024-04-23T14:35:14.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.295+0000 7fcfa2a00640 1 --2- 172.21.15.100:0/2921726976 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcf88068380 0x7fcf8806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.299+0000 7fcfa2a00640 1 --2- 172.21.15.100:0/2921726976 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcf88068380 0x7fcf8806a840 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcf94006aa0 tx=0x7fcf94006a30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:14.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.303+0000 7fcfa3e00640 1 -- 172.21.15.100:0/2921726976 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcf9c016110 con 0x7fcfac0c6d60 2024-04-23T14:35:14.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.459+0000 7fcfb2600640 1 -- 172.21.15.100:0/2921726976 --> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fcfac06d3a0 con 0x7fcf88068380 2024-04-23T14:35:14.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.459+0000 7fcfa3e00640 1 -- 172.21.15.100:0/2921726976 <== mgr.4101 v2:172.21.15.100:6832/2859853957 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcfac06d3a0 con 0x7fcf88068380 2024-04-23T14:35:14.459 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 6 to deep-scrub 2024-04-23T14:35:14.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.459+0000 7fcfb2600640 1 -- 172.21.15.100:0/2921726976 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcf88068380 msgr2=0x7fcf8806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.459+0000 7fcfb2600640 1 --2- 172.21.15.100:0/2921726976 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcf88068380 0x7fcf8806a840 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcf94006aa0 tx=0x7fcf94006a30 comp rx=0 tx=0).stop 2024-04-23T14:35:14.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.459+0000 7fcfb2600640 1 -- 172.21.15.100:0/2921726976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfac0c6d60 msgr2=0x7fcfac153cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.459+0000 7fcfb2600640 1 --2- 172.21.15.100:0/2921726976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfac0c6d60 0x7fcfac153cd0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c01aba0 tx=0x7fcf9c01ad30 comp rx=0 tx=0).stop 2024-04-23T14:35:14.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.459+0000 7fcfb2600640 1 -- 172.21.15.100:0/2921726976 shutdown_connections 2024-04-23T14:35:14.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.459+0000 7fcfb2600640 1 --2- 172.21.15.100:0/2921726976 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7fcf88068380 0x7fcf8806a840 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.459+0000 7fcfb2600640 1 --2- 172.21.15.100:0/2921726976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfac15b660 0x7fcfac0ad6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.459+0000 7fcfb2600640 1 --2- 172.21.15.100:0/2921726976 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcfac0d06d0 0x7fcfac0ad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.459+0000 7fcfb2600640 1 --2- 172.21.15.100:0/2921726976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfac0c6d60 0x7fcfac153cd0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.459+0000 7fcfb2600640 1 -- 172.21.15.100:0/2921726976 >> 172.21.15.100:0/2921726976 conn(0x7fcfac0b5310 msgr2=0x7fcfac1544b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:14.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.459+0000 7fcfb2600640 1 -- 172.21.15.100:0/2921726976 shutdown_connections 2024-04-23T14:35:14.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.459+0000 7fcfb2600640 1 -- 172.21.15.100:0/2921726976 wait complete. 2024-04-23T14:35:14.482 INFO:tasks.ceph:Scrubbing osd.7 2024-04-23T14:35:14.482 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-23T14:35:14.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.619+0000 7f96e3600640 1 Processor -- start 2024-04-23T14:35:14.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.619+0000 7f96e3600640 1 -- start start 2024-04-23T14:35:14.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.623+0000 7f96e3600640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f96dc0d0a70 0x7f96dc0ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.623+0000 7f96e3600640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96dc06bbe0 con 0x7f96dc15ae60 2024-04-23T14:35:14.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.623+0000 7f96e3600640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96dc06bec0 con 0x7f96dc06c9a0 2024-04-23T14:35:14.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.623+0000 7f96e3600640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f96dc15b240 con 0x7f96dc0d0a70 2024-04-23T14:35:14.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.623+0000 7f96dbe00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f96dc0d0a70 0x7f96dc0ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.623+0000 7f96e0c00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f96dc15ae60 0x7f96dc0cf730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42258/0 (socket says 172.21.15.100:42258) 2024-04-23T14:35:14.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.623+0000 7f96e0c00640 1 -- 172.21.15.100:0/3715757546 learned_addr learned my addr 172.21.15.100:0/3715757546 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:14.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.623+0000 7f96dbe00640 1 -- 172.21.15.100:0/3715757546 >> v1:172.21.15.151:6789/0 conn(0x7f96dc06c9a0 legacy=0x7f96dc06cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:14.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.623+0000 7f96dbe00640 1 -- 172.21.15.100:0/3715757546 >> v1:172.21.15.100:6789/0 conn(0x7f96dc15ae60 legacy=0x7f96dc0cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.623+0000 7f96dbe00640 1 -- 172.21.15.100:0/3715757546 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96dc06c1a0 con 0x7f96dc0d0a70 2024-04-23T14:35:14.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.623+0000 7f96db400640 1 -- 172.21.15.100:0/3715757546 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 119528145 0 0) 0x7f96dc06bbe0 con 0x7f96dc15ae60 2024-04-23T14:35:14.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.623+0000 7f96dbe00640 1 --2- 172.21.15.100:0/3715757546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96dc0d0a70 0x7f96dc0ca830 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f96c80097c0 tx=0x7f96c801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21fb9a6b12c00a16 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:14.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.623+0000 7f96db400640 1 -- 172.21.15.100:0/3715757546 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c80b0070 con 0x7f96dc0d0a70 2024-04-23T14:35:14.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.623+0000 7f96db400640 1 -- 172.21.15.100:0/3715757546 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c80ab600 con 0x7f96dc0d0a70 2024-04-23T14:35:14.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.623+0000 7f96db400640 1 -- 172.21.15.100:0/3715757546 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c80ab9d0 con 0x7f96dc0d0a70 2024-04-23T14:35:14.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e3600640 1 -- 172.21.15.100:0/3715757546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96dc0d0a70 msgr2=0x7f96dc0ca830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e3600640 1 --2- 172.21.15.100:0/3715757546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96dc0d0a70 0x7f96dc0ca830 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f96c80097c0 tx=0x7f96c801c3d0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e3600640 1 -- 172.21.15.100:0/3715757546 shutdown_connections 2024-04-23T14:35:14.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e3600640 1 --2- 172.21.15.100:0/3715757546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96dc0d0a70 0x7f96dc0ca830 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e3600640 1 -- 172.21.15.100:0/3715757546 >> 172.21.15.100:0/3715757546 conn(0x7f96dc0b5200 msgr2=0x7f96dc0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:14.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e3600640 1 -- 172.21.15.100:0/3715757546 shutdown_connections 2024-04-23T14:35:14.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e3600640 1 -- 172.21.15.100:0/3715757546 wait complete. 2024-04-23T14:35:14.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e3600640 1 Processor -- start 2024-04-23T14:35:14.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e3600640 1 -- start start 2024-04-23T14:35:14.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e3600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96dc06c9a0 0x7f96dc15ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e3600640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f96dc0d0a70 0x7f96dc15cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e3600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96dc15ae60 0x7f96dc0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e3600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc0cce30 con 0x7f96dc06c9a0 2024-04-23T14:35:14.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e3600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc0cccb0 con 0x7f96dc15ae60 2024-04-23T14:35:14.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e3600640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f96dc0ccfb0 con 0x7f96dc0d0a70 2024-04-23T14:35:14.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96dc06c9a0 0x7f96dc15ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96dc06c9a0 0x7f96dc15ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41206/0 (socket says 172.21.15.100:41206) 2024-04-23T14:35:14.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e0c00640 1 -- 172.21.15.100:0/325484436 learned_addr learned my addr 172.21.15.100:0/325484436 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:14.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96dbe00640 1 --2- 172.21.15.100:0/325484436 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f96dc0d0a70 0x7f96dc15cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e1600640 1 --2- 172.21.15.100:0/325484436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96dc15ae60 0x7f96dc0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e0c00640 1 -- 172.21.15.100:0/325484436 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f96dc0d0a70 msgr2=0x7f96dc15cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e0c00640 1 --2- 172.21.15.100:0/325484436 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f96dc0d0a70 0x7f96dc15cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e0c00640 1 -- 172.21.15.100:0/325484436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96dc15ae60 msgr2=0x7f96dc0c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e0c00640 1 --2- 172.21.15.100:0/325484436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96dc15ae60 0x7f96dc0c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e0c00640 1 -- 172.21.15.100:0/325484436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96dc0c68e0 con 0x7f96dc06c9a0 2024-04-23T14:35:14.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96e1600640 1 --2- 172.21.15.100:0/325484436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96dc15ae60 0x7f96dc0c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:14.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.627+0000 7f96dbe00640 1 --2- 172.21.15.100:0/325484436 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f96dc0d0a70 0x7f96dc15cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:14.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.631+0000 7f96e0c00640 1 --2- 172.21.15.100:0/325484436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96dc06c9a0 0x7f96dc15ddf0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f96d000c480 tx=0x7f96d000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:14.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.631+0000 7f96d8c00640 1 -- 172.21.15.100:0/325484436 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0018070 con 0x7f96dc06c9a0 2024-04-23T14:35:14.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.631+0000 7f96d8c00640 1 -- 172.21.15.100:0/325484436 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96d0007500 con 0x7f96dc06c9a0 2024-04-23T14:35:14.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.631+0000 7f96d8c00640 1 -- 172.21.15.100:0/325484436 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0004a30 con 0x7f96dc06c9a0 2024-04-23T14:35:14.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.631+0000 7f96e3600640 1 -- 172.21.15.100:0/325484436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96dc0c7df0 con 0x7f96dc06c9a0 2024-04-23T14:35:14.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.631+0000 7f96e3600640 1 -- 172.21.15.100:0/325484436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96dc0c83b0 con 0x7f96dc06c9a0 2024-04-23T14:35:14.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.631+0000 7f96e3600640 1 -- 172.21.15.100:0/325484436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f96dc068ad0 con 0x7f96dc06c9a0 2024-04-23T14:35:14.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.631+0000 7f96d8c00640 1 -- 172.21.15.100:0/325484436 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f96d0004bd0 con 0x7f96dc06c9a0 2024-04-23T14:35:14.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.631+0000 7f96d8c00640 1 --2- 172.21.15.100:0/325484436 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f96bc068380 0x7f96bc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.631+0000 7f96d8c00640 1 -- 172.21.15.100:0/325484436 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f96d0085ab0 con 0x7f96dc06c9a0 2024-04-23T14:35:14.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.631+0000 7f96dbe00640 1 --2- 172.21.15.100:0/325484436 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f96bc068380 0x7f96bc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.631+0000 7f96d8c00640 1 --2- 172.21.15.100:0/325484436 >> [v2:172.21.15.151:6816/389311850,v1:172.21.15.151:6817/389311850] conn(0x7f96bc06e820 0x7f96bc070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.631+0000 7f96d8c00640 1 -- 172.21.15.100:0/325484436 --> [v2:172.21.15.151:6816/389311850,v1:172.21.15.151:6817/389311850] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f96bc071330 con 0x7f96bc06e820 2024-04-23T14:35:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.631+0000 7f96d8c00640 1 -- 172.21.15.100:0/325484436 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f96d008a2a0 con 0x7f96dc06c9a0 2024-04-23T14:35:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.631+0000 7f96e1600640 1 --2- 172.21.15.100:0/325484436 >> [v2:172.21.15.151:6816/389311850,v1:172.21.15.151:6817/389311850] conn(0x7f96bc06e820 0x7f96bc070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.631+0000 7f96dbe00640 1 --2- 172.21.15.100:0/325484436 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f96bc068380 0x7f96bc06a840 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f96c80ad000 tx=0x7f96c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.635+0000 7f96e1600640 1 --2- 172.21.15.100:0/325484436 >> [v2:172.21.15.151:6816/389311850,v1:172.21.15.151:6817/389311850] conn(0x7f96bc06e820 0x7f96bc070c60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.635+0000 7f96d8c00640 1 -- 172.21.15.100:0/325484436 <== osd.7 v2:172.21.15.151:6816/389311850 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96bc071330 con 0x7f96bc06e820 2024-04-23T14:35:14.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.647+0000 7f96e3600640 1 -- 172.21.15.100:0/325484436 --> [v2:172.21.15.151:6816/389311850,v1:172.21.15.151:6817/389311850] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f96dc15e410 con 0x7f96bc06e820 2024-04-23T14:35:14.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.647+0000 7f96d8c00640 1 -- 172.21.15.100:0/325484436 <== osd.7 v2:172.21.15.151:6816/389311850 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f96dc15e410 con 0x7f96bc06e820 2024-04-23T14:35:14.649 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-23T14:35:14.649 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T14:35:14.649 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-23T14:35:14.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.647+0000 7f96e3600640 1 -- 172.21.15.100:0/325484436 >> [v2:172.21.15.151:6816/389311850,v1:172.21.15.151:6817/389311850] conn(0x7f96bc06e820 msgr2=0x7f96bc070c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.647+0000 7f96e3600640 1 --2- 172.21.15.100:0/325484436 >> [v2:172.21.15.151:6816/389311850,v1:172.21.15.151:6817/389311850] conn(0x7f96bc06e820 0x7f96bc070c60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.647+0000 7f96e3600640 1 -- 172.21.15.100:0/325484436 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f96bc068380 msgr2=0x7f96bc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.647+0000 7f96e3600640 1 --2- 172.21.15.100:0/325484436 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f96bc068380 0x7f96bc06a840 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f96c80ad000 tx=0x7f96c8008040 comp rx=0 tx=0).stop 2024-04-23T14:35:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.647+0000 7f96e3600640 1 -- 172.21.15.100:0/325484436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96dc06c9a0 msgr2=0x7f96dc15ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.647+0000 7f96e3600640 1 --2- 172.21.15.100:0/325484436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96dc06c9a0 0x7f96dc15ddf0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f96d000c480 tx=0x7f96d000c940 comp rx=0 tx=0).stop 2024-04-23T14:35:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.647+0000 7f96e1600640 1 -- 172.21.15.100:0/325484436 reap_dead start 2024-04-23T14:35:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.647+0000 7f96e3600640 1 -- 172.21.15.100:0/325484436 shutdown_connections 2024-04-23T14:35:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.647+0000 7f96e3600640 1 -- 172.21.15.100:0/325484436 >> 172.21.15.100:0/325484436 conn(0x7f96dc0b5200 msgr2=0x7f96dc15a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.647+0000 7f96e3600640 1 -- 172.21.15.100:0/325484436 shutdown_connections 2024-04-23T14:35:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.647+0000 7f96e3600640 1 -- 172.21.15.100:0/325484436 wait complete. 2024-04-23T14:35:14.667 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-23T14:35:14.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.803+0000 7f4b48c00640 1 Processor -- start 2024-04-23T14:35:14.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.803+0000 7f4b48c00640 1 -- start start 2024-04-23T14:35:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.803+0000 7f4b48c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b440d0190 0x7f4b440d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.803+0000 7f4b48c00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b440c5160 0x7f4b440c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.803+0000 7f4b48c00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b440cee00 con 0x7f4b4415b660 2024-04-23T14:35:14.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.803+0000 7f4b48c00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f4b4415ba40 con 0x7f4b440c5160 2024-04-23T14:35:14.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.803+0000 7f4b48c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4b440d0b30 con 0x7f4b440d0190 2024-04-23T14:35:14.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.803+0000 7f4b42800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b440c5160 0x7f4b440c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.803+0000 7f4b42800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b440c5160 0x7f4b440c55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41210/0 (socket says 172.21.15.100:41210) 2024-04-23T14:35:14.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.803+0000 7f4b42800640 1 -- 172.21.15.100:0/4114368941 learned_addr learned my addr 172.21.15.100:0/4114368941 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:14.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.803+0000 7f4b39400640 1 --2- 172.21.15.100:0/4114368941 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b440d0190 0x7f4b440d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.803+0000 7f4b42800640 1 -- 172.21.15.100:0/4114368941 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b440d0190 msgr2=0x7f4b440d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:14.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.803+0000 7f4b42800640 1 --2- 172.21.15.100:0/4114368941 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b440d0190 0x7f4b440d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.803+0000 7f4b42800640 1 -- 172.21.15.100:0/4114368941 >> v1:172.21.15.151:6789/0 conn(0x7f4b4415b660 legacy=0x7f4b440c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.803+0000 7f4b42800640 1 -- 172.21.15.100:0/4114368941 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4406bc50 con 0x7f4b440c5160 2024-04-23T14:35:14.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.807+0000 7f4b42800640 1 --2- 172.21.15.100:0/4114368941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b440c5160 0x7f4b440c55f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009e50 tx=0x7f4b3c01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed05cf2ffcb73923 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:14.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.807+0000 7f4b41400640 1 -- 172.21.15.100:0/4114368941 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0a7ae0 con 0x7f4b440c5160 2024-04-23T14:35:14.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.807+0000 7f4b41400640 1 -- 172.21.15.100:0/4114368941 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b3c0a7c80 con 0x7f4b440c5160 2024-04-23T14:35:14.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.807+0000 7f4b41400640 1 -- 172.21.15.100:0/4114368941 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0af610 con 0x7f4b440c5160 2024-04-23T14:35:14.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.807+0000 7f4b48c00640 1 -- 172.21.15.100:0/4114368941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b440c5160 msgr2=0x7f4b440c55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.807+0000 7f4b48c00640 1 --2- 172.21.15.100:0/4114368941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b440c5160 0x7f4b440c55f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009e50 tx=0x7f4b3c01ec10 comp rx=0 tx=0).stop 2024-04-23T14:35:14.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.807+0000 7f4b48c00640 1 -- 172.21.15.100:0/4114368941 shutdown_connections 2024-04-23T14:35:14.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.807+0000 7f4b48c00640 1 --2- 172.21.15.100:0/4114368941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b440c5160 0x7f4b440c55f0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.807+0000 7f4b48c00640 1 --2- 172.21.15.100:0/4114368941 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b440d0190 0x7f4b440d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.807+0000 7f4b48c00640 1 -- 172.21.15.100:0/4114368941 >> 172.21.15.100:0/4114368941 conn(0x7f4b440b5310 msgr2=0x7f4b440b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.807+0000 7f4b48c00640 1 -- 172.21.15.100:0/4114368941 shutdown_connections 2024-04-23T14:35:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.807+0000 7f4b48c00640 1 -- 172.21.15.100:0/4114368941 wait complete. 2024-04-23T14:35:14.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b48c00640 1 Processor -- start 2024-04-23T14:35:14.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b48c00640 1 -- start start 2024-04-23T14:35:14.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b48c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b440c5160 0x7f4b44152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b48c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b440d0190 0x7f4b4415d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b48c00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b4415b660 0x7f4b4415d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b48c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4b440c6f30 con 0x7f4b440d0190 2024-04-23T14:35:14.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b48c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4406c7d0 con 0x7f4b440c5160 2024-04-23T14:35:14.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b48c00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4b440c70b0 con 0x7f4b4415b660 2024-04-23T14:35:14.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b39400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b440d0190 0x7f4b4415d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b39400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b440d0190 0x7f4b4415d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41220/0 (socket says 172.21.15.100:41220) 2024-04-23T14:35:14.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b39400640 1 -- 172.21.15.100:0/2633413650 learned_addr learned my addr 172.21.15.100:0/2633413650 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:14.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b41e00640 1 --2- 172.21.15.100:0/2633413650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b440c5160 0x7f4b44152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b42800640 1 --2- 172.21.15.100:0/2633413650 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b4415b660 0x7f4b4415d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b39400640 1 -- 172.21.15.100:0/2633413650 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b4415b660 msgr2=0x7f4b4415d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b39400640 1 --2- 172.21.15.100:0/2633413650 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b4415b660 0x7f4b4415d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b39400640 1 -- 172.21.15.100:0/2633413650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b440c5160 msgr2=0x7f4b44152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b39400640 1 --2- 172.21.15.100:0/2633413650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b440c5160 0x7f4b44152170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b39400640 1 -- 172.21.15.100:0/2633413650 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4415dd20 con 0x7f4b440d0190 2024-04-23T14:35:14.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b39400640 1 --2- 172.21.15.100:0/2633413650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b440d0190 0x7f4b4415d1d0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00e930 tx=0x7f4b2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:14.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b3aa00640 1 -- 172.21.15.100:0/2633413650 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c017070 con 0x7f4b440d0190 2024-04-23T14:35:14.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b3aa00640 1 -- 172.21.15.100:0/2633413650 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b2c00ca00 con 0x7f4b440d0190 2024-04-23T14:35:14.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b3aa00640 1 -- 172.21.15.100:0/2633413650 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c00cdb0 con 0x7f4b440d0190 2024-04-23T14:35:14.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b48c00640 1 -- 172.21.15.100:0/2633413650 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4415e000 con 0x7f4b440d0190 2024-04-23T14:35:14.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b48c00640 1 -- 172.21.15.100:0/2633413650 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b4415e5c0 con 0x7f4b440d0190 2024-04-23T14:35:14.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.811+0000 7f4b48c00640 1 -- 172.21.15.100:0/2633413650 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b44068ad0 con 0x7f4b440d0190 2024-04-23T14:35:14.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.815+0000 7f4b3aa00640 1 -- 172.21.15.100:0/2633413650 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4b2c01a070 con 0x7f4b440d0190 2024-04-23T14:35:14.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.815+0000 7f4b3aa00640 1 --2- 172.21.15.100:0/2633413650 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4b20068380 0x7f4b2006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:14.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.815+0000 7f4b3aa00640 1 -- 172.21.15.100:0/2633413650 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f4b2c086c30 con 0x7f4b440d0190 2024-04-23T14:35:14.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.815+0000 7f4b41e00640 1 --2- 172.21.15.100:0/2633413650 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4b20068380 0x7f4b2006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:14.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.815+0000 7f4b41e00640 1 --2- 172.21.15.100:0/2633413650 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4b20068380 0x7f4b2006a840 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4b340015b0 tx=0x7f4b34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:14.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.819+0000 7f4b3aa00640 1 -- 172.21.15.100:0/2633413650 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4b2c057410 con 0x7f4b440d0190 2024-04-23T14:35:14.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.975+0000 7f4b48c00640 1 -- 172.21.15.100:0/2633413650 --> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f4b440c7300 con 0x7f4b20068380 2024-04-23T14:35:14.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.975+0000 7f4b3aa00640 1 -- 172.21.15.100:0/2633413650 <== mgr.4101 v2:172.21.15.100:6832/2859853957 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4b440c7300 con 0x7f4b20068380 2024-04-23T14:35:14.977 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 7 to deep-scrub 2024-04-23T14:35:14.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.979+0000 7f4b48c00640 1 -- 172.21.15.100:0/2633413650 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4b20068380 msgr2=0x7f4b2006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.979+0000 7f4b48c00640 1 --2- 172.21.15.100:0/2633413650 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4b20068380 0x7f4b2006a840 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4b340015b0 tx=0x7f4b34009040 comp rx=0 tx=0).stop 2024-04-23T14:35:14.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.979+0000 7f4b48c00640 1 -- 172.21.15.100:0/2633413650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b440d0190 msgr2=0x7f4b4415d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:14.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.979+0000 7f4b48c00640 1 --2- 172.21.15.100:0/2633413650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b440d0190 0x7f4b4415d1d0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00e930 tx=0x7f4b2c00edf0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.979+0000 7f4b48c00640 1 -- 172.21.15.100:0/2633413650 shutdown_connections 2024-04-23T14:35:14.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.979+0000 7f4b48c00640 1 --2- 172.21.15.100:0/2633413650 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f4b20068380 0x7f4b2006a840 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.979+0000 7f4b48c00640 1 --2- 172.21.15.100:0/2633413650 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4b4415b660 0x7f4b4415d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.979+0000 7f4b48c00640 1 --2- 172.21.15.100:0/2633413650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b440d0190 0x7f4b4415d1d0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.979+0000 7f4b48c00640 1 --2- 172.21.15.100:0/2633413650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b440c5160 0x7f4b44152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:14.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.979+0000 7f4b48c00640 1 -- 172.21.15.100:0/2633413650 >> 172.21.15.100:0/2633413650 conn(0x7f4b440b5310 msgr2=0x7f4b44072080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:14.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.979+0000 7f4b48c00640 1 -- 172.21.15.100:0/2633413650 shutdown_connections 2024-04-23T14:35:14.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:14.979+0000 7f4b48c00640 1 -- 172.21.15.100:0/2633413650 wait complete. 2024-04-23T14:35:14.998 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:35:15.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.131+0000 7f07d0e00640 1 Processor -- start 2024-04-23T14:35:15.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.131+0000 7f07d0e00640 1 -- start start 2024-04-23T14:35:15.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.131+0000 7f07d0e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f07cc073610 0x7f07cc0adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:15.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.131+0000 7f07d0e00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07cc0af080 con 0x7f07cc15b640 2024-04-23T14:35:15.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.131+0000 7f07d0e00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07cc0afbc0 con 0x7f07cc0b2a10 2024-04-23T14:35:15.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.131+0000 7f07d0e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f07cc15ba20 con 0x7f07cc073610 2024-04-23T14:35:15.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.131+0000 7f07c9e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f07cc15b640 0x7f07cc072f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42262/0 (socket says 172.21.15.100:42262) 2024-04-23T14:35:15.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.131+0000 7f07c9e00640 1 -- 172.21.15.100:0/3954662217 learned_addr learned my addr 172.21.15.100:0/3954662217 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:15.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.131+0000 7f07ca800640 1 --2- 172.21.15.100:0/3954662217 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f07cc073610 0x7f07cc0adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:15.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.131+0000 7f07c8a00640 1 -- 172.21.15.100:0/3954662217 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3885739646 0 0) 0x7f07cc0af080 con 0x7f07cc15b640 2024-04-23T14:35:15.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.131+0000 7f07c8a00640 1 -- 172.21.15.100:0/3954662217 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07b0003530 con 0x7f07cc15b640 2024-04-23T14:35:15.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.135+0000 7f07c8a00640 1 -- 172.21.15.100:0/3954662217 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2513174398 0 0) 0x7f07cc0afbc0 con 0x7f07cc0b2a10 2024-04-23T14:35:15.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.135+0000 7f07c8a00640 1 -- 172.21.15.100:0/3954662217 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07cc0af080 con 0x7f07cc0b2a10 2024-04-23T14:35:15.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.135+0000 7f07c8a00640 1 -- 172.21.15.100:0/3954662217 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2284019791 0 0) 0x7f07b0003530 con 0x7f07cc15b640 2024-04-23T14:35:15.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.135+0000 7f07ca800640 1 -- 172.21.15.100:0/3954662217 >> v1:172.21.15.151:6789/0 conn(0x7f07cc0b2a10 legacy=0x7f07cc06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:15.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.135+0000 7f07ca800640 1 -- 172.21.15.100:0/3954662217 >> v1:172.21.15.100:6789/0 conn(0x7f07cc15b640 legacy=0x7f07cc072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:15.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.135+0000 7f07ca800640 1 -- 172.21.15.100:0/3954662217 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07cc0af320 con 0x7f07cc073610 2024-04-23T14:35:15.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.135+0000 7f07ca800640 1 --2- 172.21.15.100:0/3954662217 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f07cc073610 0x7f07cc0adfc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f07c4009dc0 tx=0x7f07c401c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=192b06e2caf68216 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:15.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.135+0000 7f07c8a00640 1 -- 172.21.15.100:0/3954662217 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c40b2070 con 0x7f07cc073610 2024-04-23T14:35:15.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.135+0000 7f07c8a00640 1 -- 172.21.15.100:0/3954662217 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07c40a56c0 con 0x7f07cc073610 2024-04-23T14:35:15.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.135+0000 7f07c8a00640 1 -- 172.21.15.100:0/3954662217 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc001680 con 0x7f07cc073610 2024-04-23T14:35:15.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07d0e00640 1 -- 172.21.15.100:0/3954662217 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f07cc073610 msgr2=0x7f07cc0adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:15.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07d0e00640 1 --2- 172.21.15.100:0/3954662217 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f07cc073610 0x7f07cc0adfc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f07c4009dc0 tx=0x7f07c401c880 comp rx=0 tx=0).stop 2024-04-23T14:35:15.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07d0e00640 1 -- 172.21.15.100:0/3954662217 shutdown_connections 2024-04-23T14:35:15.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07d0e00640 1 --2- 172.21.15.100:0/3954662217 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f07cc073610 0x7f07cc0adfc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:15.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07d0e00640 1 -- 172.21.15.100:0/3954662217 >> 172.21.15.100:0/3954662217 conn(0x7f07cc0b1e80 msgr2=0x7f07cc0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:15.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07d0e00640 1 -- 172.21.15.100:0/3954662217 shutdown_connections 2024-04-23T14:35:15.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07d0e00640 1 -- 172.21.15.100:0/3954662217 wait complete. 2024-04-23T14:35:15.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07d0e00640 1 Processor -- start 2024-04-23T14:35:15.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07d0e00640 1 -- start start 2024-04-23T14:35:15.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07d0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc073610 0x7f07cc15c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:15.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07d0e00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f07cc0b2a10 0x7f07cc13e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:15.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07d0e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07cc15b640 0x7f07cc1f7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:15.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07d0e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc1691c0 con 0x7f07cc073610 2024-04-23T14:35:15.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07d0e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc06f160 con 0x7f07cc15b640 2024-04-23T14:35:15.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07d0e00640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f07cc169340 con 0x7f07cc0b2a10 2024-04-23T14:35:15.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07c9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc073610 0x7f07cc15c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:15.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07c9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc073610 0x7f07cc15c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41224/0 (socket says 172.21.15.100:41224) 2024-04-23T14:35:15.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07c9e00640 1 -- 172.21.15.100:0/3492641894 learned_addr learned my addr 172.21.15.100:0/3492641894 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:15.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07ca800640 1 --2- 172.21.15.100:0/3492641894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07cc15b640 0x7f07cc1f7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:15.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07c9400640 1 --2- 172.21.15.100:0/3492641894 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f07cc0b2a10 0x7f07cc13e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:15.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07c9e00640 1 -- 172.21.15.100:0/3492641894 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f07cc0b2a10 msgr2=0x7f07cc13e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:15.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07c9e00640 1 --2- 172.21.15.100:0/3492641894 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f07cc0b2a10 0x7f07cc13e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:15.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07c9e00640 1 -- 172.21.15.100:0/3492641894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07cc15b640 msgr2=0x7f07cc1f7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:15.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07c9e00640 1 --2- 172.21.15.100:0/3492641894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07cc15b640 0x7f07cc1f7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:15.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07c9e00640 1 -- 172.21.15.100:0/3492641894 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07cc15cbb0 con 0x7f07cc073610 2024-04-23T14:35:15.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07c9400640 1 --2- 172.21.15.100:0/3492641894 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f07cc0b2a10 0x7f07cc13e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:35:15.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07c9e00640 1 --2- 172.21.15.100:0/3492641894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc073610 0x7f07cc15c670 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f07bc00c730 tx=0x7f07bc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:15.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.139+0000 7f07c2000640 1 -- 172.21.15.100:0/3492641894 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc017070 con 0x7f07cc073610 2024-04-23T14:35:15.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.143+0000 7f07c2000640 1 -- 172.21.15.100:0/3492641894 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07bc005b00 con 0x7f07cc073610 2024-04-23T14:35:15.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.143+0000 7f07d0e00640 1 -- 172.21.15.100:0/3492641894 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07cc13e960 con 0x7f07cc073610 2024-04-23T14:35:15.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.143+0000 7f07c2000640 1 -- 172.21.15.100:0/3492641894 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc007070 con 0x7f07cc073610 2024-04-23T14:35:15.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.143+0000 7f07d0e00640 1 -- 172.21.15.100:0/3492641894 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07cc15d6f0 con 0x7f07cc073610 2024-04-23T14:35:15.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.143+0000 7f07d0e00640 1 -- 172.21.15.100:0/3492641894 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07cc068ad0 con 0x7f07cc073610 2024-04-23T14:35:15.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.143+0000 7f07c2000640 1 -- 172.21.15.100:0/3492641894 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f07bc01a070 con 0x7f07cc073610 2024-04-23T14:35:15.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.143+0000 7f07c2000640 1 --2- 172.21.15.100:0/3492641894 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f07ac068380 0x7f07ac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:15.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.143+0000 7f07c2000640 1 -- 172.21.15.100:0/3492641894 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f07bc085bb0 con 0x7f07cc073610 2024-04-23T14:35:15.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.143+0000 7f07c9400640 1 --2- 172.21.15.100:0/3492641894 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f07ac068380 0x7f07ac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:15.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.143+0000 7f07c9400640 1 --2- 172.21.15.100:0/3492641894 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f07ac068380 0x7f07ac06a840 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f07cc071eb0 tx=0x7f07b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:15.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.147+0000 7f07c2000640 1 -- 172.21.15.100:0/3492641894 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f07bc055d20 con 0x7f07cc073610 2024-04-23T14:35:15.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.299+0000 7f07d0e00640 1 -- 172.21.15.100:0/3492641894 --> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f07cc0b0100 con 0x7f07ac068380 2024-04-23T14:35:15.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.303+0000 7f07c2000640 1 -- 172.21.15.100:0/3492641894 <== mgr.4101 v2:172.21.15.100:6832/2859853957 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13354 (secure 0 0 0) 0x7f07cc0b0100 con 0x7f07ac068380 2024-04-23T14:35:15.303 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:35:15.303 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-04-23T14:35:15.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.303+0000 7f07d0e00640 1 -- 172.21.15.100:0/3492641894 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f07ac068380 msgr2=0x7f07ac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:15.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.303+0000 7f07d0e00640 1 --2- 172.21.15.100:0/3492641894 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f07ac068380 0x7f07ac06a840 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f07cc071eb0 tx=0x7f07b8009040 comp rx=0 tx=0).stop 2024-04-23T14:35:15.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.303+0000 7f07d0e00640 1 -- 172.21.15.100:0/3492641894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc073610 msgr2=0x7f07cc15c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:15.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.303+0000 7f07d0e00640 1 --2- 172.21.15.100:0/3492641894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc073610 0x7f07cc15c670 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f07bc00c730 tx=0x7f07bc00cbf0 comp rx=0 tx=0).stop 2024-04-23T14:35:15.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.303+0000 7f07d0e00640 1 -- 172.21.15.100:0/3492641894 shutdown_connections 2024-04-23T14:35:15.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.303+0000 7f07d0e00640 1 --2- 172.21.15.100:0/3492641894 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f07ac068380 0x7f07ac06a840 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:15.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.303+0000 7f07d0e00640 1 --2- 172.21.15.100:0/3492641894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07cc15b640 0x7f07cc1f7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:15.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.303+0000 7f07d0e00640 1 --2- 172.21.15.100:0/3492641894 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f07cc0b2a10 0x7f07cc13e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:15.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.303+0000 7f07d0e00640 1 --2- 172.21.15.100:0/3492641894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc073610 0x7f07cc15c670 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:15.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.303+0000 7f07d0e00640 1 -- 172.21.15.100:0/3492641894 >> 172.21.15.100:0/3492641894 conn(0x7f07cc0b1e80 msgr2=0x7f07cc0ae590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:15.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.303+0000 7f07d0e00640 1 -- 172.21.15.100:0/3492641894 shutdown_connections 2024-04-23T14:35:15.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:15.303+0000 7f07d0e00640 1 -- 172.21.15.100:0/3492641894 wait complete. 2024-04-23T14:35:15.324 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-04-23T14:35:14.773286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":230964,"kb_used_data":6472,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754743756,"statfs":{"total":773094113280,"available":772857606144,"internally_reserved":0,"allocated":6627328,"data_stored":3618600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.339411"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":237,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-23T14:35:06.634831+0000","last_change":"2024-04-23T14:31:58.314235+0000","last_active":"2024-04-23T14:35:06.634831+0000","last_peered":"2024-04-23T14:35:06.634831+0000","last_clean":"2024-04-23T14:35:06.634831+0000","last_became_active":"2024-04-23T14:31:58.313165+0000","last_became_peered":"2024-04-23T14:31:58.313165+0000","last_unstale":"2024-04-23T14:35:06.634831+0000","last_undegraded":"2024-04-23T14:35:06.634831+0000","last_fullsized":"2024-04-23T14:35:06.634831+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:57.140712+0000","last_clean_scrub_stamp":"2024-04-23T14:31:57.140712+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:44.170436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1273856,"data_stored":895101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27688,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":679936,"data_stored":304733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27688,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":679936,"data_stored":304733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27684,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":679936,"data_stored":304733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27684,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":679936,"data_stored":304733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27684,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":679936,"data_stored":304733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27684,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":679936,"data_stored":304733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1273856,"data_stored":895101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:35:15.327 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-23T14:31:57.140712+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=14, tm_min=31, tm_sec=57, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=14, tm_min=35, tm_sec=9, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T14:35:15.327 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-23T14:35:35.328 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:35:35.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.471+0000 7ff6c9400640 1 Processor -- start 2024-04-23T14:35:35.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.471+0000 7ff6c9400640 1 -- start start 2024-04-23T14:35:35.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.475+0000 7ff6c9400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6c415c470 0x7ff6c40b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.475+0000 7ff6c9400640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff6c40ad870 0x7ff6c406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.475+0000 7ff6c9400640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6c4160320 con 0x7ff6c4160de0 2024-04-23T14:35:35.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.475+0000 7ff6c9400640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7ff6c40cc7e0 con 0x7ff6c40ad870 2024-04-23T14:35:35.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.475+0000 7ff6c9400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff6c41605e0 con 0x7ff6c415c470 2024-04-23T14:35:35.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.475+0000 7ff6c2800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6c415c470 0x7ff6c40b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.475+0000 7ff6c3200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff6c40ad870 0x7ff6c406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.475+0000 7ff6c2800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6c415c470 0x7ff6c40b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.100:39006/0 (socket says 172.21.15.100:39006) 2024-04-23T14:35:35.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.475+0000 7ff6c2800640 1 -- 172.21.15.100:0/4196986388 learned_addr learned my addr 172.21.15.100:0/4196986388 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:35.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.475+0000 7ff6c3200640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff6c40ad870 0x7ff6c406bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:53366/0 (socket says 172.21.15.100:53366) 2024-04-23T14:35:35.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.475+0000 7ff6b9e00640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7ff6c4160de0 0x7ff6c40ad160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.100:51596/0 (socket says 172.21.15.100:51596) 2024-04-23T14:35:35.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.475+0000 7ff6c2800640 1 -- 172.21.15.100:0/4196986388 >> v2:172.21.15.151:3300/0 conn(0x7ff6c40ad870 msgr2=0x7ff6c406bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:35.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.475+0000 7ff6c2800640 1 --2- 172.21.15.100:0/4196986388 >> v2:172.21.15.151:3300/0 conn(0x7ff6c40ad870 0x7ff6c406bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.475+0000 7ff6c2800640 1 -- 172.21.15.100:0/4196986388 >> v1:172.21.15.151:6789/0 conn(0x7ff6c4160de0 legacy=0x7ff6c40ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.475+0000 7ff6c2800640 1 -- 172.21.15.100:0/4196986388 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c41608c0 con 0x7ff6c415c470 2024-04-23T14:35:35.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.479+0000 7ff6c2800640 1 --2- 172.21.15.100:0/4196986388 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6c415c470 0x7ff6c40b29a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009e50 tx=0x7ff6b401c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8bc3054d3249451 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:35.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.479+0000 7ff6c1e00640 1 -- 172.21.15.100:0/4196986388 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b40a5770 con 0x7ff6c415c470 2024-04-23T14:35:35.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.479+0000 7ff6c1e00640 1 -- 172.21.15.100:0/4196986388 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b40a5910 con 0x7ff6c415c470 2024-04-23T14:35:35.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.479+0000 7ff6c1e00640 1 -- 172.21.15.100:0/4196986388 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b40a5ce0 con 0x7ff6c415c470 2024-04-23T14:35:35.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.479+0000 7ff6c9400640 1 -- 172.21.15.100:0/4196986388 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6c415c470 msgr2=0x7ff6c40b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.479+0000 7ff6c9400640 1 --2- 172.21.15.100:0/4196986388 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6c415c470 0x7ff6c40b29a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009e50 tx=0x7ff6b401c9a0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.479+0000 7ff6c9400640 1 -- 172.21.15.100:0/4196986388 shutdown_connections 2024-04-23T14:35:35.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.479+0000 7ff6c9400640 1 --2- 172.21.15.100:0/4196986388 >> v2:172.21.15.151:3300/0 conn(0x7ff6c40ad870 0x7ff6c406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.479+0000 7ff6c9400640 1 --2- 172.21.15.100:0/4196986388 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6c415c470 0x7ff6c40b29a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.479+0000 7ff6c9400640 1 -- 172.21.15.100:0/4196986388 >> 172.21.15.100:0/4196986388 conn(0x7ff6c40b1e80 msgr2=0x7ff6c40b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:35.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.479+0000 7ff6c9400640 1 -- 172.21.15.100:0/4196986388 shutdown_connections 2024-04-23T14:35:35.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.479+0000 7ff6c9400640 1 -- 172.21.15.100:0/4196986388 wait complete. 2024-04-23T14:35:35.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.479+0000 7ff6c9400640 1 Processor -- start 2024-04-23T14:35:35.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.479+0000 7ff6c9400640 1 -- start start 2024-04-23T14:35:35.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6c9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6c40ad870 0x7ff6c40ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6c9400640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6c415c470 0x7ff6c40ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6c9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c4160de0 0x7ff6c40cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6c9400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c415ca60 con 0x7ff6c4160de0 2024-04-23T14:35:35.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6c9400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c40cc580 con 0x7ff6c40ad870 2024-04-23T14:35:35.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6c9400640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff6c4169540 con 0x7ff6c415c470 2024-04-23T14:35:35.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6c3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c4160de0 0x7ff6c40cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6c3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c4160de0 0x7ff6c40cf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59756/0 (socket says 172.21.15.100:59756) 2024-04-23T14:35:35.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6c3200640 1 -- 172.21.15.100:0/1702818933 learned_addr learned my addr 172.21.15.100:0/1702818933 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:35.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6b9e00640 1 --2- 172.21.15.100:0/1702818933 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6c415c470 0x7ff6c40ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6c2800640 1 --2- 172.21.15.100:0/1702818933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6c40ad870 0x7ff6c40ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6c3200640 1 -- 172.21.15.100:0/1702818933 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6c415c470 msgr2=0x7ff6c40ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6c3200640 1 --2- 172.21.15.100:0/1702818933 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6c415c470 0x7ff6c40ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6c3200640 1 -- 172.21.15.100:0/1702818933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6c40ad870 msgr2=0x7ff6c40ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6c3200640 1 --2- 172.21.15.100:0/1702818933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6c40ad870 0x7ff6c40ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6c3200640 1 -- 172.21.15.100:0/1702818933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c40cd560 con 0x7ff6c4160de0 2024-04-23T14:35:35.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6c3200640 1 --2- 172.21.15.100:0/1702818933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c4160de0 0x7ff6c40cf1c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00e7f0 tx=0x7ff6bc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:35.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6bb400640 1 -- 172.21.15.100:0/1702818933 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc017070 con 0x7ff6c4160de0 2024-04-23T14:35:35.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6bb400640 1 -- 172.21.15.100:0/1702818933 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6bc00c730 con 0x7ff6c4160de0 2024-04-23T14:35:35.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6bb400640 1 -- 172.21.15.100:0/1702818933 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc00cb00 con 0x7ff6c4160de0 2024-04-23T14:35:35.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6c9400640 1 -- 172.21.15.100:0/1702818933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c40cd840 con 0x7ff6c4160de0 2024-04-23T14:35:35.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6c9400640 1 -- 172.21.15.100:0/1702818933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c40cd1a0 con 0x7ff6c4160de0 2024-04-23T14:35:35.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6c9400640 1 -- 172.21.15.100:0/1702818933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6c4068ad0 con 0x7ff6c4160de0 2024-04-23T14:35:35.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.483+0000 7ff6bb400640 1 -- 172.21.15.100:0/1702818933 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff6bc01a070 con 0x7ff6c4160de0 2024-04-23T14:35:35.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.487+0000 7ff6bb400640 1 --2- 172.21.15.100:0/1702818933 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff6a00683b0 0x7ff6a006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.487+0000 7ff6bb400640 1 -- 172.21.15.100:0/1702818933 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7ff6bc086de0 con 0x7ff6c4160de0 2024-04-23T14:35:35.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.487+0000 7ff6c2800640 1 --2- 172.21.15.100:0/1702818933 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff6a00683b0 0x7ff6a006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.487+0000 7ff6c2800640 1 --2- 172.21.15.100:0/1702818933 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff6a00683b0 0x7ff6a006a870 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40ae000 tx=0x7ff6b40a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:35.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.491+0000 7ff6bb400640 1 -- 172.21.15.100:0/1702818933 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff6bc056d90 con 0x7ff6c4160de0 2024-04-23T14:35:35.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.643+0000 7ff6c9400640 1 -- 172.21.15.100:0/1702818933 --> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff6c406fea0 con 0x7ff6a00683b0 2024-04-23T14:35:35.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.643+0000 7ff6bb400640 1 -- 172.21.15.100:0/1702818933 <== mgr.4101 v2:172.21.15.100:6832/2859853957 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13359 (secure 0 0 0) 0x7ff6c406fea0 con 0x7ff6a00683b0 2024-04-23T14:35:35.645 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:35:35.645 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-04-23T14:35:35.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.647+0000 7ff6c9400640 1 -- 172.21.15.100:0/1702818933 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff6a00683b0 msgr2=0x7ff6a006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.647+0000 7ff6c9400640 1 --2- 172.21.15.100:0/1702818933 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff6a00683b0 0x7ff6a006a870 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40ae000 tx=0x7ff6b40a7070 comp rx=0 tx=0).stop 2024-04-23T14:35:35.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.647+0000 7ff6c9400640 1 -- 172.21.15.100:0/1702818933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c4160de0 msgr2=0x7ff6c40cf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.647+0000 7ff6c9400640 1 --2- 172.21.15.100:0/1702818933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c4160de0 0x7ff6c40cf1c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00e7f0 tx=0x7ff6bc00ecb0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.647+0000 7ff6c9400640 1 -- 172.21.15.100:0/1702818933 shutdown_connections 2024-04-23T14:35:35.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.647+0000 7ff6c9400640 1 --2- 172.21.15.100:0/1702818933 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7ff6a00683b0 0x7ff6a006a870 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.647+0000 7ff6c9400640 1 --2- 172.21.15.100:0/1702818933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c4160de0 0x7ff6c40cf1c0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.647+0000 7ff6c9400640 1 --2- 172.21.15.100:0/1702818933 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6c415c470 0x7ff6c40ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.647+0000 7ff6c9400640 1 --2- 172.21.15.100:0/1702818933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6c40ad870 0x7ff6c40ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.647+0000 7ff6c9400640 1 -- 172.21.15.100:0/1702818933 >> 172.21.15.100:0/1702818933 conn(0x7ff6c40b1e80 msgr2=0x7ff6c41433c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:35.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.647+0000 7ff6c9400640 1 -- 172.21.15.100:0/1702818933 shutdown_connections 2024-04-23T14:35:35.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.647+0000 7ff6c9400640 1 -- 172.21.15.100:0/1702818933 wait complete. 2024-04-23T14:35:35.666 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-04-23T14:35:34.777416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":230952,"kb_used_data":6472,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754743768,"statfs":{"total":773094113280,"available":772857618432,"internally_reserved":0,"allocated":6627328,"data_stored":3618600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002443"},"pg_stats":[{"pgid":"1.0","version":"56'119","reported_seq":248,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-23T14:35:16.001326+0000","last_change":"2024-04-23T14:35:16.001363+0000","last_active":"2024-04-23T14:35:16.001326+0000","last_peered":"2024-04-23T14:35:16.001326+0000","last_clean":"2024-04-23T14:35:16.001326+0000","last_became_active":"2024-04-23T14:31:58.313165+0000","last_became_peered":"2024-04-23T14:31:58.313165+0000","last_unstale":"2024-04-23T14:35:16.001326+0000","last_undegraded":"2024-04-23T14:35:16.001326+0000","last_fullsized":"2024-04-23T14:35:16.001326+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'119","last_scrub_stamp":"2024-04-23T14:35:16.001246+0000","last_deep_scrub":"56'119","last_deep_scrub_stamp":"2024-04-23T14:35:16.001246+0000","last_clean_scrub_stamp":"2024-04-23T14:35:16.001246+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:16:39.732977+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1273856,"data_stored":895101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27688,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":679936,"data_stored":304733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27688,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":679936,"data_stored":304733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27684,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":679936,"data_stored":304733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27684,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":679936,"data_stored":304733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27684,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":679936,"data_stored":304733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27684,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":679936,"data_stored":304733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1273856,"data_stored":895101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:35:35.667 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-23T14:35:35.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.799+0000 7f69f7800640 1 Processor -- start 2024-04-23T14:35:35.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.799+0000 7f69f7800640 1 -- start start 2024-04-23T14:35:35.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.803+0000 7f69f7800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f69f015ae60 0x7f69f00cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.803+0000 7f69f7800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f69f00d0a70 0x7f69f00ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.803+0000 7f69f7800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69f00cfc70 con 0x7f69f006c9a0 2024-04-23T14:35:35.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.803+0000 7f69f7800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f69f015b240 con 0x7f69f00d0a70 2024-04-23T14:35:35.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.803+0000 7f69f7800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f69f00cff10 con 0x7f69f015ae60 2024-04-23T14:35:35.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.803+0000 7f69efe00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f69f00d0a70 0x7f69f00ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.803+0000 7f69efe00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f69f00d0a70 0x7f69f00ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59760/0 (socket says 172.21.15.100:59760) 2024-04-23T14:35:35.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.803+0000 7f69efe00640 1 -- 172.21.15.100:0/1325780371 learned_addr learned my addr 172.21.15.100:0/1325780371 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:35.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.803+0000 7f69f4e00640 1 --2- 172.21.15.100:0/1325780371 >> v2:172.21.15.151:3300/0 conn(0x7f69f015ae60 0x7f69f00cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.803+0000 7f69efe00640 1 -- 172.21.15.100:0/1325780371 >> v2:172.21.15.151:3300/0 conn(0x7f69f015ae60 msgr2=0x7f69f00cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:35.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.803+0000 7f69efe00640 1 --2- 172.21.15.100:0/1325780371 >> v2:172.21.15.151:3300/0 conn(0x7f69f015ae60 0x7f69f00cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.803+0000 7f69efe00640 1 -- 172.21.15.100:0/1325780371 >> v1:172.21.15.100:6789/0 conn(0x7f69f006c9a0 legacy=0x7f69f006cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.803+0000 7f69efe00640 1 -- 172.21.15.100:0/1325780371 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f006bbe0 con 0x7f69f00d0a70 2024-04-23T14:35:35.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.807+0000 7f69efe00640 1 --2- 172.21.15.100:0/1325780371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69f00d0a70 0x7f69f00ca830 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f69f0071600 tx=0x7f69dc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a9912e3c4f2d47 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:35.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.807+0000 7f69ef400640 1 -- 172.21.15.100:0/1325780371 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc0b0070 con 0x7f69f00d0a70 2024-04-23T14:35:35.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.807+0000 7f69ef400640 1 -- 172.21.15.100:0/1325780371 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69dc0ab600 con 0x7f69f00d0a70 2024-04-23T14:35:35.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.807+0000 7f69ef400640 1 -- 172.21.15.100:0/1325780371 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc0ab9d0 con 0x7f69f00d0a70 2024-04-23T14:35:35.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.807+0000 7f69f7800640 1 -- 172.21.15.100:0/1325780371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69f00d0a70 msgr2=0x7f69f00ca830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.807+0000 7f69f7800640 1 --2- 172.21.15.100:0/1325780371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69f00d0a70 0x7f69f00ca830 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f69f0071600 tx=0x7f69dc01c490 comp rx=0 tx=0).stop 2024-04-23T14:35:35.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.807+0000 7f69f7800640 1 -- 172.21.15.100:0/1325780371 shutdown_connections 2024-04-23T14:35:35.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.807+0000 7f69f7800640 1 --2- 172.21.15.100:0/1325780371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69f00d0a70 0x7f69f00ca830 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.807+0000 7f69f7800640 1 --2- 172.21.15.100:0/1325780371 >> v2:172.21.15.151:3300/0 conn(0x7f69f015ae60 0x7f69f00cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.807+0000 7f69f7800640 1 -- 172.21.15.100:0/1325780371 >> 172.21.15.100:0/1325780371 conn(0x7f69f00b5200 msgr2=0x7f69f00b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:35.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.807+0000 7f69f7800640 1 -- 172.21.15.100:0/1325780371 shutdown_connections 2024-04-23T14:35:35.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.807+0000 7f69f7800640 1 -- 172.21.15.100:0/1325780371 wait complete. 2024-04-23T14:35:35.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.807+0000 7f69f7800640 1 Processor -- start 2024-04-23T14:35:35.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.807+0000 7f69f7800640 1 -- start start 2024-04-23T14:35:35.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.807+0000 7f69f7800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69f006c9a0 0x7f69f014d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.807+0000 7f69f7800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f69f00d0a70 0x7f69f0156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.807+0000 7f69f7800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f015ae60 0x7f69f0157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.807+0000 7f69f7800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f69f00caef0 con 0x7f69f006c9a0 2024-04-23T14:35:35.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.807+0000 7f69f7800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f69f00cad70 con 0x7f69f015ae60 2024-04-23T14:35:35.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.807+0000 7f69f7800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f69f00cca20 con 0x7f69f00d0a70 2024-04-23T14:35:35.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.811+0000 7f69f4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69f006c9a0 0x7f69f014d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.811+0000 7f69f4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69f006c9a0 0x7f69f014d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59762/0 (socket says 172.21.15.100:59762) 2024-04-23T14:35:35.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.811+0000 7f69f4e00640 1 -- 172.21.15.100:0/1604120256 learned_addr learned my addr 172.21.15.100:0/1604120256 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:35.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.811+0000 7f69efe00640 1 --2- 172.21.15.100:0/1604120256 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f69f00d0a70 0x7f69f0156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.811+0000 7f69f5800640 1 --2- 172.21.15.100:0/1604120256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f015ae60 0x7f69f0157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.811+0000 7f69f4e00640 1 -- 172.21.15.100:0/1604120256 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f69f00d0a70 msgr2=0x7f69f0156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.811+0000 7f69f4e00640 1 --2- 172.21.15.100:0/1604120256 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f69f00d0a70 0x7f69f0156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.811+0000 7f69f4e00640 1 -- 172.21.15.100:0/1604120256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f015ae60 msgr2=0x7f69f0157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.811+0000 7f69f4e00640 1 --2- 172.21.15.100:0/1604120256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f015ae60 0x7f69f0157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.811+0000 7f69f4e00640 1 -- 172.21.15.100:0/1604120256 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f0155450 con 0x7f69f006c9a0 2024-04-23T14:35:35.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.811+0000 7f69efe00640 1 --2- 172.21.15.100:0/1604120256 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f69f00d0a70 0x7f69f0156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:35:35.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.811+0000 7f69f4e00640 1 --2- 172.21.15.100:0/1604120256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69f006c9a0 0x7f69f014d300 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f69e0007e70 tx=0x7f69e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:35.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.811+0000 7f69ecc00640 1 -- 172.21.15.100:0/1604120256 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e0014620 con 0x7f69f006c9a0 2024-04-23T14:35:35.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.811+0000 7f69ecc00640 1 -- 172.21.15.100:0/1604120256 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e0019070 con 0x7f69f006c9a0 2024-04-23T14:35:35.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.811+0000 7f69f7800640 1 -- 172.21.15.100:0/1604120256 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f0155730 con 0x7f69f006c9a0 2024-04-23T14:35:35.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.811+0000 7f69ecc00640 1 -- 172.21.15.100:0/1604120256 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e0014a00 con 0x7f69f006c9a0 2024-04-23T14:35:35.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.811+0000 7f69f7800640 1 -- 172.21.15.100:0/1604120256 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f0155cf0 con 0x7f69f006c9a0 2024-04-23T14:35:35.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.811+0000 7f69f7800640 1 -- 172.21.15.100:0/1604120256 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f0068ad0 con 0x7f69f006c9a0 2024-04-23T14:35:35.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.811+0000 7f69ecc00640 1 -- 172.21.15.100:0/1604120256 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f69e0014ba0 con 0x7f69f006c9a0 2024-04-23T14:35:35.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.815+0000 7f69ecc00640 1 --2- 172.21.15.100:0/1604120256 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f69d00683b0 0x7f69d006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.815+0000 7f69ecc00640 1 -- 172.21.15.100:0/1604120256 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f69e0087160 con 0x7f69f006c9a0 2024-04-23T14:35:35.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.815+0000 7f69efe00640 1 --2- 172.21.15.100:0/1604120256 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f69d00683b0 0x7f69d006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.815+0000 7f69efe00640 1 --2- 172.21.15.100:0/1604120256 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f69d00683b0 0x7f69d006a870 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0ad000 tx=0x7f69dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:35.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.819+0000 7f69ecc00640 1 -- 172.21.15.100:0/1604120256 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f69e0011260 con 0x7f69f006c9a0 2024-04-23T14:35:35.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.979+0000 7f69f7800640 1 -- 172.21.15.100:0/1604120256 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f69f00b0d30 con 0x7f69f006c9a0 2024-04-23T14:35:35.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.983+0000 7f69ecc00640 1 -- 172.21.15.100:0/1604120256 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) ==== 123+0+0 (secure 0 0 0) 0x7f69e00577f0 con 0x7f69f006c9a0 2024-04-23T14:35:35.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.983+0000 7f69ecc00640 1 -- 172.21.15.100:0/1604120256 <== mon.0 v2:172.21.15.100:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f69e00570f0 con 0x7f69f006c9a0 2024-04-23T14:35:35.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.983+0000 7f69f7800640 1 -- 172.21.15.100:0/1604120256 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f69d00683b0 msgr2=0x7f69d006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.983+0000 7f69f7800640 1 --2- 172.21.15.100:0/1604120256 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f69d00683b0 0x7f69d006a870 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0ad000 tx=0x7f69dc008040 comp rx=0 tx=0).stop 2024-04-23T14:35:35.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.983+0000 7f69f7800640 1 -- 172.21.15.100:0/1604120256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69f006c9a0 msgr2=0x7f69f014d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.983+0000 7f69f7800640 1 --2- 172.21.15.100:0/1604120256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69f006c9a0 0x7f69f014d300 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f69e0007e70 tx=0x7f69e000e4b0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.983+0000 7f69f7800640 1 -- 172.21.15.100:0/1604120256 shutdown_connections 2024-04-23T14:35:35.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.983+0000 7f69f7800640 1 --2- 172.21.15.100:0/1604120256 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f69d00683b0 0x7f69d006a870 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.983+0000 7f69f7800640 1 --2- 172.21.15.100:0/1604120256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f015ae60 0x7f69f0157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.983+0000 7f69f7800640 1 --2- 172.21.15.100:0/1604120256 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f69f00d0a70 0x7f69f0156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.983+0000 7f69f7800640 1 --2- 172.21.15.100:0/1604120256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69f006c9a0 0x7f69f014d300 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.983+0000 7f69f7800640 1 -- 172.21.15.100:0/1604120256 >> 172.21.15.100:0/1604120256 conn(0x7f69f00b5200 msgr2=0x7f69f00b1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:35.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.983+0000 7f69f7800640 1 -- 172.21.15.100:0/1604120256 shutdown_connections 2024-04-23T14:35:35.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:35.987+0000 7f69f7800640 1 -- 172.21.15.100:0/1604120256 wait complete. 2024-04-23T14:35:36.006 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-23T14:35:36.007 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:35:36.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.139+0000 7f481b800640 1 Processor -- start 2024-04-23T14:35:36.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.139+0000 7f481b800640 1 -- start start 2024-04-23T14:35:36.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f481b800640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f48140b2a10 0x7f48140adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f481b800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4814073610 0x7f481406df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f481b800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48140af080 con 0x7f481415b640 2024-04-23T14:35:36.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f481b800640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f48140afbc0 con 0x7f48140b2a10 2024-04-23T14:35:36.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f481b800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f481415ba20 con 0x7f4814073610 2024-04-23T14:35:36.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4818e00640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f481415b640 0x7f4814072f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46952/0 (socket says 172.21.15.100:46952) 2024-04-23T14:35:36.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4818e00640 1 -- 172.21.15.100:0/4286171555 learned_addr learned my addr 172.21.15.100:0/4286171555 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-23T14:35:36.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f480be00640 1 --2- 172.21.15.100:0/4286171555 >> v2:172.21.15.151:3300/0 conn(0x7f48140b2a10 0x7f48140adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4819800640 1 --2- 172.21.15.100:0/4286171555 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4814073610 0x7f481406df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4813e00640 1 -- 172.21.15.100:0/4286171555 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 234989076 0 0) 0x7f48140af080 con 0x7f481415b640 2024-04-23T14:35:36.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4813e00640 1 -- 172.21.15.100:0/4286171555 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47f8003530 con 0x7f481415b640 2024-04-23T14:35:36.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4813e00640 1 -- 172.21.15.100:0/4286171555 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4276890663 0 0) 0x7f47f8003530 con 0x7f481415b640 2024-04-23T14:35:36.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4813e00640 1 -- 172.21.15.100:0/4286171555 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4814073610 msgr2=0x7f481406df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4813e00640 1 --2- 172.21.15.100:0/4286171555 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4814073610 0x7f481406df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4813e00640 1 -- 172.21.15.100:0/4286171555 >> v2:172.21.15.151:3300/0 conn(0x7f48140b2a10 msgr2=0x7f48140adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4813e00640 1 --2- 172.21.15.100:0/4286171555 >> v2:172.21.15.151:3300/0 conn(0x7f48140b2a10 0x7f48140adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4813e00640 1 -- 172.21.15.100:0/4286171555 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48140af320 con 0x7f481415b640 2024-04-23T14:35:36.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4813e00640 1 -- 172.21.15.100:0/4286171555 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2752290266 0 0) 0x7f4804001680 con 0x7f481415b640 2024-04-23T14:35:36.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4813e00640 1 -- 172.21.15.100:0/4286171555 >> v1:172.21.15.100:6789/0 conn(0x7f481415b640 legacy=0x7f4814072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4813e00640 1 --2- 172.21.15.100:0/4286171555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47f80039c0 0x7f47f8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4813e00640 1 --2- 172.21.15.100:0/4286171555 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4814073610 0x7f47f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4813e00640 1 --2- 172.21.15.100:0/4286171555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47f8004d50 0x7f47f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4813e00640 1 -- 172.21.15.100:0/4286171555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f48140afbc0 con 0x7f47f80039c0 2024-04-23T14:35:36.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4813e00640 1 -- 172.21.15.100:0/4286171555 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f481415ba20 con 0x7f47f8004d50 2024-04-23T14:35:36.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4813e00640 1 -- 172.21.15.100:0/4286171555 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f47f800b9d0 con 0x7f4814073610 2024-04-23T14:35:36.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4818e00640 1 --2- 172.21.15.100:0/4286171555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47f80039c0 0x7f47f8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f480be00640 1 --2- 172.21.15.100:0/4286171555 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4814073610 0x7f47f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4819800640 1 --2- 172.21.15.100:0/4286171555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47f8004d50 0x7f47f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4818e00640 1 -- 172.21.15.100:0/4286171555 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4814073610 msgr2=0x7f47f8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4818e00640 1 --2- 172.21.15.100:0/4286171555 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4814073610 0x7f47f8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4818e00640 1 -- 172.21.15.100:0/4286171555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47f8004d50 msgr2=0x7f47f80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4818e00640 1 --2- 172.21.15.100:0/4286171555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47f8004d50 0x7f47f80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.143+0000 7f4818e00640 1 -- 172.21.15.100:0/4286171555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f8003530 con 0x7f47f80039c0 2024-04-23T14:35:36.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.147+0000 7f4818e00640 1 --2- 172.21.15.100:0/4286171555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47f80039c0 0x7f47f8003de0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f480400c2f0 tx=0x7f480401cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d368bdc5b57a973 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:36.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.147+0000 7f4813e00640 1 -- 172.21.15.100:0/4286171555 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804007d60 con 0x7f47f80039c0 2024-04-23T14:35:36.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.147+0000 7f4813e00640 1 -- 172.21.15.100:0/4286171555 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f48040a9020 con 0x7f47f80039c0 2024-04-23T14:35:36.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.147+0000 7f4813e00640 1 -- 172.21.15.100:0/4286171555 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804005b00 con 0x7f47f80039c0 2024-04-23T14:35:36.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.147+0000 7f481b800640 1 -- 172.21.15.100:0/4286171555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47f80039c0 msgr2=0x7f47f8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.147+0000 7f481b800640 1 --2- 172.21.15.100:0/4286171555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47f80039c0 0x7f47f8003de0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f480400c2f0 tx=0x7f480401cd10 comp rx=0 tx=0).stop 2024-04-23T14:35:36.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.147+0000 7f4819800640 1 -- 172.21.15.100:0/4286171555 reap_dead start 2024-04-23T14:35:36.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.147+0000 7f481b800640 1 -- 172.21.15.100:0/4286171555 shutdown_connections 2024-04-23T14:35:36.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.147+0000 7f481b800640 1 -- 172.21.15.100:0/4286171555 >> 172.21.15.100:0/4286171555 conn(0x7f48140b1e80 msgr2=0x7f48140b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:36.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.147+0000 7f481b800640 1 -- 172.21.15.100:0/4286171555 shutdown_connections 2024-04-23T14:35:36.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.147+0000 7f481b800640 1 -- 172.21.15.100:0/4286171555 wait complete. 2024-04-23T14:35:36.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f481b800640 1 Processor -- start 2024-04-23T14:35:36.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f481b800640 1 -- start start 2024-04-23T14:35:36.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f481b800640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f47f80039c0 0x7f481414b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f481b800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4814099ac0 0x7f48141f94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f481b800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4814157840 0x7f4814157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f481b800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f481406eb50 con 0x7f4814157840 2024-04-23T14:35:36.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f481b800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f481406e9d0 con 0x7f4814099ac0 2024-04-23T14:35:36.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f481b800640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f481406ecd0 con 0x7f47f80039c0 2024-04-23T14:35:36.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f4819800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4814157840 0x7f4814157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f4819800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4814157840 0x7f4814157cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59784/0 (socket says 172.21.15.100:59784) 2024-04-23T14:35:36.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f4819800640 1 -- 172.21.15.100:0/3815091236 learned_addr learned my addr 172.21.15.100:0/3815091236 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-23T14:35:36.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f480be00640 1 --2- 172.21.15.100:0/3815091236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4814099ac0 0x7f48141f94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f4818e00640 1 --2- 172.21.15.100:0/3815091236 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f47f80039c0 0x7f481414b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f4819800640 1 -- 172.21.15.100:0/3815091236 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f47f80039c0 msgr2=0x7f481414b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f4819800640 1 --2- 172.21.15.100:0/3815091236 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f47f80039c0 0x7f481414b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f4819800640 1 -- 172.21.15.100:0/3815091236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4814099ac0 msgr2=0x7f48141f94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f4819800640 1 --2- 172.21.15.100:0/3815091236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4814099ac0 0x7f48141f94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f4819800640 1 -- 172.21.15.100:0/3815091236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481415de70 con 0x7f4814157840 2024-04-23T14:35:36.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f4819800640 1 --2- 172.21.15.100:0/3815091236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4814157840 0x7f4814157cc0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f480c011b40 tx=0x7f480c011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:36.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f4811600640 1 -- 172.21.15.100:0/3815091236 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c00fc40 con 0x7f4814157840 2024-04-23T14:35:36.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f4811600640 1 -- 172.21.15.100:0/3815091236 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f480c00fde0 con 0x7f4814157840 2024-04-23T14:35:36.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f481b800640 1 -- 172.21.15.100:0/3815091236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481415e150 con 0x7f4814157840 2024-04-23T14:35:36.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f4811600640 1 -- 172.21.15.100:0/3815091236 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c015690 con 0x7f4814157840 2024-04-23T14:35:36.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f481b800640 1 -- 172.21.15.100:0/3815091236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f481415d8f0 con 0x7f4814157840 2024-04-23T14:35:36.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.151+0000 7f481b800640 1 -- 172.21.15.100:0/3815091236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4814068ad0 con 0x7f4814157840 2024-04-23T14:35:36.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.155+0000 7f4811600640 1 -- 172.21.15.100:0/3815091236 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f480c015830 con 0x7f4814157840 2024-04-23T14:35:36.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.155+0000 7f4811600640 1 --2- 172.21.15.100:0/3815091236 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f47f0068380 0x7f47f006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.155+0000 7f4811600640 1 -- 172.21.15.100:0/3815091236 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f480c0874e0 con 0x7f4814157840 2024-04-23T14:35:36.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.155+0000 7f4818e00640 1 --2- 172.21.15.100:0/3815091236 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f47f0068380 0x7f47f006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.155+0000 7f4818e00640 1 --2- 172.21.15.100:0/3815091236 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f47f0068380 0x7f47f006a840 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f48040aa000 tx=0x7f4804001a50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.159+0000 7f4811600640 1 -- 172.21.15.100:0/3815091236 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f480c019030 con 0x7f4814157840 2024-04-23T14:35:36.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.355+0000 7f481b800640 1 -- 172.21.15.100:0/3815091236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48140d0e10 con 0x7f4814157840 2024-04-23T14:35:36.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.355+0000 7f4811600640 1 -- 172.21.15.100:0/3815091236 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+3881 (secure 0 0 0) 0x7f480c010040 con 0x7f4814157840 2024-04-23T14:35:36.355 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-23T14:35:36.355 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":30,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5528,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2774610498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2774610498},{"type":"v1","addr":"172.21.15.151:6833","nonce":2774610498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5929,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1347883282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1347883282},{"type":"v1","addr":"172.21.15.151:6835","nonce":1347883282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":6177,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3319827520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3319827520},{"type":"v1","addr":"172.21.15.100:6836","nonce":3319827520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":6669,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2255757215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2255757215},{"type":"v1","addr":"172.21.15.100:6837","nonce":2255757215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[]} 2024-04-23T14:35:36.355 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 30 2024-04-23T14:35:36.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.355+0000 7f481b800640 1 -- 172.21.15.100:0/3815091236 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f47f0068380 msgr2=0x7f47f006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.355+0000 7f481b800640 1 --2- 172.21.15.100:0/3815091236 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f47f0068380 0x7f47f006a840 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f48040aa000 tx=0x7f4804001a50 comp rx=0 tx=0).stop 2024-04-23T14:35:36.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.355+0000 7f481b800640 1 -- 172.21.15.100:0/3815091236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4814157840 msgr2=0x7f4814157cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.355+0000 7f481b800640 1 --2- 172.21.15.100:0/3815091236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4814157840 0x7f4814157cc0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f480c011b40 tx=0x7f480c011f40 comp rx=0 tx=0).stop 2024-04-23T14:35:36.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.355+0000 7f481b800640 1 -- 172.21.15.100:0/3815091236 shutdown_connections 2024-04-23T14:35:36.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.355+0000 7f481b800640 1 --2- 172.21.15.100:0/3815091236 >> [v2:172.21.15.100:6832/2859853957,v1:172.21.15.100:6833/2859853957] conn(0x7f47f0068380 0x7f47f006a840 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.355+0000 7f481b800640 1 --2- 172.21.15.100:0/3815091236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4814157840 0x7f4814157cc0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.355+0000 7f481b800640 1 --2- 172.21.15.100:0/3815091236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4814099ac0 0x7f48141f94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.355+0000 7f481b800640 1 --2- 172.21.15.100:0/3815091236 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f47f80039c0 0x7f481414b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.355+0000 7f481b800640 1 -- 172.21.15.100:0/3815091236 >> 172.21.15.100:0/3815091236 conn(0x7f48140b1e80 msgr2=0x7f48140b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:36.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.355+0000 7f481b800640 1 -- 172.21.15.100:0/3815091236 shutdown_connections 2024-04-23T14:35:36.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-23T14:35:36.355+0000 7f481b800640 1 -- 172.21.15.100:0/3815091236 wait complete. 2024-04-23T14:35:36.375 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-23T14:35:36.375 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-23T14:35:36.376 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-23T14:35:36.376 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:35:36.388 INFO:tasks.ceph.mds.a:Stopped 2024-04-23T14:35:36.388 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-23T14:35:36.388 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:35:36.442 INFO:tasks.ceph.mds.c:Stopped 2024-04-23T14:35:36.442 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-23T14:35:36.442 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:35:36.456 INFO:tasks.ceph.mds.b:Stopped 2024-04-23T14:35:36.456 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-23T14:35:36.456 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:35:36.513 INFO:tasks.ceph.mds.d:Stopped 2024-04-23T14:35:36.513 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-23T14:35:36.513 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-23T14:35:36.514 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:35:36.568 INFO:tasks.ceph.osd.0:Stopped 2024-04-23T14:35:36.568 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-23T14:35:36.568 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:35:38.048 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:35:38.052 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:35:38.465 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-04-23T14:35:42.670 INFO:tasks.ceph.osd.1:Stopped 2024-04-23T14:35:42.671 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-23T14:35:42.671 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:35:42.738 INFO:tasks.ceph.osd.2:Stopped 2024-04-23T14:35:42.739 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-23T14:35:42.739 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:35:44.372 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-04-23T14:35:44.372 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-04-23T14:35:48.841 INFO:tasks.ceph.osd.3:Stopped 2024-04-23T14:35:48.841 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-23T14:35:48.842 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:35:48.904 INFO:tasks.ceph.osd.4:Stopped 2024-04-23T14:35:48.904 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-23T14:35:48.904 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:35:50.078 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-04-23T14:35:50.078 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-04-23T14:35:55.006 INFO:tasks.ceph.osd.5:Stopped 2024-04-23T14:35:55.007 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-23T14:35:55.007 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:35:55.084 INFO:tasks.ceph.osd.6:Stopped 2024-04-23T14:35:55.085 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-23T14:35:55.085 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:35:55.173 INFO:tasks.ceph.osd.7:Stopped 2024-04-23T14:35:55.173 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-23T14:35:55.173 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-23T14:35:55.173 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:35:55.232 INFO:tasks.ceph.mgr.y:Stopped 2024-04-23T14:35:55.232 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-23T14:35:55.232 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:35:55.287 INFO:tasks.ceph.mgr.x:Stopped 2024-04-23T14:35:55.287 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-23T14:35:55.287 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-23T14:35:55.287 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:35:55.306 INFO:tasks.ceph.mon.a:Stopped 2024-04-23T14:35:55.307 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-23T14:35:55.307 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:35:55.323 INFO:tasks.ceph.mon.b:Stopped 2024-04-23T14:35:55.323 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-23T14:35:55.324 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:35:55.388 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-04-23T14:35:55.388 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-04-23T14:35:55.388 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-04-23T14:35:55.388 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-04-23T14:35:55.388 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-04-23T14:35:55.388 INFO:tasks.ceph.mon.c:Stopped 2024-04-23T14:35:55.389 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-23T14:35:55.389 DEBUG:teuthology.orchestra.run.smithi100:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-23T14:35:55.423 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:35:55.423 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-23T14:35:55.747 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:35:55.748 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-23T14:35:55.976 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:35:55.976 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-23T14:35:56.180 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:35:56.180 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-23T14:35:56.364 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi151.front.sepia.ceph.com 2024-04-23T14:35:56.364 DEBUG:teuthology.orchestra.run.smithi151:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-23T14:35:56.609 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi151.front.sepia.ceph.com 2024-04-23T14:35:56.609 DEBUG:teuthology.orchestra.run.smithi151:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-23T14:35:56.782 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi151.front.sepia.ceph.com 2024-04-23T14:35:56.782 DEBUG:teuthology.orchestra.run.smithi151:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-23T14:35:56.941 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi151.front.sepia.ceph.com 2024-04-23T14:35:56.941 DEBUG:teuthology.orchestra.run.smithi151:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-23T14:35:57.090 INFO:tasks.ceph:Archiving mon data... 2024-04-23T14:35:57.091 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669587/data/mon.a.tgz 2024-04-23T14:35:57.091 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-04-23T14:35:57.100 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.epvTqeI05i 2024-04-23T14:35:57.101 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.epvTqeI05i 2024-04-23T14:35:57.234 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.epvTqeI05i 2024-04-23T14:35:57.313 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.epvTqeI05i is 367KB 2024-04-23T14:35:57.376 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.epvTqeI05i 2024-04-23T14:35:57.384 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669587/data/mon.b.tgz 2024-04-23T14:35:57.384 DEBUG:teuthology.orchestra.run.smithi151:> mktemp 2024-04-23T14:35:57.393 INFO:teuthology.orchestra.run.smithi151.stdout:/tmp/tmp.Li06uSZ0Qm 2024-04-23T14:35:57.393 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Li06uSZ0Qm 2024-04-23T14:35:57.548 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 0666 /tmp/tmp.Li06uSZ0Qm 2024-04-23T14:35:57.623 DEBUG:teuthology.orchestra.remote:smithi151:/tmp/tmp.Li06uSZ0Qm is 482KB 2024-04-23T14:35:57.685 DEBUG:teuthology.orchestra.run.smithi151:> rm -fr /tmp/tmp.Li06uSZ0Qm 2024-04-23T14:35:57.693 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669587/data/mon.c.tgz 2024-04-23T14:35:57.693 DEBUG:teuthology.orchestra.run.smithi151:> mktemp 2024-04-23T14:35:57.742 INFO:teuthology.orchestra.run.smithi151.stdout:/tmp/tmp.0fiNQfko3M 2024-04-23T14:35:57.742 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.0fiNQfko3M 2024-04-23T14:35:57.894 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 0666 /tmp/tmp.0fiNQfko3M 2024-04-23T14:35:57.967 DEBUG:teuthology.orchestra.remote:smithi151:/tmp/tmp.0fiNQfko3M is 482KB 2024-04-23T14:35:58.029 DEBUG:teuthology.orchestra.run.smithi151:> rm -fr /tmp/tmp.0fiNQfko3M 2024-04-23T14:35:58.037 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-23T14:35:58.038 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-23T14:35:58.041 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-23T14:35:58.109 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-23T14:35:58.146 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-23T14:35:58.147 INFO:tasks.ceph:Archiving crash dumps... 2024-04-23T14:35:58.148 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669587/remote/smithi100/crash 2024-04-23T14:35:58.149 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-23T14:35:58.173 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669587/remote/smithi151/crash 2024-04-23T14:35:58.174 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-23T14:35:58.205 INFO:tasks.ceph:Shutting down logrotate 2024-04-23T14:35:58.205 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:35:58.224 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:35:58.296 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-23T14:35:58.317 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-23T14:35:58.335 INFO:tasks.ceph:Compressing logs... 2024-04-23T14:35:58.336 DEBUG:teuthology.orchestra.run.smithi100:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T14:35:58.364 DEBUG:teuthology.orchestra.run.smithi151:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T14:35:58.395 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39153.log 2024-04-23T14:35:58.396 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41296.log 2024-04-23T14:35:58.397 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39870.log 2024-04-23T14:35:58.398 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39153.log.gz 2024-04-23T14:35:58.398 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40374.log 2024-04-23T14:35:58.399 INFO:teuthology.orchestra.run.smithi100.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41296.log.gz 2024-04-23T14:35:58.399 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.39870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37678.log 2024-04-23T14:35:58.400 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39359.log 2024-04-23T14:35:58.400 INFO:teuthology.orchestra.run.smithi100.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.40374.log.gz 2024-04-23T14:35:58.400 INFO:teuthology.orchestra.run.smithi100.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.39870.log.gz 2024-04-23T14:35:58.401 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36334.log 2024-04-23T14:35:58.401 INFO:teuthology.orchestra.run.smithi100.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37678.log.gz 2024-04-23T14:35:58.402 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41452.log 2024-04-23T14:35:58.402 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39359.log.gz 2024-04-23T14:35:58.402 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36334.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-23T14:35:58.403 INFO:teuthology.orchestra.run.smithi100.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36334.log.gz 2024-04-23T14:35:58.403 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42207.log 2024-04-23T14:35:58.404 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41452.log.gz 2024-04-23T14:35:58.404 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33215.log 2024-04-23T14:35:58.405 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40557.log 2024-04-23T14:35:58.405 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42207.log.gz 2024-04-23T14:35:58.406 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33444.log 2024-04-23T14:35:58.406 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33215.log.gz 2024-04-23T14:35:58.406 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41426.log 2024-04-23T14:35:58.407 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40557.log.gz 2024-04-23T14:35:58.407 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34552.log 2024-04-23T14:35:58.407 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33444.log.gz 2024-04-23T14:35:58.408 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36094.log 2024-04-23T14:35:58.409 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41426.log.gz 2024-04-23T14:35:58.409 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34026.log 2024-04-23T14:35:58.410 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34552.log.gz 2024-04-23T14:35:58.411 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38385.log 2024-04-23T14:35:58.411 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36094.log.gz 2024-04-23T14:35:58.411 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40052.log 2024-04-23T14:35:58.411 INFO:teuthology.orchestra.run.smithi100.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34026.log.gz 2024-04-23T14:35:58.418 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33341.log 2024-04-23T14:35:58.419 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27697.log 2024-04-23T14:35:58.419 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.3.33341.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-23T14:35:58.419 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.27697.log: 91.6% -- replaced with /var/log/ceph/ceph-client.3.33341.log.gzgzip 2024-04-23T14:35:58.419 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-23T14:35:58.419 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27697.log.gz 2024-04-23T14:35:58.419 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33222.log 2024-04-23T14:35:58.420 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27769.log 2024-04-23T14:35:58.420 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.1.33222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27793.log 2024-04-23T14:35:58.420 INFO:teuthology.orchestra.run.smithi151.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.1.33222.log.gz 2024-04-23T14:35:58.420 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.27769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27673.log 2024-04-23T14:35:58.420 INFO:teuthology.orchestra.run.smithi151.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27769.log.gz 2024-04-23T14:35:58.420 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.27793.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-23T14:35:58.420 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27793.log.gz 2024-04-23T14:35:58.420 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.27673.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-23T14:35:58.421 INFO:teuthology.orchestra.run.smithi151.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27673.log.gz 2024-04-23T14:35:58.421 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27721.log 2024-04-23T14:35:58.427 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35334.log 2024-04-23T14:35:58.427 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38385.log.gz 2024-04-23T14:35:58.427 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37415.log 2024-04-23T14:35:58.428 INFO:teuthology.orchestra.run.smithi100.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.40052.log.gz 2024-04-23T14:35:58.428 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37787.log 2024-04-23T14:35:58.428 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35334.log.gz 2024-04-23T14:35:58.428 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37363.log 2024-04-23T14:35:58.428 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37415.log.gz 2024-04-23T14:35:58.428 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36516.log 2024-04-23T14:35:58.428 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37787.log.gz 2024-04-23T14:35:58.429 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42451.log 2024-04-23T14:35:58.429 INFO:teuthology.orchestra.run.smithi100.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37363.log.gz 2024-04-23T14:35:58.429 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40991.log 2024-04-23T14:35:58.429 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36516.log.gz 2024-04-23T14:35:58.429 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41886.log 2024-04-23T14:35:58.429 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42451.log.gz 2024-04-23T14:35:58.429 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36941.log 2024-04-23T14:35:58.430 INFO:teuthology.orchestra.run.smithi100.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40991.log.gz 2024-04-23T14:35:58.430 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43847.log 2024-04-23T14:35:58.430 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36941.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.41886.log.gz 2024-04-23T14:35:58.430 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40322.log 2024-04-23T14:35:58.430 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36941.log.gz 2024-04-23T14:35:58.430 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42582.log 2024-04-23T14:35:58.430 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43847.log.gz 2024-04-23T14:35:58.431 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.40322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33313.log 2024-04-23T14:35:58.431 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36568.log 2024-04-23T14:35:58.431 INFO:teuthology.orchestra.run.smithi100.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42582.log.gz 2024-04-23T14:35:58.431 INFO:teuthology.orchestra.run.smithi100.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.40322.log.gz 2024-04-23T14:35:58.431 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33313.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-23T14:35:58.431 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33313.log.gz 2024-04-23T14:35:58.431 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38665.log 2024-04-23T14:35:58.432 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36568.log.gz 2024-04-23T14:35:58.432 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43433.log 2024-04-23T14:35:58.432 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42180.log 2024-04-23T14:35:58.432 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38665.log.gz 2024-04-23T14:35:58.435 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-23T14:35:58.435 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.27721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27745.log 2024-04-23T14:35:58.435 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.27721.log.gz 2024-04-23T14:35:58.435 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-23T14:35:58.436 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.27745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27649.log 2024-04-23T14:35:58.436 INFO:teuthology.orchestra.run.smithi151.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27745.log.gz 92.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-23T14:35:58.436 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-04-23T14:35:58.436 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-mon.c.log: 93.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-23T14:35:58.436 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-23T14:35:58.436 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.27649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27817.log 2024-04-23T14:35:58.436 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.27649.log.gz 2024-04-23T14:35:58.437 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-23T14:35:58.437 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.27817.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-23T14:35:58.437 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27817.log.gz 2024-04-23T14:35:58.437 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-23T14:35:58.437 INFO:teuthology.orchestra.run.smithi151.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-23T14:35:58.437 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-23T14:35:58.437 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-mon.b.log: 92.1% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-23T14:35:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43433.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.43433.log.gz 2024-04-23T14:35:58.443 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39048.log 2024-04-23T14:35:58.444 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33392.log 2024-04-23T14:35:58.444 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42180.log.gz 2024-04-23T14:35:58.444 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36915.log 2024-04-23T14:35:58.444 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39048.log.gz 2024-04-23T14:35:58.444 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42311.log 2024-04-23T14:35:58.444 INFO:teuthology.orchestra.run.smithi100.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33392.log.gz 2024-04-23T14:35:58.444 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44079.log 2024-04-23T14:35:58.445 INFO:teuthology.orchestra.run.smithi100.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36915.log.gz 2024-04-23T14:35:58.445 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42233.log 2024-04-23T14:35:58.445 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42311.log.gz 2024-04-23T14:35:58.445 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36360.log 2024-04-23T14:35:58.445 INFO:teuthology.orchestra.run.smithi100.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44079.log.gz 2024-04-23T14:35:58.445 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37730.log 2024-04-23T14:35:58.445 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42233.log.gz 2024-04-23T14:35:58.445 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35543.log 2024-04-23T14:35:58.446 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36360.log.gz 2024-04-23T14:35:58.446 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38612.log 2024-04-23T14:35:58.446 INFO:teuthology.orchestra.run.smithi100.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37730.log.gz 2024-04-23T14:35:58.446 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35543.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35543.log.gz 2024-04-23T14:35:58.446 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40131.log 2024-04-23T14:35:58.446 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40939.log 2024-04-23T14:35:58.446 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38612.log.gz 2024-04-23T14:35:58.447 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44029.log 2024-04-23T14:35:58.447 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40131.log.gz 2024-04-23T14:35:58.447 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34604.log 2024-04-23T14:35:58.447 INFO:teuthology.orchestra.run.smithi100.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40939.log.gz 2024-04-23T14:35:58.447 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34859.log 2024-04-23T14:35:58.447 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44029.log.gz 2024-04-23T14:35:58.447 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39205.log 2024-04-23T14:35:58.447 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34604.log.gz 2024-04-23T14:35:58.448 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40105.log 2024-04-23T14:35:58.448 INFO:teuthology.orchestra.run.smithi100.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34859.log.gz 2024-04-23T14:35:58.451 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39469.log 2024-04-23T14:35:58.451 INFO:teuthology.orchestra.run.smithi100.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.39205.log.gz 2024-04-23T14:35:58.451 INFO:teuthology.orchestra.run.smithi100.stderr:gzip/var/log/ceph/ceph-client.admin.40105.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41270.log 2024-04-23T14:35:58.451 INFO:teuthology.orchestra.run.smithi100.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40105.log.gz 2024-04-23T14:35:58.451 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42259.log 2024-04-23T14:35:58.451 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39469.log.gz 2024-04-23T14:35:58.451 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41244.log 2024-04-23T14:35:58.452 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41270.log.gz 2024-04-23T14:35:58.452 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41043.log 2024-04-23T14:35:58.452 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42259.log.gz 2024-04-23T14:35:58.452 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37995.log 2024-04-23T14:35:58.452 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41244.log.gz 2024-04-23T14:35:58.452 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41043.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.41043.log.gz 2024-04-23T14:35:58.452 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36308.log 2024-04-23T14:35:58.455 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40783.log 2024-04-23T14:35:58.455 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37995.log.gz 2024-04-23T14:35:58.455 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36994.log 2024-04-23T14:35:58.455 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36308.log.gz 2024-04-23T14:35:58.455 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42936.log 2024-04-23T14:35:58.455 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40783.log.gz 2024-04-23T14:35:58.455 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41218.log 2024-04-23T14:35:58.456 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36994.log.gz 2024-04-23T14:35:58.457 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35360.log 2024-04-23T14:35:58.457 INFO:teuthology.orchestra.run.smithi100.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42936.log.gz 2024-04-23T14:35:58.458 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37072.log 2024-04-23T14:35:58.458 INFO:teuthology.orchestra.run.smithi100.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41218.log.gz 2024-04-23T14:35:58.458 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37020.log 2024-04-23T14:35:58.458 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35360.log.gz 2024-04-23T14:35:58.459 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37072.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37072.log.gz 2024-04-23T14:35:58.459 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34868.log 2024-04-23T14:35:58.460 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39922.log 2024-04-23T14:35:58.460 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37020.log.gz 2024-04-23T14:35:58.461 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34868.log: 91.8% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-23T14:35:58.461 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40887.log 2024-04-23T14:35:58.461 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34868.log.gz 2024-04-23T14:35:58.462 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37046.log 2024-04-23T14:35:58.462 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39922.log.gz 2024-04-23T14:35:58.463 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38821.log 2024-04-23T14:35:58.463 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40887.log.gz 2024-04-23T14:35:58.464 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35069.log 2024-04-23T14:35:58.464 INFO:teuthology.orchestra.run.smithi100.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37046.log.gz 2024-04-23T14:35:58.464 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35412.log 2024-04-23T14:35:58.464 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38821.log.gz 2024-04-23T14:35:58.466 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40478.log 2024-04-23T14:35:58.466 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35069.log.gz 2024-04-23T14:35:58.466 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35256.log 2024-04-23T14:35:58.466 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35412.log.gz 2024-04-23T14:35:58.466 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43925.log 2024-04-23T14:35:58.467 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40478.log.gz 2024-04-23T14:35:58.467 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-osd.4.log: 92.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-23T14:35:58.469 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43717.log 2024-04-23T14:35:58.469 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35256.log.gz 2024-04-23T14:35:58.469 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43206.log 2024-04-23T14:35:58.470 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43925.log.gz 2024-04-23T14:35:58.470 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36490.log 2024-04-23T14:35:58.470 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43717.log.gz 2024-04-23T14:35:58.470 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36256.log 2024-04-23T14:35:58.472 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43206.log.gz 2024-04-23T14:35:58.472 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36412.log 2024-04-23T14:35:58.472 INFO:teuthology.orchestra.run.smithi100.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36490.log.gz 2024-04-23T14:35:58.472 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36256.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-23T14:35:58.472 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36256.log.gz 2024-04-23T14:35:58.473 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41192.log 2024-04-23T14:35:58.473 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36412.log.gz 2024-04-23T14:35:58.474 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33634.log 2024-04-23T14:35:58.474 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40266.log 2024-04-23T14:35:58.475 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41192.log.gz 2024-04-23T14:35:58.475 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33470.log 2024-04-23T14:35:58.475 INFO:teuthology.orchestra.run.smithi100.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.33634.log.gz 2024-04-23T14:35:58.476 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38638.log 2024-04-23T14:35:58.477 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40266.log.gz 2024-04-23T14:35:58.477 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37652.log 2024-04-23T14:35:58.477 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33470.log.gz 2024-04-23T14:35:58.478 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43509.log 2024-04-23T14:35:58.478 INFO:teuthology.orchestra.run.smithi100.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38638.log.gz 2024-04-23T14:35:58.478 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34643.log 2024-04-23T14:35:58.479 INFO:teuthology.orchestra.run.smithi100.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37652.log.gz 2024-04-23T14:35:58.479 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43509.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43509.log.gz 2024-04-23T14:35:58.479 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36698.log 2024-04-23T14:35:58.481 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34526.log 2024-04-23T14:35:58.481 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34643.log.gz 2024-04-23T14:35:58.481 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43310.log 2024-04-23T14:35:58.481 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36698.log.gz 2024-04-23T14:35:58.483 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37181.log 2024-04-23T14:35:58.483 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34526.log.gz 2024-04-23T14:35:58.483 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36861.log 2024-04-23T14:35:58.483 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43310.log.gz 2024-04-23T14:35:58.484 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34261.log 2024-04-23T14:35:58.485 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37181.log.gz 2024-04-23T14:35:58.485 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34653.log 2024-04-23T14:35:58.485 INFO:teuthology.orchestra.run.smithi100.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36861.log.gz 2024-04-23T14:35:58.486 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43665.log 2024-04-23T14:35:58.486 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34261.log.gz 2024-04-23T14:35:58.487 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39757.log 2024-04-23T14:35:58.487 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34653.log.gz 2024-04-23T14:35:58.488 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34448.log 2024-04-23T14:35:58.488 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43665.log.gz 2024-04-23T14:35:58.488 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38586.log 2024-04-23T14:35:58.488 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39757.log.gz 2024-04-23T14:35:58.490 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34235.log 2024-04-23T14:35:58.490 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34448.log.gz 2024-04-23T14:35:58.490 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38586.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38586.log.gz 2024-04-23T14:35:58.490 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-23T14:35:58.491 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41782.log 2024-04-23T14:35:58.492 INFO:teuthology.orchestra.run.smithi100.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34235.log.gz 2024-04-23T14:35:58.492 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39439.log 2024-04-23T14:35:58.493 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41782.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41782.log.gz 2024-04-23T14:35:58.493 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35699.log 2024-04-23T14:35:58.494 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36776.log 2024-04-23T14:35:58.494 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35699.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39439.log.gz 2024-04-23T14:35:58.495 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39731.log 2024-04-23T14:35:58.496 INFO:teuthology.orchestra.run.smithi100.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.35699.log.gz 2024-04-23T14:35:58.496 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39311.log 2024-04-23T14:35:58.496 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36776.log.gz 2024-04-23T14:35:58.497 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37626.log 2024-04-23T14:35:58.497 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39731.log.gz 2024-04-23T14:35:58.498 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.39311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35760.log 2024-04-23T14:35:58.499 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36967.log 2024-04-23T14:35:58.499 INFO:teuthology.orchestra.run.smithi100.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37626.log.gz 2024-04-23T14:35:58.500 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35760.log: 92.1% -- replaced with /var/log/ceph/ceph-client.0.39311.log.gz 2024-04-23T14:35:58.500 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42504.log 2024-04-23T14:35:58.500 INFO:teuthology.orchestra.run.smithi100.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35760.log.gz 2024-04-23T14:35:58.501 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34648.log 2024-04-23T14:35:58.501 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36967.log.gz 2024-04-23T14:35:58.502 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38047.log 2024-04-23T14:35:58.502 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42504.log.gz 2024-04-23T14:35:58.503 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43154.log 2024-04-23T14:35:58.504 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34648.log: /var/log/ceph/ceph-client.admin.38047.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34648.log.gz 2024-04-23T14:35:58.504 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40209.log 2024-04-23T14:35:58.504 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38047.log.gz 2024-04-23T14:35:58.505 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33790.log 2024-04-23T14:35:58.505 INFO:teuthology.orchestra.run.smithi100.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.43154.log.gz 2024-04-23T14:35:58.506 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39413.log 2024-04-23T14:35:58.506 INFO:teuthology.orchestra.run.smithi100.stderr: 89.5%/var/log/ceph/ceph-client.admin.33790.log: -- replaced with /var/log/ceph/ceph-client.admin.40209.log.gz 2024-04-23T14:35:58.507 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34474.log 2024-04-23T14:35:58.508 INFO:teuthology.orchestra.run.smithi100.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.33790.log.gz 2024-04-23T14:35:58.508 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33712.log 2024-04-23T14:35:58.509 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39413.log.gz 2024-04-23T14:35:58.509 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35885.log 2024-04-23T14:35:58.509 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34474.log.gz 2024-04-23T14:35:58.510 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41559.log 2024-04-23T14:35:58.510 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33712.log.gz 2024-04-23T14:35:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.35885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41587.log 2024-04-23T14:35:58.511 INFO:teuthology.orchestra.run.smithi100.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.0.35885.log.gz 2024-04-23T14:35:58.512 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33148.log 2024-04-23T14:35:58.513 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41559.log.gz 2024-04-23T14:35:58.513 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42609.log 2024-04-23T14:35:58.513 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41587.log.gz 2024-04-23T14:35:58.514 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33148.log: gzip 89.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.33148.log.gz 2024-04-23T14:35:58.514 INFO:teuthology.orchestra.run.smithi100.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43821.log 2024-04-23T14:35:58.515 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36438.log 2024-04-23T14:35:58.516 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42609.log: /var/log/ceph/ceph-client.admin.43821.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42609.log.gz 2024-04-23T14:35:58.516 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36464.log 2024-04-23T14:35:58.516 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43821.log.gz 2024-04-23T14:35:58.517 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33764.log 2024-04-23T14:35:58.518 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36438.log.gz 2024-04-23T14:35:58.518 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42076.log 2024-04-23T14:35:58.519 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36464.log.gz 2024-04-23T14:35:58.519 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41505.log 2024-04-23T14:35:58.519 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33764.log.gz 2024-04-23T14:35:58.520 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41361.log 2024-04-23T14:35:58.521 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42076.log.gz 2024-04-23T14:35:58.521 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40026.log 2024-04-23T14:35:58.522 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41505.log: /var/log/ceph/ceph-client.admin.41361.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41505.log.gz 2024-04-23T14:35:58.523 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38992.log 2024-04-23T14:35:58.523 INFO:teuthology.orchestra.run.smithi100.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.41361.log.gz 2024-04-23T14:35:58.523 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40157.log 2024-04-23T14:35:58.524 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40026.log.gz 2024-04-23T14:35:58.524 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40809.log 2024-04-23T14:35:58.525 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38992.log: /var/log/ceph/ceph-client.admin.40157.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38992.log.gz 2024-04-23T14:35:58.526 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43407.log 2024-04-23T14:35:58.526 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40157.log.gz 2024-04-23T14:35:58.527 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40757.log 2024-04-23T14:35:58.527 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40809.log.gz 2024-04-23T14:35:58.527 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42530.log 2024-04-23T14:35:58.528 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40757.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43407.log.gz 2024-04-23T14:35:58.529 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39387.log 2024-04-23T14:35:58.529 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40757.log.gz 2024-04-23T14:35:58.530 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41531.log 2024-04-23T14:35:58.530 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42530.log.gz 2024-04-23T14:35:58.531 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41994.log 2024-04-23T14:35:58.531 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39387.log: /var/log/ceph/ceph-client.admin.41531.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39387.log.gz 2024-04-23T14:35:58.532 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35647.log 2024-04-23T14:35:58.532 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41531.log.gz 2024-04-23T14:35:58.532 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.41994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36724.log 2024-04-23T14:35:58.533 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43125.log 2024-04-23T14:35:58.534 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35647.log.gz 2024-04-23T14:35:58.534 INFO:teuthology.orchestra.run.smithi100.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.41994.log.gz 2024-04-23T14:35:58.535 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39548.log 2024-04-23T14:35:58.535 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36724.log.gz 2024-04-23T14:35:58.536 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40965.log 2024-04-23T14:35:58.536 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43125.log.gz 2024-04-23T14:35:58.536 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39548.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39548.log.gz 2024-04-23T14:35:58.537 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42401.log 2024-04-23T14:35:58.538 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39679.log 2024-04-23T14:35:58.538 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40965.log.gz 2024-04-23T14:35:58.539 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.42401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43535.log 2024-04-23T14:35:58.540 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40692.log 2024-04-23T14:35:58.540 INFO:teuthology.orchestra.run.smithi100.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.0.42401.log.gz 2024-04-23T14:35:58.540 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39679.log.gz 2024-04-23T14:35:58.541 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38795.log 2024-04-23T14:35:58.541 INFO:teuthology.orchestra.run.smithi100.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43535.log.gz 2024-04-23T14:35:58.542 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42478.log 2024-04-23T14:35:58.542 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40692.log: /var/log/ceph/ceph-client.admin.38795.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40692.log.gz 2024-04-23T14:35:58.543 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35076.log 2024-04-23T14:35:58.543 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38795.log.gz 2024-04-23T14:35:58.544 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36750.log 2024-04-23T14:35:58.544 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42478.log.gz 2024-04-23T14:35:58.545 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35076.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-23T14:35:58.545 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35076.log.gz 2024-04-23T14:35:58.546 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40861.log 2024-04-23T14:35:58.546 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36750.log.gz 2024-04-23T14:35:58.547 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43899.log 2024-04-23T14:35:58.548 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43691.log 2024-04-23T14:35:58.548 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40861.log.gz 2024-04-23T14:35:58.549 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40913.log 2024-04-23T14:35:58.549 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43899.log.gz 2024-04-23T14:35:58.550 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36835.log 2024-04-23T14:35:58.550 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43691.log.gz 2024-04-23T14:35:58.552 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33339.log 2024-04-23T14:35:58.552 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40913.log.gz 2024-04-23T14:35:58.552 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37865.log 2024-04-23T14:35:58.553 INFO:teuthology.orchestra.run.smithi100.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36835.log.gz 2024-04-23T14:35:58.554 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41329.log 2024-04-23T14:35:58.554 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33339.log.gz 2024-04-23T14:35:58.555 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35063.log 2024-04-23T14:35:58.555 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37865.log.gz 2024-04-23T14:35:58.556 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36230.log 2024-04-23T14:35:58.556 INFO:teuthology.orchestra.run.smithi100.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41329.log.gz 2024-04-23T14:35:58.557 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35071.log 2024-04-23T14:35:58.557 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35063.log.gz 2024-04-23T14:35:58.558 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42661.log 2024-04-23T14:35:58.558 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36230.log.gz 2024-04-23T14:35:58.559 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35308.log 2024-04-23T14:35:58.560 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35071.log.gz 2024-04-23T14:35:58.560 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36152.log 2024-04-23T14:35:58.561 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42661.log.gz 2024-04-23T14:35:58.561 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36672.log 2024-04-23T14:35:58.562 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35308.log.gz 2024-04-23T14:35:58.562 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34104.log 2024-04-23T14:35:58.563 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36152.log.gz 2024-04-23T14:35:58.564 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36620.log 2024-04-23T14:35:58.564 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36672.log.gz 2024-04-23T14:35:58.565 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43021.log 2024-04-23T14:35:58.565 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34104.log.gz 2024-04-23T14:35:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42884.log 2024-04-23T14:35:58.566 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36620.log.gz 2024-04-23T14:35:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42635.log 2024-04-23T14:35:58.567 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43021.log.gz 2024-04-23T14:35:58.568 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36594.log 2024-04-23T14:35:58.568 INFO:teuthology.orchestra.run.smithi100.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42884.log.gz 2024-04-23T14:35:58.569 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35490.log 2024-04-23T14:35:58.569 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42635.log.gz 2024-04-23T14:35:58.570 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35464.log 2024-04-23T14:35:58.570 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36594.log.gz 2024-04-23T14:35:58.571 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40400.log 2024-04-23T14:35:58.571 INFO:teuthology.orchestra.run.smithi100.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35490.log.gz 2024-04-23T14:35:58.572 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37704.log 2024-04-23T14:35:58.573 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35464.log.gz 2024-04-23T14:35:58.574 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35569.log 2024-04-23T14:35:58.574 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40400.log.gz 2024-04-23T14:35:58.574 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43639.log 2024-04-23T14:35:58.575 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37704.log.gz 2024-04-23T14:35:58.576 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41140.log 2024-04-23T14:35:58.576 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35569.log.gz 2024-04-23T14:35:58.577 INFO:teuthology.orchestra.run.smithi151.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-23T14:35:58.577 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35386.log 2024-04-23T14:35:58.577 INFO:teuthology.orchestra.run.smithi100.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43639.log.gz 2024-04-23T14:35:58.578 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41730.log 2024-04-23T14:35:58.578 INFO:teuthology.orchestra.run.smithi100.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41140.log.gz 2024-04-23T14:35:58.579 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34850.log 2024-04-23T14:35:58.579 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35386.log.gz 2024-04-23T14:35:58.580 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39948.log 2024-04-23T14:35:58.580 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41730.log.gz 2024-04-23T14:35:58.581 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39074.log 2024-04-23T14:35:58.581 INFO:teuthology.orchestra.run.smithi100.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34850.log.gz 2024-04-23T14:35:58.582 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39948.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-23T14:35:58.582 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39948.log.gz 2024-04-23T14:35:58.583 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37441.log 2024-04-23T14:35:58.584 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39074.log.gz 2024-04-23T14:35:58.584 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40583.log 2024-04-23T14:35:58.585 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37599.log 2024-04-23T14:35:58.586 INFO:teuthology.orchestra.run.smithi100.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37441.log.gz 2024-04-23T14:35:58.586 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40583.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40583.log.gz 2024-04-23T14:35:58.587 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38307.log 2024-04-23T14:35:58.588 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37285.log 2024-04-23T14:35:58.588 INFO:teuthology.orchestra.run.smithi100.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37599.log.gz 2024-04-23T14:35:58.589 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43355.log 2024-04-23T14:35:58.590 INFO:teuthology.orchestra.run.smithi100.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38307.log.gz 2024-04-23T14:35:58.590 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37285.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37285.log.gz 2024-04-23T14:35:58.591 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37495.log 2024-04-23T14:35:58.591 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43355.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43355.log.gz 2024-04-23T14:35:58.592 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42910.log 2024-04-23T14:35:58.593 INFO:teuthology.orchestra.run.smithi100.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37495.log.gz 2024-04-23T14:35:58.594 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41478.log 2024-04-23T14:35:58.595 INFO:teuthology.orchestra.run.smithi151.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-23T14:35:58.595 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39022.log 2024-04-23T14:35:58.596 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42910.log.gz 2024-04-23T14:35:58.596 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38333.log 2024-04-23T14:35:58.596 INFO:teuthology.orchestra.run.smithi100.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41478.log.gz 2024-04-23T14:35:58.598 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37547.log 2024-04-23T14:35:58.598 INFO:teuthology.orchestra.run.smithi100.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.39022.log.gz 2024-04-23T14:35:58.599 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38333.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38333.log.gz 2024-04-23T14:35:58.600 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37547.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37547.log.gz 2024-04-23T14:35:58.600 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42128.log 2024-04-23T14:35:58.601 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43284.log 2024-04-23T14:35:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38021.log 2024-04-23T14:35:58.603 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42128.log.gz 2024-04-23T14:35:58.603 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43284.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43284.log.gz 2024-04-23T14:35:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.36024.log 2024-04-23T14:35:58.605 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38021.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38021.log.gz 2024-04-23T14:35:58.605 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42852.log 2024-04-23T14:35:58.606 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.2.36024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34422.log 2024-04-23T14:35:58.607 INFO:teuthology.orchestra.run.smithi100.stderr: 91.7%/var/log/ceph/ceph-client.admin.42852.log: -- replaced with /var/log/ceph/ceph-client.2.36024.log.gz 2024-04-23T14:35:58.607 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42852.log.gz 2024-04-23T14:35:58.608 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34422.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34422.log.gz 2024-04-23T14:35:58.609 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34313.log 2024-04-23T14:35:58.610 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40609.log 2024-04-23T14:35:58.612 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42746.log 2024-04-23T14:35:58.612 INFO:teuthology.orchestra.run.smithi100.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34313.log.gz 2024-04-23T14:35:58.612 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35621.log 2024-04-23T14:35:58.613 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40609.log.gz 2024-04-23T14:35:58.613 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42746.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42746.log.gz 2024-04-23T14:35:58.614 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39783.log 2024-04-23T14:35:58.615 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39626.log 2024-04-23T14:35:58.615 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35621.log.gz 2024-04-23T14:35:58.616 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43873.log 2024-04-23T14:35:58.616 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39783.log.gz 2024-04-23T14:35:58.617 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39626.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39626.log.gz 2024-04-23T14:35:58.618 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33608.log 2024-04-23T14:35:58.618 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43873.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43873.log.gz 2024-04-23T14:35:58.619 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34156.log 2024-04-23T14:35:58.620 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39179.log 2024-04-23T14:35:58.620 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33608.log.gz 2024-04-23T14:35:58.621 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36889.log 2024-04-23T14:35:58.622 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34156.log.gz 2024-04-23T14:35:58.623 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39179.log: gzip 88.5% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.39179.log.gz -- 2024-04-23T14:35:58.623 INFO:teuthology.orchestra.run.smithi100.stderr: /var/log/ceph/ceph-client.admin.34078.log 2024-04-23T14:35:58.624 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36889.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36889.log.gz 2024-04-23T14:35:58.624 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34644.log 2024-04-23T14:35:58.625 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34078.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34078.log.gz 2024-04-23T14:35:58.625 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34852.log 2024-04-23T14:35:58.627 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34370.log 2024-04-23T14:35:58.627 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34644.log.gz 2024-04-23T14:35:58.628 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34852.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34852.log.gz 2024-04-23T14:35:58.628 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37469.log 2024-04-23T14:35:58.629 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34370.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34370.log.gz 2024-04-23T14:35:58.629 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-23T14:35:58.630 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43561.log 2024-04-23T14:35:58.631 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37469.log.gz 2024-04-23T14:35:58.631 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33968.log 2024-04-23T14:35:58.633 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43561.log: 86.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-23T14:35:58.633 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43561.log.gz 2024-04-23T14:35:58.633 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39496.log 2024-04-23T14:35:58.634 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42556.log 2024-04-23T14:35:58.635 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39496.log: 88.3% 92.7% -- replaced with /var/log/ceph/ceph-client.admin.33968.log.gz 2024-04-23T14:35:58.636 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39496.log.gz 2024-04-23T14:35:58.636 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38463.log 2024-04-23T14:35:58.636 INFO:teuthology.orchestra.run.smithi100.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42556.log.gz 2024-04-23T14:35:58.637 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34659.log 2024-04-23T14:35:58.639 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36126.log 2024-04-23T14:35:58.639 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38463.log.gz 2024-04-23T14:35:58.640 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41017.log 2024-04-23T14:35:58.641 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34659.log.gz 2024-04-23T14:35:58.642 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36126.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36126.log.gz 2024-04-23T14:35:58.642 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37969.log 2024-04-23T14:35:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40731.log 2024-04-23T14:35:58.643 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41017.log.gz 2024-04-23T14:35:58.644 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36204.log 2024-04-23T14:35:58.644 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37969.log.gz 2024-04-23T14:35:58.645 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40731.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.40731.log.gz 2024-04-23T14:35:58.646 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.36204.log: -- /var/log/ceph/ceph-client.admin.33248.log 2024-04-23T14:35:58.646 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36204.log.gz 2024-04-23T14:35:58.647 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39255.log 2024-04-23T14:35:58.648 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27687.log 2024-04-23T14:35:58.649 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33248.log.gz 2024-04-23T14:35:58.650 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39255.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39255.log.gz 2024-04-23T14:35:58.650 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34860.log 2024-04-23T14:35:58.650 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.27687.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.27687.log.gz 2024-04-23T14:35:58.652 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40000.log 2024-04-23T14:35:58.652 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34657.log 2024-04-23T14:35:58.652 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34860.log.gz 2024-04-23T14:35:58.654 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39974.log 2024-04-23T14:35:58.654 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40000.log.gz 2024-04-23T14:35:58.654 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34657.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34657.log.gz 2024-04-23T14:35:58.656 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38281.log 2024-04-23T14:35:58.656 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39974.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39974.log.gz 2024-04-23T14:35:58.656 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33660.log 2024-04-23T14:35:58.659 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35066.log 2024-04-23T14:35:58.659 INFO:teuthology.orchestra.run.smithi100.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38281.log.gz 2024-04-23T14:35:58.659 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33660.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33660.log.gz 2024-04-23T14:35:58.659 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34578.log 2024-04-23T14:35:58.659 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35066.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35066.log.gz 2024-04-23T14:35:58.659 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39101.log 2024-04-23T14:35:58.660 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43743.log 2024-04-23T14:35:58.661 INFO:teuthology.orchestra.run.smithi100.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34578.log.gz 2024-04-23T14:35:58.661 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39101.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39101.log.gz 2024-04-23T14:35:58.662 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41834.log 2024-04-23T14:35:58.662 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43743.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43743.log.gz 2024-04-23T14:35:58.663 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43232.log 2024-04-23T14:35:58.664 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33418.log 2024-04-23T14:35:58.664 INFO:teuthology.orchestra.run.smithi100.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41834.log.gz 2024-04-23T14:35:58.665 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35072.log 2024-04-23T14:35:58.666 INFO:teuthology.orchestra.run.smithi100.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43232.log.gz 2024-04-23T14:35:58.666 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33418.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33418.log.gz 2024-04-23T14:35:58.667 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35438.log 2024-04-23T14:35:58.668 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35072.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35072.log.gz 2024-04-23T14:35:58.668 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40452.log 2024-04-23T14:35:58.669 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37155.log 2024-04-23T14:35:58.669 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35438.log.gz 2024-04-23T14:35:58.671 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40452.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40452.log.gz 2024-04-23T14:35:58.671 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38151.log 2024-04-23T14:35:58.672 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37155.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37155.log.gz 2024-04-23T14:35:58.672 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40079.log 2024-04-23T14:35:58.673 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44003.log 2024-04-23T14:35:58.673 INFO:teuthology.orchestra.run.smithi100.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38151.log.gz 2024-04-23T14:35:58.674 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40079.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40079.log.gz 2024-04-23T14:35:58.674 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34130.log 2024-04-23T14:35:58.675 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44003.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44003.log.gz 2024-04-23T14:35:58.676 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33366.log 2024-04-23T14:35:58.677 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39814.log 2024-04-23T14:35:58.677 INFO:teuthology.orchestra.run.smithi100.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34130.log.gz 2024-04-23T14:35:58.678 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33366.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33366.log.gz 2024-04-23T14:35:58.679 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39814.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.39814.log.gz 2024-04-23T14:35:58.680 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33528.log 2024-04-23T14:35:58.681 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37756.log 2024-04-23T14:35:58.683 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38437.log 2024-04-23T14:35:58.683 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33528.log.gz 2024-04-23T14:35:58.683 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37756.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37756.log.gz 2024-04-23T14:35:58.684 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38437.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38437.log.gz 2024-04-23T14:35:58.685 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35048.log 2024-04-23T14:35:58.686 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38877.log 2024-04-23T14:35:58.688 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39574.log 2024-04-23T14:35:58.688 INFO:teuthology.orchestra.run.smithi100.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35048.log.gz 2024-04-23T14:35:58.689 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40661.log 2024-04-23T14:35:58.689 INFO:teuthology.orchestra.run.smithi100.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38877.log.gz 2024-04-23T14:35:58.690 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44053.log 2024-04-23T14:35:58.690 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39574.log.gz 2024-04-23T14:35:58.691 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40661.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40661.log.gz 2024-04-23T14:35:58.692 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43613.log 2024-04-23T14:35:58.692 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44053.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44053.log.gz 2024-04-23T14:35:58.692 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40531.log 2024-04-23T14:35:58.694 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42988.log 2024-04-23T14:35:58.695 INFO:teuthology.orchestra.run.smithi100.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43613.log.gz 2024-04-23T14:35:58.695 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40531.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40531.log.gz 2024-04-23T14:35:58.696 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36386.log 2024-04-23T14:35:58.696 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42988.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42988.log.gz 2024-04-23T14:35:58.696 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37521.log 2024-04-23T14:35:58.698 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41704.log 2024-04-23T14:35:58.698 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36386.log.gz 2024-04-23T14:35:58.699 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37521.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37521.log.gz 2024-04-23T14:35:58.700 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41704.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41704.log.gz 2024-04-23T14:35:58.700 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33554.log 2024-04-23T14:35:58.702 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39522.log 2024-04-23T14:35:58.703 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34344.log 2024-04-23T14:35:58.704 INFO:teuthology.orchestra.run.smithi100.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33554.log.gz 2024-04-23T14:35:58.704 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39522.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39522.log.gz 2024-04-23T14:35:58.705 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34344.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34344.log.gz 2024-04-23T14:35:58.705 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42962.log 2024-04-23T14:35:58.707 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38255.log 2024-04-23T14:35:58.708 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39600.log 2024-04-23T14:35:58.709 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42962.log.gz 2024-04-23T14:35:58.709 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38255.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-23T14:35:58.710 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38255.log.gz 2024-04-23T14:35:58.710 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27569.log 2024-04-23T14:35:58.710 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39600.log.gz 2024-04-23T14:35:58.713 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.27569.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.27569.log.gz 2024-04-23T14:35:58.713 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-23T14:35:58.713 INFO:teuthology.orchestra.run.smithi100.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-23T14:35:58.714 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38905.log 2024-04-23T14:35:58.715 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42345.log 2024-04-23T14:35:58.717 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38905.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38905.log.gz 2024-04-23T14:35:58.718 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38940.log 2024-04-23T14:35:58.718 INFO:teuthology.orchestra.run.smithi100.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42345.log.gz 2024-04-23T14:35:58.718 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37207.log 2024-04-23T14:35:58.720 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38940.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38940.log.gz 2024-04-23T14:35:58.721 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38717.log 2024-04-23T14:35:58.721 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37207.log.gz 2024-04-23T14:35:58.722 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36802.log 2024-04-23T14:35:58.723 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38717.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38717.log.gz 2024-04-23T14:35:58.723 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.36802.log: /var/log/ceph/ceph-client.admin.43099.log 2024-04-23T14:35:58.724 INFO:teuthology.orchestra.run.smithi100.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.36802.log.gz 2024-04-23T14:35:58.726 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43099.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43099.log.gz 2024-04-23T14:35:58.726 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42154.log 2024-04-23T14:35:58.727 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-23T14:35:58.729 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42154.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42154.log.gz 2024-04-23T14:35:58.729 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33500.log 2024-04-23T14:35:58.731 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35673.log 2024-04-23T14:35:58.733 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33500.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33500.log.gz 2024-04-23T14:35:58.736 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35673.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35673.log.gz 2024-04-23T14:35:58.736 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42285.log 2024-04-23T14:35:58.739 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42285.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42285.log.gz 2024-04-23T14:35:58.739 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43951.log 2024-04-23T14:35:58.741 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43587.log 2024-04-23T14:35:58.743 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43951.log: /var/log/ceph/ceph-client.admin.43587.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43587.log.gz 2024-04-23T14:35:58.744 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43951.log.gz 2024-04-23T14:35:58.745 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37839.log 2024-04-23T14:35:58.748 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37839.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37839.log.gz 2024-04-23T14:35:58.748 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37098.log 2024-04-23T14:35:58.749 INFO:teuthology.orchestra.run.smithi100.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-23T14:35:58.750 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37098.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37098.log.gz 2024-04-23T14:35:58.751 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37389.log 2024-04-23T14:35:58.752 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37259.log 2024-04-23T14:35:58.754 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37389.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37389.log.gz 2024-04-23T14:35:58.754 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37259.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37259.log.gz 2024-04-23T14:35:58.755 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36282.log 2024-04-23T14:35:58.756 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39653.log 2024-04-23T14:35:58.758 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36282.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36282.log.gz 2024-04-23T14:35:58.758 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39653.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39653.log.gz 2024-04-23T14:35:58.759 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37891.log 2024-04-23T14:35:58.759 INFO:teuthology.orchestra.run.smithi100.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-23T14:35:58.760 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43381.log 2024-04-23T14:35:58.761 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40183.log 2024-04-23T14:35:58.761 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37891.log.gz 2024-04-23T14:35:58.763 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43381.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43381.log.gz 2024-04-23T14:35:58.763 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37917.log 2024-04-23T14:35:58.763 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40183.log.gz 2024-04-23T14:35:58.764 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38769.log 2024-04-23T14:35:58.765 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34660.log 2024-04-23T14:35:58.766 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37917.log.gz 2024-04-23T14:35:58.766 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38769.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38769.log.gz 2024-04-23T14:35:58.767 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36068.log 2024-04-23T14:35:58.768 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34660.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34660.log.gz 2024-04-23T14:35:58.768 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38073.log 2024-04-23T14:35:58.769 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43258.log 2024-04-23T14:35:58.769 INFO:teuthology.orchestra.run.smithi100.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36068.log.gz 2024-04-23T14:35:58.771 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38073.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38073.log.gz 2024-04-23T14:35:58.771 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43258.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43258.log.gz 2024-04-23T14:35:58.772 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34052.log 2024-04-23T14:35:58.773 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37573.log 2024-04-23T14:35:58.775 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38743.log 2024-04-23T14:35:58.775 INFO:teuthology.orchestra.run.smithi100.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34052.log.gz 2024-04-23T14:35:58.776 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37573.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37573.log.gz 2024-04-23T14:35:58.776 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38743.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38743.log.gz 2024-04-23T14:35:58.777 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39127.log 2024-04-23T14:35:58.778 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43769.log 2024-04-23T14:35:58.780 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38229.log 2024-04-23T14:35:58.780 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39127.log.gz 2024-04-23T14:35:58.780 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35282.log 2024-04-23T14:35:58.781 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43769.log.gz 2024-04-23T14:35:58.781 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42102.log 2024-04-23T14:35:58.782 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38229.log.gz 2024-04-23T14:35:58.783 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35282.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35282.log.gz 2024-04-23T14:35:58.783 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42713.log 2024-04-23T14:35:58.784 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42102.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42102.log.gz 2024-04-23T14:35:58.784 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43977.log 2024-04-23T14:35:58.786 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41088.log 2024-04-23T14:35:58.786 INFO:teuthology.orchestra.run.smithi100.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.42713.log.gz 2024-04-23T14:35:58.787 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43977.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43977.log.gz 2024-04-23T14:35:58.787 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38521.log 2024-04-23T14:35:58.788 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41088.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41088.log.gz 2024-04-23T14:35:58.788 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35929.log 2024-04-23T14:35:58.790 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37813.log 2024-04-23T14:35:58.790 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38521.log.gz 2024-04-23T14:35:58.790 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35929.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35929.log.gz 2024-04-23T14:35:58.792 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37813.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37813.log.gz 2024-04-23T14:35:58.792 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40835.log 2024-04-23T14:35:58.793 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35595.log 2024-04-23T14:35:58.796 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35061.log 2024-04-23T14:35:58.796 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40835.log.gz 2024-04-23T14:35:58.796 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41622.log 2024-04-23T14:35:58.797 INFO:teuthology.orchestra.run.smithi100.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35595.log.gz 2024-04-23T14:35:58.797 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38203.log 2024-04-23T14:35:58.797 INFO:teuthology.orchestra.run.smithi100.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35061.log.gz 2024-04-23T14:35:58.799 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38177.log 2024-04-23T14:35:58.799 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41622.log.gz 2024-04-23T14:35:58.800 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42687.log 2024-04-23T14:35:58.800 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38203.log.gz 2024-04-23T14:35:58.801 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38177.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38177.log.gz 2024-04-23T14:35:58.801 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34287.log 2024-04-23T14:35:58.802 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42687.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42687.log.gz 2024-04-23T14:35:58.803 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41756.log 2024-04-23T14:35:58.804 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40426.log 2024-04-23T14:35:58.804 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34287.log.gz 2024-04-23T14:35:58.805 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41756.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41756.log.gz 2024-04-23T14:35:58.805 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38966.log 2024-04-23T14:35:58.806 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40426.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40426.log.gz 2024-04-23T14:35:58.806 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38847.log 2024-04-23T14:35:58.807 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42802.log 2024-04-23T14:35:58.808 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38966.log.gz 2024-04-23T14:35:58.808 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43483.log 2024-04-23T14:35:58.809 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38847.log.gz 2024-04-23T14:35:58.811 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.42802.log: /var/log/ceph/ceph-client.admin.43483.log: 90.1% -- replaced with /var/log/ceph/ceph-client.0.42802.log.gz 2024-04-23T14:35:58.811 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43483.log.gz 2024-04-23T14:35:58.811 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34865.log 2024-04-23T14:35:58.812 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42049.log 2024-04-23T14:35:58.813 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38125.log 2024-04-23T14:35:58.814 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34865.log.gz 2024-04-23T14:35:58.815 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42049.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42049.log.gz 2024-04-23T14:35:58.816 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38125.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38125.log.gz 2024-04-23T14:35:58.816 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33738.log 2024-04-23T14:35:58.818 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33580.log 2024-04-23T14:35:58.819 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43047.log 2024-04-23T14:35:58.819 INFO:teuthology.orchestra.run.smithi100.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33738.log.gz 2024-04-23T14:35:58.820 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37233.log 2024-04-23T14:35:58.820 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33580.log.gz 2024-04-23T14:35:58.821 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43047.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43047.log.gz 2024-04-23T14:35:58.822 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41860.log 2024-04-23T14:35:58.822 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37233.log.gz 2024-04-23T14:35:58.823 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34500.log 2024-04-23T14:35:58.824 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41114.log 2024-04-23T14:35:58.825 INFO:teuthology.orchestra.run.smithi100.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41860.log.gz 2024-04-23T14:35:58.825 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34500.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34500.log.gz 2024-04-23T14:35:58.826 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40635.log 2024-04-23T14:35:58.827 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41114.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41114.log.gz 2024-04-23T14:35:58.827 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43795.log 2024-04-23T14:35:58.828 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35517.log 2024-04-23T14:35:58.828 INFO:teuthology.orchestra.run.smithi100.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40635.log.gz 2024-04-23T14:35:58.829 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38359.log 2024-04-23T14:35:58.829 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43795.log.gz 2024-04-23T14:35:58.830 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.35517.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35517.log.gz 2024-04-23T14:35:58.831 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37124.log 2024-04-23T14:35:58.831 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37311.log 2024-04-23T14:35:58.832 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38359.log.gz 2024-04-23T14:35:58.833 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37337.log 2024-04-23T14:35:58.833 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37124.log.gz 2024-04-23T14:35:58.834 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37311.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37311.log.gz 2024-04-23T14:35:58.835 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37943.log 2024-04-23T14:35:58.835 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37337.log.gz 2024-04-23T14:35:58.835 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39705.log 2024-04-23T14:35:58.837 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36178.log 2024-04-23T14:35:58.837 INFO:teuthology.orchestra.run.smithi100.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37943.log.gz 2024-04-23T14:35:58.839 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.39705.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39705.log.gz 2024-04-23T14:35:58.839 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41166.log 2024-04-23T14:35:58.839 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33840.log 2024-04-23T14:35:58.840 INFO:teuthology.orchestra.run.smithi100.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36178.log.gz 2024-04-23T14:35:58.841 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43180.log 2024-04-23T14:35:58.841 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41166.log.gz 2024-04-23T14:35:58.842 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40504.log 2024-04-23T14:35:58.842 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33840.log.gz 2024-04-23T14:35:58.844 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43180.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43180.log.gz 2024-04-23T14:35:58.844 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34183.log 2024-04-23T14:35:58.844 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40504.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40504.log.gz 2024-04-23T14:35:58.845 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38489.log 2024-04-23T14:35:58.846 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34209.log 2024-04-23T14:35:58.846 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34183.log.gz 2024-04-23T14:35:58.848 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38489.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38489.log.gz 2024-04-23T14:35:58.848 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34209.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34209.log.gz 2024-04-23T14:35:58.849 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36542.log 2024-04-23T14:35:58.850 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34656.log 2024-04-23T14:35:58.851 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38411.log 2024-04-23T14:35:58.852 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36542.log.gz 2024-04-23T14:35:58.852 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41936.log 2024-04-23T14:35:58.852 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34656.log.gz 2024-04-23T14:35:58.854 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38411.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38411.log.gz 2024-04-23T14:35:58.854 INFO:teuthology.orchestra.run.smithi100.stderr:gzip/var/log/ceph/ceph-client.admin.41936.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33686.log 2024-04-23T14:35:58.855 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41936.log.gz 2024-04-23T14:35:58.855 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22762.log 2024-04-23T14:35:58.856 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.33686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38691.log 2024-04-23T14:35:58.857 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33686.log.gz 2024-04-23T14:35:58.857 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph.tmp-client.admin.22762.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22762.log.gz 2024-04-23T14:35:58.858 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34854.log 2024-04-23T14:35:58.859 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43073.log 2024-04-23T14:35:58.859 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38691.log.gz 2024-04-23T14:35:58.860 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27788.log 2024-04-23T14:35:58.860 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34854.log.gz 2024-04-23T14:35:58.861 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43073.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43073.log.gz 2024-04-23T14:35:58.862 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.27788.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27788.log.gz 2024-04-23T14:35:58.863 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38099.log 2024-04-23T14:35:58.864 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34396.log 2024-04-23T14:35:58.865 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.38099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34849.log 2024-04-23T14:35:58.866 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38099.log.gz 2024-04-23T14:35:58.866 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34396.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34396.log.gz 2024-04-23T14:35:58.867 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34849.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34849.log.gz 2024-04-23T14:35:58.868 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40240.log 2024-04-23T14:35:58.868 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41674.log 2024-04-23T14:35:58.870 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.40240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36646.log 2024-04-23T14:35:58.870 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40240.log.gz 2024-04-23T14:35:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41648.log 2024-04-23T14:35:58.872 INFO:teuthology.orchestra.run.smithi100.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41674.log.gz 2024-04-23T14:35:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36646.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36646.log.gz 2024-04-23T14:35:58.873 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41808.log 2024-04-23T14:35:58.874 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41648.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41648.log.gz 2024-04-23T14:35:58.875 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41808.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41808.log.gz 2024-04-23T14:35:58.908 INFO:teuthology.orchestra.run.smithi100.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-23T14:35:59.124 INFO:teuthology.orchestra.run.smithi151.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-23T14:35:59.192 INFO:teuthology.orchestra.run.smithi151.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-23T14:35:59.193 INFO:teuthology.orchestra.run.smithi151.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-23T14:35:59.193 INFO:teuthology.orchestra.run.smithi151.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-23T14:35:59.195 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-04-23T14:35:59.196 INFO:teuthology.orchestra.run.smithi151.stderr:real 0m0.810s 2024-04-23T14:35:59.196 INFO:teuthology.orchestra.run.smithi151.stderr:user 0m3.164s 2024-04-23T14:35:59.196 INFO:teuthology.orchestra.run.smithi151.stderr:sys 0m0.299s 2024-04-23T14:35:59.230 INFO:teuthology.orchestra.run.smithi100.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-23T14:35:59.255 INFO:teuthology.orchestra.run.smithi100.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-23T14:35:59.377 INFO:teuthology.orchestra.run.smithi100.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-23T14:35:59.396 INFO:teuthology.orchestra.run.smithi100.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-23T14:35:59.398 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-04-23T14:35:59.398 INFO:teuthology.orchestra.run.smithi100.stderr:real 0m1.031s 2024-04-23T14:35:59.398 INFO:teuthology.orchestra.run.smithi100.stderr:user 0m3.945s 2024-04-23T14:35:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:sys 0m0.917s 2024-04-23T14:35:59.400 INFO:tasks.ceph:Archiving logs... 2024-04-23T14:35:59.400 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669587/remote/smithi100/log 2024-04-23T14:35:59.401 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T14:36:00.147 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669587/remote/smithi151/log 2024-04-23T14:36:00.148 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T14:36:00.400 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-04-23T14:36:00.400 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-04-23T14:36:00.400 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-04-23T14:36:00.400 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-04-23T14:36:00.526 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-23T14:36:00.536 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-23T14:36:00.536 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T14:36:00.539 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T14:36:00.623 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-23T14:36:00.632 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-23T14:36:00.632 DEBUG:teuthology.orchestra.run.smithi100:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T14:36:00.635 DEBUG:teuthology.orchestra.run.smithi151:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T14:36:00.645 INFO:teuthology.orchestra.run.smithi100.stdout: remote refid st t when poll reach delay offset jitter 2024-04-23T14:36:00.646 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================== 2024-04-23T14:36:00.646 INFO:teuthology.orchestra.run.smithi100.stdout:*hv01.front.sepi 44.4.53.2 2 u 34 64 377 0.098 -4.702 2.717 2024-04-23T14:36:00.646 INFO:teuthology.orchestra.run.smithi100.stdout:+hv02.front.sepi 63.231.80.2 3 u 33 64 377 0.090 +0.584 2.205 2024-04-23T14:36:00.646 INFO:teuthology.orchestra.run.smithi100.stdout:+hv03.front.sepi 74.6.168.72 3 u 30 64 377 0.072 -2.702 1.302 2024-04-23T14:36:00.646 INFO:teuthology.orchestra.run.smithi100.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-23T14:36:00.650 INFO:teuthology.orchestra.run.smithi151.stdout: remote refid st t when poll reach delay offset jitter 2024-04-23T14:36:00.651 INFO:teuthology.orchestra.run.smithi151.stdout:============================================================================== 2024-04-23T14:36:00.651 INFO:teuthology.orchestra.run.smithi151.stdout:*hv01.front.sepi 44.4.53.2 2 u 32 64 377 0.092 -1.842 0.635 2024-04-23T14:36:00.651 INFO:teuthology.orchestra.run.smithi151.stdout:+hv02.front.sepi 63.231.80.2 3 u 38 64 377 0.108 +0.513 0.794 2024-04-23T14:36:00.651 INFO:teuthology.orchestra.run.smithi151.stdout:+hv03.front.sepi 74.6.168.72 3 u 28 64 377 0.091 -1.671 0.497 2024-04-23T14:36:00.651 INFO:teuthology.orchestra.run.smithi151.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-23T14:36:00.652 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-23T14:36:00.661 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-23T14:36:00.662 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-23T14:36:00.696 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-23T14:36:00.732 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-23T14:36:00.765 INFO:teuthology.task.internal:Duration was 958.955449 seconds 2024-04-23T14:36:00.765 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-23T14:36:00.800 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-23T14:36:00.800 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T14:36:00.803 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T14:36:00.905 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-23T14:36:00.905 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi100.front.sepia.ceph.com 2024-04-23T14:36:00.905 DEBUG:teuthology.orchestra.run.smithi100:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T14:36:00.924 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi151.front.sepia.ceph.com 2024-04-23T14:36:00.925 DEBUG:teuthology.orchestra.run.smithi151:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T14:36:00.945 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-23T14:36:00.945 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T14:36:00.967 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T14:36:01.029 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-23T14:36:01.029 DEBUG:teuthology.orchestra.run.smithi100:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T14:36:01.047 DEBUG:teuthology.orchestra.run.smithi151:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T14:36:01.144 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-23T14:36:01.154 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-23T14:36:01.154 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T14:36:01.174 DEBUG:teuthology.orchestra.run.smithi151:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T14:36:01.193 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-23T14:36:01.207 DEBUG:teuthology.orchestra.run.smithi100:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T14:36:01.219 DEBUG:teuthology.orchestra.run.smithi151:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T14:36:01.233 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = core 2024-04-23T14:36:01.251 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = core 2024-04-23T14:36:01.269 DEBUG:teuthology.orchestra.run.smithi100:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T14:36:01.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:36:01.304 DEBUG:teuthology.orchestra.run.smithi151:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T14:36:01.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:36:01.317 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-23T14:36:01.327 INFO:teuthology.task.internal:Transferring archived files... 2024-04-23T14:36:01.328 DEBUG:teuthology.misc:Transferring archived files from smithi100:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669587/remote/smithi100 2024-04-23T14:36:01.328 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T14:36:01.376 DEBUG:teuthology.misc:Transferring archived files from smithi151:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669587/remote/smithi151 2024-04-23T14:36:01.376 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T14:36:01.401 INFO:teuthology.task.internal:Removing archive directory... 2024-04-23T14:36:01.401 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T14:36:01.415 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T14:36:01.447 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-23T14:36:01.460 INFO:teuthology.task.internal:Not uploading archives. 2024-04-23T14:36:01.460 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-23T14:36:01.482 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-23T14:36:01.482 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T14:36:01.485 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T14:36:01.491 INFO:teuthology.orchestra.run.smithi100.stdout: 658123 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 14:36 /home/ubuntu/cephtest 2024-04-23T14:36:01.503 INFO:teuthology.orchestra.run.smithi151.stdout: 655432 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 14:36 /home/ubuntu/cephtest 2024-04-23T14:36:01.505 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-23T14:36:01.518 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-23T14:36:01.585 INFO:teuthology.nuke:Checking targets against current locks 2024-04-23T14:36:01.613 DEBUG:teuthology.nuke:shortname: smithi100 2024-04-23T14:36:01.613 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T14:36:01.638 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669587', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 14:05:43.108657', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE/aCshXzWt2yH2pHsDWDUnQMtkY17vqfGEhcBIgWVJVdM6pUoFiqJpdrUwnY+nyeH6VB0CdTY6TY8BHoavBEJY='} 2024-04-23T14:36:01.641 DEBUG:teuthology.nuke:shortname: smithi151 2024-04-23T14:36:01.642 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T14:36:01.667 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669587', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 14:05:43.109955', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG95C7i0TtHFsiFmo24RegfJZIQu1ezh/b6jEqzBhGM/8ylKezjQ6tKnnv+fWihm7hbm1I4T7PxgWGf5ucNRTaQ='} 2024-04-23T14:36:01.695 INFO:teuthology.orchestra.console.smithi100:Power off 2024-04-23T14:36:01.695 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-23T14:36:01.718 DEBUG:teuthology.orchestra.console.smithi100:power off output: Chassis Power Control: Down/Off 2024-04-23T14:36:01.718 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T14:36:01.732 INFO:teuthology.orchestra.console.smithi151:Power off 2024-04-23T14:36:01.732 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-23T14:36:01.739 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-04-23T14:36:01.753 DEBUG:teuthology.orchestra.console.smithi151:power off output: Chassis Power Control: Down/Off 2024-04-23T14:36:01.753 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T14:36:01.776 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is on 2024-04-23T14:36:05.740 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T14:36:05.777 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T14:36:05.852 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-04-23T14:36:05.887 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is on 2024-04-23T14:36:09.854 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T14:36:09.888 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T14:36:09.966 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2024-04-23T14:36:09.998 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is off 2024-04-23T14:36:10.067 INFO:teuthology.orchestra.console.smithi100:Power off completed 2024-04-23T14:36:10.099 INFO:teuthology.orchestra.console.smithi151:Power off completed 2024-04-23T14:36:10.204 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/pool-perm} duration: 958.9554493427277 failure_reason: 'Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm)' flavor: default mon.a-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.b-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=6abe976484c14aa4b401bba1ebe056dc status: fail success: false 2024-04-23T14:36:10.204 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T14:36:10.295 INFO:teuthology.run:FAIL